I am developping an application that retrieves some data from a server.
I have two options:
Get the whole data set from the server and then work with it using the pattern 'singleton' to reduce the number of queries
For each query, get the corresponding data set from the server
What is the best choice?
In my opinion it depends.
It depends on the size of the data and if it even makes sense to return data that your user may not even need.
Personally, in my app that I am building I will be returning only the data that is required at that time. Obviously though, once I have the data I won't be fetching it again if it makes sense to keep hold of it for good or even just temporarily.
I agree with C0deAttack's answer. Your goal should be to minimize network traffic within the constraints of your app being a "good citizen" on the phone. (That means that your app does not negatively impact the user or other applications by using too many resources — including memory and file space.)
From the sound of it, I'm guessing that the data are not that voluminous. If so, I would recommend caching the response and use it locally, thus avoiding repeated queries to the server. Depending on how often the data changes, you might even consider making it persistent, so that the app doesn't have to query the server the next time it starts up. If the response includes an estimated time before it is considered outdated, that would help in establishing an update schedule. (Google's license server uses this idea.)
P.S. I don't see that this has anything (directly) to do with a singleton pattern.
How about storing your data in an sqlite database and do your queries with sql? Since you get sorting and ordering for free, it can help you writing less code. Also you have instant offline functionality if your user has no internet connection. :)
Related
I see that Firebase website has a lot of documentation to help us optimize the usage of resources, however, I have not found a detailed example of the resources used.
This could be useful to me to understand how to build my applications and to better choose the strategy in terms of performance and cost.
TAKING ANDROID AS EXAMPLE
I understand that when i do a:
query.addListenerForSingleValueEvent(...);
all the reference is "queried" to the database so that is a single query but takes down all the object.
if i do:
query.addValueEventListener(...);
the connection is kept open, but will it keep making connections on time intervals?
Or maybe is considered like a single connection in terms of billing?
And after a change on the database, will it query all the object down again?
In general how much is heavier and expensive to make a single request vs using the realtime-db feature of listening to a reference?
Maybe there is a section in the docs that explain this but I didn't found it.
query.addListenerForSingleValueEvent(...);
all the reference queried" to the database so that is a single
query but takes down all the object.
It will listen once to the objects within inside the child you are querying, not all of the objects inside the database.
query.addValueEventListener(...);
the connection is kept open, but will it keep making connections on
time intervals?
It has no intervals, instead it listens whenever a change is made into your database, lets say you change certain value from your database and that will trigger your addValueEventListener. This will only consume resources when some value changes into your database, so the usage will be a variant with your database usage, instead , addListenerForSingleValue will fire just once to query your data and we can assure that it will consume less network resources than a listener that is always listen to some changes to bring into your app
Check this usefull link : https://www.firebase.com/docs/java-api/javadoc/com/firebase/client/ValueEventListener.html
When my app is offline and I am adding or updating a document the memory increases. Also, showing lists of documents take longer to load. If I run the same code when the device is online the memory stays consistent as well as the speed of the activities that have lists of documents.
I'm currently doing saves like the following:
collectionRef.document(id).set(obj, SetOptions.merge());
Or for batching a couple of records:
batch.set(docRef1, obj1);
batch.set(docRef2, obj2);
batch.commit();
I had listeners for onComplete but in the accepted answer for this question, it seems to indicate that listeners are unnecessary in most situations and that you can't wait for it to complete anyway when you're offline.
In another question they indicate in code that a "Snapshot" is required to properly do online and offline saving: Offline issue with Firestore vs Firebase. But I can't find anywhere else indicating if that will make a difference. I think of Snapshots as being something you attach to a document or query when you want to be notified of changes to it and attaching a listener like that will result in a memory leak if it isn't removed.
Another part of all of this is how this slowness might affect data integrity. When I watch in the profiler in Android Studio I see that the FirestoreWorker can get to a point where it is constantly working even if I don't do anything in the app. I'm not just talking a few seconds, more like a minute. There isn't any ordering guarantee of writes when it is offline that I can find. Trying to stop and restart the app doesn't seem to have any effect on the slowness (although it will reset the memory).
So all of this leads to the question: what is the proper way to add/update data in Firestore when offline so that the app's memory doesn't grow unbounded and slow down?
Cloud Firestore uses SQLite for its persistence mechanism. So for intermittent periods of offline activity, you shouldn't have problems with performance or durability.
However, if you intend to use a Firestore database for very long periods of time, there are some things you should be aware of. Cloud Firestore was not built as an offline database, is an online database that continues to work when you're offline for short or longer periods of time. When offline, pending writes that have not yet been synced to the server are held in a queue. If you do too many write operations without going online to sync them, that queue will grow fast and it will not slow down only the write operations it will also slow down your read operations.
So I suggest use this database for its online capabilities. As one of the Firebase engineers said and I quote, "It is impossible to build a slow query in Firestore". So, the performance comes from the new indexing capabilities on the backend and these optimizations don't exist when you're offline.
One more thing, if you have many offline clients who are trying to write to the same document, only the last one will be actually be written to servers when the state is changed.
So, to answer your question, there is no proper way to add/update data in Firestore when offline, to have a less memory usage. Just go online and that's it!
I am planning on writing an application that saves a fair amount of data. Historically, I have simply written data directly to a server, and only used some simple key/value storage with shared preferences for local storage.
I am considering this time, instead, using SQLite to save the information at first, and sync the data to the server in the background later. This will benefit the user in a few ways: 1) can use the app offline 2) don't have to worry about data being saved right away, it happens when ever it can 3) more reliability.
My approach will be to get/set data from SQLite during UI usage, and use a background process to find new rows and put them on the server, flagging them as synced when it happens.
Does this sound reasonable?
You can use SQLIte for your scenario. But, while implementing, you can follow any one of this approach.
Approach #1: Use an Abstract Factory to Instantiate the SQLiteOpenHelper.
Approach #2: Wrap the SQLiteDatabase in a ContentProvider
Refer to this link for how to implement these 2 approaches. http://www.androiddesignpatterns.com/2012/05/correctly-managing-your-sqlite-database.html
Key points to be noted while using SQLite
Sqlite takes care of the file level locking.
Many threads can read,one can write. The locks prevent more than one
writing.
Android implements some java locking in SQLiteDatabase to help keep
things straight.
If we handle the database incorrectly from many threads and mess up the code, your
database will not be corrupted. Only few updates will be lost.
How "Multiple Threads - DB access" can be used for your scenario
The SqliteOpenHelper object holds on to one database connection.
If you try to write to the database from actual distinct connections (multiple threads) at the same time, one will fail. It will not wait till the first is done and then write. It will simply not write your change. Worse, if you don’t call the right version of insert/update on the SQLiteDatabase, you won’t get an exception. You’ll just get a message in your LogCat, and that will be it.
So recommended to write using single thread and read from multiple threads if necessary for faster access.
Does this sound reasonable?
Yes. Note that the synchronization process can get tricky (e.g., what happens if the server hiccups halfway through?), but that has mostly to do with synchronization and little to do with SQLite.
We implemented a solution that used a SQLite db on the device to sync data via a web service to the master database. We did this for a couple reasons: offline, poor connection, manual sync.
For our solution we had a flag on the table that determined if the data was pushed to the web service. Our web service also provided data back to our application to let us know if the data was received and processed correctly. This allowed us to clean up the data on the device, send notifications if there were failures, and resubmit the data if there were previous failures.
You can use push notifications as well if you have fixed the issues on the backend and have the device resend the data to the web service. This worked really well for us.
Why do we use the sqlite data base in android.I am developing an android application where the data is to be fetched from the server and do some data calculation and show it on the UI.
Is it good for me to fetch the data into the sqlite DB and update the UI on regular interval from the sqlite in evry 20 minutes or will it be good to sent the Http get request to the server and update the data from teh response on the UI.
I wanted to know which one will be better and why?Why to involve sqlite DB?
The data corresponds to some 40X40 table data on which some heavy mathematical processing is to be done and then displayed on the UI(similar to Stock market application) and data needs to be cleared after every 12 hours.
plz advice
Rgds,
Raul
It is good to use database in your case.
Pros:
If your application gets closed the in memory data will be lost, but after that you will be able to restore the state from the database if you have one
Especially for the case of complex calculations it is good to store the result once in the database and not recalculate it multiple times on demand
The database will untie your UI from the internet connection and thus you will be able to display results even if there is not internet connection
Using database you will be able to fetch the updated data from a background service, without impacting your UI
Organizing your data in database usually makes it a lot easier to manage all the application data.
Cons:
Adding database will require a bit of additional effort on your side
As you can see my list proves you SHOULD use database in your case. Maybe I am biased, but at least I provide you with things to consider.
It's really a design decision, SQLite offers a very robust way to organize and persist your data, you're only other options are to write to a file, or to save in SharedPrefs, both methods become a lot harder to manage once the size of your data begins to grow, as you must manually keep a list of objects and manage their names etc etc. 40 x 40 table data is large enough to justify using SQLite, even if you are dropping and recreating the table every 12 hours.
You might want to consider using an ORM library to make fetching and saving data from the DB simpler, ORMLite is good and compatible with Android
http://ormlite.com/
If your application relies heavily on an internet connection you don't need to buffer information in the database. However if you want to use the app where you have bad or no signal you might want to used cached values from the sqlite database.
With slow internet connection your application may be unresponsive so caching may be a good idea - but it doesn't necessarily be in the sqlite database. You should use the sqlite database for data that is required by the device frequently and that is irrelevant to your server component.
If the data is updated frequently but only while the application runs you might want to cache in the devices memory. I assume your app is not running all the time within the 12 hours but is called regularly instead to check something.
12hrs is a long time, so rather than leaving your data wander in RAM, i would suggest you to use database. Because you never know when you may need to read it again.
Otherwise, if your purpose is only to downloaded data, process it and display in activity, then you dont need to involve database because if your app is closed (due to user or low memory), in anyway your app will be downloading fresh data from server... am i right?
> update the UI on regular interval from the sqlite in evry 20 minutes
Dont expect your app to be open for such a long duration.
To precisely suggest to your case
Avoid DB
Fetch Data at app start or at appropriate time when app is opened
and save it in plain java objects.
Define Methods within it that perform operation in it.
Make map or list to save those POJO
Define Seprate Controller Classes within your project to update map of pojo at any
specific change to make fresh data available to UI.
I'm getting hung up on how to handle the data for an app I'm designing. I want to pull a list of items from the net. The data is updated routinely, and I think it would be good to store all the data on the device so the app can load quickly and refresh the data in a background thread rather than have to wait for the network on every start-up.
I think I should make the data available in an XML and have a thread parse and save into a SQLite DB, but I'm not sure if that's the "best practice." Are there other ways that people go about handling this?
The cleanest way (or at least, I think it is the cleanest way) is to implement a custom ContentProvider class that interfaces with your server. You can query the contenprovider and if it doesn't have the data in a local cache (for example a SQLite db as you said) it downloads it from your server and adds it to the local data. Why a content provider? because then you can easily access your data across apps and you have a nice and clean way to get your data when using intents.
Also, I personally prefer not to download data when the app is not running, because it will cost battery life while the user does not actively requests the data.
Sounds reasonable.
If the download of the new data takes more than some seconds, you might want to use a Service. This will allow you to continue the update even when the user has left your app.
Also, think about how you will notify the user that something is going on. Displaying some progress indicator is always a good idea. Otherwise, users might just think the data is not up to date because the app is broken.