I have a requirement in my application wherein I have to every now and then perform insert and update queries on the database.
Now, my question is shall maintain cache within the application and after specific intervals or may be after say 20 entries, commit the cache into the database?
If this is the scenario where shall I maintain the cache. Should it be on the application level or in the service?
There is a possibility that say the application abruptly crashes and hence the data that is there in the cache no longer persists.
What can be the possible scenarios for this.
I have read somewhere that everytime opening and closing the database is an overhead.
Note: my database resides in the sdcard.
Thanks.
I have a requirement in my application wherein I have to every now and then perform insert and update queries on the database. Now, my question is shall maintain cache within the application and after specific intervals or may be after say 20 entries, commit the cache into the database?
Insert operation is fairly fast in SQLite. Make sure you are closing the database in finally block to avoid db in opened state. That can block further inserts. As much as possible, use batch inserts, they are much faster. See here
If this is the scenario where shall I maintain the cache. Should it be on the application level or in the service?
Service also comes at the application level :) . There is a possibility of maintaining the cache in a service and making the service live in a separate process. See here.
There is a possibility that say the application abruptly crashes and hence the data that is there in the cache no longer persists.
The only option that stays is, either maintain the cache in other process(which can again crash anytime) or maintain it on disk, may be internal storage. Maintaining a cache on internal storage is as good or bad as writing to sqlite.
What can be the possible scenarios for this.
You implement cache or you dont.
I have read somewhere that everytime opening and closing the database is an overhead. Note: my database resides in the sdcard.
AFAIK, Opening an SQLite is comparable to opening a file(a little slower though) and not comparable to opening an Oracle database. So most of the times it is affordable to open the database, but batching inserts to improve performance will never hurt ;)
Remember that in Android, your application may be killed at any time. So if you want to make sure that all your data is saved, you should store it into the database every time some data is entered and not cache it.
There is an overhead in accessing the database and for that reason make sure you don't update the database on the UI thread. If you do it correctly, performance should not deteriorate for the user.
Related
I'm currently working on an Android application that requires reading from call history and text history. And for further analysis, I've extracted these the huge amount of entries from the corresponding content provider, created and inserted all of them to a SQLite database.
But the problem I've encountered is when this is running on a phone that has been used for years (meaning there's an enormous amount of data generated from calls and texts), the retrieval process and database building process takes too much time and may even cause the app to crash. Even if i tried to put these process in a AsyncTask, the problem still exists. So my question is:
Am i doing it in a good way to just put any time consuming operations away from Main UI, OR What's a better way, if any, to handle very very large amount of data in Android?
Use pagination logic. Fetch only the most recent and relevant data and load older data if the user requests it.
Call history on most android phones is limited to 500 entries CallLog.Calls, while SMS provider has not such limits, you can query the count of the tables and limit your queries to 50 at a time and pass it to a separate thread for processing. Also make sure you run this in a background service with low priority so as to not disturb any other operations ongoing in the device.
I am planning on writing an application that saves a fair amount of data. Historically, I have simply written data directly to a server, and only used some simple key/value storage with shared preferences for local storage.
I am considering this time, instead, using SQLite to save the information at first, and sync the data to the server in the background later. This will benefit the user in a few ways: 1) can use the app offline 2) don't have to worry about data being saved right away, it happens when ever it can 3) more reliability.
My approach will be to get/set data from SQLite during UI usage, and use a background process to find new rows and put them on the server, flagging them as synced when it happens.
Does this sound reasonable?
You can use SQLIte for your scenario. But, while implementing, you can follow any one of this approach.
Approach #1: Use an Abstract Factory to Instantiate the SQLiteOpenHelper.
Approach #2: Wrap the SQLiteDatabase in a ContentProvider
Refer to this link for how to implement these 2 approaches. http://www.androiddesignpatterns.com/2012/05/correctly-managing-your-sqlite-database.html
Key points to be noted while using SQLite
Sqlite takes care of the file level locking.
Many threads can read,one can write. The locks prevent more than one
writing.
Android implements some java locking in SQLiteDatabase to help keep
things straight.
If we handle the database incorrectly from many threads and mess up the code, your
database will not be corrupted. Only few updates will be lost.
How "Multiple Threads - DB access" can be used for your scenario
The SqliteOpenHelper object holds on to one database connection.
If you try to write to the database from actual distinct connections (multiple threads) at the same time, one will fail. It will not wait till the first is done and then write. It will simply not write your change. Worse, if you don’t call the right version of insert/update on the SQLiteDatabase, you won’t get an exception. You’ll just get a message in your LogCat, and that will be it.
So recommended to write using single thread and read from multiple threads if necessary for faster access.
Does this sound reasonable?
Yes. Note that the synchronization process can get tricky (e.g., what happens if the server hiccups halfway through?), but that has mostly to do with synchronization and little to do with SQLite.
We implemented a solution that used a SQLite db on the device to sync data via a web service to the master database. We did this for a couple reasons: offline, poor connection, manual sync.
For our solution we had a flag on the table that determined if the data was pushed to the web service. Our web service also provided data back to our application to let us know if the data was received and processed correctly. This allowed us to clean up the data on the device, send notifications if there were failures, and resubmit the data if there were previous failures.
You can use push notifications as well if you have fixed the issues on the backend and have the device resend the data to the web service. This worked really well for us.
In Android, is the main thread favored in the use of the CPU, compared to Service? If yes, can I give, for a certain period, the complete control of CPU to android service? Or more generally give different priority?
If your question is related to the problems with db access, you should revisit your db access patterns, because that's where your problem lies. Otherwise, you may play with priorities and processor affinities and services and all this stuff, make the system unresponsive and eat the battery in an hour as a result, and still get the db access problems.
When there's a service with a need of constant RW access to the db, and you try to open the same db from another place, it would most definitely fail. Therefore, don't try to access db in onClick(), but have your service to prepare the necessary data and save it somewhere to be loaded when button is clicked without accessing the db.
Sorry, could not tell you more unless you explain the details of what kind of data you use and how you want to process it.
Why do we use the sqlite data base in android.I am developing an android application where the data is to be fetched from the server and do some data calculation and show it on the UI.
Is it good for me to fetch the data into the sqlite DB and update the UI on regular interval from the sqlite in evry 20 minutes or will it be good to sent the Http get request to the server and update the data from teh response on the UI.
I wanted to know which one will be better and why?Why to involve sqlite DB?
The data corresponds to some 40X40 table data on which some heavy mathematical processing is to be done and then displayed on the UI(similar to Stock market application) and data needs to be cleared after every 12 hours.
plz advice
Rgds,
Raul
It is good to use database in your case.
Pros:
If your application gets closed the in memory data will be lost, but after that you will be able to restore the state from the database if you have one
Especially for the case of complex calculations it is good to store the result once in the database and not recalculate it multiple times on demand
The database will untie your UI from the internet connection and thus you will be able to display results even if there is not internet connection
Using database you will be able to fetch the updated data from a background service, without impacting your UI
Organizing your data in database usually makes it a lot easier to manage all the application data.
Cons:
Adding database will require a bit of additional effort on your side
As you can see my list proves you SHOULD use database in your case. Maybe I am biased, but at least I provide you with things to consider.
It's really a design decision, SQLite offers a very robust way to organize and persist your data, you're only other options are to write to a file, or to save in SharedPrefs, both methods become a lot harder to manage once the size of your data begins to grow, as you must manually keep a list of objects and manage their names etc etc. 40 x 40 table data is large enough to justify using SQLite, even if you are dropping and recreating the table every 12 hours.
You might want to consider using an ORM library to make fetching and saving data from the DB simpler, ORMLite is good and compatible with Android
http://ormlite.com/
If your application relies heavily on an internet connection you don't need to buffer information in the database. However if you want to use the app where you have bad or no signal you might want to used cached values from the sqlite database.
With slow internet connection your application may be unresponsive so caching may be a good idea - but it doesn't necessarily be in the sqlite database. You should use the sqlite database for data that is required by the device frequently and that is irrelevant to your server component.
If the data is updated frequently but only while the application runs you might want to cache in the devices memory. I assume your app is not running all the time within the 12 hours but is called regularly instead to check something.
12hrs is a long time, so rather than leaving your data wander in RAM, i would suggest you to use database. Because you never know when you may need to read it again.
Otherwise, if your purpose is only to downloaded data, process it and display in activity, then you dont need to involve database because if your app is closed (due to user or low memory), in anyway your app will be downloading fresh data from server... am i right?
> update the UI on regular interval from the sqlite in evry 20 minutes
Dont expect your app to be open for such a long duration.
To precisely suggest to your case
Avoid DB
Fetch Data at app start or at appropriate time when app is opened
and save it in plain java objects.
Define Methods within it that perform operation in it.
Make map or list to save those POJO
Define Seprate Controller Classes within your project to update map of pojo at any
specific change to make fresh data available to UI.
I would like to read some data from the webview.db database that is created when your app uses a WebView. However, that database is managed by classes that are mostly package level or do not provide much in the way of access through apis.
Since I can't share their database objects or locks, how can I safely read content from that database without getting "database is locked" errors or similar exceptions?
I only need to read, I do not need to write.
If there is no way to safely aquire sql queries/cursors on it, is there a way to safely copy the actual webview.db file in a thread safe manner? Is there any danger of getting a corrupt copy?
Thanks much
I think all the locks are at the sqlite level, not in the java code, so you should be able to get away with just opening the db and reading from it without confusing the webview.
However, for the same reason you should be prepared for lock errors and retry your queries until the lock is gone. The db will be locked when the webview does data altering statements.
Hopefully the webview doesn't put an exclusive lock on all the db for the whole time it's running, but i don't see a reason for that.