I'm currently developing an Application that uses a Database / ContentProvider to provide certain location data that's pulled from the network and stored locally in a database periodically.
I've verified that the database is created correctly in the Application, and the Tables are populated with data by exporting a copy of the app local database to the SD card for viewing in a SQL browser -- so the data not being present is not the issue here.
The odd behavior is probably better visualized through a couple of screen shots of my debugging process:
So, initially when I did my queries, all of my Cursor objects were returning -1 for the row count, even though I verified that a SELECT * FROM TABLE_NAME statement should return values, since the database I pulled off of the device contains row data. Here's a look at the debugger step:
Notice that when the cursor is initially returned, mCount displays as -1.
Out of curiosity, I added the cursor.moveToFirst(); call, and here's a screenshot from after that step:
Notice that the cursor count changed from -1 to 3, which corresponds to the correct number of rows (since the application only accounts for 3 continents atm). Unfortunately, this added line that should be unnecessary still doesn't fix my problem, since CursorAdapter.bindView() and CursorAdapter.newView() still fail to be called with the resulting cursor.
I'm confused as to what could be causing this issue, and how to fix it? If anyone could give any insight into what the problem could be, please let me know. Also, if any additional code is needed, such as the SQL create statements, etc let me know - however, as I've already mentioned the database is created fine and queries performed on the exported db copy outside of the Android application work just fine...
One last thing, the application is built for API levels 14+.
If you have an SQL query that are reading from the database you would want to delay reading that data until you are certain that you will use it.
So when you call rawQuery you get a reference to a Cursor object. But nothing is read yet. This is what themoveToFirst does. It is the first thing that tells the cursor that you are serious about running this SQL query and that you want a result.
But since you are using rawQuery nothing is passed on to the database before you try to fetch a result with moveToFirst.
Hope this answers your question.
Related
I am writing an app that needs to fetch the latest changes to android's native calendar since the last time I performed such a "sync" operation.
I am also accessing other content such as the contacts. Here I can reliably check whether the VERSION column of a RawContact changes over time, however I fail to find similar options for calendar events.
Registering a ContentObserver only triggers once something changes, but does not yield the rows that changed. I thought about querying the Calendar.Events ContentProvider for any rows that have the DIRTY column set to 1, however I would be at the mercy of Google's sync adapter, which might potentially be called more quickly (?) than my query is being processed.
Likewise when no network connection is present on the android device, the dirty columns will remain 1, which yields unsuitable results for my use-case.
Is there a better way to listen for changes to the native android calendar which returns correct results, even when no network connection is present?
Apparently, there is no simple way to do this.
Here is how I worked around this issue:
I am keeping a local cache database for contacts that I found the last time I queried the database. Whenever a change occurs (notified by a call to ContentObserver::onChange()) I compare the entries in my cache database to those of CalendarContract.Contacts using a CursorJoiner.
Ids that exist in both cursors need to be compared for changes. To do so I generated a hash for every content, which i stored alongside the contact Id in my cache db.
IDs only provided by the ContentProvider represent new contacts since the last "sync", while IDs only contained in the cache DB are deleted contacts.
Once the differences are obtained, the cache database needs to be updated accordingly.
Me and my Android team have a problem. We have an app that present the user's contact book, with extended information.
Current setup
Our app reads the Contacts Provider of the Android OS. Sends this information to our Server that calculates a couple of necessary fields for us. This information is later fetched by our app and we save this information in an SQLite database. What we end up with in our database is two tables. One with all Numbers and all the extra information that the server calculated for us. The other table is one with all Contacts (one contact can have multiple numbers). This Contacts table was created only for performance; we can have a Cursor selecting all rows in this Contacts table in our CursorAdapter when presenting the contact book for the user.
Hence, when presenting the contact book to the user, we only need to read from our own SQLite database and only one table (e.g. no JOINs).
The main problem
There is a lot of syncing going on. Since, data is duplicated, we need to check for adds/changes/removes and need to sync all the f-ing time. Moreover, when we now are about to change a particular thing in our presentation layer, we need to change our Contacts table to include this particular information.
Priority for us
1st: Performance when presenting the contact book to the user.
2nd: Code maintainability.
Hence, do not comment "Do not duplicate data--it's the root of all problems". It is more important to us that the user does not have performance issues than if we as developers have to spend some extra time writing good synchronization algorithms.
Solutions?
I don't know why, but I've always thought that a CursorAdapter (reading all rows from one table) is performing much better than an ArrayAdapter with a List of objects (held in memory). Anyone know if this is true? Because one solution which will help us at least half the way is to, on start up, join the Contacts Provider (native contact book) and our extended information, save this in a List in memory and present this with an ArrayAdapter.
Creating your own Content Providers? I know little about creating your own content provider. Anyone tried to create a content provider that extend the information of the native contact book and join these. Maybe with the implementation of this interface: ContactsContract.DataColumnsWithJoins? Anyone tried anything similar? How's the performance when presenting this information in a CursorAdapter?
Please ask for any more information I might have forgot and I will update the question!
Thanks a lot in advance for all helpful tips and solutions!
I have come to conclusion (working on my app JReader which relies on fast DB operations a lot) that SQLite in Android is pretty quick as in other platforms but there are some Android specific issues.
Some advises regarding database performance and the questions you have asked:
Content providers are mostly useless if you are not planning sharing your data through them. But they provide at least 2 advantages. First you get data change notifications and you cursor gets updated automatically, and the second and important one: CursorLoaders require a Content provider, and if the performance matters to you, I would strongly suggest using them for loading your cursor;
Accessing Collections is much faster that accessing database, but it is a double work, since you have to persist the data anyway, and DB access is quite fast even for fetching data for a super-fast scrolling list and especially from a single table, it shouldn't be a problem;
Design your DB properly (with JOINS, indexes, etc) :) BUT DO NOT USE JOIN QUERIES IN CURSOR QUERIES! I have had lots of performance issues with that on multiple Android platforms (including 4.0+), not always though. The way I access joint tables is by simply getting foreign key first and then querying child table.
In my experience, I've had the situations when the DB performance was very poor, but in the end I have always managed to tweak the code and as a result would gain 10-100 fold performance increase. So keep profiling you DB code, and you will definitely achieve the desired performance.
Dan's 2nd comment is true .. Android Uses Cursor Window below the screens to Cache data from the Cursor (approx 1M ) . see Android docs on Cursor Window, Which is how the Cursor Adapter is quicker .
If you do not prefer joining two tables seperately you could consider a CursorJoiner which is faster, This can go to your custom contentProvider where one of the provider is pointing to the Contacts and returns a Cursor , similarly the second one points to your own table and returns a cursor . The cursorjoiner can join both these cursors .
(Though it is a complicated process )
I have an SQLite database with records being inserted into it for my Android app.
I am inserting the answers to on screen surveys, with one answer saved per row in my table.
My surveys have 20 questions, and it inserts the answers correctly to the database for the first survey, but then when I insert the answers for the second survey (programatically the same questions, just done again) it malforms the database, but never throws an exception or anything. I run an integrity check before and after and it always returns true.
The unusual thing is that it is only malformed when I try to look at the table on my computer. I tried using the SQLite Database Browser tool, TKSQLite tool, and then opening the DB from the supplied sqlite3.exe, and they all seem to show the DB as malformed. Although the TKSQLite tool notifies me that the table is malformed, I can get the data in other tables. The SQLite Database Browser just shows no data in the table, and sqlite3.exe won't open any tables.
The very unusual thing is if I query the table in the app and print the cursor rows out to log cat, or use the data within the app, all the data is there.
Has anyone seen only partial malformation without an exception being thrown? I would like my database to be pristine...but if my data is there can I hope that it won't get any more malformed?
I worked with SQLite databases for a few projects now doing similar things, and this is the first time I am seeing only partial malformation.
If it will help I am using the Acer a501 tablet programming on Honeycomb 3.2
Additionally, I am accessing the database using a static singleton, to avoid opening and closing the database too often for each insert, and originally inserted via SQL commands using execSQL(), but have also tried using the SQLiteDatabase method insert() to no avail.
We figured out why the database was malforming but not on the device. The tablet we are using is counted as a Portable Media Device according to Windows, and there was some issue when we copied it from the device to the computer. Using a thumb drive to copy to from the device and then from the thumb drive to Windows solved the problem.
Is it okay to hit many "Duplicate row on insert" when inserting into the sql db or is there a better way?
I am downloading "comments" from my server and displaying them in a listview on my android phone..
Here is how I do it:
1 - Whenever I display the listview, I fetch that last 15 comments via xml from the server..
2 - I insert the comments into a local sql database on the phone
3 - I update the listview by using an adapter that loads the data from the local sql database.
Every 30 seconds I am repeating steps 1, 2 and 3.. I also repeat steps 1, 2, and 3 when the activity hits onResume...
Unless another android client has uploaded a new comment, all comments fetched from server are duplicates and inserting them into the db returns an SQLConstraintException because I don't allow duplicates of the same comment.
Is this okay? I am throwing this exception for 10-15 rows every 30 seconds.. Is there a better way that wont hurt performance? I don't see the point of checking if the row exists before insert, since that will surely be slower.. I could potentially pass a variable to the server when I query it, telling it what my newest comment is..
I'm sure many many people have reached this crossroads. Where should we go from here?
Thanks for the discusson..
If your server's db has a sequential, incremental key then yes, I would just query with an additional condition "...AND SeqIdField > [my last id]". If no records are returned then you just reset your timer and wait to query again.
Even if there's no sequential id but you can get records inserted after a particular date/time then just make sure that date/time is included in your results and pass the latest one back in as a parameter. If you don't want / need to store that date/time on your local device then just save the latest somewhere so you know where to pick back up when your app runs again.
I have been testing my app which uses the dbadapter from Reto Meir's earthquake example. Everything was going ok for several days but when debugging with the app on the device today I got the 'no such table' error. I changed the name of the database and all runs well again. This doesn't give me much confidence regarding potential other users.
Since it seems impossible to see the database on the phone, by design I suppose, I can't see how to find out what caused the problem and take steps to avoid it. The database appears to open ok at the start of the program but errors when handling a select query. Just changing the name of the table doesn't fix it, it has to be a new database name.
As the change of name allows it to run ok I can't see that the code is wrong. I wonder if the data becomes corrupted.
I've also found that after successfully inserting a row, then later getting a cursor to allitems sometimes produces a -1 error against a get for one of the column names. How can a column name drop out of the columns index?
I've googled this type of problem and whilst there are a lot of folks with the problem and a lot of replies I can't find anything which informs on the underlying reason for these problems - which is what I am after.
I was having the same problem with this. I was changing the version number, uninstalling application etc... in the end it was due to me not submitting the transaction of the sql call. It might be worth checking out the answer in the following for some clarity - https://stackoverflow.com/a/13568419/1634369
The database is available on the phone and if you can 'adb shell' to the phone, you can also use sqlite3 command line shell to connect, examine, query and modify your database. It's pretty much the same functionality as you'd get with oracle's or mysql's command line tools.
You can see some more details here for example:
http://www.infinitezest.com/articles/using-sqlite-from-shell-in-android.aspx
As for "no such table" case - it really depends on where you store your database. You can choose (at DB creation time) where the database is stored - it might be the internal /data/data (usually) folder where your application resides or you could (mistakenly) create it in cache space - cache can be cleaned at any time whenever Android's OS finds that it needs more space, so this might explain why your database disappeared.
Another possibility is that instead of reinstalling the application, you uninstalled it and installed in two steps. Uninstalling application also deletes all the private data for the application (on most phones, but not on all), whereas reinstalling application (For example by clicking Run in eclipse) does not delete the data. In any case - in you application's onCreate you should recreate the database including all tables - always if you find it has not been yet created - because you never know if you run application for the first or subsequent time.
Please explicitly link to the example you reference.
The way I understand it, you create your own database at launch, unless the database already exists on the device, in which case the database creation is skipped. Correct?
If that is correct, then the only thing you need to do is uninstall the application whenever you update the database design (to remove the old database, with the old table names, etc) and re-install, so the database is recreated with the new table names.
Otherwise you will have updated application code that tries to work with an outdated database.