Develop Reference

Where to store values to be inserted to SQLite at installation

I am new for SQLite and here is the story;
I'm developing an apk which is a kind of quiz application. I want to store the questions and answers in SQLite database. ( I am also not sure if this is the best way to store data, in SQLite )
In onCreate() method I should create the db and insert all questions, answers etc. So, I'm not sure where to put the data that will be inserted to db before insert them to db tables.
I mean;
values.put("key", "What is abc");
** insert
values.put("key", "What is def");
** insert
values.put("key", "What is ghi");
** insert
values.put("key", "What is xyz");
** insert
...
where should I get those strings (What is bla bla) to insert to table ?
If this was a windows application, I would put them in a .txt file, read from there, and insert rows.
In android, I don't know.
Thanks for any suggestions.

I am also not sure if this is the best way to store data, in SQLite
There are worse choices. :-)
In onCreate() method i should create the db and insert all questions, answers etc.
I'd prepare the database ahead of time, package it with your app, and deploy it using SQLiteAssetHelper. This would be easier to write, easier to maintain, and faster to run.

You can use this to read a text file.
Once you have that in your SQLiteOpenHelper#onCreate() method you would add your default values.

One way to do this would be:
Store the data as XML file in your res/raw folder.
Create a Parser. (DOM would work for a small file. SAX would be better if the file size is large)
Parse the data in onCreate and insert the values.
For example:
` public void onCreate(SQLiteDatabase db) {
ArrayList quizDataList;
quizDataList = ParseQuizData.getQuizData(context);
ContentValues values = new ContentValues();
try {
// Create Database
db.execSQL(TABLE_CREATE);
for (Iterator<QuizData> iterator = quizDataList.iterator(); iterator.hasNext();) {
QuizData quizData = iterator.next();
values.put(COLUMN1, quizData.getAttributeFirst());
values.put(COLUMN2, quizData.getAttributeSecond());
db.insertOrThrow(TABLE_NAME, null, values);
}
} catch (Exception e) {
e.printStackTrace();
}
`

Upgrading Sqlite database in android?

I have an android application which is using Sqlite as database.It has following tables:
Hotels
Locations
Favorites
I am keeping my raw database file in assests folder and when user installs my app i just copies this database to /data/data/package_name/databases directory.Initially Favorites table is empty and it gets populated after user start liking hotels.My problem is that I want to launch my updated version of app with some bug fixes and some new hotels added to the database, so I need to update database of existing users with new hotels and locations without affecting the favorites table.Now if I keep my old approach and update the Database Version Number then application will remove the old database and use the new database but all data in favorites table will be lost.I don't want it to happen.Now problem is how do I update Hotels and Locations table without loosing data in Favorites table.

I know this question was asked long ago, but I had a similar issue and wanted to share my solution, seems to do the trick for me. I'm a novice so feel free to give input-
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
//code to keep table data
List<obj> objList = new ArrayList<obj>();
String selectQuery = "SELECT score,list_name,quiz_length FROM obj_table";
Cursor cursor = db.rawQuery(selectQuery, null);
// looping through all rows and adding to list
if (cursor.moveToFirst()) {
do {
obj o = new obj();
o.final_score = cursor.getInt(0);
o.quiz_name = cursor.getString(1);
o.quiz_length = cursor.getInt(2);
objList.add(o);
} while (cursor.moveToNext());
}
//done storing data, now upgrade DB from asset file
try {
//my db file is upgraded here
copyDataBase();
} catch (IOException e) {
}
//now insert our saved table data
for (Score obj_rec: objList){
ContentValues values = new ContentValues();
values.put("score", obj_rec.final_score);
values.put("list_name", obj_rec.quiz_name);
values.put("quiz_length", obj_rec.quiz_length);
db.insert("obj_table", null, values);
}
}

Before updating write the contents of you previous table to a file and save it on the sdcard.
Then you may update your database with new version.
And after doing that copy back the data from the backup file(from sdcard) to the updated database. After the successful copying of the backup, delete the file from the sdcard.

Usualy upgrade a database has to be done with SQLiteOpenHelper class. I would advise You to do some tests at Your own device before publish it. You have to increment Your Database Version and call "ALTER TABLE" method from sqlite. This has been discussed in many threads here, the clearest one I think is this one:
Difficulty in upgrading SQlite table
and here is even a article with some solution:
http://joshhendo.blogspot.de/2012/03/android-sqlite-database-upgrade.html
However, a safe way would be to save the old database in a tempfolder, that the user can get back the old one if anything is running into chaos.

Fastest and most efficient way to pre-populate database in Android

If you want to pre-populate a database (SQLite) in Android, this is not that easy as one might think.
So I found this tutorial which is often referenced here on Stack Overflow as well.
But I don't really like that way of pre-populating the database since you take the control from the database handler and create the files yourself. I would prefer to not touch the file system and let the database handler do everything on its own.
So what I thought one could do is create the database in the database handler's onCreate() as usual but then load a file (.sql) from /assets which contains the statements to fill in the values:
INSERT INTO testTable (name, pet) VALUES ('Mike', 'Tiger');
INSERT INTO testTable (name, pet) VALUES ('Tom', 'Cat');
...
But calling execSQL() in the handler's onCreate() doesn't really work. It seems that the /assets file must not have more than 1MB and the execSQL() only executes the first statement (Mike - Tiger).
What would you do do pre-populate the database?

I suggest the following:
Wrap all of your INSERT logic into a transaction (BEGIN... COMMIT, or via the beginTransaction()... endTransaction() APIs)
As already suggested, utilize the bind APIs and recycle objects.
Don't create any indexes until after this bulk insert is complete.
Additionally take a look at Faster bulk inserts in sqlite3?

Your question states, that you want the fastest way - but you don't like the way it's done in the article - you don't want to manually replace the DB file (even though, it may be actually faster than filling empty DB with queries).
I had exaclty the same thoughts - and I figured out, that populating via SQL statements and prepopulating can both be the best solution - but it depends on the way you will use the DB.
In my application I need to have about 2600 rows (with 4 columns) in DB at the very first run - it's the data for autocompletion and few other things. It will be modified quite rarely (users can add custom records, but most of the time - they don't need to) and is quite big. Populating it from SQL statements takes not only significantly more time, but more space in the APK (assuming I would store data inside it, alternatively I could download it from the internet).
This is the very simple case (the "Big" insert can take place only once and only at first startup) and I decided to go with copying prepopulated DB file. Sure, it may not be the nicest way - but it's faster. I want my users to be able to use the app as quickly as it's possible and treat speed as a priority - and they really like it. On the contrary, I doubt they would be glad when app would slow down because I thought that slower and nicer solution is actually better.
If instead of 2600 my table would have initially ~50 rows, I would go with SQL statements, since speed and size difference wouldn't be so big.
You have to decide which solution fits your case better. If you foresee any problems that may arise from using "prepopulated db" option - don't use it. If you are not sure about these problems - ask, providing more details on how you will use (and eventually, upgrade) contents of the DB. If you aren't quite sure which solution will be faster - benchmark it. And don't be afraid of that copying file method - it can work really well, if used wisely.

You can have your cake and eat it too. Here is a solution that can both respect the use of your db adapter and also use a simple (and much faster) copy process for a pre-populated database.
I'm using a db adapter based on one of Google's examples. It includes an internal class dbHelper() that extends Android's SQLiteOpenHelper() class. The trick is to override it's onCreate() method. This method is only called when the helper can't find the DB you are referencing and it has to create the DB for you. This should only happen the first time it is called on any given device installation, which is the only time you want to copy the DB. So override it like this -
#Override
public void onCreate(SQLiteDatabase db) {
mNeedToCopyDb = true;
}
Of course make sure you have first declared and initialized this flag in the DbHelper -
private Boolean mNeedToCopyDb = false;
Now, in your dbAdapter's open() method you can test to see if you need to copy the DB. If you do then close the helper, copy the DB and then finally open a new helper (see below code). All future attempts to open the db using the db adapter will find your (copied) DB and therefor the onCreate() method of the internal DbHelper class will not be called and the flag mNeedToCopyDb will remain false.
/**
* Open the database using the adapter. If it cannot be opened, try to
* create a new instance of the database. If it cannot be created,
* throw an exception to signal the failure.
*
* #return this (self reference, allowing this to be chained in an
* initialization call)
* #throws SQLException if the database could neither be opened nor created
*/
public MyDbAdapter open() throws SQLException {
mDbHelper = new DatabaseHelper(mCtx);
mDb = mDbHelper.getReadableDatabase();
if (mDbHelper.mNeedToCopyDb == true){
mDbHelper.close();
try {
copyDatabase();
} catch (IOException e) {
e.printStackTrace();
} finally {
mDbHelper = new DatabaseHelper(mCtx);
mDb = mDbHelper.getReadableDatabase();
}
}
return this;
}
Just place some code to do your database copy inside of your db adapter in a method named copyDatabase() as used above. You can use the value of mDb that was updated by the first instance of DbHelper (when it created the stub DB) to get the path to use for your output stream when you do the copy.
Construct your input stream like this
dbInputStream = mCtx.getResources().openRawResource(R.raw.mydatabase);
[note: If your DB file is too large to copy in one gulp then just break it up into a few pieces.]
This works very fast and puts all of the db access code (including the copying of the DB if needed) into your db adapter.

I wrote a DbUtils class similar to the previous answer. It is part of the ORM tool greenDAO and is available on github. The difference is that it will try to find statement boundaries using a simple regular expression, not just line endings. If you have to rely on a SQL file, I doubt that there's a faster way.
But, if you can supply the data in another format, it should be significantly faster than using a SQL script. The trick is to use a compiled statement. For each data row, you bind the parsed values to the statement and execute the statement. And, of course, you need to do this inside a transaction. I would recommend a simple delimiter separated file format (for example CSV) because it can be parsed faster than XML or JSON.
We did some performance tests for greenDAO. For our test data, we had insert rates of about 5000 rows per second. And for some reason, the rate dropped to half with Android 4.0.

ye, the assets maybe has size limit, so if bigger than the limit, you can cut to more files.
and exesql support more sql sentence, here give you a example:
BufferedReader br = null;
try {
br = new BufferedReader(new InputStreamReader(asManager.open(INIT_FILE)), 1024 * 4);
String line = null;
db.beginTransaction();
while ((line = br.readLine()) != null) {
db.execSQL(line);
}
db.setTransactionSuccessful();
} catch (IOException e) {
FLog.e(LOG_TAG, "read database init file error");
} finally {
db.endTransaction();
if (br != null) {
try {
br.close();
} catch (IOException e) {
FLog.e(LOG_TAG, "buffer reader close error");
}
}
}
above example require the INIT_FILE need every line is a sql sentence.
Also, if your sql sentences file is big, you can create the database out site of android(sqlite support for windows, linux, so you can create the database in your os, and copy the database file to your assets folder, if big, you can zip it)
when your application run, you can get the database file from assets, directed to save to your application's database folder (if you zip it, you can unzip to the application's database folder)
hope can help you -):

I used this method. First create your sqlite database there are a few programs you can use I like SqliteBrowser. Then copy your database file into your assets folder. Then you can use this code in the constructor of SQLiteOpenHelper.
final String outFileName = DB_PATH + NAME;
if(! new File(outFileName).exists()){
this.getWritableDatabase().close();
//Open your local db as the input stream
final InputStream myInput = ctx.getAssets().open(NAME, Context.MODE_PRIVATE);
//Open the empty db as the output stream
final OutputStream myOutput = new FileOutputStream(outFileName);
//final FileOutputStream myOutput = context.openFileOutput(outFileName, Context.MODE_PRIVATE);
//transfer bytes from the inputfile to the outputfile
final byte[] buffer = new byte[1024];
int length;
while ((length = myInput.read(buffer))>0){
myOutput.write(buffer, 0, length);
}
//Close the streams
myOutput.flush();
((FileOutputStream) myOutput).getFD().sync();
myOutput.close();
myInput.close();
}
} catch (final Exception e) {
// TODO: handle exception
}
DB_PATH is something like /data/data/com.mypackage.myapp/databases/
NAME is whatever database name you choose "mydatabase.db"
I know there are many improvements on this code but it worked so well and is VERY FAST. So I left it alone. Like this might be even better in the onCreate() method. Also checking if the file exists every time is probably not the best. Anyway like I said it works, it's fast and reliable.

If the data is not private then simply host it on your website then download it on first run. That way you can keep it up to date. So long as you remember to take app version into account when you upload it to your webserver.

Android does not update SQLite database

I am experiencing some trouble with an SQLIte database in my Android application.
The issue is that the database is never updated, not even on multiple restarts of the emulator, of Eclipse or after deletion from DDMS.
This is my onCreate method, located in a class that extends SQLiteOpenHelper:
public void onCreate(SQLiteDatabase database) {
try {
database.execSQL(ESSENCE_TABLE_CREATE);
database.execSQL(PACCO_TABLE_CREATE);
database.execSQL(TAVOLE_TABLE_CREATE);
database.rawQuery("insert into essenza values(1, 'Rovere')",
null); // added later
} catch (SQLException e) {
Log.e("DB", e.getMessage());
}
}
After instantiating the helper, I request a reference to the database:
helper = new DBHelper(context, dbpath + "/" + DATABASE_NAME);
database = helper.getWritableDatabase();
It seems that the rawQuery statement (which was added at a later time) is not executed and that the database in use is instead cached from a previous version. I also tried to change the version of the database, but it did not work. Am I missing something? Thanks in advance.

You have two options:
Use DDMs to delete the database file from your device (look in /data/data/). This will force Android to run onCreate again.
In your constructor, increment the database version you pass to SQLiteOpenHelper. Add your raw query to onUpgrade.
You probably want option 1. Option 2 is better if you have users of your app whose databases you want to update.

SQLiteException: no such table

I am using my own SQLite3 database as opposed to creating a new one each time my app runs, as I have a few tables with static data that I am trying to display. I created my database and placed it in my assets folder. I then created my database helper and when I start my app I can open my database without problem but when I try to open my first table using the following code
private Cursor getData()
{
try
{
myDbHelper = new DatabaseHelper(this);
SQLiteDatabase db = myDbHelper.getReadableDatabase();
Cursor cursor = db.query("exhibitor", FROM, null, null, null,null, ORDER_BY);
startManagingCursor(cursor);
return cursor;
}
catch(SQLiteException e)
{
String err = e.toString();
return null;
}
}
It throws an error saying android.database.sqlite.SQLiteException: no such table: exhibitor: , while compiling: SELECT _id, EXHIBITOR FROM exhibitor ORDER BY EXHIBITOR but when I check the database exhibitor is there.
What am I missing?

Settings -> Applications -> Manage Applications -> (Click on your application) -> Clear data

Whenever you create new table in an existing database the table doesnt create because the OnCreate function of database handler will not be called everytime but only if required(like database not initiated). If that is not the case your newly created table actually hasnt created. Clear the data to force the db to instantiate itself.

Have you moved the database from the assets folder to /data/data/YOUR_PACKAGE/databases/ on the emulator?
This is a good detailed post about moving the database to /data/data/YOUR_PACKAGE/databases/ if it does not exist.
Here is another short and simple solution to it.

Clear Data and uninstall application from your device and re-install application in device...
Settings -> Applications -> Manage Applications -> Your Application Name -> Clear data

Using SQLiteOpenHelper.onCreate() does not create a new database every time your app starts - rather it creates a database if it does not already exist.
It sounds like you might have an empty database created by or for your helper (if there was no database at all, I might expect a different exception), and your separate database that you created outside of Android. Looks like you and your app are not looking in the same place for your data.

Just clearing the data did not work for me.
What worked was:
Uninstall app
Restart device or emulator
Disable Instant run (Not just the main group but all individual instant run settings)
Build -> Clean Project
Build -> Rebuild Project

hapend to me once
change your DATABASE_VERSION
if your DATABASE_VERSION =1 it will see just three table
if your DATABASE_VERSION = 2 it will see just more table but i really didn't know how many
good luck

I just had a simple mistake.
Reason:
Solution:

Just happened to me. I don't exactly know why but changing the DB_VERSION attribute to a bigger number made it work. The thing is: each time i'm changing the fields of the DB (attributes of the SQLiteDB class), i need to change that number.

Here is my answer according to the description of #ReivieraKid.
No, Uninstall, No Restart, Just checking the app memory if it is your real database, if return false, then copy the database to the memory again. Then You will All set. But to apply this method, you have to know the minimum size of your database.
https://stackoverflow.com/a/73332470/7608371