Develop Reference

Unusual behaviour in app using existing database

I'm experimenting with using an existing database that is provided to the app in the Assets folder.
When I run the app for the very first time, I get an NullPointerException on an InputSteam object.
If I run the app a second time, then the input stream is not null but this time, I get an SQLiteException: no such table.
Very unusual, I'm wondering if anyone could help find the cause.
In this project, I created a simple SQLite database file and stored it in the Assets folder. It is called Customers.db and contains one table called CustomerList. The table columns are ID (integer primary key), CustomerName and Country.
In the DatabaseHelper object, the method loadDatabase() loads the database from the assets folder into the phone's internal memory.
DatabaseHelper method getRecords() returns an Array of Customer objects and these are the fields of these customer objects are listed in a RecyclerView in the MainActivity. To simply the experiment, as a first step, the getRecords() method return all the rows of the table.
When run for the first time the following exception is reported:
java.lang.RuntimeException: Unable to start activity ComponentInfo{com.mo.useexistingdatabasedemo/com.mo.useexistingdatabasedemo.MainActivity}: java.lang.NullPointerException: Attempt to invoke virtual method 'int java.io.InputStream.read(byte[], int, int)' on a null object reference
And the second time it's run, the input stream seems to be no longer null, but instead an sqliteexception is reported:
java.lang.RuntimeException: Unable to start activity ComponentInfo{com.mo.useexistingdatabasedemo/com.mo.useexistingdatabasedemo.MainActivity}: android.database.sqlite.SQLiteException: no such table: CustomerList (code 1 SQLITE_ERROR): , while compiling: SELECT * FROM CustomerList
In ,my project class DatabaseHelper is defined as follows (and its methods are called from the MainActivity).
package com.mo.useexistingdatabasedemo;
import android.content.Context;
import android.database.Cursor;
import android.database.sqlite.SQLiteDatabase;
import android.database.sqlite.SQLiteOpenHelper;
import android.util.Log;
import androidx.annotation.Nullable;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.Buffer;
import java.util.ArrayList;
public class DatabaseHelper extends SQLiteOpenHelper {
public static ArrayList<Customer> arrayList = new ArrayList<>();
public static final String DATABASE_NAME = "Customers.db";
public static final String TABLE_NAME = "CustomerList";
public static final int DATABASE_VERSION = 1;
public static final String DATABASE_DIRECTORY_PATH = "data/data/com.mo" +
".useexistingdatabasedemo/databases";
public static final String DATABASE_FILE_PATH = DATABASE_DIRECTORY_PATH + DATABASE_NAME;
Context context;
InputStream inputStream;
OutputStream outputStream;
Buffer buffer;
public static final String SELECT_ALL_TABLE = "SELECT * FROM " + TABLE_NAME;
public DatabaseHelper(#Nullable Context context) {
super(context, DATABASE_NAME, null, 2);
this.context = context;
}
public void loadDatabase() {
String path = DATABASE_DIRECTORY_PATH + DATABASE_NAME;
File dbFile = new File(path); // the descriptor of a file in internal memory to which
// we will write the db in the assets folder.
// if the database isn't already in internal memory, copy it over from assets.
if (!dbFile.exists()) {
try {
inputStream = context.getAssets().open(DATABASE_NAME);
} catch (IOException e) {
Log.e("IOException input stream: ",
"Exception opening input stream " + e.getMessage());
}
OutputStream outputStream = null;
try {
outputStream = new FileOutputStream(path);
} catch (FileNotFoundException e) {
Log.e("IOException, output stream: ",
"Exception creating output stream " + e.getMessage());
}
// create a buffer of 1024 bytes length
byte[] buffer = new byte[1024];
int len;
try {
while ((len = inputStream.read(buffer, 0, 1024)) > 0)
outputStream.write(buffer, 0, len);
outputStream.flush();
inputStream.close();
} catch (IOException e) {
Log.e("IOException", "Exception occurred in while block" + e.getMessage());
}
}
Log.i("Load database", "The method public loadDatabase() executed succesffully");
}
#Override
public void onCreate(SQLiteDatabase db) {
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
public ArrayList getRecords() {
SQLiteDatabase database = getWritableDatabase();
Log.i("The attached database is :", database.getAttachedDbs().toString());
Cursor cursor = database.rawQuery("SELECT * FROM " + TABLE_NAME, null);
while (cursor.moveToNext()) {
Customer customer = new Customer();
customer.customerName = cursor.getColumnName(1).toString();
customer.country = cursor.getColumnName(2).toString();
arrayList.add(customer);
}
return arrayList;
}
}

Unusual behaviour in app using existing database
The Issue(s)
Not at all unusual. You are not copying the database to it's expected location, which is data/data/your_package/databases (directory) However no need to know or even code that (see later).
First run and the copy fails BUT the file is created.
Second run as the file exists, no attempt is made to copy. So theSQLiteOpenHelper subclass can't find the database, as it's not in the expected location, so it invokes the onCreate method and creates the database without any tables and hence the table not found.
The Fix
I would suggest NOT hard coding the path but getting the path via the Context's getDatabasePath method. e.g.
String dbpath = context.getDatabasePath(name).getPath();
or as a File :-
File db = context.getDatabasePath(name);
After acquiring I suggest checking if the parent directory exists and if not then issuing a mkdirs so that the databases folder is created if it doesn't exist (for older API's it will not and it's non existence prevents the copy working but with the file existing, I believe I saw that later versions create the directory). e.g. something like :-
File db = context.getDatabasePath(name);
if (!db.getParentFile().exists()) db.mkdirs();
String dbpath = db.getPath(); // may not be needed as the File object is what is then required
Here's is an example of a pretty bulletproof SQLiteOpenHelper subclass that will copy a pre-existing database.

Updating SQLite database when releasing new version of the android app on the store

I'm using sqlite in my android app, my database file is called data.db. Let's suppose that I have two table in it one is having static values(read only) and second is saves dynamic values and pushed the app to the playStore.
In the next version I updated the data.db and I updated the values in first table, added new columns in second table and added third new table and push the app to the PlayStore. So how I can check if user is updating the app and what is best possible way to save existing data and how I can update data.db programmatically when it is a update not a fresh install?

You can use SQLiteOpenHelper's onUpgrade method. In the onUpgrade method, you get the oldVersion as one of the parameters.
In the onUpgrade use Switch case and in each of the cases use the version number to keep track of the current version of Database that was sent for each new Version of Database.
Its best that you loop over from oldVersion to newVersion, incrementing version by 1 at a time and then upgrade the database stepbystep. This is very helpful when someone with Database version 1 upgrades the app after a long time, to a version using database version 7 and the app starts crashing because of certain incompatible changes.
Then the updates in database will be done stepwise, covering all possible cases i.e incorporating the changes in the database done for each new version and thereby preventing your application from Crashing.

There are two ways
1] Changed database version so that when user updates an app SQLiteOpenHelper's onUpgrade() method gets executed in this method you can write a code to drop tables n create new tables according to new schema and send network calls to fetch data.
2] Push app with pre-installed db.
Create database with db version and save it in assets folder.
Whenever app runs compare versions of assets's db & app's db.
if asset's db version is higher than app's db then this means you need to update database.
Directly copy Assete's db into folder structor and sends delta call to fetch data.

In this example you have add a new column so the changes in this case should be like this (here you gonna not lose your data).
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
String sql = "ALTER TABLE " + TABLE_SECRET + " ADD COLUMN " +
"name_of_column_to_be_added" + " INTEGER";
db.execSQL(sql);
}

I advice you to use http://www.activeandroid.com/ library. It supports migrations. You just need to write [n].sql files in assets/migrations folder for n version of you database model. Active android carries about existing version and applies required migration.

private static String DB_PATH = Environment.getDataDirectory()+"/data/package-name/databases/";
private static String DB_NAME = "name_of_the_database"; // student.db
private void copyDataBase() throws IOException
{
// Open your local db as the input stream
InputStream myInput = myContext.getAssets().open(DB_NAME);
// Path to the just created empty d inb
String outFileName = DB_PATH + DB_NAME;
// Open the empty db as the output stream
OutputStream myOutput = new FileOutputStream(outFileName);
// transfer bytes from the inputfile to the outputfile
byte[] buffer = new byte[1024];
int length;
while ((length = myInput.read(buffer)) > 0)
{
myOutput.write(buffer, 0, length);
}
// Close the streams
myOutput.flush();
myOutput.close();
myInput.close();
//Copy successful
outFileName = DB_PATH + DB_SUCCESS;
myOutput = new FileOutputStream(outFileName);
myOutput.write(1);
myOutput.flush();
myOutput.close();
}

package com.example.sqllite_db;
import android.content.Context;
import android.database.sqlite.SQLiteDatabase;
import android.database.sqlite.SQLiteDatabase.CursorFactory;
import android.database.sqlite.SQLiteOpenHelper;
public class DatabseAdapter extends SQLiteOpenHelper
{
public static String DatabaseName="NewDatabase";
public static int DatabaseVersion=1;
public DatabseAdapter(Context context)
{
super(context,DatabaseName,null,DatabaseVersion);
// TODO Auto-generated constructor stub
}
/*public static String TableUser="tb1";
public static String keyId="id";
public static String keyName="name";
public static String keyPass="pass";*/
#Override
public void onCreate(SQLiteDatabase db)
{
// TODO Auto-generated method stub
String createQuery="create table tb1(name text,pass text);";
db.execSQL(createQuery);
//String createQuery1="create table tb2(pass text);";//("+keyName+"text,"+keyPass+"text);";
//db.execSQL(createQuery1);
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion)
{
// TODO Auto-generated method stub
db.execSQL("drop table tb1");
onCreate(db);
//db.execSQL("drop table tb2");
//onCreate(db);
}
}
drop the database in on Upgrade method then again call on Create method in it and your database will update when you upload new database.

Replace sql database but keep old records

After a lot of work I managed to build my first app, but I stick with one question. For my app I am using a sql database.. Suppose I want to add 30 records to a certain table. How is it possible that when I put a new version in the android market with a new sql table to use this one for the future, but to keep the records of the previous database?
Does it has to do something with:
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
EDIT
my databasehelper code:
public class DataBaseHelper extends SQLiteOpenHelper {
private static String DB_PATH = "/data/data/com.test.com/databases/";
private static String DB_NAME = "quizDb";
private SQLiteDatabase myDataBase;
private final Context myContext;
private Cursor c;
static int numberOfLevels = 10;
private final static int DB_VERSION = 2; // = until level 10
/**
* Constructor Takes and keeps a reference of the passed context in order to
* access to the application assets and resources.
*
* #param context
*/
public DataBaseHelper(Context context) {
super(context, DB_NAME, null, DB_VERSION);
this.myContext = context;
}
/**
* Creates a empty database on the system and rewrites it with your own
* database.
* */
public void createDataBase() throws IOException {
boolean dbExist = checkDataBase();
if (!dbExist) {
// By calling this method and empty database will be created into
// the default system path
// of your application so we are gonna be able to overwrite that
// database with our database.
this.getReadableDatabase();
try {
copyDataBase();
} catch (IOException e) {
throw new Error("Error copying database");
}
}
}
/**
* Check if the database already exist to avoid re-copying the file each
* time you open the application.
*
* #return true if it exists, false if it doesn't
*/
private boolean checkDataBase() {
File dbFile = new File(DB_PATH + DB_NAME);
return dbFile.exists();
}
/**
* Copies your database from your local assets-folder to the just created
* empty database in the system folder, from where it can be accessed and
* handled. This is done by transfering bytestream.
* */
private void copyDataBase() throws IOException {
// Open your local db as the input stream
InputStream myInput = myContext.getAssets().open(DB_NAME);
// Path to the just created empty db
String outFileName = DB_PATH + DB_NAME;
// Open the empty db as the output stream
OutputStream myOutput = new FileOutputStream(outFileName);
// transfer bytes from the inputfile to the outputfile
byte[] buffer = new byte[1024];
int length;
while ((length = myInput.read(buffer)) > 0) {
myOutput.write(buffer, 0, length);
}
// Close the streams
myOutput.flush();
myOutput.close();
myInput.close();
}
public void openDataBase() throws SQLException {
// Open the database
String myPath = DB_PATH + DB_NAME;
myDataBase = SQLiteDatabase.openDatabase(myPath, null,
SQLiteDatabase.OPEN_READONLY);
}
#Override
public synchronized void close() {
if (c != null)
c.close();
if (myDataBase != null)
myDataBase.close();
super.close();
}
#Override
public void onCreate(SQLiteDatabase db) {
}
public File getDatabasePath(String name) {
File file = myContext.getDatabasePath(name);
return file;
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
db.execSQL("ATTACH DATABASE ? as AttachedDB",
new String[] { getDatabasePath("quizDbNew").getPath() });
db.execSQL("INSERT OR IGNORE INTO questions (_id, file, answer, level) SELECT _id, file, answer, level FROM AttachedDB.questions");
db.execSQL("DETACH AttachedDB");
}

The concept of using "DROP TABLE" in onUpgrade() is as primitive as database management gets, however more useful techniques require more SQL savvy. A smarter way to upgrade your databases by using "ALTER TABLE" to add new columns or otherwise finagle the old data into your new schema.
Addition
Below in the comments you stated (more or less):
I want to copy the content from my backup file of Db v1 into my current Db v2
So let's set up a couple hypothetical tables:
Database Version One (DBv1):
CREATE TABLE Foo(_id INTEGER PRIMARY KEY, bar TEXT, bar2 TEXT, bar3 TEXT);
Database Version Two (DBv2):
CREATE TABLE Foo(_id INTEGER PRIMARY KEY, bar2 TEXT, bar4 INTEGER);
First let's see a regular upgrade from DBv1 to DBv2. SQLite only supports ADD COLUMN and RENAME TO, not REMOVE COLUMN or anythings else. So we have to re-create the entire table:
#Override // DBv1 => DBv2
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
db.execSQL("ALTER TABLE Foo RENAME TO OldFoo");
db.execSQL("CREATE TABLE Foo(_id INTEGER PRIMARY KEY, bar2 TEXT, bar4 INTEGER)");
db.execSQL("INSERT INTO Foo (_id, bar) SELECT _id, bar2 FROM OldFoo");
db.execSQL("DROP TABLE OldFoo");
}
Again this created a table with DBv2's schema and kept all of the valid, existing data from DBv1 by inserting the appropriate columns into DBv2. (Then it removed the old data by dropping the old table.)
You have wisely chosen to backup your database over time in a separate file, but now you want to bring the old data into the new table schema. To start make sure that your backup SQLite file is in the same directory as your current SQLite file (data/data/<reverse.package.name>/databases/). It will obviously need a unique name, let's call it DBBackup. Now let's attach DBBackup to your current database and perform a similar action from above:
// DBBackupv1 => DBv2
public void restore(SQLiteDatabase db) {
db.execSQL("ATTACH DATABASE ? as AttachedDB", new String[] {getDatabasePath("DBBackup").getPath()});
db.execSQL("INSERT OR IGNORE INTO Foo (_id, bar2) SELECT _id, bar2 FROM AttachedDB.Foo");
db.execSQL("DETACH AttachedDB");
}
I used INSERT OR IGNORE to restore any rows that were deleted but left the current existing rows untouched. You can use INSERT OR REPLACE to revert to the backed up version. There are many more options to suit your needs.

Android SQLite Database Connection not succesfull

i encountered a little problem with my current project. I am doing an android application which needs to connect to a SQLite database to work through some statements. I believe the statements etc are fine, my only problem is the fact that the connection to the database is not succesfull.
LogCat Error:
04-18 08:20:30.688: E/Database(304): sqlite3_open_v2("jdbc:sqlite:res/raw/randomdb.db", &handle, 1, NULL) failed
So my code so far for connecting to the database is like this:
String url = "jdbc:sqlite:res/raw/randomdb.db";
SQLiteDatabase db;
db = SQLiteDatabase.openDatabase(url, null, SQLiteDatabase.OPEN_READONLY);
As you can see, i am trying to acces a database which is located in my project/res/raw folder. Does anyone see the mistake?
!!!UPDATE!!!
*I tried to go the way with SQLiteOpenHelper, but still encouner an error i cannot seem to solver. Here is my new code:*
public class DatabaseAdapter extends SQLiteOpenHelper {
private static String dbPath= "data/data/com.YourPackageName/applicationDb/";
private static String dbName = "YourDBName";
private SQLiteDatabase applicationDatabase;
private final Context applicationContext;
private boolean checkDataBase(){
File dbFile = new File( dbPath + dbName);
return dbFile.exists();
}
public void openDataBase() throws SQLException{
String fullDbPath= dbPath + dbName;
applicationDatabase = SQLiteDatabase.openDatabase( fullDbPath,null,SQLiteDatabase.OPEN_READONLY);
}
#Override
public void onCreate(SQLiteDatabase db) {
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
}
I get this error:
Implicit super constructor SQLiteOpenHelper() is undefined for default constructor. Must define an explicit constructor
Any ideas? Would be great!

if you want to connect your android application with SQLite database then you need to extends SQLiteOpenHelper
public class AbcClass extends SQLiteOpenHelper
after that you need to Override these method:
onCreate and onUpgrade
and constructor of AbcClass looks like:
public AbcClass(Context context) {
super(context, DB_NAME, null, VERSION); //public static final String DB_NAME = "test.sqlite";
}

public boolean databaseExist()
{
File dbFile = new File(DB_PATH + DB_NAME);
return dbFile.exists();
}
This is the solution...
OR-----------------------
private boolean checkDataBase(){
SQLiteDatabase checkDB = null;
try{
String myPath = DB_PATH + DB_NAME;
checkDB = SQLiteDatabase.openDatabase(myPath, null, SQLiteDatabase.OPEN_READONLY);
}catch(SQLiteException e){
//database does't exist yet.
}
if(checkDB != null){
checkDB.close();
}
return checkDB != null ? true : false;
}

I think the path you specify is not the correct one and so as mentioned in one of the other answers the database is not found. However I had never used the method openDatabase to read from the raw folder so I do not know which is the correct path.
However once upon a time I had shipped the database along with my application. It resided in the assets folder and once the application started I copied it in the private application storage (much like any database created with SqliteOpenHelper). From then on I used the usual way with SqliteOpenHelper to access the database.
Basically for this I followed the blog mentioned in this thread and because my database file was bigger than 1MB I used the technique described in this thread. Hopefully combining those two you will get your database running!
EDIT Btw you are wrong, openDatabase does not accept url, but path.

How to create Android Database application?

I am new to Databases and do not know if this would be the best way to do what I want to achieve. I want to create a custom list of restaurants around me, then search them, and sort them on ranking, title, or location.
Would I need to create a database for this? I technically could use a text file and Arrays but I feel like this is very inefficient. I would need a Node to contain the following data: Name of Establishment, Address, Phone Number, Ranking (Based of Our Ranking System).
How should I go about doing this?

Yes, you definitely want to use a database. If you use a database local to the phone, you need to use an SQLite database. Here is a good place to start.
If you want a database that is preloaded in the phone, put it in your assets folder. Here is an example of a database helper class with the database packaged with the phone:
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.text.DateFormat;
import java.util.Calendar;
import java.util.Date;
import java.util.Vector;
import android.content.ContentValues;
import android.content.Context;
import android.database.Cursor;
import android.database.SQLException;
import android.database.sqlite.SQLiteDatabase;
import android.database.sqlite.SQLiteException;
import android.database.sqlite.SQLiteOpenHelper;
import android.util.Log;
public class DataBaseHelper extends SQLiteOpenHelper{
private static final String TAG = "DataBaseHelper";
//The Androids default system path of your application database.
private static String DB_PATH = "/data/data/yourpackage/databases/";
private static String DB_NAME = "DatabaseName";
public SQLiteDatabase myDataBase;
private final Context myContext;
/**
* Constructor
* Takes and keeps a reference of the passed context in order to access to the application assets and resources.
* #param context
*/
public DataBaseHelper(Context context) {
super(context, DB_NAME, null, 1);
this.myContext = context;
}
/**
* Creates a empty database on the system and rewrites it with your own database.
* */
public void createDataBase() throws IOException{
boolean dbExist = checkDataBase();
if(dbExist){
//do nothing - database already exist
}else{
//By calling this method and empty database will be created into the default system path
//of your application so we are gonna be able to overwrite that database with our database.
this.getWritableDatabase();
try {
copyDataBase();
} catch (IOException e) {
throw new Error("Error copying database");
}
}
}
/**
* Check if the database already exist to avoid re-copying the file each time you open the application.
* #return true if it exists, false if it doesnt
*/
private boolean checkDataBase(){
SQLiteDatabase checkDB = null;
try{
String myPath = DB_PATH + DB_NAME;
checkDB = SQLiteDatabase.openDatabase(myPath, null, SQLiteDatabase.OPEN_READWRITE);
}catch(SQLiteException e){
//database doest exist yet.
}
if(checkDB != null){
checkDB.close();
}
return checkDB != null ? true : false;
}
/**
* Copies your database from your local assets-folder to the just created empty database in the
* system folder, from where it can be accessed and handled.
* This is done by transfering bytestream.
* */
private void copyDataBase() throws IOException{
//Open your local db as the input stream
InputStream myInput = myContext.getAssets().open(DB_NAME);
Log.d(TAG, "found the database");
// Path to the just created empty db
String outFileName = DB_PATH + DB_NAME;
//Open the empty db as the output stream
OutputStream myOutput = new FileOutputStream(outFileName);
//transfer bytes from the inputfile to the outputfile
byte[] buffer = new byte[2048];
int length;
while ((length = myInput.read(buffer))>0){
myOutput.write(buffer, 0, length);
}
//Close the streams
myOutput.flush();
myOutput.close();
myInput.close();
}
public void openDataBase() throws SQLException{
//Open the database
String myPath = DB_PATH + DB_NAME;
myDataBase = SQLiteDatabase.openDatabase(myPath, null, SQLiteDatabase.OPEN_READWRITE);
}
#Override
public synchronized void close() {
if(myDataBase != null)
myDataBase.close();
super.close();
}
#Override
public void onCreate(SQLiteDatabase db) {
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
// Add your public helper methods to access and get content from the database.
// You could return cursors by doing "return myDataBase.query(....)" so itd be easy
// to you to create adapters for your views.

A SQLite database would be an excellent way to accomplish your application goals. Based on what you said, a single table called "Restaurants" containing 5 fields: _id, Name, Address, Phone, Rank would do the trick.
The SQLite SQL documentation is located here: http://www.sqlite.org/lang.html . Some of the commands you'll need to be familiar with are CREATE TABLE, INSERT, UPDATE and SELECT. Sqlite for Android is a well-documented standard found across the web. You should have no trouble finding examples of how to accomplish your task.
Remember to close any database objects that you create else, you'll create a leak.
Good luck!

What you want to do is create a Web Service, this lives in the "cloud" on the Internets somewhere. Then you have your mobile app periodically contact the web service and request restaurant data perhaps based on the users current location, so the app passes the gps coordinates to the web service and the Web Service checks its database for restaurants within X distance of that location, then the Web Service returns the list to the app in a format such as Json or XML, the App parses this info and displays it to the user.
Ideally you would also have a database on the phone that would store results on the device this reduces the amount of requests you have to pull from the server, saving on battery, data and server resources. This gets a little complicated as you have to decide what to ask for intelligently and the server has to decide what to send you.
There was a Google I/O talk on how to do this.
http://www.youtube.com/watch?v=xHXn3Kg2IQE