I'm trying to use a prepopulated database in my Android project. My pipeline is this:
I'm creating a new database and processing a large JSON file to populate it (on Android). It takes a lot of time (~15 minutes or so), so I'd like to pack this created database and distribute it with the app.
I upload the database to Firebase Storage and download it manually on my PC. This database file, when checked with a browser from https://sqlitebrowser.org/, appears correct.
I add the downloaded database to Assets folder, and copy it over to original database path.
The database should work at this point, but it doesn't. There's no error message. It appears empty, but the copied file has proper size and everything. Restarting the app doesn't help, and calling new Room.databaseBuilders doesn't help either.
Code:
Database:
#Database(entities = {...}, version = 1)
public abstract class MyDatabase extends RoomDatabase {
public abstract DbDao dbDao();
private static MyDatabase instance = null;
public static MyDatabase getInstance(Context context) {
if (instance == null){
instance =
Room.databaseBuilder(context, MyDatabase.class, "db")
.build();
}
return instance;
}
Uploading:
public void uploadDB(){
String DBPath = mContext.getDatabasePath("db").getAbsolutePath();
File file = new File(DBPath);
StorageReference storageRef = FirebaseStorage.getInstance().getReference().child("db/my_database.db");
BufferedInputStream bis;
try {
bis = new BufferedInputStream(new FileInputStream(file));
} catch (FileNotFoundException e){
e.printStackTrace();
return;
}
storageRef.putStream(bis);
}
Copying:
public void loadDbFromAssets() throws IOException {
InputStream in = mContext.getAssets().open("databases/my_database.db");
String db_path = mContext.getDatabasePath("db").getAbsolutePath();
File out_file = new File(db_path);
if (out_file.exists()){
boolean deleted = out_file.delete();
if (!deleted) {
DebugLog.log("Old DB not deleted!");
return;
}
}
OutputStream out = new FileOutputStream(out_file);
copy(in, out);
File in_file = new File(db_path);
DebugLog.log("Copied file size: " + in_file.length() + "b");
}
public static void copy(InputStream in, OutputStream out) throws IOException{
try {
try {
byte[] buff = new byte[1024];
int len;
while ((len = in.read(buff)) > 0){
out.write(buff, 0, len);
}
} finally {
out.flush();
out.close();
}
} finally {
in.close();
}
}
Am I missing something?
EDIT: Solved using https://github.com/humazed/RoomAsset.
Related
I am copying SQLite Database from assets folder to android app. I am using below mentioned code. And it was working fine and I am doing my code smoothly. Suddenly, it stopped copying database to android app.
public class PreCreateDB {
public static void copyDB (Context context) {
try {
String destPath = "/data/data/"+ context.getPackageName()+ "/databases";
File f = new File(destPath);
if (!f.exists()) {
f.mkdir();
rawCopy(context.getAssets().open("dbname.db"), new FileOutputStream(destPath + "/dbname.db"));
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public static void rawCopy(InputStream inputStream, OutputStream outputStream) throws IOException{
byte[] buffer = new byte[1024];
int length;
while ((length=inputStream.read(buffer)) > 0) {
outputStream.write(buffer, 0, length);
}
inputStream.close();
outputStream.close();
}
}
I found db in Device File Explorer and I see the following lines in it (not whole tables). Kindly help
SQLite format 3 # .�
� � W--ctableandroid_metadataandroid_metadataCREATE TABLE android_metadata (locale TEXT)
� � en_IN
I change my code to the following one, now it's working.
public static void copyDB (Context context) throws IOException {
try (SQLiteDatabase dbe = SQLiteDatabase.openDatabase("/data/data/com.app.example/databases/dbname.db", null, 0)) {
dbe.close();
} catch (Exception e) {
AssetManager am = context.getAssets();
OutputStream os = new FileOutputStream("/data/data/com.app.example/databases/dbname.db");
byte[] b = new byte[100];
int r;
InputStream is = am.open("dbname.db");
while ((r = is.read(b)) != -1) {
os.write(b, 0, r);
}
is.close();
os.close();
}
}
In my activity I create an object to copy a database from asset folder to app database, everything works fine in emulator but in the device, I get no such file or directory Error on
OutputStream os = new FileOutputStream(dbFile);
I have permission needed:
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
I call in MainActivity:
db = new ExternalDB(this);
and in ExternalDB (Is SqliteOpenHelper):
ExternalDB(Context context){
super(context, DB_Name, null, DATABASE_VERSION);
mycontext = context;
AssetDatabaseOpenHelper adb = new AssetDatabaseOpenHelper(context,DB_Name);
db = adb.OpenDatabase();
}
And AssetDatabaseOpenHelper:
public class AssetDatabaseOpenHelper {
private static String DB_NAME;
private Context context;
AssetDatabaseOpenHelper(Context context, String DB_NAME){
this.context = context;
AssetDatabaseOpenHelper.DB_NAME = DB_NAME;
}
public SQLiteDatabase OpenDatabase(){
File dbFile = context.getDatabasePath(DB_NAME);
if(!dbFile.exists()){
try{
CopyDatabase(dbFile);
}
catch (IOException e){
throw new RuntimeException("Error Creating source database", e);
}
}
// copyDataBase();
return SQLiteDatabase.openDatabase(dbFile.getPath(),null,SQLiteDatabase.OPEN_READWRITE);
}
private void CopyDatabase(File dbFile) throws IOException{
InputStream is = context.getAssets().open(DB_NAME);
OutputStream os = new FileOutputStream(dbFile);
byte[] buffer = new byte[1024];
while(is.read(buffer)>0){
os.write(buffer);
}
os.flush();
os.close();
is.close();
}
}
As I Mentioned i get this error on this line:
OutputStream os = new FileOutputStream(dbFile);
CopyDatabase get's called when dbFile does not exist. Right? And then you tell the FileOutputStream to open the dbFile which we have established does not exist. Thus, no such file or directory Error. Seems legit, doesn't it?
The error is arising because the folder "databases" does not exist so the database cannot be copied
Try:
public SQLiteDatabase OpenDatabase() {
File dbFile = context.getDatabasePath(DB_NAME);
if (!dbFile.exists()) {
try {
//check if "databases" folder exists and create it if needed
File destDir = context.getDatabasePath(DB_NAME).getParentFile();
if(!destDir.exists()){
destDir.mkdirs();
}
CopyDatabase(dbFile);
} catch (IOException e) {
throw new RuntimeException("Error Creating source database", e);
}
} // copyDataBase();
return SQLiteDatabase.openDatabase(dbFile.getPath(), null, SQLiteDatabase.OPEN_READWRITE);
}
For Any One having same problem, Hard Coding database paths didn't work for me and finaly writing copy function like this solved my problem:
/**
* Copy database file from assets folder inside the apk to the system database path.
* #param context Context
* #param databaseName Database file name inside assets folder
* #param overwrite True to rewrite on the database if exists
* #return True if the database have copied successfully or if the database already exists without overwrite, false otherwise.
*/
private boolean copyDatabaseFromAssets(Context context, String databaseName , boolean overwrite) {
File outputFile = context.getDatabasePath(databaseName);
if (outputFile.exists() && !overwrite) {
return true;
}
outputFile = context.getDatabasePath(databaseName + ".temp");
outputFile.getParentFile().mkdirs();
try {
InputStream inputStream = context.getAssets().open(databaseName);
OutputStream outputStream = new FileOutputStream(outputFile);
// transfer bytes from the input stream into the output stream
byte[] buffer = new byte[1024];
int length;
while ((length = inputStream.read(buffer)) > 0) {
outputStream.write(buffer, 0, length);
}
// Close the streams
outputStream.flush();
outputStream.close();
inputStream.close();
outputFile.renameTo(context.getDatabasePath(databaseName));
} catch (IOException e) {
if (outputFile.exists()) {
outputFile.delete();
}
return false;
}
return true;
}
I had to create the folder databases first and then try to create database file.
from this answer: https://stackoverflow.com/a/29058717/4225644
I have used pre-existing database in my application which placed in assets folder and I am using that pre-existing structured for storing data. this is working fine in all the Android devices.
But Somehow it is not working in Android P beta version. When I unzip the database and store into the internal memory and when closed that database object, database file has being corrupted so, after closed the database only "android-metadata" table is left in that database file other tables has been removed automatically.
Please advice.!! what will be the cause. Here is my code
public void open() throws SQLException {
try {
boolean isExist = mDatabaseHelper.checkDataBase();
if (isExist == false) {
mDatabase = mDatabaseHelper.getWritableDatabase();
mDatabaseHelper.copyFromZipFile();
if (mDatabase.isOpen()) {
mDatabase.close();
}
}
mDatabase = mDatabaseHelper.getWritableDatabase();
} catch (Exception e) {
Logger.d(TAG, e.getMessage());
e.printStackTrace();
}
}
/**
* This method is used to close the dataHelper object.
*/
public void close() {
try {
if (mDatabase != null && mDatabase.isOpen())
mDatabase.close();
} catch (Exception e) {
e.printStackTrace();
}
}
public boolean checkDataBase() {
PACKAGE = mContext.getApplicationContext().getPackageName();
DB_PATH = DATA + PACKAGE + "/databases/";
Logger.d(TAG, DB_PATH);
File f = new File(DB_PATH + mDatabaseName);
return f.exists();
}
public void copyFromZipFile() throws IOException {
InputStream is = mContext.getAssets().open("xyz.zip");
// Path to the just created empty db
PACKAGE = mContext.getApplicationContext().getPackageName();
Logger.d(TAG, DB_PATH);
File outFile = new File(DB_PATH, mDatabaseName);
//Open the empty db as the output stream
OutputStream myOutput = new FileOutputStream(outFile.getAbsolutePath());
ZipInputStream zis = new ZipInputStream(new BufferedInputStream(is));
try {
while (zis.getNextEntry() != null) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int count;
while ((count = zis.read(buffer)) != -1) {
baos.write(buffer, 0, count);
}
baos.writeTo(myOutput);
}
} finally {
zis.close();
myOutput.flush();
myOutput.close();
is.close();
}
}
Context
I am copying a sqlite db file out of the assets to internal storage.
The file opens fine, but I wanted a additional layer of safety. There is
a slim possibility that when the file gets copied out it doesn't complete.
I decided to favor a check sum technique: specifically the MessageDigest
Java thing. so here's the code:
public ZipCode(Context ctx) {
this.ctx = ctx;
if (!databaseExist(ctx)) {
Log.d("ZipCode", "DB DNE");
inflate_db(ctx);
check_DB(ctx);
} else {
Log.d("ZipCode", "DB Exsits");
check_DB(ctx);
}
}
private static void inflate_db(Context ctx) {
byte[] buffer = new byte[2048];
int length;
AssetManager am = ctx.getAssets();
try {
BufferedInputStream is = new BufferedInputStream(
am.open(ZIPCODE_SQLITE_FAUX_FILE));
GZIPInputStream zis = new GZIPInputStream(is);
File dbfile = ctx.getDatabasePath(ZIPCODE_SQLITE);
FileOutputStream out = new FileOutputStream(dbfile);
while ((length = zis.read(buffer, 0, buffer.length)) > 0) {
out.write(buffer, 0, length);
}
out.flush();
out.close();
zis.close();
} catch (IOException e) {
e.printStackTrace();
Log.d("ERROR", e.getMessage());
}
}
private static void check_DB(Context ctx) {
File dbfile = ctx.getDatabasePath(ZIPCODE_SQLITE);
FileInputStream fis;
MessageDigest digester;
byte[] bytes = new byte[8192];
int byteCount;
try {
digester = MessageDigest.getInstance("MD5");
fis = new FileInputStream(dbfile);
while ((byteCount = fis.read(bytes)) > 0) {
digester.update(bytes, 0, byteCount);
}
String digest = Base64.encodeBytes(digester.digest());
Log.d("MD5 Sum", digest);
fis.close();
return;
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
catch (NoSuchAlgorithmException e1) {
e1.printStackTrace();
}
}
Now for the Question
Why is it that on fresh creation the check_DB's Log.d("MD5 Sum", digest);
has one output and say on relaunch (i.e. the DB file exists in internal storage)
the check_DB's Log.d("MD5 Sum", digest); has a different output.
Note:
databaseExist(ctx) checks for the DB file's existence per Android conventions.
private static boolean databaseExist(Context ctx) {
File dbFile = ctx.getDatabasePath(ZIPCODE_SQLITE);
return dbFile.exists();
}
why don't you calculate MD5 of your asset file (in your project directory) to find out which MD5 is right and which is wrong? then it'll be easier to find the problem part.
also, i'd suggest to replace (byteCount = fis.read(bytes)) > 0 with (byteCount = fis.read(bytes)) != -1 according to the FileInputStream reference manual.
The code for copying database:
public static final String DB_PATH = Environment.getDataDirectory() + "/data/MyPackageName/databases/";
public static final String DB_NAME = "MyDB.sqlite";
private void copyDataBase(String dbPath){
try{
InputStream inputStream = context.getAssets().open(dbName);
OutputStream appDB = new FileOutputStream(dbPath,false);
byte[] buffer = new byte[1024];
int length;
while ((length = inputStream.read(buffer)) > 0) {
appDB.write(buffer, 0, length);
}
appDB.flush();
appDB.close();
inputStream.close();
}catch(IOException e){
e.printStackTrace();
}
}
this code is work fine in every phone and every android version, but i get a SQLiteDiskIOException in some phones (for example Galaxy S Plus) in above code or this line:
SQLiteDatabase db = super.getWritableDatabase();
everyone can help me in this problem?
Thanks my friend,
my problem is solved with using SQLiteAssetHelper introduced in https://github.com/jgilfelt/android-sqlite-asset-helper