Method exceeds compiler instruction limit - android
Before I wrote this question, i've searched in all proposed answers, "stackoverflow" shows me but nothing fit my case.
My problem:
I wrote Room migration to copy old data from a table and add it to another table, and i need to specify all the columns in the query. I have something like 130 columns.
database.execSQL("INSERT INTO newTable(col1, ..n) SELECT col1, .... coln FROM oldTable")
When i executed my migration, i got this error:
method exceeds compiler instruction limit: 16602 in androidx.room.RoomOpenHelper$ValidationResult
Thx for your helps.
Running a test using 150 columns (151 including the id column) and thus :-
2022-05-11 06:29:50.050 16308-16308/a.a.so72188145javaroommaxcolumns D/INSERTSQL: INSERT INTO newtable (id,col1,col2,col3,col4,col5,col6,col7,col8,col9,col10,col11,col12,col13,col14,col15,col16,col17,col18,col19,col20,col21,col22,col23,col24,col25,col26,col27,col28,col29,col30,col31,col32,col33,col34,col35,col36,col37,col38,col39,col40,col41,col42,col43,col44,col45,col46,col47,col48,col49,col50,col51,col52,col53,col54,col55,col56,col57,col58,col59,col60,col61,col62,col63,col64,col65,col66,col67,col68,col69,col70,col71,col72,col73,col74,col75,col76,col77,col78,col79,col80,col81,col82,col83,col84,col85,col86,col87,col88,col89,col90,col91,col92,col93,col94,col95,col96,col97,col98,col99,col100,col101,col102,col103,col104,col105,col106,col107,col108,col109,col110,col111,col112,col113,col114,col115,col116,col117,col118,col119,col120,col121,col122,col123,col124,col125,col126,col127,col128,col129,col130,col131,col132,col133,col134,col135,col136,col137,col138,col139,col140,col141,col142,col143,col144,col145,col146,col147,col148,col149,col150) SELECT id,col1,col2,col3,col4,col5,col6,col7,col8,col9,col10,col11,col12,col13,col14,col15,col16,col17,col18,col19,col20,col21,col22,col23,col24,col25,col26,col27,col28,col29,col30,col31,col32,col33,col34,col35,col36,col37,col38,col39,col40,col41,col42,col43,col44,col45,col46,col47,col48,col49,col50,col51,col52,col53,col54,col55,col56,col57,col58,col59,col60,col61,col62,col63,col64,col65,col66,col67,col68,col69,col70,col71,col72,col73,col74,col75,col76,col77,col78,col79,col80,col81,col82,col83,col84,col85,col86,col87,col88,col89,col90,col91,col92,col93,col94,col95,col96,col97,col98,col99,col100,col101,col102,col103,col104,col105,col106,col107,col108,col109,col110,col111,col112,col113,col114,col115,col116,col117,col118,col119,col120,col121,col122,col123,col124,col125,col126,col127,col128,col129,col130,col131,col132,col133,col134,col135,col136,col137,col138,col139,col140,col141,col142,col143,col144,col145,col146,col147,col148,col149,col150 FROM oldtable;
Worked fine (Room 2.5.0-aplha01 and Android 30 emulator). I suspect that the issue may not be the number of columns but elsewhere in the SQL.
The above SQL was extracted from the log, it was output to the log exactly as was then used (see testing code below).
A work-around IF you have columns that can be null and a sufficient number of them to circumvent the issue. Have an INSERT that inserts a manageable number of columns allowing the other columns to default to null, followed by 1 or more UPDATEs to apply the subsequent values to override the nulls.
Another work-around could be to extract all columns SELECT * .... into a Cursor and then build a ContentValues from the Cursor (on a per row basis) and use the SupportSQliteDatabase's insert method. Something along the lines of:-
Cursor csr = database.query("SELECT * FROM oldtable");
database.beginTransaction();
ContentValues cv = new ContentValues();
while (csr.moveToNext()) {
cv.clear();
for (int i=0; i < csr.getColumnCount(); i++) {
if (!csr.getColumnName(i).equals("col133")) { //<<<<< skip col133 as an example
cv.put(csr.getColumnName(i),csr.getString(i));
}
}
database.insert("newTable", SQLiteDatabase.CONFLICT_IGNORE,cv);
}
database.setTransactionSuccessful();
database.endTransaction();
Full code used for testing
Note that the only change between the 2 is the version number. So initially the code is run (after uninstalling the App) with version 1 (so no Migration). To test it's just a matter, of then changing the version to 2.
The #Entity annotated class OldTable:-
#Entity
class OldTable {
#PrimaryKey
Long id=null;
String col1;
String col2;
String col3;
.... (i.e. all missing numbers)
String col147;
String col148;
String col149;
String col150;
}
The #Dao annotated interface (not actually used) AllDao
#Dao
interface AllDao {
#Insert(onConflict = OnConflictStrategy.IGNORE)
long insert(OldTable oldTable);
}
The #Database annotated class TheDatabase :-
#Database(entities = {OldTable.class},version = MainActivity.DATABASE_VERSION,exportSchema = false)
abstract class TheDatabase extends RoomDatabase {
abstract AllDao getAllDao();
private static volatile TheDatabase INSTANCE = null;
static TheDatabase getINSTANCE(Context context) {
if (INSTANCE == null) {
INSTANCE = Room.databaseBuilder(context,TheDatabase.class,MainActivity.DATABASE_NAME)
.allowMainThreadQueries()
.addMigrations(MIGRATION_1_2)
.build();
}
return INSTANCE;
}
static Migration MIGRATION_1_2 = new Migration(1,2) {
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
StringBuilder sb = new StringBuilder();
StringBuilder sb2 = new StringBuilder();
ArrayList<String> columns = new ArrayList<>();
for (int i=0; i < MainActivity.column_count;i++) {
columns.add("col" + (i+1));
if (i>0) {
sb.append(",");
sb2.append(",");
}
sb.append(columns.get(i));
sb2.append(columns.get(i)).append(" TEXT");
}
String columnsAsCSV = sb.toString();
String columnDefs = sb2.toString();
database.execSQL("DROP TABLE IF EXISTS newtable");
database.execSQL("CREATE TABLE IF NOT EXISTS newtable (id INTEGER PRIMARY KEY," + columnDefs + ");");
String insertSQL = "INSERT INTO newtable (id,"+columnsAsCSV+") SELECT id,"+columnsAsCSV+" FROM oldtable;";
Log.d("INSERTSQL",insertSQL);
database.execSQL(insertSQL);
}
};
}
And finally putting it alltogether MainActivity :-
public class MainActivity extends AppCompatActivity {
public static final String DATABASE_NAME = "the_database.db";
public static final int DATABASE_VERSION = 2;
public static final int column_count = 150;
private ArrayList<String> column_names = new ArrayList<>();
private String allColumnsAsCSV = "";
TheDatabase db;
AllDao dao;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
BuildColumns();
db = TheDatabase.getINSTANCE(this);
dao = db.getAllDao();
SupportSQLiteDatabase supdb = db.getOpenHelper().getWritableDatabase();
StringBuilder sb = new StringBuilder();
for (int i=0;i < 100; i++) {
sb = new StringBuilder();
sb.append("INSERT INTO oldtable (").append(allColumnsAsCSV).append(") VALUES(");
for(int ii=0;ii<column_count;ii++) {
if (ii > 0) {
sb.append(",");
}
sb.append("'").append(column_names.get(ii)).append(i).append(ii).append("'");
}
sb.append(")");
supdb.execSQL(sb.toString());
}
}
void BuildColumns() {
column_names.clear();
allColumnsAsCSV = "";
StringBuilder sb = new StringBuilder();
for (int i=0; i < column_count; i++) {
column_names.add("col" + (i+1));
if (i>0) {
sb.append(",");
}
sb.append(column_names.get(i));
}
allColumnsAsCSV = sb.toString();
}
}
Related
createFromAsset Migration but keep specific Columns
I have kind of a quiz app and I have a database with all the questions in tables, for each question there is a column solved that I update if the answer was correct, so I can filter with SQL WHERE to only show unsolved questions. Now every once in a while I have to correct typos in the questions or might want to add some new ones, so How do I employ the corrected database (questions.db) from the assets to the saved one on the user device while keeping the solved columns? I thought of and tried the following things without success: Currently, i use a self-crafted solution to replace the database on the device (destructive) but between updates keep the solved info https://github.com/ueen/RoomAsset Put solved info (question id solved y/n) in a separate table and LEFT JOIN to filter unsolved questions, this only complicated matters Have an extra database for the solved questions, it seems there's no easy way to attach two Room Databases So in essence, this may be inspiration for the Room dev team, I would like to have a proper migration strategy for createFromAsset with ability to specify certain columns/tables to be kept. Thanks for your great work so far, I really enjoy using Android Jetpack and Room especially! Also, I'm happy about any workaround I could employ to resolve this issue :)
I believe the following does what you want #Database(version = DatabaseConstants.DBVERSION, entities = {Question.class}) public abstract class QuestionDatabase extends RoomDatabase { static final String DBNAME = DatabaseConstants.DBNAME; abstract QuestionDao questionsDao(); public static QuestionDatabase getInstance(Context context) { copyFromAssets(context,false); if (getDBVersion(context,DatabaseConstants.DBNAME) < DatabaseConstants.DBVERSION) { copyFromAssets(context,true); } return Room.databaseBuilder(context,QuestionDatabase.class,DBNAME) .addCallback(callback) .allowMainThreadQueries() .addMigrations(Migration_1_2) .build(); } private static RoomDatabase.Callback callback = new Callback() { #Override public void onCreate(#NonNull SupportSQLiteDatabase db) { super.onCreate(db); } #Override public void onOpen(#NonNull SupportSQLiteDatabase db) { super.onOpen(db); } #Override public void onDestructiveMigration(#NonNull SupportSQLiteDatabase db) { super.onDestructiveMigration(db); } }; private static Migration Migration_1_2 = new Migration(1, 2) { #Override public void migrate(#NonNull SupportSQLiteDatabase database) { } }; private static boolean doesDatabaseExist(Context context) { if (new File(context.getDatabasePath(DBNAME).getPath()).exists()) return true; if (!(new File(context.getDatabasePath(DBNAME).getPath()).getParentFile()).exists()) { new File(context.getDatabasePath(DBNAME).getPath()).getParentFile().mkdirs(); } return false; } private static void copyFromAssets(Context context, boolean replaceExisting) { boolean dbExists = doesDatabaseExist(context); if (dbExists && !replaceExisting) return; //First Copy if (!replaceExisting) { copyAssetFile(context); return; } //Subsequent Copies File originalDBPath = new File(context.getDatabasePath(DBNAME).getPath()); // Open and close the original DB so as to checkpoint the WAL file SQLiteDatabase originalDB = SQLiteDatabase.openDatabase(originalDBPath.getPath(),null,SQLiteDatabase.OPEN_READWRITE); originalDB.close(); //1. Rename original database String preservedDBName = "preserved_" + DBNAME; File preservedDBPath = new File (originalDBPath.getParentFile().getPath() + preservedDBName); (new File(context.getDatabasePath(DBNAME).getPath())) .renameTo(preservedDBPath); //2. Copy the replacement database from the assets folder copyAssetFile(context); //3. Open the newly copied database SQLiteDatabase copiedDB = SQLiteDatabase.openDatabase(originalDBPath.getPath(),null,SQLiteDatabase.OPEN_READWRITE); SQLiteDatabase preservedDB = SQLiteDatabase.openDatabase(preservedDBPath.getPath(),null,SQLiteDatabase.OPEN_READONLY); //4. get the orignal data to be preserved Cursor csr = preservedDB.query( DatabaseConstants.QUESTION_TABLENAME,DatabaseConstants.EXTRACT_COLUMNS, null,null,null,null,null ); //5. Apply preserved data to the newly copied data copiedDB.beginTransaction(); ContentValues cv = new ContentValues(); while (csr.moveToNext()) { cv.clear(); for (String s: DatabaseConstants.PRESERVED_COLUMNS) { switch (csr.getType(csr.getColumnIndex(s))) { case Cursor.FIELD_TYPE_INTEGER: cv.put(s,csr.getLong(csr.getColumnIndex(s))); break; case Cursor.FIELD_TYPE_STRING: cv.put(s,csr.getString(csr.getColumnIndex(s))); break; case Cursor.FIELD_TYPE_FLOAT: cv.put(s,csr.getDouble(csr.getColumnIndex(s))); break; case Cursor.FIELD_TYPE_BLOB: cv.put(s,csr.getBlob(csr.getColumnIndex(s))); break; } } copiedDB.update( DatabaseConstants.QUESTION_TABLENAME, cv, DatabaseConstants.QUESTION_ID_COLUMN + "=?", new String[]{ String.valueOf( csr.getLong( csr.getColumnIndex(DatabaseConstants.QUESTION_ID_COLUMN ) ) ) } ); } copiedDB.setTransactionSuccessful(); copiedDB.endTransaction(); csr.close(); //6. Cleanup copiedDB.close(); preservedDB.close(); preservedDBPath.delete(); } private static void copyAssetFile(Context context) { int buffer_size = 8192; byte[] buffer = new byte[buffer_size]; int bytes_read = 0; try { InputStream fis = context.getAssets().open(DBNAME); OutputStream os = new FileOutputStream(new File(context.getDatabasePath(DBNAME).getPath())); while ((bytes_read = fis.read(buffer)) > 0) { os.write(buffer,0,bytes_read); } os.flush(); os.close(); fis.close(); } catch (IOException e) { e.printStackTrace(); throw new RuntimeException("Unable to copy from assets"); } } private static int getDBVersion(Context context, String databaseName) { SQLiteDatabase db = SQLiteDatabase.openDatabase( context.getDatabasePath(databaseName).getPath(),null,SQLiteDatabase.OPEN_READONLY); int rv = db.getVersion(); db.close(); return rv; } } This manages the Asset File copy (in this case directly from the assets folder) outside of Room and before the database is built doing it's own version and database existence checking. Although ATTACH could be used, the solution keeps the original and the new databases seperate when updating the new using a Cursor. Some flexibility/adaptability has been included in that the columns to be preserved can be expanded upon. In the test runs DatabaseConstants includes :- public static final String[] PRESERVED_COLUMNS = new String[] { QUESTION_SOLVED_COLUMN }; public static final String[] EXTRACT_COLUMNS = new String[] { QUESTION_ID_COLUMN, QUESTION_SOLVED_COLUMN }; thus additional columns to be preserved can be added (of any type as per 5. in the copyFromAssets method). The columns to be extracted can also be specified, in the case above, the ID column uniquely identifies the question so that is extracted in addition to the solved column for use by the WHERE clause. Testing The above has been tested to :- Original Copy the first version of the database from the assets when DBVERSION is 1. Note that this originaly contains 3 questions as per Part of the code (in the invoking activity checks to to see if all the solved values are 0 if so, then it chages the solved status of the question with an id of 2) Not copy the database, but use the existing database when DBVERSION is 1 on a subseuent run(s). ID 2 remains solved. New After renaming the original asset from to be prefixed with original_, editing the database to be as below and after copying it to the assets file :- Without changing the DBVERSION (still 1) run and the original database is still in use. After changing DBVERSION to 2 running copies the changed asset file and restores/preserves the solved status. For subsequent runs the solved status for the new data remains. For testing the invoking activity consisted of :- public class MainActivity extends AppCompatActivity { QuestionDatabase questionDatabase; #Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); questionDatabase = QuestionDatabase.getInstance(this); int solvedCount = 0; for (Question q: questionDatabase.questionsDao().getAll()) { if (q.isSolved()) solvedCount++; q.logQuestion(); } if (solvedCount == 0) { questionDatabase.questionsDao().setSolved(true,2); } for (Question q: questionDatabase.questionsDao().getAll()) { q.logQuestion(); } } } For each run it outputs all of the questions to the log twice. After the first if there are no solved questions it solves the question with an id of 2. The output from the last run was :- 2020-01-08 09:14:37.689 D/QUESTIONINFO: ID is 1 Question is Editted What is x Answers Are :- a b x Correct Answer is 3 Is Solved false 2020-01-08 09:14:37.689 D/QUESTIONINFO: ID is 2 Question is Edited What is a Answers Are :- a b c Correct Answer is 1 Is Solved false 2020-01-08 09:14:37.689 D/QUESTIONINFO: ID is 3 Question is Edited What is b Answers Are :- a b c Correct Answer is 2 Is Solved false 2020-01-08 09:14:37.689 D/QUESTIONINFO: ID is 4 Question is New Question What is d Answers Are :- e f d Correct Answer is 3 Is Solved false 2020-01-08 09:14:37.692 D/QUESTIONINFO: ID is 1 Question is Editted What is x Answers Are :- a b x Correct Answer is 3 Is Solved false 2020-01-08 09:14:37.692 D/QUESTIONINFO: ID is 2 Question is Edited What is a Answers Are :- a b c Correct Answer is 1 Is Solved true 2020-01-08 09:14:37.692 D/QUESTIONINFO: ID is 3 Question is Edited What is b Answers Are :- a b c Correct Answer is 2 Is Solved false 2020-01-08 09:14:37.693 D/QUESTIONINFO: ID is 4 Question is New Question What is d Answers Are :- e f d Correct Answer is 3 Is Solved false Additional - Improved Version This is an approved version that caters for multiple tables and columns. To cater for tables a class TablePreserve has been added that allows a table, the columns to preserve, the columns to extract and the columns for the where clause. As per :- public class TablePreserve { String tableName; String[] preserveColumns; String[] extractColumns; String[] whereColumns; public TablePreserve(String table, String[] preserveColumns, String[] extractColumns, String[] whereColumns) { this.tableName = table; this.preserveColumns = preserveColumns; this.extractColumns = extractColumns; this.whereColumns = whereColumns; } public String getTableName() { return tableName; } public String[] getPreserveColumns() { return preserveColumns; } public String[] getExtractColumns() { return extractColumns; } public String[] getWhereColumns() { return whereColumns; } } You create an Array of TablePreserve objects and they are looped through e.g. public final class DatabaseConstants { public static final String DBNAME = "question.db"; public static final int DBVERSION = 2; public static final String QUESTION_TABLENAME = "question"; public static final String QUESTION_ID_COLUMN = "id"; public static final String QUESTION_QUESTION_COLUMN = QUESTION_TABLENAME; public static final String QUESTION_ANSWER1_COLUMN = "answer1"; public static final String QUESTION_ANSWER2_COLUMN = "answer2"; public static final String QUESTION_ANSWER3_COLUMN = "answer3"; public static final String QUESTION_CORRECTANSWER_COLUMN = "correctAsnwer"; public static final String QUESTION_SOLVED_COLUMN = "solved"; public static final TablePreserve questionTablePreserve = new TablePreserve( QUESTION_TABLENAME, new String[]{QUESTION_SOLVED_COLUMN}, new String[]{QUESTION_ID_COLUMN,QUESTION_SOLVED_COLUMN}, new String[]{QUESTION_ID_COLUMN} ); public static final TablePreserve[] TABLE_PRESERVELIST = new TablePreserve[] { questionTablePreserve }; } Then QuestionsDatabase becomes :- #Database(version = DatabaseConstants.DBVERSION, entities = {Question.class}) public abstract class QuestionDatabase extends RoomDatabase { static final String DBNAME = DatabaseConstants.DBNAME; abstract QuestionDao questionsDao(); public static QuestionDatabase getInstance(Context context) { if (!doesDatabaseExist(context)) { copyFromAssets(context,false); } if (getDBVersion(context, DatabaseConstants.DBNAME) < DatabaseConstants.DBVERSION) { copyFromAssets(context, true); } return Room.databaseBuilder(context,QuestionDatabase.class,DBNAME) .addCallback(callback) .allowMainThreadQueries() .addMigrations(Migration_1_2) .build(); } private static RoomDatabase.Callback callback = new Callback() { #Override public void onCreate(#NonNull SupportSQLiteDatabase db) { super.onCreate(db); } #Override public void onOpen(#NonNull SupportSQLiteDatabase db) { super.onOpen(db); } #Override public void onDestructiveMigration(#NonNull SupportSQLiteDatabase db) { super.onDestructiveMigration(db); } }; private static Migration Migration_1_2 = new Migration(1, 2) { #Override public void migrate(#NonNull SupportSQLiteDatabase database) { } }; private static boolean doesDatabaseExist(Context context) { if (new File(context.getDatabasePath(DBNAME).getPath()).exists()) return true; if (!(new File(context.getDatabasePath(DBNAME).getPath()).getParentFile()).exists()) { new File(context.getDatabasePath(DBNAME).getPath()).getParentFile().mkdirs(); } return false; } private static void copyFromAssets(Context context, boolean replaceExisting) { boolean dbExists = doesDatabaseExist(context); if (dbExists && !replaceExisting) return; //First Copy if (!replaceExisting) { copyAssetFile(context); setDBVersion(context,DBNAME,DatabaseConstants.DBVERSION); return; } //Subsequent Copies File originalDBPath = new File(context.getDatabasePath(DBNAME).getPath()); // Open and close the original DB so as to checkpoint the WAL file SQLiteDatabase originalDB = SQLiteDatabase.openDatabase(originalDBPath.getPath(),null,SQLiteDatabase.OPEN_READWRITE); originalDB.close(); //1. Rename original database String preservedDBName = "preserved_" + DBNAME; File preservedDBPath = new File (originalDBPath.getParentFile().getPath() + File.separator + preservedDBName); (new File(context.getDatabasePath(DBNAME).getPath())) .renameTo(preservedDBPath); //2. Copy the replacement database from the assets folder copyAssetFile(context); //3. Open the newly copied database SQLiteDatabase copiedDB = SQLiteDatabase.openDatabase(originalDBPath.getPath(),null,SQLiteDatabase.OPEN_READWRITE); SQLiteDatabase preservedDB = SQLiteDatabase.openDatabase(preservedDBPath.getPath(),null,SQLiteDatabase.OPEN_READONLY); //4. Apply preserved data to the newly copied data copiedDB.beginTransaction(); for (TablePreserve tp: DatabaseConstants.TABLE_PRESERVELIST) { preserveTableColumns( preservedDB, copiedDB, tp.getTableName(), tp.getPreserveColumns(), tp.getExtractColumns(), tp.getWhereColumns(), true ); } copiedDB.setVersion(DatabaseConstants.DBVERSION); copiedDB.setTransactionSuccessful(); copiedDB.endTransaction(); //5. Cleanup copiedDB.close(); preservedDB.close(); preservedDBPath.delete(); } private static void copyAssetFile(Context context) { int buffer_size = 8192; byte[] buffer = new byte[buffer_size]; int bytes_read = 0; try { InputStream fis = context.getAssets().open(DBNAME); OutputStream os = new FileOutputStream(new File(context.getDatabasePath(DBNAME).getPath())); while ((bytes_read = fis.read(buffer)) > 0) { os.write(buffer,0,bytes_read); } os.flush(); os.close(); fis.close(); } catch (IOException e) { e.printStackTrace(); throw new RuntimeException("Unable to copy from assets"); } } private static int getDBVersion(Context context, String databaseName) { SQLiteDatabase db = SQLiteDatabase.openDatabase( context.getDatabasePath(databaseName).getPath(),null,SQLiteDatabase.OPEN_READONLY); int rv = db.getVersion(); db.close(); return rv; } private static void setDBVersion(Context context, String databaseName, int version) { SQLiteDatabase db = SQLiteDatabase.openDatabase( context.getDatabasePath(databaseName).getPath(),null,SQLiteDatabase.OPEN_READWRITE); db.setVersion(version); db.close(); } private static boolean preserveTableColumns( SQLiteDatabase originalDatabase, SQLiteDatabase newDatabase, String tableName, String[] columnsToPreserve, String[] columnsToExtract, String[] whereClauseColumns, boolean failWithException) { StringBuilder sb = new StringBuilder(); Cursor csr = originalDatabase.query("sqlite_master",new String[]{"name"},"name=? AND type=?",new String[]{tableName,"table"},null,null,null); if (!csr.moveToFirst()) { sb.append("\n\tTable ").append(tableName).append(" not found in database ").append(originalDatabase.getPath()); } csr = newDatabase.query("sqlite_master",new String[]{"name"},"name=? AND type=?",new String[]{tableName,"table"},null,null,null); if (!csr.moveToFirst()) { sb.append("\n\tTable ").append(tableName).append(" not found in database ").append(originalDatabase.getPath()); } if (sb.length() > 0) { if (failWithException) { throw new RuntimeException("Both databases are required to have a table named " + tableName + sb.toString()); } return false; } for (String pc: columnsToPreserve) { boolean preserveColumnInExtractedColumn = false; for (String ec: columnsToExtract) { if (pc.equals(ec)) preserveColumnInExtractedColumn = true; } if (!preserveColumnInExtractedColumn) { if (failWithException) { StringBuilder sbpc = new StringBuilder().append("Column in Columns to Preserve not found in Columns to Extract. Cannot continuue." + "\n\tColumns to Preserve are :-"); } throw new RuntimeException("Column " + pc + " is not int the Columns to Extract."); } return false; } sb = new StringBuilder(); for (String c: whereClauseColumns) { sb.append(c).append("=? "); } String[] whereargs = new String[whereClauseColumns.length]; csr = originalDatabase.query(tableName,columnsToExtract,sb.toString(),whereClauseColumns,null,null,null); ContentValues cv = new ContentValues(); while (csr.moveToNext()) { cv.clear(); for (String pc: columnsToPreserve) { switch (csr.getType(csr.getColumnIndex(pc))) { case Cursor.FIELD_TYPE_INTEGER: cv.put(pc,csr.getLong(csr.getColumnIndex(pc))); break; case Cursor.FIELD_TYPE_STRING: cv.put(pc,csr.getString(csr.getColumnIndex(pc))); break; case Cursor.FIELD_TYPE_FLOAT: cv.put(pc,csr.getDouble(csr.getColumnIndex(pc))); break; case Cursor.FIELD_TYPE_BLOB: cv.put(pc,csr.getBlob(csr.getColumnIndex(pc))); } } int waix = 0; for (String wa: whereClauseColumns) { whereargs[waix] = csr.getString(csr.getColumnIndex(wa)); } newDatabase.update(tableName,cv,sb.toString(),whereargs); } csr.close(); return true; } }
I debugged and modified the code by MikeT a bit, now heres the final kotlin library with an easy databaseBuilder https://github.com/ueen/RoomAssetHelper Please read the documentation and report if you encounter any issue, enjoy :)
How to encrypt sqlite texts without 3rd party libraries (Android)
I'm using an external database inside my android application and it directly embeds inside the apk package after compiling. As I want to implement in app purchase in order to access some of its data I don't want to leave it without any encryption. I used Sqlcipher library but it makes the app too big and slow. Isn't there any other way to do this? For example an algorithm to encrypt the strings so I put the encrypted text in database and decrypt it inside application code?
The following is an example App that Encrypts part of the data that can then be turned on. It is based upon the code in my comment. Stage 1 - The Master Database. To start with you need the database that is to be the basis of the encrypted database (i.e. the MASTER database that IS NOT include in the App, it's use is to create the Encrypted database (or databases, perhaps a library, each database with a unique password/secret key if you wanted greater security)) in part consider this (as is used throughout the example) :- As you can see this one will work by having a table called FreeData and another called PaidData. The tables definitions are the same EXCEPT that for the PaidData there is no ID column (the intention of this method is to decrypt the rows in the PaidData into the FreeData when/if the requested and the SecretKey (password) is valid.). So The FreeData table looks like :- The PaidData table looks like :- So the only difference between the tables is the actual data contained within and that the id column is missing. The id's will be generated when the encrypted data is extracted from the PaidData table, decrypted and the inserted into the FreeData table. Thus just one decryption is required to get access to the data. Stage 2 - Generating the Encrypted Database for distribution with the App This is done by a App just for this purpose using the EncryptDecrypt class very similar to the one at Encrypt data in SQLite as per EncryptDecrypt.java class EncryptDecrypt { private Cipher cipher; private static SecretKeySpec secretKeySpec; private static IvParameterSpec ivParameterSpec; private boolean do_encrypt = true; /** * Construct EncryptDecrypt instance that does not check user login-in * mode, thus the assumption is that this user is NOT the special user * NOUSER that doesn't require a password to login; this constructor * is designed to ONLY be used when a user has been added by NOUSER, * and to then encrypt the data using the enccryptForced method solely * to encrypt any existing card data for the new user that has a password. * * #param context The context, required for database usage (user) * #param skey The secret key to be used to encrypt/decrypt */ EncryptDecrypt(Context context, String skey) { //DBUsersMethods users = new DBUsersMethods(context); String saltasString = "there is no dark side of the moon it is all dark."; String paddedskey = (skey + saltasString).substring(0,16); secretKeySpec = new SecretKeySpec(paddedskey.getBytes(),"AES/CBC/PKCS5Padding"); ivParameterSpec = new IvParameterSpec((saltasString.substring(0,16)).getBytes()); try { cipher = Cipher.getInstance("AES/CBC/PKCS5Padding"); } catch (Exception e){ //e.printStackTrace(); } } /** * Normal encryption routine that will not encrypt data if the user is * the special case NOUSER (i.e LOGIN mode is NOUSER), otherwise data * is encrypted. * * #Param toEncrypt The string to be encrypted * #return The encryted (or not if NOUSER) data as a string */ String encrypt(String toEncrypt) { if (!do_encrypt) { return toEncrypt; } byte[] encrypted; try { cipher.init(Cipher.ENCRYPT_MODE,secretKeySpec,ivParameterSpec); encrypted = cipher.doFinal(toEncrypt.getBytes()); } catch (Exception e) { //e.printStackTrace(); return null; } return Base64.encodeToString(encrypted, Base64.DEFAULT); } /** * Encryption, irrespective of the USER type, noting that this should * only be used in conjunction with an EncryptDecrypt instance created * using the 2nd/extended constructor * * #param toEncrypt The string to be encrypted * #return The encrypted data as a string */ String encryptForced(String toEncrypt) { byte[] encrypted; try { cipher.init(Cipher.ENCRYPT_MODE,secretKeySpec,ivParameterSpec); encrypted = cipher.doFinal(toEncrypt.getBytes()); } catch (Exception e) { //e.printStackTrace(); return null; } return Base64.encodeToString(encrypted,Base64.DEFAULT); } /** * Decrypt an encrypted string * #param toDecrypt The encrypted string to be decrypted * #return The decrypted string */ String decrypt(String toDecrypt) { if (!do_encrypt) { return toDecrypt; } byte[] decrypted; try { cipher.init(Cipher.DECRYPT_MODE,secretKeySpec,ivParameterSpec); decrypted = cipher.doFinal(Base64.decode(toDecrypt,Base64.DEFAULT)); } catch (Exception e) { //e.printStackTrace(); return null; } return new String(decrypted); } } As this was designed for user login and multiple users where a salt was part of the database the salt has been hard coded using :- String saltasString = "there is no dark side of the moon it is all dark.";, The phrase can be changed as long as it is at least 16 characters in length (only the first 16 bytes are used). A class is used to cater for potential flexibility/expansion where multiple tables can be specified or not for encryption and for multiple columns that can be encrypted, copied asis or skipped (e.g. id's would probably be skipped (in the example it's not even defined as a column).). This class is TableColumnConvertList.java and is :- public class TableColumnConvertList { private ArrayList<TableEntry> tables; public TableColumnConvertList() { this.tables = new ArrayList<>(); } public String[] getTables() { String[] tableList = new String[tables.size()]; int ix = 0; for (TableEntry te: this.tables) { tableList[ix++] = te.getSourceTableName(); } return tableList; } public String[] getTableColumnNamesToEncrypt(String tableName) { String[] rv = null; for(TableEntry te: this.tables) { if (te.getSourceTableName().equals(tableName)) { rv = new String[te.getColumnNamesToEncrypt().size()]; int ix=0; for (String s: te.getColumnNamesToEncrypt()) { rv[ix++] = s; } } } return rv; } public String[] getTableColumnNamesToCopyAsis(String tableName) { String[] rv = null; for (TableEntry te: this.tables) { if (te.getSourceTableName().equals(tableName)) { rv = new String[te.getColumnNamesToCopyAsis().size()]; int ix=0; for (String s: te.getColumnNamesToCopyAsis()) { rv[ix++] = s; } } } return rv; } public String[] getTableColumnNamesToSkip(String tableName) { String[] rv = null; for (TableEntry te: this.tables) { if (te.sourceTableName.equals(tableName)) { rv = new String[te.getColumnNamesToSkip().size()]; int ix =0; for (String s: te.getColumnNamesToSkip()) { rv[ix++] = s; } } } return rv; } public void addTable( String sourceTableName, String destinationTableName, String[] columnNamesToEncrypt, String[] columnNamesToCopyAsis, String[] columnNamesToSkip ) { tables.add( new TableEntry( sourceTableName, destinationTableName, columnNamesToEncrypt, columnNamesToCopyAsis, columnNamesToSkip ) ); } private class TableEntry { private String sourceTableName; private String destinationTableName; private ArrayList<String> columnNamesToEncrypt; private ArrayList<String> columnNamesToCopyAsis; private ArrayList<String> columnNamesToSkip; private TableEntry() {} private TableEntry(String sourceTableName, String destinationTableName, String[] columnNamesToEncrypt, String[] columnNamesToCopyAsis, String[] columnNamesToSkip ) { this.sourceTableName = sourceTableName; this.destinationTableName = destinationTableName; this.columnNamesToEncrypt = new ArrayList<>(); if (columnNamesToEncrypt != null && columnNamesToEncrypt.length > 0) { for (String s: columnNamesToEncrypt) { addColumn(s); } } } private void addColumn(String s) { this.columnNamesToEncrypt.add(s); } private String getSourceTableName() { return sourceTableName; } public String getDestinationTableName() { return destinationTableName; } public void setSourceTableName(String sourceTableName) { this.sourceTableName = sourceTableName; } public void setDestinationTableName(String destinationTableName) { this.destinationTableName = destinationTableName; } private ArrayList<String> getColumnNamesToEncrypt() { return columnNamesToEncrypt; } public void setColumnNamesToEncrypt(ArrayList<String> columnNamesToEncrypt) { this.columnNamesToEncrypt = columnNamesToEncrypt; } private ArrayList<String> getColumnNamesToCopyAsis() { return columnNamesToCopyAsis; } public void setColumnNamesToCopyAsis(ArrayList<String> columnNamesToCopyAsis) { this.columnNamesToCopyAsis = columnNamesToCopyAsis; } public ArrayList<String> getColumnNamesToSkip() { return columnNamesToSkip; } public void setColumnNamesToSkip(ArrayList<String> columnNamesToSkip) { this.columnNamesToSkip = columnNamesToSkip; } } } The rest of this basic App, at present, is all in a single activity that uses two input's (EditTexts) :- The secret key used to encrypt The database name (file name) of the encrypted database. Code in the App prevents using the same name as the base database, which needs to be copied into the assets folder. and a Button, to initiate the Encryption if the input is good (to a fashion aka with limited validation). This the layout xml activiy_main.xml is :- <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:orientation="vertical" android:layout_width="match_parent" android:layout_height="match_parent" android:padding="10dp" tools:context=".MainActivity"> <TextView android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="Database EncryptTool" /> <EditText android:id="#+id/secretkey" android:layout_width="wrap_content" android:layout_height="wrap_content" android:hint="Secret Key to use to Encrypt the Database." > </EditText> <EditText android:id="#+id/databasename" android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="MyDatabase" android:hint="Database Name" > </EditText> <Button android:id="#+id/encrypt" android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="ENCRYPT" android:visibility="gone" > </Button> </LinearLayout> MainActivity.java where the work is done is :- public class MainActivity extends AppCompatActivity { public static final String ASSETDB_NAME = "basedb.db"; public static final int ASSETDB_NOT_FOUND = -10; public static final int ASSETFILE_OPEN_ERROR = -11; public static final int ASSETDB_OPEN_ERROR = -12; public static final int ASSETDB_COPY_ERROR = -13; public static final int ASSETDB_FLUSH_ERROR = -14; public static final int ASSETDB_CLOSE_ERROR = -15; public static final int ASSETFILE_CLOSE_ERROR = -16; public static final int ASSETDB_CREATED_SUCCESSFULLY = 0; public static final int BUFFERSIZE = 1024 * 4; EditText mSecretKey, mDBName; Button mEncryptButton; TableColumnConvertList mTCCL = new TableColumnConvertList(); #Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); mDBName = this.findViewById(R.id.databasename); mSecretKey = this.findViewById(R.id.secretkey); mEncryptButton = this.findViewById(R.id.encrypt); //<<<<<<<<< set what data to encrypt i.e. table(s) and the column(s) in the table >>>>>>>>> mTCCL.addTable( "PaidData", "FreeData", new String[]{"theData"}, new String[]{}, new String[]{"id"} ); if (getDBFromAsset() >= 0) { mEncryptButton.setVisibility(View.VISIBLE); mEncryptButton.setOnClickListener(new View.OnClickListener() { #Override public void onClick(View v) { if (mDBName.getText().toString().length() < 1) { Toast.makeText( v.getContext(), "The Database Name cannot be blank.", Toast.LENGTH_LONG ).show(); mDBName.requestFocus(); return; } if (mDBName.getText().toString().equals(ASSETDB_NAME)) { Toast.makeText( v.getContext(), "Database Name cannot be " + ASSETDB_NAME + ". Please change the name.", Toast.LENGTH_LONG ).show(); mDBName.requestFocus(); return; } if (mSecretKey.getText().toString().length() < 1) { Toast.makeText( v.getContext(), "The Secret Key cannot be blank.", Toast.LENGTH_LONG ).show(); mSecretKey.requestFocus(); return; } if (createEncryptedDatabase(mTCCL, mDBName.getText().toString(), mSecretKey.getText().toString() ) == 0) { Toast.makeText(v.getContext(),"Successfully Encrypted Database " + mDBName + " using Secret Key " + mSecretKey,Toast.LENGTH_LONG).show(); } } }); } } private boolean checkIfDataBaseExists(String databaseName) { File dbFile = new File(this.getDatabasePath(databaseName).getPath()); if (dbFile.exists()) { return true; } else { if (!dbFile.getParentFile().exists()) { dbFile.getParentFile().mkdirs(); } } return false; } private boolean checkIfAssetDBExists() { try { InputStream is = this.getAssets().open(ASSETDB_NAME); is.close(); return true; } catch (IOException e) { return false; } } private int getDBFromAsset() { int rv = ASSETDB_NOT_FOUND; File dbFile = new File(this.getDatabasePath(ASSETDB_NAME).getPath()); InputStream is; FileOutputStream os; int read_length; byte[] buffer = new byte[BUFFERSIZE]; if (!checkIfAssetDBExists()) { return ASSETDB_NOT_FOUND; } if (checkIfDataBaseExists(ASSETDB_NAME)) { dbFile.delete(); } try { rv = ASSETFILE_OPEN_ERROR; is = this.getAssets().open(ASSETDB_NAME); rv = ASSETDB_OPEN_ERROR; os = new FileOutputStream(dbFile); rv = ASSETDB_COPY_ERROR; while ((read_length = is.read(buffer)) > 0) { os.write(buffer,0,read_length); } rv = ASSETDB_FLUSH_ERROR; os.flush(); rv = ASSETDB_CLOSE_ERROR; os.close(); rv = ASSETFILE_CLOSE_ERROR; is.close(); rv = ASSETDB_CREATED_SUCCESSFULLY; } catch (IOException e) { e.printStackTrace(); } return rv; } private int createEncryptedDatabase(TableColumnConvertList tableColumnConvertList, String databaseName, String key) { File copiedAssetDB = new File(this.getDatabasePath(ASSETDB_NAME).getPath()); File encryptedDB = new File(this.getDatabasePath(databaseName).getPath()); if (encryptedDB.exists()) { encryptedDB.delete(); } try { byte[] buffer = new byte[BUFFERSIZE]; int read_length; InputStream is = new FileInputStream(copiedAssetDB); OutputStream os = new FileOutputStream(encryptedDB); while ((read_length = is.read(buffer)) > 0) { os.write(buffer,0,read_length); } os.flush(); os.close(); is.close(); } catch (IOException e) { e.printStackTrace(); return -1; } SQLiteDatabase db = SQLiteDatabase.openDatabase(encryptedDB.getPath(),null,SQLiteDatabase.OPEN_READWRITE); EncryptDecrypt ed = new EncryptDecrypt(this,key); int errorcount = 0; db.beginTransaction(); for (String t: tableColumnConvertList.getTables()) { ContentValues cv = new ContentValues(); String[] columnsToEncrypt = tableColumnConvertList.getTableColumnNamesToEncrypt(t); String[] columnOriginalValues = new String[columnsToEncrypt.length]; Cursor c = db.query(true,t,columnsToEncrypt,null,null,null,null,null, null); int totalRows = c.getCount(); int updatedRows = 0; while (c.moveToNext()) { cv.clear(); int ovix=0; StringBuilder whereClause = new StringBuilder(); for (String s: c.getColumnNames()) { for (String ec: columnsToEncrypt ) { if (s.equals(ec)) { cv.put(s,ed.encrypt(c.getString(c.getColumnIndex(s)))); columnOriginalValues[ovix++] = c.getString(c.getColumnIndex(s)); if (whereClause.length() > 0) { whereClause.append(" AND "); } whereClause.append(s).append("=?"); } } } updatedRows += db.update(t,cv,whereClause.toString(),columnOriginalValues); } c.close(); Log.d("ENCRYPTRESULT","Read " + totalRows + " DISTINCT ROWS. Updated " + updatedRows); errorcount += totalRows - updatedRows; } if (errorcount == 0) { db.setTransactionSuccessful(); } else { Toast.makeText( this, "Errors encountered Encrypting Database. Rolled back (not changed)", Toast.LENGTH_LONG ).show(); } db.endTransaction(); return errorcount; } } Of importance is this line/code :- TableColumnConvertList mTCCL = new TableColumnConvertList(); .......... //<<<<<<<<< set what data to encrypt i.e. table(s) and the column(s) in the table >>>>>>>>> mTCCL.addTable( "PaidData", "FreeData", new String[]{"theData"}, new String[]{}, new String[]{"id"} ); This adds a table to the List of tables to be encrypted. It's parameters are :- The name of the table that is to be included in Encryption. the name of the table into which the encrypted data is to be stored. Note this functionality is not present but could be added. As such the value is ignored. The list of columns that are to be encrypted. The list of columns that are to be copied asis. This functionality is not present but could be added. As such the list is ignored. The list of columns that are to be skipped (e.g. id columns). Although coded, the functionality is not present. As such the list is ignored. What the App does. The final result is an database as per the database in the assets folder (named basedb.db) that has the data in the theData column of the PaidData table encrypted, but the the FreeData table is unchanged. This database could then be copied (e.g. using device explorer) and then included as an asset in the App that is to be distributed. That App could include a reversal of the Encryption using the secret key and the decryption part of the EncryptDecrypt class. e.g. The FreeData table :- The PaidData table :- When the App is started if copies the database (hard coded as basedb.db) from the assets folder it it exists and makes the Encrypt button visible. If the Encrypt button isn't visible then the asset file was not located. So it's time to correct the issue (provide the correct database file). Note as this is just a demo many checks/options that could/should be done or added are skipped for brevity. If the Encrypt button appears then encryption is just a matter of hitting the button. After hitting the button createEncryptedDatabase method is called. This creates a copy, this will be the encrypted database, of the database copied from the assets folder by copying the file to it's new name (as per the given database name which MUST be different to the asset's file name). Using the copied database it queries the table(s) defined in mTCCL (an instance of the TableColumnConvertList class) . The query will extract data only for the columns that have been specified as those to be encrypted. The query only obtain distinct rows (i.e if multiple rows exist that has the same data in the columns then only one of the rows is extracted). For each extracted row :- The commonly used ContentValues instance is cleared. The whereClause StringBuilder is cleared. Each column in the Cursor is checked to see if it is a column, defined in the table being processed (it should be as only column t be encrypted are extracted). if not then it is skipped. The original value is saved in the appropriate element of the string array columnOriginalValues (this to be used as the bind parameters for the WHERE clause of the UPDATE) An element of the ContentValues instance is added with the current column name and the encrypted data. This is done as per cv.put(s,ed.encrypt(c.getString(c.getColumnIndex(s)))); If the length of the whereClause is greater than 0 then AND is added to the whereClause, then the column name suffixed with =? is added to the whereClause being built. After all columns have been processed the SQLiteDatabase update method is called to update the columns setting the values to the encrypted values WHERE all the column match the original data. After all rows have been processed the Cursor is closed and the next table processed. If after all tables have been processed then error count is 0 then the transaction is set as successful, otherwise a message is Toasted Errors encountered Encrypting Database. Rolled back (not changed). The transaction is then ended (if not set as successful then the data is not updated but rolled back). The database will be in the data/data/package_name/databases folder e.g. :-
Example Decryption App for Answer 1 Note that the database (MyDatabase) was copied from data/data/package_name/databases of the App from the previous answer (i.e. the encrypted database) into the assets folder of this app The following is a very basic App that initially only has the Free Data, but as an Edit Text and a Button that allows the Paid Data to be decrypted and retrieved. The available data (Free Data initially) is listed in a ListView; after decryption the PaidData having been copied to the FreeData is then avialable and listed. Notes the data can be decrypted numerous times and each such successful attempt will add and display more rows. EncryptDEcrypt.java is identical to the one used in the Encrypt tool. The Database Helper is :- public class DBHelper extends SQLiteOpenHelper { public static final String DBNAME = "MyDatabase"; public static final int DBVERSION = 1; public static final String TBL_FREEDATA = "FreeData"; public static final String COL_FREEDATA_ID = "id"; public static final String COL_THEDATA = "theData"; SQLiteDatabase mDB; public DBHelper(Context context) { super(context, DBNAME, null, DBVERSION); loadDBFromAssets(context); mDB = this.getWritableDatabase(); } #Override public void onCreate(SQLiteDatabase db) { } #Override public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) { } public long insertFreeDataRow(String theData) { ContentValues cv = new ContentValues(); cv.put(COL_THEDATA,theData); return mDB.insert(TBL_FREEDATA,null,cv); } public Cursor getAllAvialableData() { return mDB.query(TBL_FREEDATA,new String[]{"*",COL_FREEDATA_ID + " AS " + BaseColumns._ID}, null,null,null,null,null ); } public void decryptAndLoadPaidData(Context context, String secretKey) { EncryptDecrypt ed = new EncryptDecrypt(context,secretKey); mDB.beginTransaction(); Cursor c = mDB.query("PaidData",null,null,null,null,null,null); while (c.moveToNext()) { String decrypted_data = ed.decrypt(c.getString(c.getColumnIndex(COL_THEDATA))); if (decrypted_data != null) { insertFreeDataRow(decrypted_data); } else { Toast.makeText(context,"Naughty, that's not the password.",Toast.LENGTH_LONG).show(); } } c.close(); mDB.setTransactionSuccessful(); mDB.endTransaction(); } private boolean loadDBFromAssets(Context context) { File dbFile = new File(context.getDatabasePath(DBNAME).getPath()); byte[] buffer = new byte[1024 * 4]; int read_length = 0; if (dbFile.exists()) return true; if (!dbFile.getParentFile().exists()) { dbFile.getParentFile().mkdirs(); } try { InputStream assetdb = context.getAssets().open(DBNAME); OutputStream realdb = new FileOutputStream(dbFile); while ((read_length = assetdb.read(buffer)) > 0) { realdb.write(buffer,0,read_length); } realdb.flush(); realdb.close(); assetdb.close(); } catch (IOException e) { e.printStackTrace(); return false; } return true; } } MainActivity.java is :- public class MainActivity extends AppCompatActivity { ListView mListView; EditText mSecretKeyInput; Button mDecrypt; SimpleCursorAdapter mSCA; Cursor mAllTheData; DBHelper mDBhlpr; #Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); mListView = this.findViewById(R.id.list); mSecretKeyInput = this.findViewById(R.id.secretKey); mDecrypt = this.findViewById(R.id.decrypt); mDBhlpr = new DBHelper(this); manageListView(); manageDecryptButton(); } private void manageListView() { mAllTheData = mDBhlpr.getAllAvialableData(); if (mSCA == null) { mSCA = new SimpleCursorAdapter( this,android.R.layout.simple_list_item_1,mAllTheData,new String[]{DBHelper.COL_THEDATA},new int[]{android.R.id.text1},0); mListView.setAdapter(mSCA); } else { mSCA.swapCursor(mAllTheData); } } private void manageDecryptButton() { mDecrypt.setOnClickListener(new View.OnClickListener() { #Override public void onClick(View v) { if (mSecretKeyInput.getText().toString().length() > 0) { mDBhlpr.decryptAndLoadPaidData(v.getContext(),mSecretKeyInput.getText().toString()); manageListView(); } } }); } } Result When first run the App only shows the Free Data as per :- If the correct password/secret key is input and the Get Paid Data button is pressed then the extra data is added :- If an incorrect password is provided, then the data is not loaded and a toast appears indicating the wrong password.
Database copied to local file system and Room Query won't work
I am trying to query data with Androids persistence library Room from a existing Database. No compilation error or runtime error. There is just no result in the cursor. The query is checked and looks fine (also in the generated Java as mentioned in the answer of 'Pinakin' Android Room Database DAO debug log The database existis in the /data/data/com.project/databases/ folder. The database has the correct size but there are no other files such as mentioned in the answer of 'kosas' Android Room database file is empty Similar style of copying as shown here https://android.jlelse.eu/room-persistence-library-with-pre-populated-database-5f17ef103d3d. I only changed the constructor to: grammarDatabase = GrammarDatabase.getInstance(appContext); because my GrammarDatabase looks like this: #Database( entities = { Term.class }, version = 1, exportSchema = false ) public abstract class GrammarDatabase extends RoomDatabase { private static final String DB_NAME = "database.db"; private static volatile GrammarDatabase instance; public static synchronized GrammarDatabase getInstance(Context context) { if (instance == null) { instance = create(context); } return instance; } private static GrammarDatabase create(final Context context) { return Room.databaseBuilder(context, GrammarDatabase.class, DB_NAME).fallbackToDestructiveMigration().allowMainThreadQueries().build(); } public abstract TermDao getTermDao(); } The .fallbackToDestructiveMigration() and .allowMainThreadQueries is just for testing purposes. The interface 'TermDao_Impl' (query string shortened): #Override public Cursor getSearchContent(String searchTerm, String languageCode) { final String _sql = "SELECT T._id, T.term FROM term AS T INNER JOIN ... INNER JOIN language AS L INNER JOIN ... AND L.code= ? AND T.term LIKE ?"; final RoomSQLiteQuery _statement = RoomSQLiteQuery.acquire(_sql, 2); int _argIndex = 1; if (languageCode == null) { _statement.bindNull(_argIndex); } else { _statement.bindString(_argIndex, languageCode); } _argIndex = 2; if (searchTerm == null) { _statement.bindNull(_argIndex); } else { _statement.bindString(_argIndex, searchTerm); } final Cursor _tmpResult = __db.query(_statement); return _tmpResult; } Call in MainActivity: First: database = DatabaseHelper.getInstance(context).getGrammarDatabase(); Then: Cursor c = database.getTermDao().getSearchContent("'"+searchTerm+"'", "'"+currentLanguageCode+"'"); if (c.moveToNext()) { System.out.println("Found!"); System.out.println(c.getString(c.getColumnIndex("T.term"))); } Tried to call with " ' " around the arguments and without. So the cursor is empty eventhough the query will have a result (tested in DB Browser for SQLite). "Found!" Is not printed.
Multiple Insert or Replace Query in Android SQLite
I am using below query to insert or replace "number" and "time" in "Contacts" table. Is there any way I can insert or replace multiple record in SQLite for Android? "Insert or replace into Contacts (_id, number, status) values ((select _id from Contacts where number = '123456'), '123456','sent');"
You can improve speed for Multiple/Batch Database Inserting or Replacing operation using concept of transaction and compileStatement so your query will be compiled only once. For Example: db.beginTransaction(); try { String sql = "Insert or Replace into Items (itemNumber, description,unitOfMeasure, weight) values(?,?,?,?)"; ArrayList<ItemMaster> itemsList = // Retrieve your items list here for(int i=0;i<itemsList.size();i++) { SQLiteStatement insert = db.compileStatement(sql); insert.bindString(1, item.getItemNumber()); insert.bindString(2, item.getItemDescription1()); insert.bindString(3, item.getSellingUOM()); insert.bindDouble(4, item.getWeight()); insert.execute(); } db.setTransactionSuccessful(); } finally { db.endTransaction(); }
Try this code for insert multiple entries public void insertIntoTable(ArrayList<YourModelClass> alist) { SQLiteDatabase db = this.getWritableDatabase(); String sql = "insert into tableName (colomname1,colomnname2) values(?,?)"; db.beginTransaction(); SQLiteStatement stmt = db.compileStatement(sql); for (int i = 0; i < alist.size(); i++) { stmt.bindString(1, alist.get(i).getMethod1()); stmt.bindString(2, alist.get(i).getMethod2()); stmt.execute(); stmt.clearBindings(); } db.setTransactionSuccessful(); db.endTransaction(); db.close(); } Also Make a Model class to specify your variable names. eg: class ModelClass { String var1,var2; public void setVar1(String var1) { this.var1=var1; } public String getVar1() { return var1; } public void setVar2(String var2) { this.var2=var2; } public String getVar2() { return var2; } }
Not sure I understand the query, but it looks like you can just update instead of insert or replace (the select will return null if number is not found) Update Contacts set status = 'sent' where number in (...)
Android SQLite: attempt to re-open an already-closed object
I'm trying to get certain book data from my Inventory table based on the ISBN. However, I'm getting an error: "attempt to re-open an already-closed object". The error only occurs when I click a listView object, go to a different screen, go back to this page via "finish()", and then try to click on another listView object. I moved the String searchEntries[] = InventoryAdapter.getInventoryEntriesByISBN(searchQuery, isbn[position]); from the onClickListener to the previous for loop before the onClickListener and now it works. Why does it not work if I try to getInventoryEntriesByISBN after returning to this activity from another activity via "finish()"? The error occurs at SearchResultsScreen: String searchEntries[] = InventoryAdapter.getInventoryEntriesByISBN(searchQuery, isbn[position]); and by extension, occurs at InventoryAdapter: Cursor cursor = db.rawQuery(query, new String[] {ISBN}); SearchResultsScreen.java // Set up search array for(int i = 0; i < isbn.length; i++) { searchArray.add(new InventoryItem(isbn[i], InventoryAdapter.getTitleAndAuthorByISBN(isbn[i]))); } Toast.makeText(getApplicationContext(), "searchArray.size()="+searchArray.size(), Toast.LENGTH_LONG).show(); // add data in custom adapter adapter = new CustomAdapter(this, R.layout.list, searchArray); ListView dataList = (ListView) findViewById(R.id.list); dataList.setAdapter(adapter); // On Click ======================================================== dataList.setOnItemClickListener(new OnItemClickListener() { #Override public void onItemClick(AdapterView<?> parent, View view, int position, long id) { String searchEntries[] = InventoryAdapter.getInventoryEntriesByISBN(searchQuery, isbn[position]); InventoryAdapter.java (Most relevant parts) public String[] getInventoryEntriesByISBN(String search, String ISBN) { String[] searchEntry = new String [9]; //Query String query = "select * from INVENTORY where ISBN = ?"; Cursor cursor = db.rawQuery(query, new String[] {ISBN}); if(cursor.getCount()<1) // title Not Exist { cursor.close(); for(int i = 0; i < 9; i++) searchEntry[i] = "Not Found"; return searchEntry; } cursor.moveToFirst(); //put data into respective variable int publish = cursor.getInt(cursor.getColumnIndex("PUBLISH_DATE")); String publishdate = ((Integer)publish).toString(); String title = cursor.getString(cursor.getColumnIndex("TITLE")); String author = cursor.getString(cursor.getColumnIndex("AUTHOR")); String callNumber = cursor.getString(cursor.getColumnIndex("CALL_NUMBER")); int available = cursor.getInt(cursor.getColumnIndex("AVAILABLE_COUNT")); String availablecount = ((Integer)available).toString(); int inventory = cursor.getInt(cursor.getColumnIndex("INVENTORY_COUNT")); String inventorycount = ((Integer)inventory).toString(); int due = cursor.getInt(cursor.getColumnIndex("DUE_PERIOD")); String dueperiod = ((Integer)due).toString(); int checkoutcount = cursor.getInt(cursor.getColumnIndex("COUNT")); String count = ((Integer)checkoutcount).toString(); //combine variables into one array searchEntry[0] = ISBN; searchEntry[1] = title; searchEntry[2] = author; searchEntry[3] = publishdate; searchEntry[4] = callNumber; searchEntry[5] = availablecount; searchEntry[6] = inventorycount; searchEntry[7] = dueperiod; searchEntry[8] = count; cursor.close(); return searchEntry; } public String getTitleAndAuthorByISBN(String ISBN) { int entriesFound = getNumSearchEntries(ISBN); if(entriesFound==0) entriesFound = 1; String searchEntry; //Query String query = "select * from INVENTORY where ISBN = ?"; Cursor cursor = db.rawQuery(query, new String[] {ISBN}); if(cursor.getCount()<1) // title Not Exist { cursor.close(); searchEntry = "Not Found"; return searchEntry; } cursor.moveToFirst(); //put data into respective variable String title = cursor.getString(cursor.getColumnIndex("TITLE")); String author = cursor.getString(cursor.getColumnIndex("AUTHOR")); //combine variables into one String searchEntry = title + " / " + author; //close cursor and return cursor.close(); return searchEntry; } DataBaseHelper.java public class DataBaseHelper extends SQLiteOpenHelper { // Database Version private static final int DATABASE_VERSION = 1; // Database Name private static final String DATABASE_NAME = "database.db"; // ============================ End Variables =========================== public DataBaseHelper(Context context, String name, CursorFactory factory, int version) { super(context, name, factory, version); } public DataBaseHelper(Context context) { super(context, DATABASE_NAME, null, DATABASE_VERSION); } // Called when no database exists in disk and the helper class needs // to create a new one. #Override public void onCreate(SQLiteDatabase _db) { _db.execSQL(LoginDataBaseAdapter.USER_TABLE_CREATE); _db.execSQL(CheckOutDataBaseAdapter.CHECKOUT_TABLE_CREATE); _db.execSQL(InventoryAdapter.INVENTORY_TABLE_CREATE); _db.execSQL(StatisticsAdapter.STATISTICS_TABLE_CREATE); } // Called when there is a database version mismatch meaning that the version // of the database on disk needs to be upgraded to the current version. #Override public void onUpgrade(SQLiteDatabase _db, int _oldVersion, int _newVersion) { // Log the version upgrade. Log.w("TaskDBAdapter", "Upgrading from version " +_oldVersion + " to " +_newVersion + ", which will destroy all old data"); // Upgrade the existing database to conform to the new version. Multiple // previous versions can be handled by comparing _oldVersion and _newVersion // values. // on upgrade drop older tables _db.execSQL("DROP TABLE IF EXISTS " + LoginDataBaseAdapter.USER_TABLE_CREATE); _db.execSQL("DROP TABLE IF EXISTS " + CheckOutDataBaseAdapter.CHECKOUT_TABLE_CREATE); _db.execSQL("DROP TABLE IF EXISTS " + InventoryAdapter.INVENTORY_TABLE_CREATE); _db.execSQL("DROP TABLE IF EXISTS " + StatisticsAdapter.STATISTICS_TABLE_CREATE); // Create a new one. onCreate(_db); } }
Check Database Connection before executing query: if (!dbHelper.db.isOpen()) dbHelper.open(); you can also use cursor.requery(); for again same query. and in last you have to close the cursor and database also. cursor.close(); db.close(); Edited: I have created DBHelper class which extends SQLiteOpenHelper, this class is inner class of DatabaseHelper class and that class have following methods. /** For OPEN database **/ public synchronized DatabaseHelper open() throws SQLiteException { dbHelper = new DBHelper(context); db = dbHelper.getWritableDatabase(); return this; } /** For CLOSE database **/ public void close() { dbHelper.close(); } If you have still doubt then feel free to ping me. Thank you.
The error only occurs when I click an item, go to a different screen, go back to this page via "finish()", and then try to click on another listView object. I moved the String searchEntries[] = InventoryAdapter.getInventoryEntriesByISBN(searchQuery, isbn[position]); from the onClickListener to the previous for loop before the onClickListener and now it works. The correct SearchResultsScreen is below: SearchResultsScreen.java // Set up search array final String Entries[][] = new String[isbn.length][9]; for(int i = 0; i < isbn.length; i++) { searchArray.add(new InventoryItem(isbn[i], InventoryAdapter.getTitleAndAuthorByISBN(isbn[i]))); Entries[i] = InventoryAdapter.getInventoryEntriesByISBN(searchQuery, isbn[i]); } Toast.makeText(getApplicationContext(), "searchArray.size()="+searchArray.size(), Toast.LENGTH_LONG).show(); // add data in custom adapter adapter = new CustomAdapter(this, R.layout.list, searchArray); ListView dataList = (ListView) findViewById(R.id.list); dataList.setAdapter(adapter); // On Click ======================================================== dataList.setOnItemClickListener(new OnItemClickListener() { #Override public void onItemClick(AdapterView<?> parent, View view, int position, long id) { String searchEntries[] = Entries[position];
This is your problem if(cursor.getCount()<1) // title Not Exist { cursor.close(); for(int i = 0; i < 9; i++) searchEntry[i] = "Not Found"; return searchEntry; } cursor.moveToFirst(); cursor.close(); Change to for (cursor.moveToFirst(); !cursor.isAfterLast(); cursor.moveToNext()) { { String title = cursor.getString(cursor.getColumnIndex("TITLE")); String author = cursor.getString(cursor.getColumnIndex("AUTHOR")); //combine variables into one String searchEntry = title + " / " + author; }
public String[] getInventoryEntriesByISBN(String search, String ISBN) { String[] searchEntry = new String [9]; //Query String query = "select * from INVENTORY where ISBN = ?"; Cursor cursor = db.rawQuery(query, new String[] {ISBN}); Add SQLiteDatabase db = this.getWritableDatabase(); in this code before executing the raw Query