Develop Reference

How to pre-populate an existing DB with data?

The app I'm building currently has a Workout database.
Workout DB has a Workout table and a WoroutSets table.
The data in these two tables is inserted(saved) through user input.
And that's where some data is stored.
By the way, I want to put pre-populated data called WorkoutList into this Workout DB.
I consulted the docs for this.
I exported Workout.db and pre-populated it with data in DB Browser for SQLite.
And we are going to use createFromAsset("Workout.db") as per the docs. (Haven't tried yet)
However, what I am concerned about is whether there is a conflict between the Work DB of the existing app and the Workout DB to which the WorkoutList table has been added.

Assuming that you want to preserve each app users workouts and workoutsetss that they have input then you would not want to overwrite them by using createFromAsset.
Rather I suspect that what you want to do is introduce a new workoutlist table that is populated with predefined/pre-existing rows in the workoutlist as per a database supplied as an asset. In this case you do not want to use the createFromAsset method (although you could potentially have a second database created from the asset, attach it to the original and then merge the data - this would be more complex than it need be).
You also have to consider how to handle new installs, in which case there will be no existing user input workouts and workoutsetss, in which case you could use createFromAsset method. However, you would not want any other user's workouts and workoutsetss rows.
Based upon this assumption perhaps consider this demo that introduces a new table (workoutlist) whose data is retrieved from an asset maintaining the original user data in the other tables (workout and workoutset) but for a new install of the App creates database from the asset.
the schema is made up so will very likely differ from yours but the principle applies.
Java has been used but it would take little to change it to Kotlin
Workout
#Entity
class Workout {
#PrimaryKey
Long workoutId=null;
String workoutName;
Workout(){};
#Ignore
Workout(String workoutName) {
this.workoutId=null;
this.workoutName=workoutName;
}
}
WorkoutSet (plural not used but easily changed)
#Entity
class WorkoutSet {
#PrimaryKey
Long workoutSetId=null;
String workoutSetName;
long workoutIdMap;
WorkoutSet(){}
#Ignore
WorkoutSet(String workoutSetName, long parentWorkout) {
this.workoutSetId=null;
this.workoutSetName = workoutSetName;
this.workoutIdMap = parentWorkout;
}
}
WorkkoutList (the new table)
#Entity
class WorkoutList {
#PrimaryKey
Long workoutListId=null;
String workoutListName;
}
AllDAO (just for completeness)
#Dao
abstract class AllDAO {
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(Workout workout);
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(WorkoutSet workoutSet);
#Query("SELECT count(*) FROM workout")
abstract long getNumberOfWorkouts();
}
WorkoutDatabase the #Database annotated class
#Database(entities = {Workout.class,WorkoutSet.class, WorkoutList.class /*<<<<<<<<<< ADDED for V2 */}, exportSchema = false, version = MainActivity.DATABASE_VERSION)
abstract class WorkoutDatabase extends RoomDatabase {
abstract AllDAO getAllDAO();
private static Context passed_context;
private static volatile WorkoutDatabase INSTANCE;
static WorkoutDatabase getInstance(Context context) {
passed_context = context;
if (INSTANCE==null && MainActivity.DATABASE_VERSION == 1) {
INSTANCE = Room.databaseBuilder(context,WorkoutDatabase.class,MainActivity.DATABASE_NAME)
.allowMainThreadQueries()
.build();
}
if (INSTANCE ==null && MainActivity.DATABASE_VERSION > 1) {
INSTANCE = Room.databaseBuilder(context,WorkoutDatabase.class,MainActivity.DATABASE_NAME)
.allowMainThreadQueries()
.createFromAsset(MainActivity.DATABASE_ASSET_NAME) /* so new App installs use asset */
.addMigrations(MIGRATION_1_TO_2) /* to handle migration */
.build();
}
return INSTANCE;
}
static Migration MIGRATION_1_TO_2 = new Migration(1,2) {
#SuppressLint("Range")
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
/* Create the new table */
database.execSQL("CREATE TABLE IF NOT EXISTS `WorkoutList` (`workoutListId` INTEGER, `workoutListName` TEXT, PRIMARY KEY(`workoutListId`))");
/* Cater for copying the data from the asset */
String tempDBName = "temp_" + MainActivity.DATABASE_NAME; /* name of the temporary/working database NOT an SQLITE TEMP database */
String newTableName = "workoutlist"; /* The table name */
String qualifiedNewTableName = tempDBName + "." + newTableName; /* The fully qualified new table name for the attached temp/wrk db */
String tempDBPath = passed_context.getDatabasePath(MainActivity.DATABASE_NAME).getParent() + File.separator + tempDBName; /* path to temp/wrk db */
try {
/* Copy the asset to a second DB */
InputStream asset = passed_context.getAssets().open(MainActivity.DATABASE_ASSET_NAME); /* open the asset */
File tempDB_File = new File(tempDBPath); /* File for temp/wrk database */
OutputStream tempdb = new FileOutputStream(tempDB_File); /* now an output stream ready for the copy */
int bufferLength = 1024 * 8; /* length of buffer set to 8k */
byte[] buffer = new byte[bufferLength]; /* the buffer for the copy */
/* copy the temp/wrk database from the asset to it's location */
while(asset.read(buffer) > 0) {
tempdb.write(buffer);
}
/* clean up after copy */
tempdb.flush();
tempdb.close();
asset.close();
/*Use the temporary/working database to populate the actual database */
/* Issues with WAL file change because migration is called within a transaction as per */
/* java.lang.IllegalStateException: Write Ahead Logging (WAL) mode cannot be enabled or disabled while there are transactions in progress. .... */
/* SO COMMENTED OUT */
//database.execSQL("ATTACH DATABASE '" + tempDBPath + "' AS " + tempDBName);
//database.execSQL("INSERT INTO " + newTableName + " SELECT * FROM " + qualifiedNewTableName);
//database.execSQL("DETACH " + tempDBName);
/* Alternative to ATTACH */
SQLiteDatabase assetdb = SQLiteDatabase.openDatabase(tempDB_File.getPath(),null,SQLiteDatabase.OPEN_READONLY);
Cursor csr = assetdb.query(newTableName,null,null,null,null,null,null);
ContentValues cv = new ContentValues();
while (csr.moveToNext()) {
cv.clear();
for (String s: csr.getColumnNames()) {
cv.put(s,csr.getString(csr.getColumnIndex(s)));
}
database.insert(newTableName,SQLiteDatabase.CONFLICT_IGNORE,cv);
}
assetdb.close();
tempDB_File.delete(); /* delete the temporary/working copy of the asset */
} catch (Exception e) {
/* handle issues here e.g. no asset, unable to read/write an so on */
e.printStackTrace();
}
}
};
}
This has been written to allow easy switching/running of the App with either version, simply two changes to run as old or new version of the App.
to run as version 1 DATABASE_VERSION (in MainActivity) is set to 1
AND the WorkoutSet class is commented out in the #Database annotation.
the Mirgration handles the copy of the data from the asset if the database already exists, otherwise for a new file then createFromAssets is used to copy the database from the asset.
MainActivity the activity code that does something with the database to ensure that it is opened/accessed
public class MainActivity extends AppCompatActivity {
public static final String DATABASE_NAME = "workout.db";
public static final int DATABASE_VERSION = 2;
public static final String DATABASE_ASSET_NAME = "testit.db"/*DATABASE_NAME*/ /* could be different */;
WorkoutDatabase wdb;
AllDAO dao;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
wdb = WorkoutDatabase.getInstance(this);
dao = wdb.getAllDAO();
if (dao.getNumberOfWorkouts() < 1) addInitialData();
//wdb.close(); // FORCE close so WAL is full checkpointed (i.e. no -wal or -shm)
}
private void addInitialData() {
String prefix = String.valueOf(System.currentTimeMillis()); // to differentiate devices
dao.insert(new Workout(prefix + "-W001"));
dao.insert(new Workout(prefix + "-W002"));
dao.insert(new WorkoutSet(prefix + "-WS001",1));
dao.insert(new WorkoutSet(prefix + "-WS002",2));
}
}
note the wdb.close() is uncommented when creating the database to be loaded into the SQlite tool.
testit.db the SQLite database file as modified to introduce the new workoutList table. This first copied from a run of the App at Version 1 and then adding the new table (according SQL copied from the createAllTables method of WorkoutDataabase_Impl class in the generated java (aka the exact table as per Room's expectations)).
Note really the rows should be deleted as they are user specific. Instead -WRONG has been added to the end of the data (helps to prove the some points)
as above
The new table
Navicat was the SQLite tool used rather than DB Browser.
Run 1 running at version 1 to populate the database
DATABASE_VERSION = 1
#Database(entities = {Workout.class,WorkoutSet.class/*, WorkoutList.class*/ /*<<<<<<<<<< ADDED for V2 */}, exportSchema = false, version = MainActivity.DATABASE_VERSION)
i.e. the WorkoutList table has been excluded
App Inspection shows:-
and
Impotantly no WorkoutList table
Run 2 version 2 and introduction of new workoutlist table
DATABASE_VERSION = 2
#Database(entities = {Workout.class,WorkoutSet.class, WorkoutList.class /*<<<<<<<<<< ADDED for V2 */}, exportSchema = false, version = MainActivity.DATABASE_VERSION)
i.e. WorkoutList now introduced
App Inspection shows:-
old rows retained (they do not have -WRONG as per the asset)
old rows retained
the new table, populated as per the asset
Run 3 new install (after App uninstalled) at VERSION 2
as an be seen the incorrectly left in the assets rows exist (obviously you would delete the rows in real life but leaving them in shows that they are from the asset rather than generated by the code in the activity)
likewise
Conclusion
So the new install of the App correctly copies the asset database via createFromAsset whilst if migrating only the new data in the workoutlist table is copied from the asset.

Room migration testing MigrationTestHelper version

Well, i am trying to test my database migration. Unfortunatly something looks like wrong.
#RunWith(AndroidJUnit4ClassRunner.class)
public class MigrationTest {
private static final String TEST_DB = "migration-test";
#Rule
public MigrationTestHelper helper;
public MigrationTest() {
helper = new MigrationTestHelper(InstrumentationRegistry.getInstrumentation(),
AppDatabase.class.getCanonicalName(),
new FrameworkSQLiteOpenHelperFactory());
}
#Test
public void migrateAll() throws IOException {
// Create earliest version of the database.
SupportSQLiteDatabase db = helper.createDatabase(TEST_DB, 1);
db.close();
// Open latest version of the database. Room will validate the schema
// once all migrations execute.
AppDatabase appDb = Room.databaseBuilder(
InstrumentationRegistry.getInstrumentation().getTargetContext(),
AppDatabase.class,
TEST_DB)
.addMigrations(ALL_MIGRATIONS).build();
appDb.getOpenHelper().getWritableDatabase();
appDb.close();
}
// Array of all migrations
private static final Migration[] ALL_MIGRATIONS = new Migration[]{MIGRATION_1_2};
}
Migration code.
public static final Migration MIGRATION_1_2 = new Migration(1, 2) {
#Override
public void migrate(SupportSQLiteDatabase database) {
database.execSQL("ALTER TABLE mytable ADD COLUMN reference_code TEXT");
}
};
All is working fine with real migration but in junit test case i have the following error.
E/SQLiteLog: (1) duplicate column name: reference_code
E/TestRunner: failed: migrateAll(com.apps.MigrationTest)
E/TestRunner: ----- begin exception -----
E/TestRunner: android.database.sqlite.SQLiteException: duplicate column name: reference_code (code 1 SQLITE_ERROR): , while compiling: ALTER TABLE mytable ADD COLUMN reference_code TEXT
at android.database.sqlite.SQLiteConnection.nativePrepareStatement(Native Method)
at android.database.sqlite.SQLiteConnection.acquirePreparedStatement(SQLiteConnection.java:986)
at a
As i understand, it looks like SupportSQLiteDatabase db = helper.createDatabase(TEST_DB, 1); is creating the schema V2 of my database (and not the version 1).
As a result the new column is taggued as duplicate.
To fix it i have to rollback my version = 1 to #Database class and then start my junit test again.
Can anyone help me on it ?
I follow the google guide here : https://developer.android.com/training/data-storage/room/migrating-db-versions.html

Well, i finally found it. It looks like a wrong schema has been generated in my assets folder.
To fix the issue, here is what i did.
Delete 1.json and 2.json files from the assets folder (each file contains the structure of the database version)
Rollback to the version 1 database (in my code), build > make projet
You will see the 1.json in your assets folder
Made my changes, i mean adding my new column in my Table.java file
Build > make projet
You will see the 2.json in your assets folder
Run the junit test, working now
Here is the difference bewteen my java object and database version 1 et 2
#Entity(tableName = "Table")
public class Table implements Parcelable {
#ColumnInfo(name = COLUMN_REFERENCE_CODE)
private String referenceCode;
}
Hope this will help.

Why Room select query is not working via c++?

I create the database via Room and gave the path to C++ developers via JNI. All data are inserting from C++ side by using sqlpp, then i am executing SQL query and not getting fresh datas. If i kill app then start again, SQL query is returning all datas.
#Provides
#Singleton
LocalDatabase provideLocalDatabase(#DatabaseInfo String dbName, Context context) {
return Room.databaseBuilder(context, LocalDatabase.class, dbName)
.setJournalMode(RoomDatabase.JournalMode.WRITE_AHEAD_LOGGING)
.build();
}
MessageDao
#Dao
public abstract class MessageDao implements BaseDao<Message> {
#Query("SELECT * FROM messages")
public abstract Flowable<Message> getMessage();
}
Sending db path to C++
String uiDbPath = MyApp.applicationContext.getDatabasePath(AppConstants.DB_NAME).getAbsolutePath();
send_db_path_to_c_plus_plus(uiDbPath);
We are using JournalMode.WRITE_AHEAD_LOGGING mode, after inserting data to database i am checking the database to make sure all data are exits. As a result i am seeing all data, but select query is not returning. How can i solve the issue please help? Thanks in advance.
EDIT 1 C++ parts
int64_t insert(manager::db::connection & conn, int64_t conv_id, int64_t author_id, const std::string & data)
{
auto conn_guard = manager::db::make_connection_guard(conn);
{
LOG_TRACE("db::insert(in_msg) - BEGIN conv_id={0} author_id={1}", conv_id, author_id);
const auto tab_in_msg = manager::db::tables::InMsgs{};
auto tx = start_transaction(conn_guard.native());
conn_guard.native()(insert_into(tab_in_msg).set(tab_in_msg.conversationId = conv_id, tab_in_msg.authorId = author_id, tab_in_msg.data = data));
int64_t rowid = conn_guard.native().last_insert_id();
tx.commit();
LOG_TRACE("db::insert(in_msg) - END rowid={0}", rowid);
return rowid;
}
}

Not able to update sqlite_sequence table using RoomDatabase.query

We try to update sqlite_sequence with the following code.
WeNoteRoomDatabase weNoteRoomDatabase = WeNoteRoomDatabase.instance();
weNoteRoomDatabase.query(new SimpleSQLiteQuery("UPDATE sqlite_sequence SET seq = 0 WHERE name = 'attachment'"));
However, it has no effect at all. I exam the sqlite_sequence table content using SQLite browser. The counter is not reset to 0.
If we try to run the same query manually using SQLite browser on same SQLite file, it works just fine.
Our Room database is pretty straightforward.
#Database(
entities = {Attachment.class},
version = 6
)
public abstract class WeNoteRoomDatabase extends RoomDatabase {
private volatile static WeNoteRoomDatabase INSTANCE;
private static final String NAME = "wenote";
public abstract AttachmentDao attachmentDao();
public static WeNoteRoomDatabase instance() {
if (INSTANCE == null) {
synchronized (WeNoteRoomDatabase.class) {
if (INSTANCE == null) {
INSTANCE = Room.databaseBuilder(
WeNoteApplication.instance(),
WeNoteRoomDatabase.class,
NAME
)
.build();
}
}
}
return INSTANCE;
}
}
Any idea what we had missed out?
Additional information : clearing sqlite_sequence is not working in android room

Room doesn't use SQLiteDatabase - but it uses SupportSQLiteDatabase, while it's source code states, that it delegates all calls to an implementation of {#link SQLiteDatabase}... I could even dig further - but I'm convinced, that this is a consistency feature and not a bug.
SQLiteDatabase.execSQL() still works fine, but with SupportSQLiteDatabase.execSQL() the same UPDATE or DELETE queries against internal tables have no effect and do not throw errors.
my MaintenanceHelper is available on GitHub. it is important that one initially lets Room create the database - then one can manipulate the internal tables with SQLiteDatabase.execSQL(). while researching I've came across annotation #SkipQueryVerification, which could possibly permit UPDATE or DELETE on table sqlite_sequence; I've only managed to perform a SELECT with Dao... which in general all hints for the internal tables are being treated as read-only, from the perspective of the publicly available API; all manipulation attempts are being silently ignored.

i think query is wrong, you should try below query
weNoteRoomDatabase.query(new SimpleSQLiteQuery("UPDATE sqlite_sequence SET seq = 0 WHERE name = attachment"));

I'm using room database version 2.2.5
Here I'm unable to execute this query using Room Dao structure, so make one simple class and access method as like this and I got successful outcomes so this one is tested result. I'm using RxJava and RxAndroid for same.
public class SqlHelper {
private static SqlHelper helper = null;
public static SqlHelper getInstance() {
if (helper == null) {
helper = new SqlHelper();
}
return helper;
}
public Completable resetSequence(Context context) {
return Completable.create(emitter -> {
try {
AppDatabase.getDatabase(context)
.getOpenHelper()
.getWritableDatabase()
.execSQL("DELETE FROM sqlite_sequence WHERE name='<YOUR_TABLE_NAME>'");
emitter.onComplete();
} catch (Exception e) {
emitter.onError(e);
}
});
}
}
Execute:
SqlHelper.getInstance()
.resetQuizSequence(context)
.subscribeOn(Schedulers.io()
.observeOn(AndroidSchedulers.mainThread())
.subscribe(() -> {}, error -> {});

Table sql_sequence is not managed by Room, so you need to edit it using a SupportSQLiteDatabase.
Try this:
String sqlQuery = "DELETE FROM sqlite_sequence WHERE name='attachment'";
weNoteRoomDatabase().getOpenHelper().getWritableDatabase().execSQL(sqlQuery);

This works for me - Room 2.2.6
String sqlQuery = "DELETE FROM sqlite_sequence WHERE name='attachment'";
<YourDatabase>.getInstance(mContext).getOpenHelper().getWritableDatabase().execSQL(sqlQuery);

how to delete a record from ORMLITE?

I need to delete a record from ORMLite Database I can delete a record by id using as below
#Override
public void Delete(int id) throws SQLException {
this.dao.deleteById(id);
}
but what if I have to delete a record from same table not by id but by name or any other field
I want something like
public void Deletefromcanteen(String name,MealType mealtype) {
this.dao.deletebyName(name);
}
what query should i write using querybuilder to delete a record where name = name and mealtype = say (lunch)
I tried something like this in my databasehelper class
public void deletefromcanteen(int id, String mealtype) {
try {
Dao<CanteenLog, Integer> canteenDao = getCanteen();
DeleteBuilder<CanteenLog, Integer> deleteBuilder = canteenDao
.deleteBuilder();
deleteBuilder.where().eq("user_id", id).and().eq("meal", mealtype);
canteenDao.delete(deleteBuilder.prepare());
} catch (Exception e) {
...
}
}

deleteBuilder.where().eq("FIELD_NAME", arg);
deleteBuilder.delete();
Update:
For example :
//Get helper
DatabaseHelper helper = OpenHelperManager.getHelper(App.getContext(), DatabaseHelper.class);
//get dao
Dao dao = helper.getDao(YOUR_CLASS.class);
//delete elements from table in field by arg
DeleteBuilder<CanteenLog, Integer> deleteBuilder = dao.deleteBuilder();
deleteBuilder.where().eq("FIELD_NAME", arg);
deleteBuilder.delete();
Good luck!

To save building a query, you can do a select to find the ID, then do a delete by ID. Thsi will be straightforward if you already have the select query set up for this table.

It can be done also like that
DatabaseHelper.getInstance().getDao(YourObject.class).delete(yourObjectInstance);

If you are using Kotlin, you will need to do something like this instead since the type inference is not currently working correctly for the fluent syntax:
val deleteBuilder = dao.deleteBuilder()
val where = deleteBuilder.where().eq("address", address)
deleteBuilder.setWhere(where)
deleteBuilder.delete()

If you already have an instance of the object (or object list) you want to delete, just do :
//Get helper
DatabaseHelper helper =
OpenHelperManager.getHelper(App.getContext(), DatabaseHelper.class);
// delete 'em !
helper.getDao(YourObject.class).delete(yourObjectInstance);
If you don't have the instance to delete or you want to do it SQL way(!):
//Get helper
DatabaseHelper helper =
OpenHelperManager.getHelper(App.getContext(), DatabaseHelper.class);
//get dao
Dao dao = helper.getDao(YOUR_CLASS.class);
//delete elements from table in field by arg
DeleteBuilder<CanteenLog, Integer> deleteBuilder = dao.deleteBuilder();
deleteBuilder.where().eq("FIELD_NAME", "FIELD_VALUE");
deleteBuilder.delete();
If you need a more complicated Where logic, then use deleteBuilder.where().or() or deleteBuilder.where().and() to add more constraints.