Develop Reference

Can Android Room manage multiple databases and create databases from a template database?

Any thoughts, suggestions, or ideas on best practice or option for the following?
Problem: With Android Room, what is the practical way to approach creating and handling multiple databases in a single Android app?
What I'm trying to do: I have an Android app intended to manage multiple research topics. The idea is an user can create databases specific to subjects or research and able to store sources, attachments and notes on those subjects. For example, an user can have a database specific to the history of modern music, a second database on the subject of the history of hunting, and even subjects as deep as micro-biological research.
My thoughts were to have separate databases vs. one database that stores all this data. Especially since attachments can be stored and take up space quite quickly. And these databases can be shared between the phone / tablet app and a desktop version. There is a Java desktop version of this being used.
What I've Done" I've really only searched here and googled some but appear kind-of vague. I'm familiar with and have migrated changes to a database but wasn't sure if this would always be the best way to create a new database, as well as renaming, etc.
This Android app comes with a predefined and pre-popuated database as a demonstration. This database hasn't changed for 2 years now. So, the idea was possibly having a "template.db" that could be used to create new databases and rename them accordingly.

With Android Room, what is the practical way to approach creating and handling multiple databases in a single Android app?
You can certainly handle multiple databases and multiple databases based upon the same schema.
The issue is how to ascertain what databases there are that can be used. If all the databases were located in the same path (or even multiple paths) then this could be used. Another methodology could be to have a database of the databases.
Here's an example that utilises the databases of databases (the "MasterDatabase") and allows access to x databases.
So first the MasterDatabase which has a simple table with an id column (could be dispensed with) and a column for the database name. The table (#Entity) being named MasterDatabaseList as per :-
#Entity(
indices = { #Index(value = "databaseName", unique = true)
}
)
class MasterDatabaseList {
#PrimaryKey
Long id;
String databaseName;
public MasterDatabaseList() {}
#Ignore
public MasterDatabaseList(String databaseName) {
this.databaseName = databaseName;
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getDatabaseName() {
return databaseName;
}
public void setDatabaseName(String databaseName) {
this.databaseName = databaseName;
}
}
Note the unique index on the databaseName column
Accompanying the table is MasterDao an #Dao class :-
#Dao
abstract class MasterDao {
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(MasterDatabaseList masterDatabaseList);
#Query("SELECT * FROM masterdatabaselist")
abstract List<MasterDatabaseList> getAllDatabases();
}
allows rows to be inserted or extracted.
a duplicated database will be ignored and thus not added.
MasterDatabase is the #Database class (which ties the previous classes to the database) and includes a method to get an instance of the database from which the MasterDao can be accessed :-
#Database(
entities = {MasterDatabaseList.class},
version = 1
)
abstract class MasterDatabase extends RoomDatabase {
abstract MasterDao getMasterDao();
static volatile MasterDatabase instance = null;
public static MasterDatabase getInstance(Context context) {
if (instance == null) {
instance = Room.databaseBuilder(context,MasterDatabase.class,"master.db")
.allowMainThreadQueries()
.build();
}
return instance;
}
}
Now the template database, Base???? (a simple single table database for the demo). First the table BaseTable #Entity class:-
#Entity
class BaseTable {
#PrimaryKey
Long id;
String mydata;
public BaseTable(){}
#Ignore
public BaseTable(String myData) {
this.mydata = myData;
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getMydata() {
return mydata;
}
public void setMydata(String mydata) {
this.mydata = mydata;
}
}
a very simple table with an id column and a column that holds some string data.
Accompanying is the #Dao class BaseDao :-
#Dao
abstract class BaseDao {
#Insert
abstract long insert(BaseTable baseTable);
#Query("SELECT * FROM basetable")
abstract List<BaseTable> getAllBaseTables();
#Update
abstract int update(BaseTable baseTable);
}
with very basic insert, extract and update
and as before the #Database class BaseDatabase :-
#Database(
entities = {BaseTable.class},
version = 1
)
abstract class BaseDatabase extends RoomDatabase {
abstract BaseDao getBaseDao();
public static BaseDatabase getInstance(Context context, String databaseName) {
BaseDatabase instance = null;
if (databaseName != null) {
return Room.databaseBuilder(context, BaseDatabase.class, databaseName)
.allowMainThreadQueries()
.build();
}
return instance;
}
}
note how the database name needs to be passed, that is basically the crux of catering for multiple databases.
With all of that a Demo Activity.
public class MainActivity extends AppCompatActivity {
private static final String TAG = "DBINFO";
MasterDatabase masterDB;
MasterDao masterDao;
/* 3 Lists that need to be synchronised index wise */
/* i.e. each index position should hold the respective name/databaseobject/dao
/* List of the known databases (their names) */
List<MasterDatabaseList> masterDatabaseListList = null;
/* List of the BaseDatabase objects */
ArrayList<BaseDatabase> baseDatabaseList = new ArrayList<>();
/* List of the BaseDao's */
ArrayList<BaseDao> baseDaoList = new ArrayList<>();
/* The current database */
int currentBaseIndex = -1; /* Index into the three Lists */
BaseDatabase currentDB = null;
BaseDao currentDao = null;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
masterDB = MasterDatabase.getInstance(this);
masterDao = masterDB.getMasterDao();
masterDatabaseListList = masterDao.getAllDatabases();
// Add default db1 if it does not exist
if (masterDatabaseListList.size() < 1) {
addBaseDB("db1");
}
buildBaseLists();
/* Add some data to db1 IF it exists (it should) */
setCurrentIndexDBandDao("db1");
if (currentBaseIndex > -1) {
currentDao.insert(new BaseTable("Blah for db1"));
}
/* Add some data to db2 (it will not exist) */
/* noting that the database will be created if it does not exist */
setCurrentIndexDBandDao("db2");
if (currentBaseIndex == -1) {
addBaseDB("db2");
}
if (currentBaseIndex > -1) {
currentDao.insert(new BaseTable("Blah for db2"));
}
/* Extract and Log Data for ALL the BaseDatabase databases i.e. db1 and db2 */
for(MasterDatabaseList masterdb: masterDao.getAllDatabases()) {
Log.d(TAG,"Database is " + masterdb.getDatabaseName());
setCurrentIndexDBandDao(masterdb.databaseName);
if (currentBaseIndex > -1) {
for(BaseTable bt: currentDao.getAllBaseTables()) {
Log.d(TAG,"Extracted Base Table row where MyData is" + bt.getMydata());
}
}
}
}
/* Add a new Database */
/* Note that it assumes that it will now be the current */
/* so the current values are set */
private void addBaseDB(String baseDBName) {
masterDao.insert(new MasterDatabaseList(baseDBName));
buildBaseLists();
setCurrentIndexDBandDao(baseDBName);
}
/* Build/ReBuild the 3 Lists according to the master database */
/* This could be better managed so as to not rebuild existing database/dao objects */
private void buildBaseLists() {
int ix = 0;
baseDatabaseList.clear();
baseDaoList.clear();
masterDatabaseListList = masterDao.getAllDatabases();
// Loop through the databases defined in the master database adding the database and dao to the respective lists
for (MasterDatabaseList masterDB: masterDao.getAllDatabases()) {
BaseDatabase baseDB = BaseDatabase.getInstance(this, masterDB.getDatabaseName());
baseDatabaseList.add(baseDB);
baseDaoList.add(baseDB.getBaseDao());
ix++;
}
}
/* Set the current trio according to the database name that is:*/
/* 1.the currentBaseIndex for the 3 Lists */
/* 2. the BaseDatabase object */
/* 3. the BaseDao */
/* The index value (currentBaseIndex) is also returned */
private int setCurrentIndexDBandDao(String baseDBName) {
currentBaseIndex = getListIndexByBaseDBName(baseDBName);
if (currentBaseIndex > -1) {
currentDB = baseDatabaseList.get(currentBaseIndex);
currentDao = baseDaoList.get(currentBaseIndex);
}
return currentBaseIndex;
}
/* Get the index according to the database name passed */
/* note -1 signifies not know/found */
private int getListIndexByBaseDBName(String baseDBName) {
masterDatabaseListList = masterDao.getAllDatabases(); // OverKill????
int rv = -1; // default to not found
for(int i=0; i < masterDatabaseListList.size();i++) {
if (masterDatabaseListList.get(i).databaseName.equals(baseDBName)) {
rv = i;
break;
}
}
return rv;
}
/* Output all rows from the BaseTable for data extracted by the BaseDaos getAllBaseTables */
private void logBaseData(List<BaseTable> baseTableList) {
Log.d(TAG,"Current Database Index is " + currentBaseIndex + " DB name is " + masterDatabaseListList.get(currentBaseIndex).getDatabaseName());
for(BaseTable bt: baseTableList) {
Log.d(TAG,"\tMyData value is " + bt.getMydata());
}
}
}
Result
When the above is run for the first time the log includes:-
2021-09-16 11:39:30.262 D/DBINFO: Database is db1
2021-09-16 11:39:30.278 D/DBINFO: Extracted Base Table row where MyData isBlah for db1
2021-09-16 11:39:30.278 D/DBINFO: Database is db2
2021-09-16 11:39:30.284 D/DBINFO: Extracted Base Table row where MyData isBlah for db2
And via Android Studio's App Inspector the databases :-
and for the db2 BaseTable :-
Note The above is only intended to cover the basics of utilising multiple databases in what is intended to be a simplistic explanation, as such the code has been kept short and simple. It would probably be unacceptable, as it is, for an App that would be distribtued.

Room: error: Not sure how to convert a Cursor to this method's return type (void)

I have this query set up to return all the records from these tables and display the information on a recyclerview in android. the DB is set up using the Room persistence library aka SQLITE.
#Query
("SELECT moodBeforetable.userId,
moodBeforetable.moodBefore,
moodBeforetable.cbtId,
cbtTable.automaticThought,
cbtTable.twistedThinkingPK,
cbtTable.challengeThought,
cbtTable.rationalThought,
cbtTable.date,
moodAfterTable.moodAfter,
twistedThinkingTable.twistedThinkingPK,
twistedThinkingTable.allOrNothing,
twistedThinkingTable.blamingOthers,
twistedThinkingTable.catastrophizing,
twistedThinkingTable.emotionalReasoning,
twistedThinkingTable.fortuneTelling,
twistedThinkingTable.labelling,
twistedThinkingTable.magnifyingTheNegative,
twistedThinkingTable.mindReading,
twistedThinkingTable.minimisingThePositive,
twistedThinkingTable.overGeneralisation,
twistedThinkingTable.selfBlaming,
twistedThinkingTable.shouldStatement
FROM moodBeforetable
JOIN cbtTable ON moodBeforetable.cbtId = cbtTable.cbtId
JOIN twistedThinkingTable ON cbtTable.cbtId = twistedThinkingTable.cbtId
JOIN moodAfterTable ON moodAfterTable.cbtId = cbtTable.cbtId
WHERE moodBeforetable.date >= datetime('now', '-1 year')
AND moodBeforetable.userId = :userId
ORDER BY :date DESC")
LiveData<List<MoodBeforeTable>> moodLogsAll (int userId, String date);
When I try to compile the app I get the following error:
The query returns some columns which are not used by com.example.feelingfit.persistence.tables.MoodBeforeTable.
You can use #ColumnInfo annotation on the fields to specify the mapping.
Could anyone help me debug this and find out why the app wont compile?

Problem is Room cannot map the result from your custom query to existing MoodBeforeTable. It is because your return type is List<MoodBeforeTable> but you have used joins using TwistedThinkingTable and MoodAfterTable and so on.
What you should do is create a new POJO like below:
public class MoodLogPojo() {
private int userId;
private String moodBefore;
zprivate int cbtId;
private String automaticThought;
private int twistedThinkingPK;
private String challengeThought;
private String rationalThought;
private String date;
private String moodAfter;
private int twistedThinkingPK;
private String allOrNothing;
private String blamingOthers;
private String catastrophizing;
private String emotionalReasoning;
private String fortuneTelling;
private String labelling;
private String magnifyingTheNegative;
private String mindReading;
private String minimisingThePositive;
private String overGeneralisation;
private String selfBlaming;
private String shouldStatement;
// generate getters and setters too
public void setSelfBlaming(Stirng selfBlming) {
this.selfBlming = selfBlming
}
public String getSelfBlaming() { return selfBlaming; }
// and so on ...
}
Then use this class as the return type like :
LiveData<List<MoodLogPojo>> moodLogsAll (int userId, String date);.
NOTE: Mind the MoodLogPojo class. Modify it accoring to the corresponding data types from each Entity.

How to get timestamp(rowversion) from sql azure to android with mobileservice

i have a problem getting timestamp(rowversion) from my SQL Azure database.
In my tables there is a column with datatype timestamp. This timestamp isn't similar to datetime, it's more like a rowversion.
I can get all other data in this table with the query from MobileServiceTable, there is no problem.
But this special datatype is a problem.
My class for this table looks like:
public class ArbeitsgangBezeichnung {
#com.google.gson.annotations.SerializedName("id")
private int ID;
#com.google.gson.annotations.SerializedName("ABZ_ArbeitsgangBezeichnungID")
private int ABZ_ArbeitsgangBezeichnungID;
#com.google.gson.annotations.SerializedName("ABZ_Bezeichnung")
private String ABZ_Bezeichnung;
#com.google.gson.annotations.SerializedName("ABZ_RowVersion")
private StringMap<Number> ABZ_RowVersion;
//constructor, getter, setter, etc....
}
If i login in Azure and look at the table, there are my example values and the automatic generated timestamp. The timestamp value looks like "AAAAAAAAB/M=". If i login in sql database and let me show the data, then for timestamp there is only "binarydata" (in pointed brackets) and not that value as it is shown in Azure.
The variable "ABZ_RowVersion" should include this timestamp, but the data in the StringMap doesn't look like the one in Azure. I tried String and Byte as datatype for the StringMap, but it doesn't helped.
I tried byte[] for ABZ_RowVersion, but then i got an exception in the callback method.
Then i tried Object for ABZ_RowVersion, that time i found out, that it is a StringMap, but nothing more.
Does anybody know, how to get the data from timestamp, i need it for comparison.
Thanks already

When you create a timestamp column in a table, it's essentially a varbinary(8) column. In the node SQL driver, it's mapped to a Buffer type (the usual node.js type used for binary data). The object which you see ({"0":0, "1":0, ..., "length":8}) is the way that a buffer is stringified into JSON. That representation doesn't map to the default byte array representation from the Gson serializer in Android (or to the byte[] in the managed code).
To be able to use timestamp columns, the first thing you need to do is to "teach" the serializer how to understand the format of the column returned by the server. You can do that with a JsonDeserializer<byte[]> class:
public class ByteArrayFromNodeBufferGsonSerializer
implements JsonDeserializer<byte[]> {
#Override
public byte[] deserialize(JsonElement element, Type type,
JsonDeserializationContext context) throws JsonParseException {
if (element == null || element.isJsonNull()) {
return null;
} else {
JsonObject jo = element.getAsJsonObject();
int len = jo.get("length").getAsInt();
byte[] result = new byte[len];
for (int i = 0; i < len; i++) {
String key = Integer.toString(i);
result[i] = jo.get(key).getAsByte();
}
return result;
}
}
}
Now you should be able to read data. There's still another problem, though. On insert and update operations, the value of the column is sent by the client, and SQL doesn't let you set them in them. So let's take this class:
public class Test {
#SerializedName("id")
private int mId;
#SerializedName("name")
private String mName;
#SerializedName("version")
private byte[] mVersion;
public int getId() { return mId; }
public void setId(int id) { this.mId = id; }
public String getName() { return mName; }
public void setName(String name) { this.mName = name; }
public byte[] getVersion() { return mVersion; }
public void setVersion(byte[] version) { this.mVersion = version; }
}
On the insert and update operations, the first thing we need to do in the server-side script is to remove that property from the object. And there's another issue: after the insert is done, the runtime doesn't return the rowversion property (i.e., it doesn't update the item variable. So we need to perform a lookup against the DB to retrieve that column as well:
function insert(item, user, request) {
delete item.version;
request.execute({
success: function() {
tables.current.lookup(item.id, {
success: function(inserted) {
request.respond(201, inserted);
}
});
}
});
}
And the same on update:
function update(item, user, request) {
delete item.version;
request.execute({
success: function() {
tables.current.lookup(item.id, {
success: function(updated) {
request.respond(200, updated);
}
});
}
});
}
Now, this definitely is a lot of work - the support for this type of column should be better. I've created a feature request in the UserVoice page at http://mobileservices.uservoice.com/forums/182281-feature-requests/suggestions/4670504-better-support-for-timestamp-columns, so feel free to vote it up to help the team prioritize it.

Android Ormlite upgrade issue with ForeignCollectionField

While upgrading my database schema I have run into a problem with a ForeignCollectionField (ormlite 4.7) not returning rows. With a clean install of the app, rows can be added and displayed as expected.
When the app is updated to a new version, the schema is updated (see below), but when rows are added to the database the collection is not returning the added rows. (I can see the rows exist in the database)
The parent row existed before the update. What do I need to do to fix it?
Parent class with Foreign Collection defined
#DatabaseTable(tableName = "setting")
public class SettingEntity {
#DatabaseField(generatedId = true)
private long id;
…
//New field added
#ForeignCollectionField
private ForeignCollection<DistributionEntity> distribution;
public SettingEntity() {
// Required for ORMLite
}
public ForeignCollection<DistributionEntity> getDistribution() {
return distribution;
}
public void setDistribution(ForeignCollection<DistributionEntity> distribution) {
this.distribution = distribution;
}
}
Child class
#DatabaseTable(tableName = "distribution")
public class DistributionEntity {
#DatabaseField(generatedId = true)
private long id;
…
//New field added
#DatabaseField(canBeNull = true, foreign = true, index = true, foreignAutoRefresh = true, columnDefinition = "integer references setting(id) on delete cascade")
private SettingEntity setting;
public void setSetting(SettingEntity setting) {
this.setting = setting;
}
}
onUpgrade code
RuntimeExceptionDao<DistributionEntity, Integer> distributionDao = helper.getDistributionDao();
distributionDao.executeRaw("ALTER TABLE distribution ADD setting_id");
distributionDao.executeRaw("CREATE INDEX distribution_setting_idx ON distribution (setting_id)");
Debug info of the ForeignCollectionField call distribution
The code that iterates over the collection
public ArrayList<Distribution> getDistribution() {
getEntity().getDistribution().size();
final ArrayList<Distribution> items = new ArrayList<Distribution>();
final ForeignCollection<DistributionEntity> collection = getEntity().getDistribution();
for (final DistributionEntity item : collection) {
final Distribution dist = new Distribution(item, mContext);
items.add(dist);
}
return items;
}
NB getEntity() returns an instance of SettingEntity
Thanks for spending the time

More of a workaround than answer but had to get around this problem. Replicated behavior by writing code.
public List<DistributionEntity> getDistribution() {
List<DistributionEntity> distributionEntities = new ArrayList<DistributionEntity>();
try {
DatabaseHelper helper = DatabaseHelper.getInstance();
RuntimeExceptionDao<DistributionEntity, Integer> dao = helper.getDistributionDao();
QueryBuilder<DistributionEntity, Integer> queryBuilder = dao.queryBuilder();
queryBuilder.where().eq(DistributionEntity.SETTING_FIELD_NAME, Long.toString(this.getId()));
PreparedQuery<DistributionEntity> preparedQuery = queryBuilder.prepare();
distributionEntities = dao.query(preparedQuery);
} catch (SQLException e) {
e.printStackTrace();
}
return distributionEntities;
//return distribution;
}
Love to know what the true answer is

ORMLite where clausule in String Array

I use ormlite and I have a db with a field:
public static final String NAME = "name";
#DatabaseField (canBeNull = false, dataType = DataType.SERIALIZABLE, columnName = NAME)
private String[] name = new String[2];
And I would like to get all elements that name[0] and name[1] are "car". I try to add a where clausule like:
NAMEDB nameDB = null;
Dao<NAMEDB, Integer> daoName = this.getHelper().getDao(NAMEDB.class);
QueryBuilder<NAMEDB, Integer> queryName = daoName.queryBuilder();
Where<NAMEDB, Integer> where = queryName.where();
where.in(nameDb.NAME, "car");
But it doesn't work because it's an array string.
I have other fields:
public static final String MARK = "mark";
#DatabaseField (canBeNull = false, foreign = true, index = true, columnName = MARK)
private String mark = null;
And I can do this:
whereArticulo.in(nameDB.MARK, "aaa");
How can I solve my problem? Thanks.

It seems to me that a third option to store a string array (String[] someStringArray[]) in the database using Ormlite would be to define a data persister class that converts the string array to a single delimited string upon storage into the database and back again to a string array after taking it out of the database.
E.g., persister class would convert ["John Doe", "Joe Smith"] to "John Doe | Joe Smith" for database storage (using whatever delimiter character makes sense for your data) and converts back the other way when taking the data out of the database.
Any thoughts on this approach versus using Serializable or a foreign collection? Anyone tried this?
I just wrote my first persister class and it was pretty easy. I haven't been able to identify through web search or StackOverflow search that anyone has tried this.
Thanks.

As ronbo4610 suggested, it is a good idea to use a custom data persister in this case, to store the array as a string in the database separated by some kind of delimiter. You can then search the string in your WHERE clause just as you would any other string. (For example, using the LIKE operator)
I have implemented such a data persister. In order to use it, you must add the following annotation above your String[] object in your persisted class:
#DatabaseField(persisterClass = ArrayPersister.class)
In addition, you must create a new class called "ArrayPersister" with the following code:
import com.j256.ormlite.field.FieldType;
import com.j256.ormlite.field.SqlType;
import com.j256.ormlite.field.types.StringType;
import org.apache.commons.lang3.StringUtils;
public class ArrayPersister extends StringType {
private static final String delimiter = ",";
private static final ArrayPersister singleTon = new ArrayPersister();
private ArrayPersister() {
super(SqlType.STRING, new Class<?>[]{ String[].class });
}
public static ArrayPersister getSingleton() {
return singleTon;
}
#Override
public Object javaToSqlArg(FieldType fieldType, Object javaObject) {
String[] array = (String[]) javaObject;
if (array == null) {
return null;
}
else {
return StringUtils.join(array, delimiter);
}
}
#Override
public Object sqlArgToJava(FieldType fieldType, Object sqlArg, int columnPos) {
String string = (String)sqlArg;
if (string == null) {
return null;
}
else {
return string.split(delimiter);
}
}
}

Unfortunately ORMLite does not support querying fields that are the type SERIALIZABLE. It is storing the array as a serialized byte[] so you cannot query against the values with an IN query like:
where.in(nameDb.NAME, "car");
ORMLite does support foreign collections but you have to set it up yourself with another class holding the names. See the documentation with sample code:
http://ormlite.com/docs/foreign-collection