In my Android project, ORMLite is functioning as a cache. I'm downloading data from a web server and placing it in the database. I'm calling createOrUpdate on my objects, but duplicates are appearing in the database. The database entries are identical except for the primary key (which is simply an auto incremented integer). I think that since my second object doesn't yet have a primary key, ORMLite considers the two as being different, even though every other field is identical.
Does anyone know if this is true?
You should not be calling createOrUpdate unless your object already has an id field set. The way ORMLite determines whether or not it exists in the database is to do a query-by-id on it. The code does:
ID id = extractId(data);
// assume we need to create it if there is no id <<<<<<<<<<<<<<<<<<<<<<<
if (id == null || !idExists(id)) {
int numRows = create(data);
return new CreateOrUpdateStatus(true, false, numRows);
} else {
int numRows = update(data);
return new CreateOrUpdateStatus(false, true, numRows);
}
I'll expand the javadocs to explain this better. They are very weak there. Sorry. I've updated them to be:
This is a convenience method for creating an item in the database if it does not exist. The id is extracted from the data argument and a query-by-id is made on the database. If a row in the database with the same id exists then all of the columns in the database will be updated from the fields in the data parameter. If the id is null (or 0 or some other default value) or doesn't exist in the database then the object will be created in the database. This also means that your data item must have an id field defined.
It seems like you manually need to extract the id of the already existing entry and then update the database, based on your actual unique identifier. Example with unique URLs:
CacheItem newEntry = fetchEntry("..."); // fetch it here
List<CacheItem> existing = mDao.queryForEq("url", newEntry.getUrl());
if (existing.size() > 0) {
int id = existing.get(0).getId();
newEntry.setId(id);
mDao.update(newEntry);
} else mDao.create(newEntry);
set your primary key for it to work normally.
#DatabaseField(columnName = "id",uniqueIndex = true)
private int id;
Worked for me. Exceptions are thrown at background but it works :D
Related
I want to insert list of objects into my SQLite db. If I do query for each object to check if its present, it takes alot of time for long list (20-30sec).
I used function 'insertWithOnConflict()' to do that faster.
I thought that it will try to insert a row and if record with same value of column entered to function already exists, it will just replace its values (like update) otherwise it will insert new row.
But I checked my DB and it is creating duplicated objects. Any solution for this?
Important information 'COLUMN_OBJECT_ID' is not 'PRIMARY_KEY'. 'PRIMARY_KEY' is autoincrement. But 'COLUMN_OBJECT_ID' has to be unique in this case. So I want to update row with same 'COLUMN_OBJECT_ID' because data inside can change.
Code:
fun saveObjectsToCache(objects: List<Item>) {
val db = getDb()
db.transaction {
objects.forEach {item->
val cv = ContentValues()
cv.apply {
put(COLUMN_OBJECT_ID, item.id)
put(COLUMN_OBJECT_DATA, item.js.toString())
}
insertWithOnConflict(TABLE_OBJECT_CACHE, COLUMN_OBJECT_ID, cv, SQLiteDatabase.CONFLICT_REPLACE)
}
}
}
Looks like you forgot to add a unique constraint on COLUMN_OBJECT_ID.
Because of this, there is no conflict when you add a row with the same COLUMN_OBJECT_ID.
Make COLUMN_OBJECT_ID unique and it should work.
Additionally, if COLUMN_OBJECT_ID is unique, and is what you use for checking existing values in the table, you should probably make it the primary key, instead of the auto increment one you are using right now as the primary key.
It will be more efficient both in memory usage, and in performance
Conclusion: Android's database APIs work but the documentation is horribly incomplete.
I have recently run into a brain wrecking situation due to the flexibility Sqlite provides by not forcing you to specify the data type when creating the table. The problem was my mindset that assumed that every data type would be a general character sequence if not specified and therefore the way to talk to database is through java.lang.String.
But you can't blame me either when you see methods like the below:
int delete (String table,
String whereClause,
String[] whereArgs)
in the SqlDatabase class from Android docs.
I have a table consisting of Phone No(that I stored as java.lang.String) and Timestamp as a long field. When I tried deleting a record using this method, it just never got deleted despite countless debugging.
I checked everything and query was alright and table is existent and all the checklist until by chance, I discovered that removing the '' around the timestamp while querying in a raw manner instead of using the above method yields a successful deletion, something like this:
DELETE FROM messages_records_table WHERE messageTimestamp = 1508494606000;
instead of the following:
DELETE FROM messages_records_table WHERE messageTimestamp = '1508494606000';
or,
DELETE FROM messages_records_table WHERE messageTimestamp = "1508494606000";
Phone No isn't a problem; it's the timestamp that was creating the problem in INSERTION/DELETION
So, I tried running a raw deletion query with quotes removed(that are required with a string/varchar type) and it yielded successful deletion. I used the following method for this:
db.execSQL(String sql, Object[] whereArgs)
The key thing to notice here is that Object[] is different from String[] when compared to delete(). I passed a Long to Object to make it work but passing a Long.toString() in delete() seems to be useless.
So my question is, Is my analysis correct and delete() API is basically useless or have I missed some bigger picture..after all, it's provided by Android team carefully?
SQLite supports multiple data types; and while column types are not strictly enforced, values might be automatically converted in some cases (this is called affinity).
When your values are stored as numbers, you should access them as numbers, not as strings.
The Android database API does not allow you to use parameter types other than strings in most functions. This is a horrible design bug.
To search for a number, either use execSQL(), which allows you to use number parameters, or convert the string value back into a number:
db.delete(..., "timestamp = CAST(? AS NUMBER)",
new String[]{ String.valueOf(ts) });
The problem was my mindset that assumed that every data type would be
a general character sequence if not specified and therefore the way to
talk to database is through java.lang.String.
I think that's the real issue.
If you specify no type e.g.
CREATE TABLE mytable (col1,col2,col3)
Then according to Determination of Column Affinity(3.1) rule 3:-
3) If the declared type for a column contains the string "BLOB" or if no
type is specified then the column has affinity BLOB.
And then according to Section 3
A column with affinity BLOB does not prefer one storage class over
another and no attempt is made to coerce data from one storage class
into another.
I've personally never had an issue with delete. However I do have a tendency to always delete according to rowid.
Here's a working example usage that shows that delete isn't useless and is deleting according to a long. However the columns are all of type INTEGER :-
int pudeletes;
int sldeletes;
int rdeletes;
int pdeletes;
if(doesProductExist(productid)) {
// if not in a transaction then begin a transaction
if(!intransaction) {
db.beginTransaction();
}
String whereargs[] = { Long.toString(productid)};
// Delete ProductUsage rows that use this product
pudeletes = db.delete(
DBProductusageTableConstants.PRODUCTUSAGE_TABLE,
DBProductusageTableConstants.PRODUCTUSAGE_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete ShopList rows that use this product
sldeletes = db.delete(
DBShopListTableConstants.SHOPLIST_TABLE,
DBShopListTableConstants.SHOPLIST_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete Rules rows that use this product
rdeletes = db.delete(
DBRulesTableConstants.RULES_TABLE,
DBRulesTableConstants.RULES_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete the Product
pdeletes = db.delete(
DBProductsTableConstants.PRODUCTS_TABLE,
DBProductsTableConstants.PRODUCTS_ID_COL +
" = ?",
whereargs
);
// if originally not in a transaction then as one was started
// complete and end the transaction
if(!intransaction) {
db.setTransactionSuccessful();
db.endTransaction();
}
}
I'm new to app development using SQLite and Sugar ORM on Android, and have tried to read through the Sugar ORM documentation, but didn't find anything for how to update a saved object in SQLite. Can I still save the object after changing its properties? something like:
Customer myCustomer = (Customer.find(Customer.class, "id = ?", id)).get(0);
myCustomer.setName("new name");
myCustomer.setAddress("new Address");
myCustomer.save(); // is this okay for updating the object?
the save() method won't create another new object while leaving the old entry untouched, right?
Your code should update the row without issue.
From the docs - Update Entity:
Book book = Book.findById(Book.class, 1);
book.title = "updated title here"; // modify the values
book.edition = "3rd edition";
book.save(); // updates the previous entry with new values.
It will update your entity. The Sugar ORM overwriting your existing e.g Name and updated it with "new name" after the save() method call.
Updating a saved object is pretty straightforward.
Retrieve the object;
Object object= Object.findById(Object.class, ID);
Set the attributes you need to;
object.setAttr("new value");
Then finally call save;
object.save();
Alternatively, as someone mentioned above one can choose to use update() which works slightly differently and would ideally be used when changing several attributes;
First create the object and set the necessary attributes;
Object object= new Object();
object.setAttr("some data");
Then set an ID for the Object that ideally already exists in the database in order to target that item for replacement;
object.setID(ID);
And finally;
object.update();
Save and Object methods are completely different and both are really useful.
If you have an object and you say:
Object.save();
That will override all of the other fields as well for example:
column1 column2
1 1
if in your object you have only set column1 corresponding field a number like 2 you will get:
Object.save();
column1 column2
2 NULL
Object.update();
column1 column2
2 1
You don't need to use .setId() explicitly to get update working it looks for a unique item if it's found it will update that,if not it will create a new row.By default an auto increment ID column is added to each of your tables and used as unique ids for update.If you need your own fields to be unique use:
#Unique
String MyID
or for multiple of the same thing you can use:
#MultiUnique("MyFirstID,MySecondID")
public class MyClass extends SugarRecord {...
which both are name of your fields in the table.
I am currently trying to select entries using a query containing a where condition on a (byte array) property. This property/column contains a serialized UUID. Unfortunately I currently cannot change the data type of this column, as the database is created and synced by a separate module which only works properly with the current implementation. As greenDao cannot handle byte arrays as primary keys properly I am trying to work around this issue somehow. Creating my own selection queries etc. would be a solution
The property is defined during greenDAO generation as:
Entity randomEntity = schema.addEntity("RandomEntity");
...
randomEntity.addByteArrayProperty("RandomProperty");
The query is built using following line:
Query query = this.mRandomEntityDao.queryBuilder().where(RandomEntityDao.Properties.RandomProperty.eq(randomByteArray)).build();
Unfortunately I get following error with this operation:
de.greenrobot.dao.DaoException: Illegal value: found array, but simple object required
at de.greenrobot.dao.query.WhereCondition$PropertyCondition.checkValueForType(WhereCondition.java:75)
...
Is this where condition is simply not supported by greenDAO or am I missing something crucial? Unfortunately I cannot use another datatype for this certain property.
Edit:
My current workaround goes as follows:
As greenDao can handle primary keys which are strings (up to a certain point) and UUIDs can also be represented by this data type I altered the existing tables and added following columns:
db.execSQL("ALTER TABLE 'RANDOMTABLE' ADD COLUMN '_GREENID' TEXT;");
The sync module is ignoring this column so there shouldn’t be any issues with that. Then I created a trigger mapping the serialized UUID in the ID column to the new _GREENID column:
db.execSQL("CREATE TRIGGER randomtableGreen AFTER INSERT ON 'RANDOMTABLE' BEGIN " +
"UPDATE 'RANDOMTABLE' SET '_GREENID' = HEX(NEW.ID) WHERE 'RANDOMTABLE'.ID = NEW.ID; " +
"END;");
Finally I run an update on the table in case they already contained some entries prior to the creation of the trigger:
db.execSQL("UPDATE 'RANDOMTABLE' SET '_GREENID' = HEX(ID) WHERE '_GREENID' <> '';");
About byte[] aka BLOB in greendao:
Looking at de.greenrobot.dao.query.WhereCondition.PropertyCondition.checkValueForType conditions for byte[] are not supported at the moment, because the following lines will always throw an exception if value is of type byte[].
if (value != null && value.getClass().isArray()) {
throw new DaoException("Illegal value: found array, but simple object required");
}
Solution 1 - modify and contribute to greendao:
You could modify the uper lines, so that the exception is only thrown if the type of value and the type of the Property don't fit.
if (value != null) {
if (value.getClass().isArray() && !property.type.isArray()) {
throw new DaoException("Illegal value: found array, but " +
"simple object required");
}
if (!value.getClass().isArray() && property.type.isArray()) {
throw new DaoException("Illegal value: found simple object, " +
"but array required");
}
}
Maybe this will already solve the problem, but there may be other parts in greendao stopping to work with this edit or that will break the query. For example the binding of parameters to queries may not work with arrays.
Solutinon 2 - Use queryRaw(String where, String... selectionArg)
This is pretty straight forward and shouldn't be a problem with some knowledge about SQLite.
Solution 3 - Using a lookup-table
Assume the original table:
ORIG
-------------------------------
UUID BLOB
...
You can modify ORIG and add a autoincrement-primarykey:
db.execSQL("ALTER TABLE 'ORIG' " +
"ADD COLUMN 'REF_ID' INT PRIMARYKEY AUTOINCREMENT;");
The sync service should already take care about the uniqueness of ORIG.UUID and ignore the new ORIG.REF_ID-column. For inserting new UUIDs the sync service will probably use INSERT causing a new autoincremented value in ORIG.REF_ID.
For updating an existing UUID the sync service will probably use UPDATE ... WHERE UUID=? and no new ORIG.REF_ID-value will be created, but the old value will be kept.
Summarized the ORIG-table has a new Bijection between column REF_ID and column UUID.
Now you can create another table:
ORIG_IDX
------------------------------
UUID TEXT PRIMARYKEY
REF_ID INT UNIQUE
(If your data is smaller than 8 bytes it will also fit into a INT instead TEXT, but I don't know if there is a built-in cast/coversion from BLOB to INT.)
ORIG.IDX.UUID will be the String-representation of ORIG.UUID.
ORIG_IDX.REF_ID is foreign key for ORIG.REF_ID.
ORIG_IDX is filled and updated by triggers:
db.execSQL("CREATE TRIGGER T_ORIG_AI AFTER INSERT ON 'ORIG' BEGIN " +
"INSERT 'ORIG_IDX' SET 'REF_ID' = NEW.REF_ID, 'UUID' = NEW.UUID" +
"END;");
Create corresponding triggers for UPDATE and DELETE.
You can create the tables ORIG and ORIG_IDX using greendao and then query a requested uuid with:
public Orig getOrig(String uuid) {
OrigIdx origIdx = OrigIdxDao.queryBuilder().where(
QrigIdxDao.Properties.UUID.eq(uuid)).unique();
if (origIdx != null) {
return origIdx.getOrig();
}
return null;
}
I think String-primarykey is not supported yet, so dao.load(uuid) won't be available.
CONCERING AN EXTENDED TABLE:
You could use a string primarykey-column and provide conversion-methods in the keep-sections of your entity. You will have to compute the primarykey-column before you do an insert.
If there are other tools inserting data (for example your sync service) you'd have to use a trigger to compute your primary-key before the insert happens. This doesn't seem possible using SQLite. Thus the primarykey-constraint will fail on inserts by the sync service, so this solution will not work with primarykey!
I have a Language entity with a ForeignCollection of Country entities:
#ForeignCollectionField(eager = false, foreignFieldName = "language")
public ForeignCollection<Country> countries;
I create a country, say representing France, in my database. Later, I retrieve it by querying for its id and I want to add this to a Language's foreign collection of countries:
Country country = dbHelper.getCountryDao().queryForId(countryId);
if (country != null) {
country.language = language;
ForeignCollection<Country> countries = language.countries;
if (countries == null) {
countries = dbHelper.getLanguageDao().getEmptyForeignCollection("countries");
}
countries.add(country); // Illegal state exception thrown here
language.countries = countries;
dbHelper.getLanguageDao().create(language);
}
However, the marked line countries.add(country) causes an illegal state exception to be thrown:
01-27 13:39:19.112: E/AndroidRuntime(15614): Caused by:
java.sql.SQLException: inserting to database failed: INSERT INTO
countries (identifier ,name ,language_id ) VALUES (?,?,?)
01-27 13:39:19.112: E/AndroidRuntime(15614): at
com.j256.ormlite.misc.SqlExceptionUtil.create(SqlExceptionUtil.java:22)
Why does .add() trigger the recreation of an entity existing in the database and the internal DAO? And what should I be doing instead?
Why does .add() trigger the recreation of an entity existing in the database and the internal DAO?
It does this because that is how it was designed. To quote the javadocs for EagerForeignCollection#add():
Add an element to the collection. This will also add the item to the associated database table.
The country already exists in the table so an exception is thrown. You can associate an existing country with a language by doing something like the following:
country.language = language;
dbHelper.getCountryDao().update(country);
Then if you want to refresh your eager collection in the language you do:
dbHelper.getLanguageDao().refresh(language);
Right now I get an SQL Exception, if I add an already created element to the collection, because of the dao.create() in the add() of the collection realization. But for a quite some time we got a dao.createOrUpdate() method, which is better for this situation because it would do an update or simply nothing if we have an already created entity in collection. Is it possible to change this in the source code?
By the way there is an already done and closed ticket (#31) with changes that should automatically set the parent field value on the collection element. But even in the 4.45 I still have to do this manually. Did I miss something or probably do something wrong?