Develop Reference

Why Realm doesn't find object which I inserted in transaction before?

I'm trying to implement a simple chat application on web sockets in Clean Architecture. I had to choose a db for caching all information, so I decided to use Realm, because I heard it was pretty good database for any kind of mobile applications. But when I actually faced the Realm, it turned out to be really painful experience for me to implement caching logic with it.
All problems come from applying transaction to database which then must be synced on all threads working with Realm. There seems to some kind of synchronization problem with my code. For example, I want to save my object to Realm and then query it out of.
Here I have two simple functions to save and to get chat:
fun getBackgroundLooper(): Looper {
val handlerThread = HandlerThread("backgroundThread")
if (!handlerThread.isAlive)
handlerThread.start()
return handlerThread.looper
}
fun saveChat(chat: Chat): Completable {
val realmChat = ChatMapper.domainToCache(chat)
return Completable.create { e ->
val realm = Realm.getDefaultInstance()
realm.executeTransactionAsync({
it.insertOrUpdate(realmChat)
}, {
realm.close()
e.onComplete()
}, {
realm.close()
e.onError(it)
})
// Subscribe on background looper thread
// to be able to execute async transaction
}.subscribeOn(AndroidSchedulers.from(getBackgroundLooper()))
}
fun getSingleChat(chatId: String): Single<Chat> {
return Single.defer {
val realm = Realm.getDefaultInstance()
realm.isAutoRefresh = true
val realmChat = realm.where(RealmChat::class.java)
.equalTo("id", chatId).findFirstAsync()
if (realmChat.isValid) {
realmChat.load()
val chat = ChatMapper.cacheToDomain(realmChat)
realm.close()
Single.just(chat)
}
realm.close()
Single.error<Chat>(ChatNotExistException())
// Subscribe on background looper thread
// to be able to execute auto refreshing
}.subscribeOn(AndroidSchedulers.from(getBackgroundLooper()))
}
So, when I try to run simple code like this
remote.getChat().flatMap {
cache.saveChat(it) //save chat to realm
.andThen(cache.getSingleChat(it.id)) //then query it by id
}
I always get no matter of what ChatNotExistException, but if I try to run query again in another attempt or after restarting the application, then the chat object gets found
I also tried many different approaches to execute this code:
I tried to use realm.refresh() in getSingleChat or not use it at all.
I tried to query chat synchronously with findFirst() and findAll() instead of findFirstAsync().
I tried querying chat on current thread without .subscribeOn().
I tried to use realm.executeTransaction() instead of async transactions.
I tried to add thread sleep between saving and querying, so that transaction may take some time to get applied and I need to wait before attempting to query the chat
I'm begging anybody to explain me what am I doing wrong and how to make this code working. I can't change the architecture of my application and use Realm objects as my view models, I need to find solution in these conditions.

But when I actually faced the Realm, it turned out to be really painful experience for me to implement caching logic with it.
Reading the docs regarding best practices help. For example, the default idea is that you define a RealmResults using an async query on the UI thread, add a change listener to it, and observe the latest emission of the database.
There is no "caching" involved in that beyond saving to the database and observing the database. Any additional complexity is added by you and is completely optional.
All problems come from applying transaction to database which then must be synced on all threads working with Realm.
All looper threads automatically make the Realm auto-refresh, therefore if addChangeListener is used as intended in the docs, then there is no need for trickery, Realm will manage the synchronization between threads.
I want to save my object to Realm and then query it out of.
realm.executeTransactionAsync({
No reason to use executeTransactionAsync when you are already on a background thread.
try(Realm realm = Realm.getDefaultInstance()) {
realm.executeTransaction((r) -> {
// do write here
});
}
realm.where(RealmChat::class.java)
If you do import io.realm.kotlin.where, then you can do realm.where<RealmChat>().
.findFirstAsync()
No reason to use findFirstAsync() instead of findFirst() when you are already on a background thread. Also no reason to use load() when you're on a background thread, because you should be using findFirst() in the first place anyway.
You are also most likely missing a return#defer Single.just(chat) to actually return the chat if it's found. That is most likely what your original problem is.
With the handler thread things you're doing though, you might want to consider taking a look at this project called "Monarchy", as it intends to set up the ability to run queries on a background looper thread while still observing the results. It is labelled stagnant but the ideas are sound.

Secondary .realm file getInstance taking too long

By secondary .realm file I mean a realm file which is not the default.realm file.
I have two .realm files - one being the standard default.realm and the other being say aux.realm.
Things work as they should under normal circumstances, but when I perform a heavy operation (multiple tables undergo .deleteAllFromRealm() and re-sync everything) while this happens on a worker thread, the user is still free to perform any UI activities, whenever any interaction is performed involving the aux.realm instance, the app shuts with an ANR.
With some extensive debugging I found that the getAuxRealmInstance takes a lot of time to pass the instance, even though the value for it should be cached. This is in spite of the fact that its configuration already loaded lazily. Hence, it is unclear as to why it takes so much time?
I also though it might be an issue of transactions as there can be only one active transaction at a time, but what i'm not sure is that is the rule valid also through files, like can two realm files have their own transactions running in parallel?
My aux.realm file:
private const val FILE_NAME = "auxiliary.realm"
private val auxiliaryConfiguration = lazy {
RealmConfiguration.Builder()
.name(FILE_NAME)
.schemaVersion(AuxiliarySchemaVersionMappings.CURRENT_SCHEMA_VERSION)
.modules(AuxiliaryRealmModule())
.initialData {
Log.d("AuxRealm", "running initial data migration: ")
// initial version..
// migrate the AppMetaData table from base realm to aux realm
}
}
.migration(AuxiliaryRealmMigration())
.build().also { Log.d("AuxRealm", "configuration created: ") }
}
fun getAuxiliaryRealmInstance(): Realm{
return Realm.getInstance(auxiliaryConfiguration.value)
}
fun getAuxiliaryRealmInstanceAsync(callback: Realm.Callback): RealmAsyncTask{
return Realm.getInstanceAsync(auxiliaryConfiguration.value, callback)
}
PS: The ANR goes away if I load the aux realm instance in async, which as mentioned above, points to the same problem.
Env variables: Realm: 5.4.2, Kotlin 1.2.51

I solved this problem by caching the realm instance:
private Realm auxiliaryRealmInstance;
fun getAuxiliaryRealmInstance(): Realm{
return auxiliaryRealmInstance == null ? Realm.getInstance(auxiliaryConfiguration.value) : auxiliaryRealmInstance;
}
This workaround should not be necessary because here it's written that caching will not make anything more efficient. But I did not notice any disadvantages so far.

Realm: working with Clean-Architecture and RxJava2

A bit of context, I’ve tried to apply some clean-architecture to one of my projects and I’m having trouble with the (Realm) disk implementation of my repository. I have a Repository which pulls some data from different DataStores depending on some conditions (cache). This is the theory, the problem comes when mixing all of this with UseCases and RxJava2.
First I get the list of objects from Realm and then I manually create an Observable of it. But the subscribe (as expected) is executed on a different thread so realm ends up crashing… (second block of code)
This is the code I use to create the Observables (from an abstract class DiskStoreBase):
Observable<List<T>> createListFrom(final List<T> list) {
return Observable.create(new ObservableOnSubscribe<List<T>>() {
#Override
public void subscribe(ObservableEmitter<List<T>> emitter) throws Exception {
if (list != null) {
emitter.onNext(list);
emitter.onComplete();
} else {
emitter.onError(new ExceptionCacheNotFound());
}
}
});
}
How can I deal with this scenario?
More code of DiskStoreForZone:
#Override
public Observable<List<ResponseZone>> entityList() {
Realm realm = Realm.getDefaultInstance();
List<ResponseZone> result = realm.where(ResponseZone.class).findAll();
return createListFrom(result);
}
The exact crash:
E/REALM_JNI: jni: ThrowingException 8, Realm accessed from incorrect thread.
E/REALM_JNI: Exception has been thrown: Realm accessed from incorrect thread.

It doesn't work because despite using Rx, your data layer is not reactive.
Realm by its nature is a reactive datasource, and its managed objects by nature are also mutable (updated in place by Realm), and thread-confined (can only be accessed on the same thread where the Realm was opened).
For your code to work, you'd need to copy out the data from the Realm.
#Override
public Single<List<ResponseZone>> entityList() {
return Single.fromCallable(() -> {
try(Realm realm = Realm.getDefaultInstance()) {
return realm.copyFromRealm(realm.where(ResponseZone.class).findAll());
}
});
}
I took the liberty and represented your Single as a Single, considering it's not an Observable, it does not listen for changes, there is only 1 event and that is the list itself. So sending it through an ObservableEmitter doesn't really make sense as it does not emit events.
Therefore, this is why I said: your data layer is not reactive. You are not listening for changes. You are just obtaining data directly, and you are never notified of any change; despite using Rx.
I drew some pictures in paint to illustrate my point. (blue means side-effects)
in your case, you call a one-off operation to retrieve the data from multiple data-sources (cache, local, remote). Once you obtain it, you don't listen for changes; technically if you edit the data in one place and another place, the only way to update is by "forcing the cache to retrieve the new data manually"; for which you must know that you modified the data somewhere else. For which you need a way to either directly call a callback, or send a message/event - a notification for change.
So in a way, you must create a cache invalidation notification event. And if you listen to that, the solution could be reactive again. Except you're doing this manually.
----------------------------------------------------------------------
Considering Realm is already a reactive data source (similarly to SQLBrite for SQLite), it is able to provide change notifications by which you can "invalidate your cache".
In fact, if your local data source is the only source of data, and any write from network is a change that you listen to, then your "cache" can be written down as replay(1).publish().refCount() (replay latest data for new subscribers, replace data with new if new data is evaluated) which is RxReplayingShare.
Using a Scheduler created from the looper of a handler thread, you can listen to changes in the Realm on a background thread, creating a reactive data source that returns up-to-date unmanaged copies that you can pass between threads (although mapping directly to immutable domain models is preferred to copyFromRealm() if you choose this route - the route being clean architecture).
return io.reactivex.Observable.create(new ObservableOnSubscribe<List<ResponseZone>>() {
#Override
public void subscribe(ObservableEmitter<List<ResponseZone>> emitter)
throws Exception {
final Realm observableRealm = Realm.getDefaultInstance();
final RealmResults<ResponseZone> results = observableRealm.where(ResponseZone.class).findAllAsync();
final RealmChangeListener<RealmResults<ResponseZone>> listener = results -> {
if(!emitter.isDisposed()) {
if(results.isValid() && results.isLoaded()) {
emitter.onNext(observableRealm.copyFromRealm(results));
}
}
};
emitter.setDisposable(Disposables.fromRunnable(() -> {
if(results.isValid()) {
results.removeChangeListener(listener);
}
observableRealm.close();
}));
results.addChangeListener(listener);
// initial value will be handled by async query
}
}).subscribeOn(looperScheduler).unsubscribeOn(looperScheduler);
Where looper scheduler is obtained as
handlerThread = new HandlerThread("LOOPER_SCHEDULER");
handlerThread.start();
synchronized(handlerThread) {
looperScheduler = AndroidSchedulers.from(handlerThread.getLooper());
}
And that is how you create reactive clean architecture using Realm.
ADDED:
The LooperScheduler is only needed if you intend to actually enforce Clean Architecture on Realm. This is because Realm by default encourages you to use your data objects as domain models and as a benefit provides lazy-loaded thread-local views that mutate in place when updated; but Clean Architecture says you should use immutable domain models instead (independent from your data layer). So if you want to create reactive clean architecture where you copy from Realm on a background thread any time when Realm changes, then you'll need a looper scheduler (or observe on a background thread, but do the copying from a refreshed Realm on Schedulers.io()).
With Realm, generally you'd want to use RealmObjects as your domain models, and rely on lazy-evaluation. In that case, you do not use copyFromRealm() and you don't map the RealmResults to something else; but you can expose it as a Flowable or a LiveData.
You can read related stuff about this here.

Out of memory error when reading Realm database after restarting the app

I am currently using Realm database on my Android application to retrieve data according to user defined filters and to synchronize with a remote database. After making several requests the app works pretty fine (it consumes about 30 MB of memory, what is ok, taking into account I am using other components and services like maps, etc.). However, when closing the app (swipe on recent apps list) and restarting it, after two or three queries the memory consumption starts increasing considerably (above 100 MB), so it takes too long to make a query and finally the app crashes because of an OutOfMemoryError. Tracking the application instances on Android Studio I see a lot of instances of io.realm.internal.NativeObjectReference.
According to realm documentation I have tried two things, without success:
Getting a Realm instance with Realm.getDefaultInstance() in a class constructor, which is responsible to query the Realm database, and closing the database with mRealm.close()from onDestroy method on main activity.
Getting a Realm instance with Realm.getDefaultInstance() right before starting a database operation, and closing it right after the operation ends. With asynchronous operations I make something like this
mRealm = Realm.getDefaultInstance();
mRealm.executeTransactionAsync(new Realm.Transaction() {
#Override
public void execute(Realm realm) {
//perform transaction
}
}, new Realm.Transaction.OnSuccess() {
#Override
public void onSuccess() {
mRealm.close();
//notify calling function
}
}, new Realm.Transaction.OnError() {
#Override
public void onError(Throwable error) {
mRealm.close();
}
});
with non-asynchronous operations I make something like this
mRealm = Realm.getDefaultInstance();
RealmResults<DealLocal> dealLocals = mRealm.where(DealLocal.class).findAll();
mRealm.close();
I use Realm.getDefaultInstance() in two classes: one to synchronize with the remote database, and the other one to make the user defined queries. However, at this moment all the calls are made from the main activity, and I guarantee only one operation on the database is made at the same time (i.e. I synchronize first and then make the user defined queries). When a query takes too long because of an excessive memory usage and I kill the app (swipe on recent apps list), the memory usage does not go back to normal.
How would be the correct way to handle the realm instances and operations on my application in order to avoid this problem?

Is it ok to close Realm instance after each query/write?

I am trying to figure out what is the best way to open/close Realm instances. We are using Realm in 3 places - App, Activities and Worker thread. However, it might be pretty cool to access data from Realm from any place. Thus, we wrapped all our Realm queries/writes in a Data Layer. Is it ok (from performance point of view) to do the following for every query (and similar on each write)?
try (Realm realm = getRealm()) {
return realm.copyFromRealm(realm.where(Task.class).isNull("name")
.findAllSorted("createdAt", Sort.DESCENDING));
}
How do you do something similar in async transaction? Are there better approaches? (We are using RxJava).
Thank you!
EDIT: After quick response from #christian-melchior we will use it like this:
try (Realm realm = getRealm()) {
return realm.copyFromRealm(realm.where(Task.class).isNull("name")
.findAllSortedAsync("createdAt", Sort.DESCENDING));
}
EDIT2: We actually settled on this:
Realm realm = getRealm();
return realm.where(Task.class).isNull("name")
.findAllSortedAsync("createdAt", Sort.DESCENDING)
.asObservable()
.filter(RealmResults::isLoaded)
.first()
.map(realmObjects -> {
if (realmObjects.isEmpty()) {
realm.close();
return null;
}
List<Task> res = realm.copyFromRealm(realmObjects);
realm.close();
return res;
});
Since we want our onComplete to be called after each query.

Using copyFromRealm will give you an in-memory copy of all the data, so as long as you are not loading huge lists, it should work fine. You will use more memory though, and there is a slight overhead in opening and closing Realms all the time.
One idea for controlling the Realm lifecycle can be seen here: https://realm.io/docs/java/latest/#controlling-the-lifecycle-of-realm-instances
Also note you in many cases can use our async API if you are concerned about loading data on the UI thread: https://realm.io/docs/java/latest/#asynchronous-queries