I'm seeing a very weird problem with getting the count from a query not running on the main thread. I have a UI that uses the RecyclerView adapter for Realm, and it works just dandy.
I have a method that counts the number of records before a query so it can set a starting point, for some reason it will return the previous count was before the last transaction. Here is a somewhat shortened version with the output of the log:
D/SearchController: Query Count (Main Thread): 50
D/SearchController: Query Count (Rx Thread): 50
D/SearchController: Query Count (Main Thread): 100
D/SearchController: Query Count (Rx Thread): 50
public Single<Update> searchWithCriteriaForMore(SearchCriteria searchCriteria) {
Realm realmI = Realm.getDefaultInstance();
Timber.d("Query Count (Main Thread): %d", realmI.where(Ad.class).equalTo("searchId", searchCriteria.id()).count());
realmI.close();
return Single.defer(() -> {
Realm realm = Realm.getDefaultInstance();
Timber.d("Query Count (Rx Thread): %d", realm.where(Ad.class).equalTo("searchId", searchCriteria.id()).count());
realm.close();
// Stuff to add records on Rx Thread
});
}
The call looks like this:
SearchController.instance().searchWithCriteriaForMore(searchCriteria)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new SingleSubscriber<SearchController.Update>() {
...
}
Thoughts anyone? I've been trying to nail this down for a few weeks, thought it was something I was doing, but if I do the count on the main thread and pass it in, it works fine.
I'm running RxJava 1.2.7, RxAndroid 1.2.1, and Realm 3.3.1.
It feels that you have a Realm instance on RxThread somewhere which is not closed before. Since RxThread doesn't have a looper, the realm instance cannot be auto-updated. So it was locked at the version when it was created.
Realm is using ref counter internally, the getDefaultInstance() will just return the same instance you opened on the RxThread before.
The solution:
1. Find out which Realm instance was retrieved on the RxThread and close it properly. So the next time getDefaultInstance() will return a new Realm instance at the latest version data.
2. If it is intended to have a Realm instance on the RxThread, you can call realm.refresh() to manually move it to the latest data version.
Related
I'm trying to implement a simple chat application on web sockets in Clean Architecture. I had to choose a db for caching all information, so I decided to use Realm, because I heard it was pretty good database for any kind of mobile applications. But when I actually faced the Realm, it turned out to be really painful experience for me to implement caching logic with it.
All problems come from applying transaction to database which then must be synced on all threads working with Realm. There seems to some kind of synchronization problem with my code. For example, I want to save my object to Realm and then query it out of.
Here I have two simple functions to save and to get chat:
fun getBackgroundLooper(): Looper {
val handlerThread = HandlerThread("backgroundThread")
if (!handlerThread.isAlive)
handlerThread.start()
return handlerThread.looper
}
fun saveChat(chat: Chat): Completable {
val realmChat = ChatMapper.domainToCache(chat)
return Completable.create { e ->
val realm = Realm.getDefaultInstance()
realm.executeTransactionAsync({
it.insertOrUpdate(realmChat)
}, {
realm.close()
e.onComplete()
}, {
realm.close()
e.onError(it)
})
// Subscribe on background looper thread
// to be able to execute async transaction
}.subscribeOn(AndroidSchedulers.from(getBackgroundLooper()))
}
fun getSingleChat(chatId: String): Single<Chat> {
return Single.defer {
val realm = Realm.getDefaultInstance()
realm.isAutoRefresh = true
val realmChat = realm.where(RealmChat::class.java)
.equalTo("id", chatId).findFirstAsync()
if (realmChat.isValid) {
realmChat.load()
val chat = ChatMapper.cacheToDomain(realmChat)
realm.close()
Single.just(chat)
}
realm.close()
Single.error<Chat>(ChatNotExistException())
// Subscribe on background looper thread
// to be able to execute auto refreshing
}.subscribeOn(AndroidSchedulers.from(getBackgroundLooper()))
}
So, when I try to run simple code like this
remote.getChat().flatMap {
cache.saveChat(it) //save chat to realm
.andThen(cache.getSingleChat(it.id)) //then query it by id
}
I always get no matter of what ChatNotExistException, but if I try to run query again in another attempt or after restarting the application, then the chat object gets found
I also tried many different approaches to execute this code:
I tried to use realm.refresh() in getSingleChat or not use it at all.
I tried to query chat synchronously with findFirst() and findAll() instead of findFirstAsync().
I tried querying chat on current thread without .subscribeOn().
I tried to use realm.executeTransaction() instead of async transactions.
I tried to add thread sleep between saving and querying, so that transaction may take some time to get applied and I need to wait before attempting to query the chat
I'm begging anybody to explain me what am I doing wrong and how to make this code working. I can't change the architecture of my application and use Realm objects as my view models, I need to find solution in these conditions.
But when I actually faced the Realm, it turned out to be really painful experience for me to implement caching logic with it.
Reading the docs regarding best practices help. For example, the default idea is that you define a RealmResults using an async query on the UI thread, add a change listener to it, and observe the latest emission of the database.
There is no "caching" involved in that beyond saving to the database and observing the database. Any additional complexity is added by you and is completely optional.
All problems come from applying transaction to database which then must be synced on all threads working with Realm.
All looper threads automatically make the Realm auto-refresh, therefore if addChangeListener is used as intended in the docs, then there is no need for trickery, Realm will manage the synchronization between threads.
I want to save my object to Realm and then query it out of.
realm.executeTransactionAsync({
No reason to use executeTransactionAsync when you are already on a background thread.
try(Realm realm = Realm.getDefaultInstance()) {
realm.executeTransaction((r) -> {
// do write here
});
}
realm.where(RealmChat::class.java)
If you do import io.realm.kotlin.where, then you can do realm.where<RealmChat>().
.findFirstAsync()
No reason to use findFirstAsync() instead of findFirst() when you are already on a background thread. Also no reason to use load() when you're on a background thread, because you should be using findFirst() in the first place anyway.
You are also most likely missing a return#defer Single.just(chat) to actually return the chat if it's found. That is most likely what your original problem is.
With the handler thread things you're doing though, you might want to consider taking a look at this project called "Monarchy", as it intends to set up the ability to run queries on a background looper thread while still observing the results. It is labelled stagnant but the ideas are sound.
I have problems with realm my realm instances. I only use realm default instance. For each query/update operation, I get a realm instance with Realm.getDefaultInstance(), and I close it when I finish the operation.
My problem is that one of those queries is an observable query. I only close this realm instance when the subscription is finished. Meanwhile, other operations can close its realm instance. I know that realm instances are obtained from a cache, so I think that one of this operations are closing the same realm instance that I get when I create my observable query, because after some operations, my observable query stops emmiting items, and when I finish my subscription, I get an exception saying that my realm instance is already closed. Does it have sense? How can avoid that?
I think this is what is happening:
1. Start observable query -> Realm observable instance opened
2. Start database update 1 -> Realm query1 instance opened
3. Finish database update 1 -> Realm query1 instance closed
...
4. Start database update X -> Realm observable instace opened again <- PROBLEM
5. Finish database update X -> Realm observable instance closed
6. Finish observable query -> Realm observable instance closed again -> EXCEPTION
This is the code where I perform my observable query:
Realm realm = Realm.getDefaultInstance();
return realm.where(Message.class)
.findAll().asObservable()
.filter(RealmResults::isLoaded)
.map(realm::copyFromRealm)
.doOnUnsubscribe(() -> realm.close());
Is there any way to get a brand new realm instance which is not cached so that nobody can get the reference and close it?
Actually, Realm's Rx integration uses the RealmResults's RealmConfiguration to increase the Realm ref-count internally (inside the Observable), so you don't need to do such magic. You could do:
try(Realm realm = realm.getDefaultInstance()) {
return realm.where(Message.class)
.findAll()
.asObservable() // synchronous query is always loaded
.map(realm::copyFromRealm)
}
And it'd work. Although I don't know for sure if it helps solves your problem.
I am using Realm 3.0.0 as the DB of my Android app. It's like a questionnaire application, in which the user navigates inside the app a lot. When I use the app (go back and forth) continuously, I get the following error:
Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. mmap() failed: Out of memory size: 1073741824 offset: 0 in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 109
at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
at io.realm.internal.SharedRealm.(SharedRealm.java:187)
at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:229)
at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:204)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:124)
at io.realm.Realm.getDefaultInstance(Realm.java:210)
Now I know the main cause of this is not closing Realm instances. But I've already checked for that multiple times. And I am positive that I close every instance I open.
The app has many activities and fragments that all get a Realm instance on their onCreate and close it on their onDestroy. There are also other background network jobs that run to upload data that get Realm instances. These jobs close their Realm instances when they've finished running or when they cancel.
All of the above get their Realm instance thru injection via Dagger 2:
#Provides
#Nullable
static Realm realm(#Nullable RealmConfiguration configuration) {
if (configuration != null) {
Realm.setDefaultConfiguration(configuration);
return Realm.getDefaultInstance();
}
return null;
}
Configuration is also provided in the same Dagger Module.
To be more specific, a Questionnaire consists of many Question Fragments displayed in a ViewPager. Each Fragment gets injected with a realm. Many interactions in a given Question Fragment write data to the DB (some async, some blocking). These Fragments also query the database on onResume to get their updated Data. Some of this data is also copied out of Realm via realm.copyFromRealm(). Now at any given time of these happening, an upload job is most likely running and reading data from the DB and uploading it to a server. When an upload job finishes, it then writes to the DB.
I think I can have up to 7-12 fragment/activities holding a realm reference on the UI thread at a given moment. And 0-6 other references on 0-3 other threads (Background Jobs).
Moreover, I compact my realm DB via Realm.compactRealm(realmConfiguration) on every app launch (perhaps as a separate problem, this doesn't seem to do it's job consistently).
Above I've tried to describe my Realm usage descriptively without going into details. Now my problem is, when a user excessively uses the app (going back and forth between activities/fragments (realm injection + DB read query), uploading data (realm injection + DB read&write query)), I get the above posted Out of Memory Error.
I am also using Leak Canary, and it hasn't detected any leaks. (Not sure if it can anyway)
Am I using Realm in a way it's not supposed to be used? Should I close Realm instances onPause instead of onDestroy? Should I have only one realm instance in an activity and have all it's fragmetns (up to 5 in my case) use this instance? What kind of changes can I make in my app, and perhaps my app architecture to solve this problem?
I appreciate any help in trying to solve this problem.
EDIT: I'm sharing the realm open-close logic in my background threads.
All my jobs share the same realm usage, which is the following:
Realm is injected lazily via:
#Inject protected transient Lazy<Realm> lazyRealm;
The realm object reference is held at the private transient Realm realm; field. I am using Android Priority Job Queue. When the job is added:
#Override
public void onAdded() {
realm = lazyRealm.get();
realm.executeTransaction(realm1 -> {
//write some stuff into realm
});
realm.close();
}
And when the job is run realm is retreived once, and every possible ending of this method has a call to realm.close()
#Override public void onRun() throws Throwable {
synchronized (syncAdapterLock) {
realm = lazyRealm.get();
Answer answer = realm.where(Answer.class).equalTo(AnswerQuery.ID, answerId).findFirst();
if (answer == null) {
realm.close();
throw new RealmException("File not found");
}
final File photoFile = new File(answer.getFilePath());
final Response response = answerService.uploadPhotoAnswer(answerId, RequestBody.create(MediaType.parse("multipart/form-data"), photoFile)).execute();
if (!response.isSuccessful()) {
realm.close();
throw new HttpError(statusCode);
}
realm.executeTransaction(realm1 -> {
answer.setSyncStatus(SyncStatus.SYNCED.getCode());
});
}
realm.close();
}
}
As you can see, these background threads do close their realm instances properly as far as I'm concerned.
While it was true that all my background tasks did call realm.close(), one of them called it too late in it's lifecycle. That was my GPSService, which is a background service. The problem was that GPS service is initialized at the launch of the App as an Android Service, which is rarely destroyed. I was injecting a realm instance onCreate and closing it onDestroy. After the comments of #EpicPandaForce and reading his articles about using realm properly. I realized that this was the cause of the leak. A non-looper thread was keeping an open realm reference for an extremely long time, thus, the mmap was bloating every time a write transaction occures. Now that I moved the realm get/close to happen every time the service runs, my problem is fixed.
My take away is that one needs to treat background thread realm access very delicately. Thank you both for your quick responses and help!
I use Realm.getDefaultInstance() in a worker thread (in a service) and save my records with:
Realm realm = Realm.getDefaultInstance();
realm.executeTransaction(new Realm.Transaction() {
#Override
public void execute(Realm realm) {
...
realm.copyToRealmOrUpdate(...
When done (this is sync, so it is done right away), I submit an EventBus message to the UI thread that it can start the next task.
The UI thread calls to a different worker thread again which does another Realm.getDefaultInstance() and then a findAll() call. It finds nothing. Why is this? "Some time later" it will find the items fine, but not right away.
The items are in the database already: If I do the findAll() right after saving the items on the first worker thread, it does return the items. If I do it AFTER that again on the second worker thread it finds nothing. Is this some kind of caching?
The problem was based on me using a threadpool, on which threads the Realm instance was not updated. A solution for me is to not use the threadpool, so instead of
observable
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
I now use this everywhere:
observable
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread())
And it solves all Realm threading problems. It is a little less efficient probably, but fine for my purpose.
I am trying to figure out what is the best way to open/close Realm instances. We are using Realm in 3 places - App, Activities and Worker thread. However, it might be pretty cool to access data from Realm from any place. Thus, we wrapped all our Realm queries/writes in a Data Layer. Is it ok (from performance point of view) to do the following for every query (and similar on each write)?
try (Realm realm = getRealm()) {
return realm.copyFromRealm(realm.where(Task.class).isNull("name")
.findAllSorted("createdAt", Sort.DESCENDING));
}
How do you do something similar in async transaction? Are there better approaches? (We are using RxJava).
Thank you!
EDIT: After quick response from #christian-melchior we will use it like this:
try (Realm realm = getRealm()) {
return realm.copyFromRealm(realm.where(Task.class).isNull("name")
.findAllSortedAsync("createdAt", Sort.DESCENDING));
}
EDIT2: We actually settled on this:
Realm realm = getRealm();
return realm.where(Task.class).isNull("name")
.findAllSortedAsync("createdAt", Sort.DESCENDING)
.asObservable()
.filter(RealmResults::isLoaded)
.first()
.map(realmObjects -> {
if (realmObjects.isEmpty()) {
realm.close();
return null;
}
List<Task> res = realm.copyFromRealm(realmObjects);
realm.close();
return res;
});
Since we want our onComplete to be called after each query.
Using copyFromRealm will give you an in-memory copy of all the data, so as long as you are not loading huge lists, it should work fine. You will use more memory though, and there is a slight overhead in opening and closing Realms all the time.
One idea for controlling the Realm lifecycle can be seen here: https://realm.io/docs/java/latest/#controlling-the-lifecycle-of-realm-instances
Also note you in many cases can use our async API if you are concerned about loading data on the UI thread: https://realm.io/docs/java/latest/#asynchronous-queries