I have problems with realm my realm instances. I only use realm default instance. For each query/update operation, I get a realm instance with Realm.getDefaultInstance(), and I close it when I finish the operation.
My problem is that one of those queries is an observable query. I only close this realm instance when the subscription is finished. Meanwhile, other operations can close its realm instance. I know that realm instances are obtained from a cache, so I think that one of this operations are closing the same realm instance that I get when I create my observable query, because after some operations, my observable query stops emmiting items, and when I finish my subscription, I get an exception saying that my realm instance is already closed. Does it have sense? How can avoid that?
I think this is what is happening:
1. Start observable query -> Realm observable instance opened
2. Start database update 1 -> Realm query1 instance opened
3. Finish database update 1 -> Realm query1 instance closed
...
4. Start database update X -> Realm observable instace opened again <- PROBLEM
5. Finish database update X -> Realm observable instance closed
6. Finish observable query -> Realm observable instance closed again -> EXCEPTION
This is the code where I perform my observable query:
Realm realm = Realm.getDefaultInstance();
return realm.where(Message.class)
.findAll().asObservable()
.filter(RealmResults::isLoaded)
.map(realm::copyFromRealm)
.doOnUnsubscribe(() -> realm.close());
Is there any way to get a brand new realm instance which is not cached so that nobody can get the reference and close it?
Actually, Realm's Rx integration uses the RealmResults's RealmConfiguration to increase the Realm ref-count internally (inside the Observable), so you don't need to do such magic. You could do:
try(Realm realm = realm.getDefaultInstance()) {
return realm.where(Message.class)
.findAll()
.asObservable() // synchronous query is always loaded
.map(realm::copyFromRealm)
}
And it'd work. Although I don't know for sure if it helps solves your problem.
Related
I'm trying to implement a simple chat application on web sockets in Clean Architecture. I had to choose a db for caching all information, so I decided to use Realm, because I heard it was pretty good database for any kind of mobile applications. But when I actually faced the Realm, it turned out to be really painful experience for me to implement caching logic with it.
All problems come from applying transaction to database which then must be synced on all threads working with Realm. There seems to some kind of synchronization problem with my code. For example, I want to save my object to Realm and then query it out of.
Here I have two simple functions to save and to get chat:
fun getBackgroundLooper(): Looper {
val handlerThread = HandlerThread("backgroundThread")
if (!handlerThread.isAlive)
handlerThread.start()
return handlerThread.looper
}
fun saveChat(chat: Chat): Completable {
val realmChat = ChatMapper.domainToCache(chat)
return Completable.create { e ->
val realm = Realm.getDefaultInstance()
realm.executeTransactionAsync({
it.insertOrUpdate(realmChat)
}, {
realm.close()
e.onComplete()
}, {
realm.close()
e.onError(it)
})
// Subscribe on background looper thread
// to be able to execute async transaction
}.subscribeOn(AndroidSchedulers.from(getBackgroundLooper()))
}
fun getSingleChat(chatId: String): Single<Chat> {
return Single.defer {
val realm = Realm.getDefaultInstance()
realm.isAutoRefresh = true
val realmChat = realm.where(RealmChat::class.java)
.equalTo("id", chatId).findFirstAsync()
if (realmChat.isValid) {
realmChat.load()
val chat = ChatMapper.cacheToDomain(realmChat)
realm.close()
Single.just(chat)
}
realm.close()
Single.error<Chat>(ChatNotExistException())
// Subscribe on background looper thread
// to be able to execute auto refreshing
}.subscribeOn(AndroidSchedulers.from(getBackgroundLooper()))
}
So, when I try to run simple code like this
remote.getChat().flatMap {
cache.saveChat(it) //save chat to realm
.andThen(cache.getSingleChat(it.id)) //then query it by id
}
I always get no matter of what ChatNotExistException, but if I try to run query again in another attempt or after restarting the application, then the chat object gets found
I also tried many different approaches to execute this code:
I tried to use realm.refresh() in getSingleChat or not use it at all.
I tried to query chat synchronously with findFirst() and findAll() instead of findFirstAsync().
I tried querying chat on current thread without .subscribeOn().
I tried to use realm.executeTransaction() instead of async transactions.
I tried to add thread sleep between saving and querying, so that transaction may take some time to get applied and I need to wait before attempting to query the chat
I'm begging anybody to explain me what am I doing wrong and how to make this code working. I can't change the architecture of my application and use Realm objects as my view models, I need to find solution in these conditions.
But when I actually faced the Realm, it turned out to be really painful experience for me to implement caching logic with it.
Reading the docs regarding best practices help. For example, the default idea is that you define a RealmResults using an async query on the UI thread, add a change listener to it, and observe the latest emission of the database.
There is no "caching" involved in that beyond saving to the database and observing the database. Any additional complexity is added by you and is completely optional.
All problems come from applying transaction to database which then must be synced on all threads working with Realm.
All looper threads automatically make the Realm auto-refresh, therefore if addChangeListener is used as intended in the docs, then there is no need for trickery, Realm will manage the synchronization between threads.
I want to save my object to Realm and then query it out of.
realm.executeTransactionAsync({
No reason to use executeTransactionAsync when you are already on a background thread.
try(Realm realm = Realm.getDefaultInstance()) {
realm.executeTransaction((r) -> {
// do write here
});
}
realm.where(RealmChat::class.java)
If you do import io.realm.kotlin.where, then you can do realm.where<RealmChat>().
.findFirstAsync()
No reason to use findFirstAsync() instead of findFirst() when you are already on a background thread. Also no reason to use load() when you're on a background thread, because you should be using findFirst() in the first place anyway.
You are also most likely missing a return#defer Single.just(chat) to actually return the chat if it's found. That is most likely what your original problem is.
With the handler thread things you're doing though, you might want to consider taking a look at this project called "Monarchy", as it intends to set up the ability to run queries on a background looper thread while still observing the results. It is labelled stagnant but the ideas are sound.
I have read that one of the ways to query data in room database is to use Livedata which can be used on the main thread as it is asynchronous.
I would like to use LiveData instead of RxJava or AsyncTask.
For this in my repository class I have function getSomeData() which returns LiveData> and I call this function in my viewModel constructor:
private var mObservableSomeData: LiveData<List<SomeData>>
init {
mObservableSomeData = repository.getSomeData()
}
fun getSomeData(): LiveData<List<SomeData>> {
return mObservableSomeData
}
However it crashes saying:
Cannot access database on the main thread since it may potentially lock the UI for a long period of time.
What should I do?
As pointed out by #LieForBananas, most probably you are getting error while doing insertion. Whenever you have #Query and you are wrapping returned data into an observable e.g. LiveData or Flowable, your query gets executed on background thread by default.
Remember that Room Database ensure that the query returning observable is run on background thread. This is why, if you are wrapping returned value in Flowable, you don't have to explicitly write .subscribeOn(Schedulers.io) while creating observer. Whereas If you are using Flowable for network call(Single might be better because usually we need to emit only once), then you have to explicitly write .subscribeOn(Scheduler.io()) to run the call on a background thread.
Room doesn't allow database operation on the Main thread unless you allow database on the main thread with allowMainThreadQueries().
MyApp.database = Room.databaseBuilder(this,AppDatabase::class.java,"MyDatabase")
.allowMainThreadQueries()
.build()
I'm seeing a very weird problem with getting the count from a query not running on the main thread. I have a UI that uses the RecyclerView adapter for Realm, and it works just dandy.
I have a method that counts the number of records before a query so it can set a starting point, for some reason it will return the previous count was before the last transaction. Here is a somewhat shortened version with the output of the log:
D/SearchController: Query Count (Main Thread): 50
D/SearchController: Query Count (Rx Thread): 50
D/SearchController: Query Count (Main Thread): 100
D/SearchController: Query Count (Rx Thread): 50
public Single<Update> searchWithCriteriaForMore(SearchCriteria searchCriteria) {
Realm realmI = Realm.getDefaultInstance();
Timber.d("Query Count (Main Thread): %d", realmI.where(Ad.class).equalTo("searchId", searchCriteria.id()).count());
realmI.close();
return Single.defer(() -> {
Realm realm = Realm.getDefaultInstance();
Timber.d("Query Count (Rx Thread): %d", realm.where(Ad.class).equalTo("searchId", searchCriteria.id()).count());
realm.close();
// Stuff to add records on Rx Thread
});
}
The call looks like this:
SearchController.instance().searchWithCriteriaForMore(searchCriteria)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new SingleSubscriber<SearchController.Update>() {
...
}
Thoughts anyone? I've been trying to nail this down for a few weeks, thought it was something I was doing, but if I do the count on the main thread and pass it in, it works fine.
I'm running RxJava 1.2.7, RxAndroid 1.2.1, and Realm 3.3.1.
It feels that you have a Realm instance on RxThread somewhere which is not closed before. Since RxThread doesn't have a looper, the realm instance cannot be auto-updated. So it was locked at the version when it was created.
Realm is using ref counter internally, the getDefaultInstance() will just return the same instance you opened on the RxThread before.
The solution:
1. Find out which Realm instance was retrieved on the RxThread and close it properly. So the next time getDefaultInstance() will return a new Realm instance at the latest version data.
2. If it is intended to have a Realm instance on the RxThread, you can call realm.refresh() to manually move it to the latest data version.
I am using Realm 3.0.0 as the DB of my Android app. It's like a questionnaire application, in which the user navigates inside the app a lot. When I use the app (go back and forth) continuously, I get the following error:
Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. mmap() failed: Out of memory size: 1073741824 offset: 0 in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 109
at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
at io.realm.internal.SharedRealm.(SharedRealm.java:187)
at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:229)
at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:204)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:124)
at io.realm.Realm.getDefaultInstance(Realm.java:210)
Now I know the main cause of this is not closing Realm instances. But I've already checked for that multiple times. And I am positive that I close every instance I open.
The app has many activities and fragments that all get a Realm instance on their onCreate and close it on their onDestroy. There are also other background network jobs that run to upload data that get Realm instances. These jobs close their Realm instances when they've finished running or when they cancel.
All of the above get their Realm instance thru injection via Dagger 2:
#Provides
#Nullable
static Realm realm(#Nullable RealmConfiguration configuration) {
if (configuration != null) {
Realm.setDefaultConfiguration(configuration);
return Realm.getDefaultInstance();
}
return null;
}
Configuration is also provided in the same Dagger Module.
To be more specific, a Questionnaire consists of many Question Fragments displayed in a ViewPager. Each Fragment gets injected with a realm. Many interactions in a given Question Fragment write data to the DB (some async, some blocking). These Fragments also query the database on onResume to get their updated Data. Some of this data is also copied out of Realm via realm.copyFromRealm(). Now at any given time of these happening, an upload job is most likely running and reading data from the DB and uploading it to a server. When an upload job finishes, it then writes to the DB.
I think I can have up to 7-12 fragment/activities holding a realm reference on the UI thread at a given moment. And 0-6 other references on 0-3 other threads (Background Jobs).
Moreover, I compact my realm DB via Realm.compactRealm(realmConfiguration) on every app launch (perhaps as a separate problem, this doesn't seem to do it's job consistently).
Above I've tried to describe my Realm usage descriptively without going into details. Now my problem is, when a user excessively uses the app (going back and forth between activities/fragments (realm injection + DB read query), uploading data (realm injection + DB read&write query)), I get the above posted Out of Memory Error.
I am also using Leak Canary, and it hasn't detected any leaks. (Not sure if it can anyway)
Am I using Realm in a way it's not supposed to be used? Should I close Realm instances onPause instead of onDestroy? Should I have only one realm instance in an activity and have all it's fragmetns (up to 5 in my case) use this instance? What kind of changes can I make in my app, and perhaps my app architecture to solve this problem?
I appreciate any help in trying to solve this problem.
EDIT: I'm sharing the realm open-close logic in my background threads.
All my jobs share the same realm usage, which is the following:
Realm is injected lazily via:
#Inject protected transient Lazy<Realm> lazyRealm;
The realm object reference is held at the private transient Realm realm; field. I am using Android Priority Job Queue. When the job is added:
#Override
public void onAdded() {
realm = lazyRealm.get();
realm.executeTransaction(realm1 -> {
//write some stuff into realm
});
realm.close();
}
And when the job is run realm is retreived once, and every possible ending of this method has a call to realm.close()
#Override public void onRun() throws Throwable {
synchronized (syncAdapterLock) {
realm = lazyRealm.get();
Answer answer = realm.where(Answer.class).equalTo(AnswerQuery.ID, answerId).findFirst();
if (answer == null) {
realm.close();
throw new RealmException("File not found");
}
final File photoFile = new File(answer.getFilePath());
final Response response = answerService.uploadPhotoAnswer(answerId, RequestBody.create(MediaType.parse("multipart/form-data"), photoFile)).execute();
if (!response.isSuccessful()) {
realm.close();
throw new HttpError(statusCode);
}
realm.executeTransaction(realm1 -> {
answer.setSyncStatus(SyncStatus.SYNCED.getCode());
});
}
realm.close();
}
}
As you can see, these background threads do close their realm instances properly as far as I'm concerned.
While it was true that all my background tasks did call realm.close(), one of them called it too late in it's lifecycle. That was my GPSService, which is a background service. The problem was that GPS service is initialized at the launch of the App as an Android Service, which is rarely destroyed. I was injecting a realm instance onCreate and closing it onDestroy. After the comments of #EpicPandaForce and reading his articles about using realm properly. I realized that this was the cause of the leak. A non-looper thread was keeping an open realm reference for an extremely long time, thus, the mmap was bloating every time a write transaction occures. Now that I moved the realm get/close to happen every time the service runs, my problem is fixed.
My take away is that one needs to treat background thread realm access very delicately. Thank you both for your quick responses and help!
I'm trying to figure out a way to save my Parsed objects from retro into realm through rxjava observable.
Right now I have an API call:
fetchCatalogData().subscribeOn(Schedulers.io())
.subscribe(data ->
saveToRealm();
)
This doesn't work because either I'm using realm on different treads or that scheduler.io doesn't have a looper.
I'm not sure what's the best way to work with retrofit, realm and rxjava. I want to be able to save all my data when it arrives from retrofit and then access it afterwards.
You may leverage a very good library that combines all.
https://github.com/FabianTerhorst/ApiClient
"An easy to use api client that combines the power of Retrofit, Realm, Gson, Rxjava and Retrolambda in a easy to use library for Java and Android"
It's actually rather simple, all you need to do is have two different streams - one for the download, and one for displaying the data.
I'm assuming fetchCatalogData() returns the data via Retrofit, therefore unmanaged objects.
fetchCatalogData()
.subscribeOn(Schedulers.io())
.subscribe(data ->
saveToRealm(data);
)
In this case this works just fine, as long as saveToRealm() opens a thread-specific Realm-instance:
private void saveToRealm(CatalogData data) {
try(Realm r = Realm.getDefaultInstance()) {
r.executeTransaction((realm) -> {
realm.insertOrUpdate(data);
});
}
}
This would allow you to create a Realm (which gets auto-closed) on the background thread, and persist to the Realm.
For the UI thread, you would create a query for CatalogData, and via RxJava-support of Realm you get a RealmChangeListener appended to the result set, and receive a "hot Observable" which gives you the results whenever the underlying table changes.
Subscription showCatalog = realm.where(CatalogData.class)
.findAllAsync()
.filter(RealmResults::isLoaded)
.filter(RealmResults::isValid)
.subscribe((results) -> {
adapter.updateData(results);
});
Realm works with both Retrofit 1.* and 2.* out of the box, but note that Retrofit does not automatically add objects to Realm, instead you must manually add them using the realm.copyToRealm() or realm.copyToRealmOrUpdate() methods.
GitHubService service = restAdapter.create(GitHubService.class);
List<Repo> repos = service.listRepos("octocat");
// Copy elements from Retrofit to Realm to persist them.
realm.beginTransaction();
List<Repo> realmRepos = realm.copyToRealmOrUpdate(repos);
realm.commitTransaction();
Also Realm has first-class support for RxJava, which means that the following Realm classes can be exposed as an Observable: Realm, RealmResults, RealmObject, DynamicRealm and DynamicRealmObject.
// Combining Realm, Retrofit and RxJava (Using Retrolambda syntax for brevity)
// Load all persons and merge them with their latest stats from GitHub (if they have any)
Realm realm = Realm.getDefaultInstance();
GitHubService api = retrofit.create(GitHubService.class);
realm.where(Person.class).isNotNull("username").findAllAsync().asObservable()
.filter(persons.isLoaded)
.flatMap(persons -> Observable.from(persons))
.flatMap(person -> api.user(person.getGithubUserName())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(user -> showUser(user));
This this official Realm docs.