I'm having a tough time figuring out why do I observe InterruptedExceptions in the SyncAdapter#onPerformSync method.
Brief intro
My project originally contained only Java code and had a working SyncAdapter pattern implemented.
In the onPerfromSync method I did the following operations for each resource that I needed to sync:
query local sqlite database
send data to web server over HTTP
save data to local sqlite database
As the project progressed I introduced Kotlin and Coroutines and decided to use them in the SyncAdapter.
Most of the resources kept the existing logic, and for a few I decided to use Coroutines.
I had to bridge the onPerformSync method with a CoroutineScope so that I can launch coroutines. After doing so I started observing InterruptedExceptions occurring.
Code before (no exceptions)
class MySyncAdapter extends AbstractThreadedSyncAdapter {
...
#Override
public void onPerformSync(...) {
syncResourceX();
syncResourceY();
}
private void syncResourceX() {
// query db
// send to server
// store locally
}
}
Code After (with exceptions)
class MySyncAdapter(...): AbstractThreadedSyncAdapter(...) {
...
override fun onPerformSync(...) {
runBlocking {
syncResourceX();
syncResourceY();
}
}
private suspend fun syncResourceX() {
// query db
// send to server
// store locally
}
}
After this refactoring I started receiving InterruptedExceptions where runBlocking is being invoked.
Initially I thought this might be due to not performing network operations on the current thread as the docs of the sync adapter state that the system monitors network traffic and might interrupt the sync process if no traffic is generated.
But runBlocking should cause any network requests to be executed on the current thread right?
After which I started thinking that this process has existed in the Java code as well. It's just that there is nothing to report the interruption of the sync process. It is up until I started using coroutines and runBlocking that this problem has revealed it self. Or so I think now.
Question:
Any thoughts or explanations why I previously did not observe InterruptedException with the Java code and do observe InterruptedException with the Kotlin code is more than welcome.
Notes
I haven't overriden onSyncCanceled
The sync process usually takes less then 20 seconds to complete
Related
Because database fetches usually happen asynchronously by default, a variable that holds the data from the firebase database fetch will be null when used right after the fetch. To solve this I have seen people use the ".await()" feature in Kotlin coroutines but this goes against the purpose of asynchronous database queries. People also call the succeeding code from within 'addOnSuccessListener{}' but this seems to go against the purpose of MVVM, since 'addOnSuccessListener{}' will be called in the model part of MVVM, and the succeeding code that uses the fetched data will be in the ViewModel. The answer I'm looking for is maybe a listener or observer that is activated when the variable (whose value is filled from the fetched data) is given a value.
Edit:
by "succeeding code" I mean what happens after the database fetch using the fetched data.
As #FrankvanPuffelen already mentioned in his comment, that's what the listener does. When the operation for reading the data completes the listener fires. That means you know if you got the data or the operation was rejected by the Firebase servers due to improper security rules.
To solve this I have seen people use the ".await()" feature in Kotlin coroutines but this goes against the purpose of asynchronous database queries.
It doesn't. Using ".await()" is indeed an asynchronous programming technique that can help us prevent our applications from blocking. When it comes to the MVVM architecture pattern, the operation for reading the data should be done in the repository class. Since reading the data is an asynchronous operation, we need to create a suspend function. Assuming that we want to read documents that exist in a collection called "products", the following function is needed:
suspend fun getProductsFirestore(): List<Product> {
var products = listOf<Product>()
try {
products = productsRef.get().await().documents.mapNotNull { snapShot ->
snapShot.toObject(Product::class.java)
}
} catch (e: Exception) {
Log.d("TAG", e.message!!)
}
return products
}
This method can be called from within the ViewModel class:
val productsLiveData = liveData(Dispatchers.IO) {
emit(repository.getProductsFromFirestore())
}
So it can be observed in activity/fragment class:
private fun getProducts() {
viewModel.producsLiveData.observe(this, {
print(it)
//Do what you need to do with the product list
})
}
I have even written an article in which I have explained four ways in which you can read the data from Cloud Firestore:
How to read data from Cloud Firestore using get()?
Do Firebase Cloud Functions run off of the main thread on Android when initiated similar to Firestore calls?
For Firestore database calls background threading is handled by default.
i.e. Do we need to use background thread for retrieving data using firebase?
The Firebase Database client performs all network and disk operations off the main thread.
The Firebase Database client invokes all callbacks to your code on the main thread.
So network and disk access for the database are no reason to spin up your own threads or use background tasks. But if you do disk, network I/O or CPU intensive operations in the callback, you might need to perform those off the main thread yourself.
Observe
A Firebase Cloud Function is launched on an IO thread within a ViewModel in Android using Kotlin coroutines and returned on the Main thread. However, if Cloud Functions are not run on the main thread by default flowOn(Dispatchers.IO) and withContext(Dispatchers.Main) are not required to specify the threads.
SomeViewModel.kt
fun someMethod() {
repository.someCloudFunction().onEach { resource ->
withContext(Dispatchers.Main) {
// Do something with returned resource here.
}
}.flowOn(Dispatchers.IO).launchIn(viewModelScope)
}
SomeRepository.kt
fun someCloudFunction(contentSelected: FeedViewEventType.ContentSelected) = flow {
try {
val content = contentSelected.content
FirebaseFunctions.getInstance(firebaseApp(true))
.getHttpsCallable("SOME_CLOUD_FUNCTION").call(
hashMapOf(
BUILD_TYPE_PARAM to BuildConfig.BUILD_TYPE,
CONTENT_ID_PARAM to content.id,
CONTENT_TITLE_PARAM to content.title,
CONTENT_PREVIEW_IMAGE_PARAM to content.previewImage))
.continueWith { task ->
(task.result?.data as HashMap<String, String>)
}
// Use 'await' to convert callback to coroutine.
.await().also { response ->
// Do something with response here.
}
} catch (error: FirebaseFunctionsException) {
// Do something with error here.
}
}
Expect
The explicit call to run the cloud function on the IO thread and return the response on the Main thread can be removed safely given that the cloud function does not run on the main thread by default.
SomeViewModel.kt
fun someMethod() {
repository.someCloudFunction().onEach { resource ->
// Do something with returned resource here.
}.launchIn(viewModelScope)
}
When you execute:
FirebaseFunctions.getInstance(...).getHttpsCallable(...).call()
The call returns asynchronously, and the work of accessing the function happens in a non-main thread managed by the SDK. You can't change this behavior, and launching it in a different coroutine scope doesn't really change anything.
When you add a continuation with continueWith, by default the callback happens on the main thread.
It's not until you call await() on the returned Task that anything happens in the coroutine scope that you used to launch the Task. When you await the reuslts, the result is handed to the coroutine for more processing.
For your code, I wouldn't bother trying to use a continuation, since bouncing the result to the main thread isn't helpful at all here. Just await the Task returned by call, and do what you want with the raw result in the coroutine.
My answer that you linked is about the Firebase Realtime Database client, which is separate from the Firebase Functions client.
That said, all Firebase clients should follow the same rules, so I'd expect the Functions client to also execute all network I/O off the main thread.
Is that not what you're seeing?
The code for Android SDK for Functions lives here in the open-source SDK.
I'm trying to implement a simple chat application on web sockets in Clean Architecture. I had to choose a db for caching all information, so I decided to use Realm, because I heard it was pretty good database for any kind of mobile applications. But when I actually faced the Realm, it turned out to be really painful experience for me to implement caching logic with it.
All problems come from applying transaction to database which then must be synced on all threads working with Realm. There seems to some kind of synchronization problem with my code. For example, I want to save my object to Realm and then query it out of.
Here I have two simple functions to save and to get chat:
fun getBackgroundLooper(): Looper {
val handlerThread = HandlerThread("backgroundThread")
if (!handlerThread.isAlive)
handlerThread.start()
return handlerThread.looper
}
fun saveChat(chat: Chat): Completable {
val realmChat = ChatMapper.domainToCache(chat)
return Completable.create { e ->
val realm = Realm.getDefaultInstance()
realm.executeTransactionAsync({
it.insertOrUpdate(realmChat)
}, {
realm.close()
e.onComplete()
}, {
realm.close()
e.onError(it)
})
// Subscribe on background looper thread
// to be able to execute async transaction
}.subscribeOn(AndroidSchedulers.from(getBackgroundLooper()))
}
fun getSingleChat(chatId: String): Single<Chat> {
return Single.defer {
val realm = Realm.getDefaultInstance()
realm.isAutoRefresh = true
val realmChat = realm.where(RealmChat::class.java)
.equalTo("id", chatId).findFirstAsync()
if (realmChat.isValid) {
realmChat.load()
val chat = ChatMapper.cacheToDomain(realmChat)
realm.close()
Single.just(chat)
}
realm.close()
Single.error<Chat>(ChatNotExistException())
// Subscribe on background looper thread
// to be able to execute auto refreshing
}.subscribeOn(AndroidSchedulers.from(getBackgroundLooper()))
}
So, when I try to run simple code like this
remote.getChat().flatMap {
cache.saveChat(it) //save chat to realm
.andThen(cache.getSingleChat(it.id)) //then query it by id
}
I always get no matter of what ChatNotExistException, but if I try to run query again in another attempt or after restarting the application, then the chat object gets found
I also tried many different approaches to execute this code:
I tried to use realm.refresh() in getSingleChat or not use it at all.
I tried to query chat synchronously with findFirst() and findAll() instead of findFirstAsync().
I tried querying chat on current thread without .subscribeOn().
I tried to use realm.executeTransaction() instead of async transactions.
I tried to add thread sleep between saving and querying, so that transaction may take some time to get applied and I need to wait before attempting to query the chat
I'm begging anybody to explain me what am I doing wrong and how to make this code working. I can't change the architecture of my application and use Realm objects as my view models, I need to find solution in these conditions.
But when I actually faced the Realm, it turned out to be really painful experience for me to implement caching logic with it.
Reading the docs regarding best practices help. For example, the default idea is that you define a RealmResults using an async query on the UI thread, add a change listener to it, and observe the latest emission of the database.
There is no "caching" involved in that beyond saving to the database and observing the database. Any additional complexity is added by you and is completely optional.
All problems come from applying transaction to database which then must be synced on all threads working with Realm.
All looper threads automatically make the Realm auto-refresh, therefore if addChangeListener is used as intended in the docs, then there is no need for trickery, Realm will manage the synchronization between threads.
I want to save my object to Realm and then query it out of.
realm.executeTransactionAsync({
No reason to use executeTransactionAsync when you are already on a background thread.
try(Realm realm = Realm.getDefaultInstance()) {
realm.executeTransaction((r) -> {
// do write here
});
}
realm.where(RealmChat::class.java)
If you do import io.realm.kotlin.where, then you can do realm.where<RealmChat>().
.findFirstAsync()
No reason to use findFirstAsync() instead of findFirst() when you are already on a background thread. Also no reason to use load() when you're on a background thread, because you should be using findFirst() in the first place anyway.
You are also most likely missing a return#defer Single.just(chat) to actually return the chat if it's found. That is most likely what your original problem is.
With the handler thread things you're doing though, you might want to consider taking a look at this project called "Monarchy", as it intends to set up the ability to run queries on a background looper thread while still observing the results. It is labelled stagnant but the ideas are sound.
I'm using LiveData+Room+ViewModel to build a simple kotlin app. The main activity (which presents a list) is getting the required data from a ViewModel which is getting the info from a database (the data is transformed before being consumed by the activity). Now, I need to allow the user to refresh the data through a swipe. When that happens, the app should check if the current connection can be used for that and if it can't, then the app should schedule a job.
Currently, I'm delegating this work (check the current connection and the eventual job scheduling) to my ViewModel. It looks like this:
fun tryToRefreshDataFromService(){
//first, check if there's network
//If there is, call web service and then update db
//if no network, schedule a job and try to refresh from the database
if(canGetDataFromNetwork()){
Timber.d("With network access, getting data from web services")
WebServiceAsyncTask(newsManager).execute()
}
else{
//schedule job for refreshing
//no network access, setting up job
Timber.d("No network access, setting up job")
scheduleJob()
}
}
The activity will then be able to call the method from within a helper method (which handles the swiper refresh event):
private fun recoverDataForTabs(swiper: SwipeRefreshLayout? = null){
_swiper = swiper //for clearing
_viewModel.tryToRefreshDataFromService()
}
However, it seems like this is really a bad idea because it seems like ViewModels shouldn't know anything about Android framework classes (and that ends up being required for this case). So, does this mean that I should update my code so that the network checking + job scheduling is done from the activity?
Thanks
You can inject framework-related objects into your ViewModels. For example:
class MyViewModel(val networkChecker: IMyNetworkChecker, val jobSetter: IMyJobSetter, ...) {
fun tryToRefreshDataFromService(){
if(networkChecker.canGetDataFromNetwork()){
Timber.d("With network access, getting data from web services")
WebServiceAsyncTask(newsManager).execute()
}
else{
Timber.d("No network access, setting up job")
jobSetter.scheduleJob()
}
}
}
I am working on an Android application that uses greenDAO as a data persistence layer. The application downloads data from various different sources across multiple threads (determined by a thread pool), each piece of data is inserted into the database in a transaction using insertOrReplaceInTx. This is working fine.
My question is whether it is technically possible, using greenDAO, to encapsulate these different transactions (which occur on different threads) into an overall transaction, using nested transactions. I know in theory it is possible to do this if all the transactions were taking place on a single thread, however I am unsure if this possible with the insertOrReplaceInTx calls occurring on different threads.
The reason I wish to encapsulate these into a single overall transaction is because they represent a synchronisation process within an app. In the event of any single part of the import failing, I wish to abort and rollback all of the modifications within the overall transaction.
If I begin a transaction with db.beginTransaction on the main thread where I initiate the import process, this creates a deadlock when another thread tries to insertOrReplaceInTxt.
Is the correct way to counter this to ensure that all greenDAO transactions are taking place on the same thread?
Afaik, you cannot because each thread manages its own connection.
If you have such dependency between these operations, you probably want to sync them anyways.
e.g. what if Job A finishes way before Job B and Job B's db connection fails. Your data will go out of sync again. You still need some logic for the other job.
Also, writers are mutually exclusive.
I would suggest creating a utility class that can run a list of runnables in a transaction. Each job, when finished, enqueues a Runnable to this utility. These runnables include the actual database commands.
When the last one arrives (this depends on your dependency logic), the utility will run all runnables in a transaction.
A sample implementation may look like this: (I used a simple counter but you may need a more complex logic)
class DbBundle {
AtomicInteger mLatch;
List<Runnable> mRunnables = new ArrayList();
DbBundle(int numberOfTx) {
mLatch = new AtomicInteger(numberOfTx);
}
void cancel() {
mLatch.set(-1); // so decrement can never reach 0 in submit
}
boolean isCanceled() {
mLatch.count() < 0;
}
void submit(Runnable runnable) {
mRunnables.add(runnable);
if (mLatch.decrementAndGet() == 0) {
db.beginTransaction();
try {
for (Runnable r : mRunnables) r.run();
db.setTransactionSuccessful()
} finally {
db.endTransaction();
}
}
}
}
When you create each job, you pass this shared DbBundle and the last one will execute them all.
So a job would look like:
....
try {
if (!dbBundle.isCanceled()) { // avoid extra request if it is already canceled
final List<User> users = webservice.getUsers();
dbBundle.submit(new Runnable() {
void onRun() {
saveUsers(users);//which calls db. no transaction code.
});
});
} catch(Throwable t) {
dbBundle.cancel();
}