Develop Reference

How to do parallel network requests in the repository ? MVVM

I am working on an Android project and at the moment we are doing multiple network calls in a single repository, for example in the PostsRepository class there are multiple endpoints that needs to be called e.g. (/getNewspost /getPostPrice and maybe /get) then it returns a large Post data class back to the ViewModel.
Although it seems fine, the downside of this structure is being unable to do parallel network calls in the repository like the features of launch, or async/await which only exists in the ViewModel.
So question is can this logic be moved to the ViewModel so then i can do multiple network calls ? Or if this logic should stay in the repository how can we do parallel calls in the repo?

You can create coroutine in Repository class also.
Class PostsRepository{
suspend fun callAPIs() : String{
return withContext(Dispatchers.IO) {
val a = async { getPost() }
val b = async { getNews() }
return#withContext a.await() + b.await()
}
}
}

With Clean architecture , you can create a UseCase to handle this behavior
1.first way
Class GetPostsUseCase(private val postRepository : PostRepository){
suspend operator fun invoke():List<Post>{
// we assume that getPosts()
// and getPostsPricies() are also suspend functions
val posts = postRepository.getPosts()
val prices = postRepository.getPostPricies()
return build(posts , prices)
}
private fun build(posts,prices) :List<Post>{
// build your data object here
}
}
/////// OR ////////
Class GetPostsUseCase(private val postRepository : PostRepository){
suspend operator fun invoke():List<Post> = withContext(Dispatchers.IO){
val posts = async{postRepository.getPosts()}
val prices = async { postRepository.getPostPricies() }
posts.await()
prices.await()
return build(posts, prices)
}
private fun build(posts,prices) :List<Post>{
// build your data object here
}
}

You can achieve this by using suspend and withContext
class PostsRepository {
suspend fun fetchPostData(): Post {
return withContext(Dispatchers.IO) {
val fetchA = async { getA() }
val fetchB = async { getB() }
val fetchC = async { getC() }
//More if needed ...
//Then execute waitAll() to get them all as parallel
val (AResult, BResult, CResult) = awaitAll(fetchA, fetchB, fetchC)
//Finally use the result of these fetch when all of them is completed
return#withContext Post(AResult, BResult, CResult)
}
}
}

How to avoid concurrency issues with Kotlin coroutines?

I am going to implement a chat feature in the android application. In order to do that, I fetch chat messages every five seconds from the server by a coroutine flow. The problem is when I want to send a message sometimes the server receives two concurrent requests and it returns an error. How should I make sure that these requests run sequentially in my chat repository? Here is my chat repository:
class ChatRepositoryImpl #Inject constructor(
private val api: ApolloApi,
private val checkTokenIsSetDataStore: CheckTokenIsSetDataStore
) : ChatRepository {
override fun chatMessages(
lastIndex: Int,
limit: Int,
offset: Int,
channelId: Int,
): Flow<Resource<ChatMessages>> = flow {
var token = ""
checkTokenIsSetDataStore.get.first {
token = it
true
}
while (true) {
val response = ChatMessagesQuery(
lastIndex = Input.fromNullable(lastIndex),
limit = Input.fromNullable(limit),
offset = Input.fromNullable(offset),
channelId
).let {
api.getApolloClient(token)
.query(it)
.await()
}
response.data?.let {
emit(
Resource.Success<ChatMessages>(
it.chatMessages
)
)
}
if (response.data == null)
emit(Resource.Error<ChatMessages>(message = response.errors?.get(0)?.message))
delay(5000L)
}
}.flowOn(Dispatchers.IO)
override fun chatSendText(channelId: Int, text: String): Flow<Resource<ChatSendText>> = flow {
var token = ""
checkTokenIsSetDataStore.get.first {
token = it
true
}
val response = ChatSendTextMutation(
channelId = channelId,
text = text
).let {
api.getApolloClient(token)
.mutate(it)
.await()
}
response.data?.let {
return#flow emit(
Resource.Success<ChatSendText>(
it.chatSendText
)
)
}
return#flow emit(Resource.Error<ChatSendText>(message = response.errors?.get(0)?.message))
}.flowOn(Dispatchers.IO)
}

One way to limit concurrency is to use utils like Mutex or Semaphore. We can very easily solve your problem with mutex:
class ChatRepositoryImpl ... {
private val apolloMutex = Mutex()
override fun chatMessages(...) {
...
apolloMutex.withLock {
api.getApolloClient(token)
.query(it)
.await()
}
...
}
override fun chatSendText(...) {
...
apolloMutex.withLock {
api.getApolloClient(token)
.mutate(it)
.await()
}
...
}
However, this problem should not be really fixed on the client side, but on the server side. Your attempted solution doesn't protect you against concurrent requests entirely. If for some reasons two instances of the application has the same token or if the user attempts to manipulate your application, it could still send concurrent requests.
If you can't easily fix the problem properly, you can apply the same fix on the server side that you intend to apply on the client side. Just handle requests or part of requests sequentially. It is more error-proof and also more performant, because this way only part of the whole request time has to be done sequentially.

Android download multiple files with OkHttp and coroutine

In my app, I get a set of urls to some images from an api and need to create Bitmap objects out of those urls to be able do display the images in the UI. I saw that the android docs recommend using corutines for performing such async tasks, but I am not sure how to do it properly.
Using OkHttp for my http client, I tried the following approach:
GlobalScope.launch {
val gson = Gson();
val parsedRes = gson.fromJson(
response.body?.charStream(),
Array<GoodreadsBook>::class.java
);
// Create the bitmap from the imageUrl
for (i in 0 until parsedRes.size) {
val bitmap =
GlobalScope.async { createBitmapFromUrl(parsedRes[i].best_book.image_url) }
parsedRes[i].best_book.imageBitmap = bitmap.await();
}
searchResults.postValue(parsedRes)
}
Where response is what I get back from my API, and searchResults is a LiveData that hold the parsed response.
Also, here is how I am getting the images from those urls:
suspend fun createBitmapFromUrl(url: String): Bitmap? {
val client = OkHttpClient();
val req = Request.Builder().url(url).build();
val res = client.newCall(req).execute();
return BitmapFactory.decodeStream(res.body?.byteStream())
}
Even though every fetch action is done on a separate coroutine, it's still too slow. Is there a better way of doing it? I can use any other http client if there is one out there optimized for use with coroutines, although I am new to Kotlin so I don't know any.

First of all the createBitmapFromUrl(url: String) does everything synchronously, you've to first stop them from blocking the coroutine thread, you may want to use Dispatchers.IO for that because callback isn't the most idomatic thing ever in coroutines.
val client = OkHttpClient() // preinitialize the client
suspend fun createBitmapFromUrl(url: String): Bitmap? = withContext(Dispatchers.IO) {
val req = Request.Builder().url(url).build()
val res = client.newCall(req).execute()
BitmapFactory.decodeStream(res.body?.byteStream())
}
Now, when you are calling bitmap.await() you are simply saying that "Hey, wait for the deferred bitmap and once it is finished resume the loop for next iteration"
So you may want to do the assignment in the coroutine itself to stop it from suspending the loop, otherwise create another loop for that. I'd go for first option.
scope.launch {
val gson = Gson();
val parsedRes = gson.fromJson(
response.body?.charStream(),
Array<GoodreadsBook>::class.java
);
// Create the bitmap from the imageUrl
for (i in 0 until parsedRes.size) {
launch {
parsedRes[i].best_book.imageBitmap = createBitmapFromUrl(parsedRes[i].best_book.image_url)
}
}
}

Use a library like the following that doesn't use the blocking execute method and instead bridges from the async enqueue.
https://github.com/gildor/kotlin-coroutines-okhttp
suspend fun main() {
// Do call and await() for result from any suspend function
val result = client.newCall(request).await()
println("${result.code()}: ${result.message()}")
}
What this basically does is the following
public suspend fun Call.await(): Response {
return suspendCancellableCoroutine { continuation ->
enqueue(object : Callback {
override fun onResponse(call: Call, response: Response) {
continuation.resume(response)
}
override fun onFailure(call: Call, e: IOException) {
if (continuation.isCancelled) return
continuation.resumeWithException(e)
}
})
continuation.invokeOnCancellation {
try {
cancel()
} catch (ex: Throwable) {
//Ignore cancel exception
}
}
}
}

Unit Test for retrofit2.Call

Since Retrofit 2.6.0 has support for Coroutines and the library of kotlinCoroutinesAdapter is deprecated I started to refactor my API calls. I changed it and is working fine but I have problems with Unit Tests.
This is how I had my calls:
//Retrofit API
#GET("playlists/{playlistId}")
fun getPlaylistAsync(#Path("playlistId") id: String): Deferred<PlaylistData>
//DataSource
fun getPlaylistAsync(playlistId: String): Deferred<PlaylistData> = playlistApi.getPlaylistAsync(playlistId)
//Repository
override suspend fun getPlaylist(playlistId: String): PlaylistDomain {
val playlistDeferred = remoteDataSource.getPlaylistAsync(playlistId)
return playlistRemoteMapper.map(playlistDeferred.await())
}
And this is what I changed to make it work:
//Retrofit API
#GET("playlists/{playlistId}")
fun getPlaylistAsync(#Path("playlistId") id: String): Call<PlaylistData>
//DataSource
fun getPlaylistAsync(playlistId: String): Call<PlaylistData> = playlistApi.getPlaylistAsync(playlistId)
//Repository
override suspend fun getPlaylist(playlistId: String): PlaylistDomain {
val playlistDeferred = remoteDataSource.getPlaylistAsync(playlistId)
return playlistRemoteMapper.map(playlistDeferred.await())
}
Basically is the same... the await() used with Deferred and with Call is different but everything else is so similar.
As I said the problems comes with the test. Here I have what I did for the first example:
#Test
fun `getPlaylist() maps correctly the playlist and the favorite request when is favorite`() = runBlocking {
val playlist = PlaylistDataBuilder().getPlaylistData(id = PLAYLIST_ID)
every { playlistDataSource.getPlaylistAsync(PLAYLIST_ID) } answers { async { playlist } }
val response = playlistRepository.getPlaylist(PLAYLIST_ID)
assertTrue(response.isFavorite)
}
Now I want to change this to fit the second example... but async returns a Deferred and now I need Call to fit my needs. I tried to mock Call and do the following:
#Test
fun `getPlaylist() maps correctly the playlist and the favorite request when is favorite`() = runBlocking {
val playlist = PlaylistDataBuilder().getPlaylistData(id = PLAYLIST_ID)
every { playlistCall.await() }
every { playlistDataSource.getPlaylistAsync(PLAYLIST_ID) } answers { playlistCall }
val response = playlistRepository.getPlaylist(PLAYLIST_ID)
assertTrue(response.isFavorite)
}
The problem here is that playlistCall.await() must be into a suspend method so I cannot make it work this way. I would need a way to make a Unit Test for retrofit Call.

How to use Fuel with coroutines in Kotlin?

I want to get an API request and save request's data to a DB. Also want to return the data (that is written to DB). I know, this is possible in RxJava, but now I write in Kotlin coroutines, currently use Fuel instead of Retrofit (but a difference is not so large). I read How to use Fuel with a Kotlin coroutine, but don't understand it.
How to write a coroutine and methods?
UPDATE
Say, we have a Java and Retrofit, RxJava. Then we can write a code.
RegionResponse:
#AutoValue
public abstract class RegionResponse {
#SerializedName("id")
public abstract Integer id;
#SerializedName("name")
public abstract String name;
#SerializedName("countryId")
public abstract Integer countryId();
public static RegionResponse create(int id, String name, int countryId) {
....
}
...
}
Region:
data class Region(
val id: Int,
val name: String,
val countryId: Int)
Network:
public Single<List<RegionResponse>> getRegions() {
return api.getRegions();
// #GET("/regions")
// Single<List<RegionResponse>> getRegions();
}
RegionRepository:
fun getRegion(countryId: Int): Single<Region> {
val dbSource = db.getRegion(countryId)
val lazyApiSource = Single.defer { api.regions }
.flattenAsFlowable { it }
.map { apiMapper.map(it) }
.toList()
.doOnSuccess { db.updateRegions(it) }
.flattenAsFlowable { it }
.filter({ it.countryId == countryId })
.singleOrError()
return dbSource
.map { dbMapper.map(it) }
.switchIfEmpty(lazyApiSource)
}
RegionInteractor:
class RegionInteractor(
private val repo: RegionRepository,
private val prefsRepository: PrefsRepository) {
fun getRegion(): Single<Region> {
return Single.fromCallable { prefsRepository.countryId }
.flatMap { repo.getRegion(it) }
.subscribeOn(Schedulers.io())
}
}

Let's look at it layer by layer.
First, your RegionResponse and Region are totally fine for this use case, as far as I can see, so we won't touch them at all.
Your network layer is written in Java, so we'll assume it always expects synchronous behavior, and won't touch it either.
So, we start with the repo:
fun getRegion(countryId: Int) = async {
val regionFromDb = db.getRegion(countryId)
if (regionFromDb == null) {
return apiMapper.map(api.regions).
filter({ it.countryId == countryId }).
first().
also {
db.updateRegions(it)
}
}
return dbMapper.map(regionFromDb)
}
Remember that I don't have your code, so maybe the details will differ a bit. But the general idea with coroutines, is that you launch them with async() in case they need to return the result, and then write your code as if you were in the perfect world where you don't need to concern yourself with concurrency.
Now to the interactor:
class RegionInteractor(
private val repo: RegionRepository,
private val prefsRepository: PrefsRepository) {
fun getRegion() = withContext(Schedulers.io().asCoroutineDispatcher()) {
val countryId = prefsRepository.countryId
return repo.getRegion(countryId).await()
}
}
You need something to convert from asynchronous code back to synchronous one. And for that you need some kind of thread pool to execute on. Here we use thread pool from Rx, but if you want to use some other pool, so do.

After researching How to use Fuel with a Kotlin coroutine, Fuel coroutines and https://github.com/kittinunf/Fuel/ (looked for awaitStringResponse), I made another solution. Assume that you have Kotlin 1.3 with coroutines 1.0.0 and Fuel 1.16.0.
We have to avoid asynhronous requests with callbacks and make synchronous (every request in it's coroutine). Say, we want to show a country name by it's code.
// POST-request to a server with country id.
fun getCountry(countryId: Int): Request =
"map/country/"
.httpPost(listOf("country_id" to countryId))
.addJsonHeader()
// Adding headers to the request, if needed.
private fun Request.addJsonHeader(): Request =
header("Content-Type" to "application/json",
"Accept" to "application/json")
It gives a JSON:
{
"country": {
"name": "France"
}
}
To decode the JSON response we have to write a model class:
data class CountryResponse(
val country: Country,
val errors: ErrorsResponse?
) {
data class Country(
val name: String
)
// If the server prints errors.
data class ErrorsResponse(val message: String?)
// Needed for awaitObjectResponse, awaitObject, etc.
class Deserializer : ResponseDeserializable<CountryResponse> {
override fun deserialize(content: String) =
Gson().fromJson(content, CountryResponse::class.java)
}
}
Then we should create a UseCase or Interactor to receive a result synchronously:
suspend fun getCountry(countryId: Int): Result<CountryResponse, FuelError> =
api.getCountry(countryId).awaitObjectResponse(CountryResponse.Deserializer()).third
I use third to access response data. But if you wish to check for a HTTP error code != 200, remove third and later get all three variables (as Triple variable).
Now you can write a method to print the country name.
private fun showLocation(
useCase: UseCaseImpl,
countryId: Int,
regionId: Int,
cityId: Int
) {
GlobalScope.launch(Dispatchers.IO) {
// Titles of country, region, city.
var country: String? = null
var region: String? = null
var city: String? = null
val countryTask = GlobalScope.async {
val result = useCase.getCountry(countryId)
// Receive a name of the country if it exists.
result.fold({ response -> country = response.country.name }
, { fuelError -> fuelError.message })
}
}
val regionTask = GlobalScope.async {
val result = useCase.getRegion(regionId)
result.fold({ response -> region = response.region?.name }
, { fuelError -> fuelError.message })
}
val cityTask = GlobalScope.async {
val result = useCase.getCity(cityId)
result.fold({ response -> city = response.city?.name }
, { fuelError -> fuelError.message })
}
// Wait for three requests to execute.
countryTask.await()
regionTask.await()
cityTask.await()
// Now update UI.
GlobalScope.launch(Dispatchers.Main) {
updateLocation(country, region, city)
}
}
}
In build.gradle:
ext {
fuelVersion = "1.16.0"
}
dependencies {
...
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.0.0'
// Fuel.
//for JVM
implementation "com.github.kittinunf.fuel:fuel:${fuelVersion}"
//for Android
implementation "com.github.kittinunf.fuel:fuel-android:${fuelVersion}"
//for Gson support
implementation "com.github.kittinunf.fuel:fuel-gson:${fuelVersion}"
//for Coroutines
implementation "com.github.kittinunf.fuel:fuel-coroutines:${fuelVersion}"
// Gson.
implementation 'com.google.code.gson:gson:2.8.5'
}
If you want to work with coroutines and Retrofit, please, read https://medium.com/exploring-android/android-networking-with-coroutines-and-retrofit-a2f20dd40a83 (or https://habr.com/post/428994/ in Russian).

You should be able to significantly simplify your code. Declare your use case similar to the following:
class UseCaseImpl {
suspend fun getCountry(countryId: Int): Country =
api.getCountry(countryId).awaitObject(CountryResponse.Deserializer()).country
suspend fun getRegion(regionId: Int): Region =
api.getRegion(regionId).awaitObject(RegionResponse.Deserializer()).region
suspend fun getCity(countryId: Int): City=
api.getCity(countryId).awaitObject(CityResponse.Deserializer()).city
}
Now you can write your showLocation function like this:
private fun showLocation(
useCase: UseCaseImpl,
countryId: Int,
regionId: Int,
cityId: Int
) {
GlobalScope.launch(Dispatchers.Main) {
val countryTask = async { useCase.getCountry(countryId) }
val regionTask = async { useCase.getRegion(regionId) }
val cityTask = async { useCase.getCity(cityId) }
updateLocation(countryTask.await(), regionTask.await(), cityTask.await())
}
}
You have no need to launch in the IO dispatcher because your network requests are non-blocking.
I must also note that you shouldn't launch in the GlobalScope. Define a proper coroutine scope that aligns its lifetime with the lifetime of the Android activity or whatever else its parent is.