I have an application that makes requests to a 3rd party API and needs to save most of that data to a local Sqlite database (where local primary keys are autoincremented).
Those requests return a JSON response that often contains many nested objects, that I need to map and save and then link that record back to the original parent object ObjectA in this case.
{
"ObjectAList": [
{
"somefield1": "someValue1",
"someField2": "someValue2",
"nestedObjectB": {
"nestedField": "nestedValue"
},
"nestedObjectCArray": [
{
"nestedArrayField": "nestedArrayValue"
}
]
}
]
}
The app is powered by RxJava; for which I'm fairly new to. (both the networking and persistence via StorIO are reactive). I am able to set up methods to save in bulk the parent ObjectA and that works fine. However I run in to issues as I start to iterate though each of those objects using Observable.from and subsequently saving the nested child objects, collecting those records, and updating the foreign keys in their respective parent items (with usage of Observable.zip for multiple children). Often the function would be executed (according to my logs anyway), but somehow the results don't get submitted back through the rest of the chain, so I'm not sure if something went wrong or if there's an issue with my approach/implementation.
Any help or pointers on how to do this would be appreciated!
Edit some sample psudeo-code added
/*this flatmap is attached to a network request that returns
the previously defined JSON response*/
.flatMap(new Func1<List<ObjectAResponse>, Observable<List<ObjectAResponse>>>() {
#Override
public Observable<List<ObjectAResponse>> call(final List<ObjectAResponse> objectAResponse) {
return saveObjectAListToDb(someOtherKey, objectAJsonResponses)
.map(new Func1<PutResults<ObjectAEntity>, List<ObjectAResponse>>() {
#Override
public List<ObjectAResponse> call(PutResults<ObjectAEntity> putResults) {
//Returning the original response to get the nested values
return objectAResponse;
}
});
}
})
.flatMap(new Func1<List<ObjectAResponse>, Observable<ObjectAResponse>>() {
#Override
public Observable<ObjectAResponse> call(List<ObjectAResponse> objectAResponse) {
//Start emitting individual 'ObjectA' items to process its child items
return Observable.from(objectAResponse);
}
})
.flatMap(new Func1<ObjectAResponse, Observable<ObjectAEntity>>() {
#Override
public Observable<ObjectAEntity> call(ObjectAResponse objectAResponse) {
/*Saving the individual nested objects in separate functions that return an observable that contains
* the resultant entity to be updated in to the main ObjectA entity
* (in reality there could be more nested child objects to save in parallel)
*/
return Observable.zip(saveEpic(objectAResponse.nestedObjectB), saveUser(objectAResponse.nestedObjectCArray),
new Func2<ObjectBEntity, ObjectCEntity, ObjectAEntity>() {
#Override
public Object call(ObjectBEntity objectBEntity, ObjectCEntity objectCEntity) {
//Function that updates the initial object with the new entities
return updateObjectA(objectAResponse.id, objectBEntity, objectCEntity);
}
});
}
})
.collect(new Func0<List<ObjectAEntity>>() {
#Override
public List<ObjectAEntity> call() {
return new ArrayList<ObjectAEntity>();
}
},
new Action2<List<ObjectAEntity>, ObjectAEntity>() {
#Override
public void call(List<ObjectAEntity> objectAEntityList, ObjectAEntity o) {
//Collect the resultant updated entities and return the list
objectAEntityList.add(o);
}
});
Related
I am very new to RxJava and I'm working on an Android app with it. I am making a network request and I'd like my Fragment to update the UI based on the network returned data, and I'm looking for a good 'rx' way to do this. Basically I have my Fragment immediately firing to my viewmodel that it should make the server call. I need to make the server call and notify/send that data to the viewModel so that I can update it to the fragment. Normally (without rx), I'd just pass all of this data with variables, but how can I achieve this data flow using rx and observables?
Thanks everyone.
Create a separate Repository layer, access it only from your viewModels.
In this way you will have view/databinding triggering requests.
Next, have some State management inside Repository or store some data there(use LiveData)
In your ViewModel assign value to the ref of LiveData from repository. So anytime you update it inside Repository - viewModel will have the same object.
Finally, you can observe that viewModel's LiveData.
val someData = MutableLiveData<SomeObject>() - this one inside repository, now you will be able to save any network call result inside repository.
Have your ViewModel contain next one: val someData= Repository.instance.someData
And from fragment/activity use : viewModel.someData.observe(this, {...})
Going to show simple example with code. Another way of doing this using concept single source of truth (SSOT).
Activity-->ViewModel--->Repository--->Insert On Room DB
Step 01: Get all data from room database with Live Data query. And set adapter.
Step 02: Call from Activity/Fragment to remote database/repository to get data.
Step 03: After getting data from remote repository insert it to room database.
Step 04: You have already observing data with Live Query on step 01 so as soon as you
insert data on room database your live observe query will fire again and update
your listview.
Now following example is not complete. But to get a rough idea.
To call & update List using LiveData.
Activity/ Fragment:
RouteViewModel mViewModel = ViewModelProviders.of(this).get(RouteViewModel.class);
mViewModel.getAllRoutes().observe(this, new Observer<List<Route>>() {
#Override
public void onChanged(#Nullable final List<Route> items) {
// will call automatic as soon as room database update
adapter.setItems(items);
}
});
//init/write a remote call here (like you called on room database)
--View Model
public LiveData<List<Route>> getAllRoutes()
{
//call here reposatory
return mAllRoutes;
}
//also write another method here to call repo to init a remote call
---Repository
public LiveData<List<Route>> getRoutes() {
//call on Dao
return mRouteDao.getRoutes();
}
//init a remote call
public Observable<Route> getRoutesFromNetwork(int routeID) {
return new NetworkService().GetChannel().subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Observer<String>() {
#Override
public void onSubscribe(Disposable d) {
}
#Override
public void onNext(String result) {
List<Route> items = new Gson().fromJson(result, new TypeToken<List<Route>>() {
}.getType());
Completable.fromRunnable(() -> {
//insert routes
//if routes is Live data it will update ui automatic
}).subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new CompletableObserver() {
#Override
public void onSubscribe(Disposable d) {
}
#Override
public void onComplete() {
Log.v("Completed", "DONE");
Toasty.info(context,"DONE", Toast.LENGTH_SHORT,true).show();
}
#Override
public void onError(Throwable e) {
Log.v("Error", "Error");
}
});
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
}
});
}
I have a list ofFunctions that is retrieved from a local SQLite database using Room and I want to observe every function in that list. At the moment I'm doing the following:
public List<MutableLiveData<Function>> getLiveFunctions() {
if (liveFunctions == null) {
liveFunctions = new ArrayList<>();
for (Function function : functions) {
//Livedata with a default value on background thread (postValue)
liveFunctions.add(new DefaultLiveData<>(function, true));
}
}
return liveFunctions;
}
After a local fetch from the database, I can request the status of a given function using an RPC to my server. When I receive a response, I can set the new value for that function and I want my UI to observe the changes in that function.
Just some clarifications:
The difference between LiveData<List<Function>>> and List<LiveData<Function>> is that the first will only observe whether an object was added, updated or removed in the list, correct? It's not that LiveData<List<Function>>> also listens to changes on their items?
I'm using a MediatorLiveData to combine my observers to a "FunctionObserver". Is this the correct approach to handle all my function callbacks?
[code]
MediatorLiveData<LiveData<Function>> mediator = new MediatorLiveData<>();
List<MutableLiveData<Function>> functions = //functions
for (LiveData<Function> function : functions) {
mediator.addSource(function, functionObserver);
}
mediator.observe(MainActivity.this, new Observer<LiveData<Function>>() {
#Override
public void onChanged(#Nullable LiveData<Function> functionLiveData) {
Log.i(TAG, "Live data change: ");
}
});
Can my code logic be improved? I know that I can request a LiveData<List<Function>>> from Room but I'm having trouble with my parent class having a #Relation annotation which needs the type to be a List or Set (and not LiveData)
I am using Room for the database layer in my application and Retrofit for network calls - both in room and retrofit I am using RxJava2 (this is my first project with rxjava so I am still quite newbie in this area). To inject database, api etc. I am using Dagger 2.
I want to make a Network Call and add response from the network to the database. When there is no need for making another network call - I want to fetch data from the database. I am having problem with the use of Maybe / Flowable in my room repository.
This is Dao:
#Dao
public interface CoinDao {
#Query("SELECT * FROM coin")
Flowable<List<Coin>> getAllCoins();
#Insert
void insert(List<Coin> coins);
#Update
void update(Coin... coins);
#Delete
void delete(Coin... coins);
}
This is my repository:
public class CoinRepository implements Repository {
private CoinMarketCapNetworkApi api;
private final CoinDao coinDao;
public CoinRepository(CoinMarketCapNetworkApi api, CoinDao coinDao) {
System.out.println("Creating CoinRepository");
this.api = api;
this.coinDao = coinDao;
}
#Override
public Flowable<List<Coin>> getCoinResults() {
System.out.println("getting coin results");
return getCoinResultsFromDatabase().switchIfEmpty(getCoinResultsFromNetwork())
}
#Override
public Flowable<List<Coin>> getCoinResultsFromNetwork() {
System.out.println("getting results from network");
return api.getCoins().doOnNext(new Consumer<List<Coin>>() {
#Override
public void accept(List<Coin> coins) throws Exception {
System.out.println("inserting to db");
coinDao.insert(coins);
}
});
}
#Override
public Flowable<List<Coin>> getCoinResultsFromDatabase() {
System.out.println("getting coins from database");
return coinDao.getAllCoins();
}
}
I firstly run the app to fill the database only with network call
#Override
public Flowable<List<Coin>> getCoinResults() {
return getCoinResultsFromNetwork();
}
And when the network call was executed the data was successfuly added to the database - I run the app once again with only getting data from database and it was also successfull - the data was fetched from db.
#Override
public Flowable<List<Coin>> getCoinResults() {
return getCoinResultsFromDatabase();
}
However when I try now to do such a thing
return getCoinResultsFromDatabase.switchIfEmpty(getCoinResultsFromMemory));
The problem is that everytime switchIfEmpty is executed and every time "getCoinResultsFromMemory()" is executed (even though the data in database is available).
According to https://medium.com/google-developers/room-rxjava-acb0cd4f3757
I have read that the Flowable when there is no data in database will emit nothing and I should use Maybe. But why getResultsFromMemory() returns empty even though there is data in database? How exactly should I use Maybe in this scenario?
I have tried changing Flowable to Maybe
Maybe<List<Coin>> getCoinResultsFromDatabase()
and doing something like this - to access the resutl from maybe and check there if the list is empty or not, but I don't know how to return flowable in this case:
public Flowable<List<Coin>> getCoinResults() {
getCoinResultsFromDatabase().subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Consumer<List<Coin>>() {
#Override
public void accept(List<Coin> coins) throws Exception {
System.out.println("returning coins from maybe" + coins.get(0).getId());
if (coins.isEmpty()) {
System.out.println("coin list is empty");
Flowable<List<Coin>> flowable = getCoinResultsFromNetwork();
} else {
Flowable<List<Coin>> flowable = getCoinResultsFromDatabase();
}
}
});
return flowable //how to access this flowable??
}
Maybe I am missing something and there is a better and more clean solution.
There are few issues with your code:
1.Looks like in room Flowable<List<Coin>> getAllCoins() will always return some value: either list of items or empty list so Maybe will not help here
2.In this piece of code
#Override
public Flowable<List<Coin>> getCoinResults() {
System.out.println("getting coin results");
return getCoinResultsFromDatabase().switchIfEmpty(getCoinResultsFromNetwork())
}
the getCoinResultsFromNetwork is called right when you call getCoinResults method not when the flowable is empty (this is plain java method call)
You need to perform deferred call. The final solution may look like this
#Override
public Flowable<List<Coin>> getCoinResults() {
System.out.println("getting coin results");
return getCoinResultsFromDatabase()
.filter(list -> !list.isEmpty())
.switchIfEmpty(
Flowable.defer(() -> getCoinResultsFromNetwork()))
}
I have a list of transactions. Each transaction has currency and amount information among others. I want to create a list of holdings, so the current amount held by currency. I started with groupBy() and continued with reduce. It seems I have to subscribe before I can do anything with the results, because this gives me an error:
Observable.fromIterable(transactions)
.groupBy(Transaction::getCurrency)
.flatMap(t -> t.reduce(new Holding(t.getKey()), (holding, transaction) -> holding.addTransaction(transaction.getAmount()))
It says "no instance of type variable R exist so that Single conforms to ObservableSource< ? extends R>".
On the other hand if I try this:
Observable.fromIterable(transactions)
.groupBy(Transaction::getCurrency)
.subscribe((GroupedObservable<String, Transaction> r) -> r.reduce(new Holding(r.getKey()), (holding, transaction) -> holding.addTransaction(transaction.getAmount()))
.toObservable()
.subscribe(t -> {
//t is a single Holding.
}
));
I cannot get a list, because I already subscribed to the grouped stream. I could add it up, but I'm pretty sure there is a more elegant solution, but I cannot figure it out.
Solution based on akarnokd's answer:
Observable.fromIterable(transactions)
.groupBy(Transaction::getCurrency)
.flatMapSingle(Observable::toList)
.map(Holding::new)
.toList()
.subscribe(holdings -> {
whatever(holdings);
});
(From my comment to the post):
Try flatMapSingle in the upper case. Also, subscribing from within an onNext handler is a bad practice as you lose the composition properties of RxJava.
This will work for sure
public Single<Map<Integer, List<Category>>> getSubCategoryListById(List<Category> categoryList) {
return Flowable.just(categoryList)
.flatMapIterable(new Function<List<Category>, Iterable<Category>>() {
#Override public Iterable<Category> apply(List<Category> categories) throws Exception {
return categories;
}
})
.filter(new Predicate<Category>() {
#Override public boolean test(Category category) throws Exception {
return category.parent_id != 0;
}
})
.groupBy(new Function<Category, Integer>() {
#Override public Integer apply(Category category) throws Exception {
return category.category_id;
}
})
.flatMapSingle(new Function<GroupedFlowable<Integer, Category>, Single<List<Category>>>() {
#Override public Single<List<Category>> apply(
GroupedFlowable<Integer, Category> integerCategoryGroupedFlowable) throws Exception {
return integerCategoryGroupedFlowable.toList();
}
})
.toMap(new Function<List<Category>, Integer>() {
#Override public Integer apply(List<Category> categories) throws Exception {
return categories.get(0).category_id;
}
});
}
As the documentation says, the reduce function
applies a function to each item emitted by an Observable,
sequentially, and emit the final value.
This is way you get a single value (actually for each Observable of the group you get a single item).
You can defer your reduce operation after you get a list. You could probably replace your first long subscribe with this:
.subscribe(group -> group.toList()
Then you get some Observables based on the number of groups that you have, each emitting a single List of your predefined type.
NOTE: not sure about it, but probably you can replace the first subscribe with a flatMap that transforms your GroupedObservable into an Observable that emit a list of items.
I have the following observable to get list of feed ids from database ( i use sugar ORM library)
public Observable<Set<Long>> getFeedIdsFromDB() {
return Observable.create(subscriber -> {
Set<Integer> subscribedFeedIds = new HashSet<>();
//get feed ids from FeedEntity Table
for (FeedEntity feed : FeedEntity.listAll(FeedEntity.class)){
if (feed.isSubscribed()){
subscribedFeedIds.add(feed.getFeedId());
}
}
});
}
this Observable should emits ids to be used for api call in the following:
public Observable<StoryCollectionEntity> storyEntityList(final int page) {
return this.restApi.storyCollection(/* this is feed ids*/ id, page)
.distinct(storyCollectionEntity -> storyCollectionEntity)
.doOnNext(saveStoryCollectionToCacheAction)
}
i guess i should use some sort of mapping but have no idea how can i implement it.
EDIT:
i did the following modification:
// To map feed ids (retrieved from database) to getAllStoryEntityList Observable:
#Override
public Observable<StoryCollectionEntity> storyEntityList(final int page) {
return this.mNewsCache.getFeedIdsFromDB().flatMap(id -> getAllStoryEntityList(page, id));
}
//call restApi
public Observable<StoryCollectionEntity> getAllStoryEntityList(final int page, Set<Long> id){
return this.restApi.storyCollection( id, page)
.distinct(storyCollectionEntity -> storyCollectionEntity)
.doOnNext(saveStoryCollectionToCacheAction);
}
but api service is never called. something wrong in the mapping.
#GET("story")
Observable<StoryCollectionEntity> storyCollection(
#Query("feed_ids") Set<Long> feedIds,
#Query("page") int page);
The Observable created in getFeedIdsFromDB isn't emitting any items, so your flatMap and other data transformations never occur because the stream actually has no data. You can test this by subscribing directly to the returned Observable and doing something for onNext.
getFeedIdsFromDB().subscribe(feedId -> System.out.println(feedId));
You should see that nothing gets printed. When using Observable#create, the onNext method of subscriber in the anonymous class must be manually called with whatever data you wish to pass downstream. The documentation provides sample code for this.
So modifying your Observable to call onNext, we get this:
public Observable<Set<Long>> getFeedIdsFromDB() {
return Observable.create(subscriber -> {
Set<Integer> feedIds = new HashSet<>();
// get feed ids from FeedEntity Table
for (FeedEntity feed : FeedEntity.listAll(FeedEntity.class)){
feedIds.add(feed.getFeedId());
}
// emit a single Set and complete
if (subscriber.isSubscribed()) {
subscriber.onNext(feedIds);
subscriber.onCompleted();
}
});
}
Now the Set should get passed along. If your goal is to end up with a the emission of a single StoryCollectionEntity object after the transformations (and if I'm reading this properly), then your mapping looks correct.
I'm not sure what the expected output is, but I'll show you a way to do this sort of thing. Maybe you can alter it to fit your use case.
The first step is to allow id as a function parameter in storyEntityList:
public Observable<StoryCollectionEntity> storyEntityList(final int page, int id) {
return this.restApi.storyCollection(/* this is feed ids*/ id, page)
.distinct(storyCollectionEntity -> storyCollectionEntity)
.doOnNext(saveStoryCollectionToCacheAction)
Now you can use an Observable.flatMap:
public Observable<StoryCollectionEntity> getAllStoryEntityList(int page){
return getFeedIdsFromDB().flatMap(id -> storyEntityList(page, id));
}
The naming might be off, but again - I'm not sure what the entities are.