Develop Reference

How to wait/observe incoming data from server correctly with rxjava/retrofit

I have a problem with downloading data from server, to be more specific I cannot implement a "waiting/observing incoming data" behavior. I want to fill up my RecyclerView and Spinner with lists come from server, but right know, the methods return null/empty list. I understand, the problem is that the method finish before the data arrives.
I've checked these two questions which are similar, but none of the answers worked for me:
Waiting for API Calls to complete with RxJava and Retrofit
RXJava retrofit waiting having the UI wait for data to be fetched from API
My code:
API:
#GET("/api/categories")
Observable<Response<JsonObject>> getCategories();
netWorker:
public List<String> getCat() {
List<String> incomingDataSet = new ArrayList<>();
compositeDisposable.add(
apiConnector.getCategories()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(response -> {
if (response.code() >= 200 && response.code() < 300) {
JsonArray incomingArray = response.body().getAsJsonArray("category");
for (int i = 0; i < incomingArray.size(); i++) {
incomingDataSet.add(incomingArray.get(i).getAsJsonObject().get("categoryname").toString().replaceAll("\"", ""));
Log.i("LOG", "downloadCategories response: " + incomingDataSet.toString());
}
} else {
Log.i("LOG", "Response error: " + response.code());
}
})
);
Log.i("LOG", "getCat: " + incomingDataSet.toString());
return incomingDataSet;
}
MainActivity:
List<String> categorylist = new ArrayList<>();
categorylist = netWorker.getCat();
Log.i("LOG", "list:" + categorylist );
So, what I see in log:
First:
getCat: []
list: []
and then:
downloadCategories response: [cat1]
downloadCategories response: [cat1, cat2]
downloadCategories response: [cat1, cat2, cat3]

Show spinner in doOnSubscribe() and hide in doFinally(). Put code that fills RecyclerView inside subscribe() to fill it when data arrives. Otherwise you just get empty new list on Main thread and do not update RecyclerView further.

Rxjava zip operator filter second Observable by checking data with first Observable android

I am using .zip operator to combine 2 API calls
What I want
I wanted to get filtered values from 2nd Observable based on some ids from 1st Observable
Eg
1st Observable returns data like (sample data)
"categories": [
{
"category": "1",
"category_name": "Wedding Venues",
"category_photo_url": "http://www.marriager.com/uploads/album/0463373001465466151-0463467001465466151.jpeg",
"category_type_id": "1",
2nd Observable returns data like :
"data": [
{
"cat_id": "1",
"category_name": "Wedding Venues",
"status": "1",
"order_id": "1",
"category_type_id": "1"
},
I wanted to filter my 2nd Observable data to only return values that matches category_type_id from 1st Observable
My Code
Observable obsService = retrofitService.loadService(getSharedPref().getVendorId());
Observable obsCategory = retrofitService.loadCategory();
Observable<ServiceAndCategory> obsCombined = Observable.zip(obsService.observeOn(AndroidSchedulers.mainThread()).subscribeOn(Schedulers.io()), obsCategory.observeOn(AndroidSchedulers.mainThread()).subscribeOn(Schedulers.io()), new Func2<ServiceModel, CategoryModel, ServiceAndCategory>() {
#Override
public ServiceAndCategory call(ServiceModel serviceModel, CategoryModel categoryModel) {
return new ServiceAndCategory(serviceModel, categoryModel);
}
});
obsCombined.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io());
obsCombined.subscribe(new Subscriber<ServiceAndCategory>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
if (e instanceof UnknownHostException || e instanceof ConnectException) {
mPresenter.onNetworkError();
} else if (e instanceof SocketTimeoutException) {
mPresenter.onTimeOutError();
} else {
mPresenter.onServerError();
}
}
#Override
public void onNext(ServiceAndCategory model) {
mPresenter.onSuccess(model);
}
});
EDIT
basically I want to apppy the following logic
this.categoryList = combinedModel.categoryModel.getData();
serviceList = combinedModel.serviceModel.getData().getCategories();
for (int i = 0; i < serviceList.size(); i++) {
for (int j = 0; j < categoryList.size(); j++) {
if (!serviceList.get(i).getCategoryTypeId().equals(categoryList.get(j).getCategoryTypeId())) {
categoryList.remove(j);
}
}
}

You can apply this filtering with reactive approach using a map and a list, first collect all categories to a map, and all services to a list, zip them together, and then filter the services list according to categories map:
Observable<HashMap<Integer, CategoryData>> categoriesMapObservable =
obsCategory
.flatMapIterable(CategoryModel::getData)
.reduce(new HashMap<>(),
(map, categoryData) -> {
map.put(categoryData.getCategoryTypeId(), categoryData);
return map;
}
);
Observable<List<ServiceData>> serviceListObservable = obsService
.map(ServiceModel::getData);
Observable obsCombined =
Observable.zip(
categoriesMapObservable
.subscribeOn(Schedulers.io()),
serviceListObservable
.subscribeOn(Schedulers.io()),
Pair::new
)
.flatMap(hashMapListPair -> {
HashMap<Integer, CategoryData> categoriesMap = hashMapListPair.first;
return Observable.from(hashMapListPair.second)
.filter(serviceData -> categoriesMap.containsKey(serviceData.getCategoryTypeId()))
.toList();
}, (hashMapListPair, serviceDataList) -> new Pair<>(hashMapListPair.first.values(), serviceDataList));
the output result depends on you , here I apply at the end a selector of flatMap() that will create a Pair of Collection of CategoryData and a filtered list of ServiceData, you can of course create whatever custom Object you need for that.
I'm not sure you're gaining much from this, it's seems more efficient from complexity perspective, assuming HashMap is O(1), where categories are N, and services are M, you have here N + M (N constructing the map, M iterating the list and querying the map), while your naive implementation will be N x M.
as for code complexity, i'm not sure it worth it, you can apply your logic at the end of the zip for filtering, or use some library that might be doing filter more efficiently.
P.S the observerOn(AndroidSchedulers.mainThread() is unnecessary so I removed it.

Retrofit + RxJava chain multiple requests and aggregate the result

I am looking for the way how to chain multiple but same API requests with different parameters. So far my method looks like this:
#Override
public Observable<List<Entity>> getResult(Integer from, Integer to, Integer limit) {
MyService myService = restClient.getMyService();
if (null != from && null != to) {
Observable<List<Response>> responseObservable = myService.get(from, limit);
for (int i = from + 1; i <= to; i++) {
responseObservable = Observable.concat(responseObservable, myService.get(i, limit));
}
return responseObservable.map(mapResponseToEntity);
} else {
int fromParameter = null == from ? DEFAULT_FROM : from;
return myService.get(fromParameter, limit).map(mapResponseToEntity);
}
}
I expected that concat method combines Oservables data into one stream and returns combined Observable but I am getting only the last one calls result. However, in logcat I can see that correct number of calls to API was made.

Try using Observable.merge() and Observable.toList() as follows:
List<Observable<Response>> observables = new ArrayList();
// add observables to the list here...
Subscription subscription = Observable.merge(observables)
.toList()
.single()
.subscribe(...); // subscribe to List<Response>

How to retrieve more than 1000 rows from Parse.com?

I have been using Parse to retrieve a data for a list view. Unfortunately they limit requests to 100 by default to a 1000 max. I have well over that 1000 max in my class. I found a link on the web which shows a way to do it on iOS but how would you do it on Android? Web Link
I am currently adding all the data into a arraylist in a loop until all items are complete (100) then adding them to the list

I have figured out how to achieve my goal:
Declare Global Variable
private static List<ParseObject>allObjects = new ArrayList<ParseObject>();
Create Query
final ParseQuery parseQuery = new ParseQuery("Objects");
parseQuery.setLimit(1000);
parseQuery.findInBackground(getAllObjects());
Callback for Query
int skip=0;
FindCallback getAllObjects(){
return new FindCallback(){
public void done(List<ParseObject> objects, ParseException e) {
if (e == null) {
allObjects.addAll(objects);
int limit =1000;
if (objects.size() == limit){
skip = skip + limit;
ParseQuery query = new ParseQuery("Objects");
query.setSkip(skip);
query.setLimit(limit);
query.findInBackground(getAllObjects());
}
//We have a full PokeDex
else {
//USE FULL DATA AS INTENDED
}
}
};
}

Here is a JavaScript version without promises..
These are the global variables (collections are not required, just a bad habit of mine)..
///create a collection of cool things and instantiate it (globally)
var CoolCollection = Parse.Collection.extend({
model: CoolThing
}), coolCollection = new CoolCollection();
This is the "looping" function that gets your results..
//recursive call, initial loopCount is 0 (we haven't looped yet)
function getAllRecords(loopCount){
///set your record limit
var limit = 1000;
///create your eggstra-special query
new Parse.Query(CoolThings)
.limit(limit)
.skip(limit * loopCount) //<-important
.find({
success: function (results) {
if(results.length > 0){
//we do stuff in here like "add items to a collection of cool things"
for(var j=0; j < results.length; j++){
coolCollection.add(results[j]);
}
loopCount++; //<--increment our loop because we are not done
getAllRecords(loopCount); //<--recurse
}
else
{
//our query has run out of steam, this else{} will be called one time only
coolCollection.each(function(coolThing){
//do something awesome with each of your cool things
});
}
},
error: function (error) {
//badness with the find
}
});
}
This is how you call it (or you could do it other ways):
getAllRecords(0);

IMPORTANT None of the answers here are useful if you are using open
source parse server then it does limit 100 rows by default but you can
put any value in query,limit(100000) //WORKS
No need for recursive
calls just put the limit to number of rows you want.
https://github.com/parse-community/parse-server/issues/5383

JAVA
So after 5 years, 4 months the above answer of #SquiresSquire needed some changes to make it work for me, and I would like to share it with you.
private static List<ParseObject>allObjects = new ArrayList<ParseObject>();
ParseQuery<ParseObject> parseQuery = new ParseQuery<ParseObject>("CLASSNAME");
parseQuery.setLimit(1000);
parseQuery.findInBackground(getAllObjects());
FindCallback <ParseObject> getAllObjects() {
return new FindCallback <ParseObject>() {
#Override
public void done(List<ParseObject> objects, ParseException e) {
if (e == null) {
allObjects.addAll(objects);
int limit = 1000;
if (objects.size() == limit) {
skip = skip + limit;
ParseQuery query = new ParseQuery("CLASSNAME");
query.setSkip(skip);
query.setLimit(limit);
query.findInBackground(getAllObjects());
}
//We have a full PokeDex
else {
//USE FULL DATA AS INTENDED
}
}
}
};

In C# I use this recursion:
private static async Task GetAll(int count = 0, int limit = 1000)
{
if (count * limit != list.Count) return;
var res = await ParseObject.GetQuery("Row").Limit(limit).Skip(list.Count).FindAsync();
res.ToList().ForEach(x => list.Add(x));
await GetAll(++count);
}
JS version:
function getAll(list) {
new Parse.Query(Row).limit(1000).skip(list.length).find().then(function (result) {
list = list.concat(result);
if (result.length != 1000) {
//do here something with the list...
return;
}
getAll(list);
});
}
Usage: GetAll() in C#, and getAll([]) in JS.
I store all rows from the class Rowin the list. In each request I get 1000 rows and skip the current size of the list. Recursion stops when the current number of exported rows is different from the expected.

**EDIT : Below answer is redundant because open source parse server doesn't put any limit on max rows to be fetched
//instead of var result = await query.find();
query.limit(99999999999);//Any value greater then max rows you want
var result = await query.find();**
Original answer:
Javascript / Cloud Code
Here's a clean way working for all queries
async function fetchAllIgnoringLimit(query,result) {
const limit = 1000;
query.limit(limit);
query.skip(result.length);
const results = await query.find();
result = result.concat(results)
if(results.length === limit) {
return await fetchAllIgnoringLimit(query,result );
} else {
return result;
}
}
And here's how to use it
var GameScore = Parse.Object.extend("GameScore");
var query = new Parse.Query(GameScore);
//instead of var result = await query.find();
var result = await fetchAllIgnoringLimit(query,new Array());
console.log("got "+result.length+" rows")

YAS (Yet Another Solution!) Using async() and await() in javascript.
async parseFetchAll(collected = []) {
let query = new Parse.Query(GameScore);
const limit = 1000;
query.limit(limit);
query.skip(collected.length);
const results = await query.find();
if(results.length === limit) {
return await parseFetchAll([ ...collected, ...results ]);
} else {
return collected.concat(results);
}
}

A Swift 3 Example:
var users = [String] ()
var payments = [String] ()
///set your record limit
let limit = 29
//recursive call, initial loopCount is 0 (we haven't looped yet)
func loadAllPaymentDetails(_ loopCount: Int){
///create your NEW eggstra-special query
let paymentsQuery = Payments.query()
paymentsQuery?.limit = limit
paymentsQuery?.skip = limit*loopCount
paymentsQuery?.findObjectsInBackground(block: { (objects, error) in
if let objects = objects {
//print(#file.getClass()," ",#function," loopcount: ",loopCount," #ReturnedObjects: ", objects.count)
if objects.count > 0 {
//print(#function, " no. of objects :", objects.count)
for paymentsObject in objects {
let user = paymentsObject[Utils.name] as! String
let amount = paymentsObject[Utils.amount] as! String
self.users.append(user)
self.payments.append(amount)
}
//recurse our loop with increment because we are not done
self.loadAllPaymentDetails(loopCount + 1); //<--recurse
}else {
//our query has run out of steam, this else{} will be called one time only
//if the Table had been initially empty, lets inform the user:
if self.users.count == 1 {
Utils.createAlert(self, title: "No Payment has been made yet", message: "Please Encourage Users to make some Payments", buttonTitle: "Ok")
}else {
self.tableView.reloadData()
}
}
}else if error != nil {
print(error!)
}else {
print("Unknown Error")
}
})
}
adapted from #deLux_247's example above.

You could achieve this using CloudCode... Make a custom function you can call that will enumerate the entire collection and build a response from that but a wiser choice would be to paginate your requests, and fetch the records 1000 (or even less) at a time, adding them into your list dynamically as required.

GENERIC VERSION For SWIFT 4:
Warning: this is not tested!
An attempt to adapt nyxee's answer to be usable for any query:
func getAllRecords(for query: PFQuery<PFObject>, then doThis: #escaping (_ objects: [PFObject]?, _ error: Error?)->Void) {
let limit = 1000
var objectArray : [PFObject] = []
query.limit = limit
func recursiveQuery(_ loopCount: Int = 0){
query.skip = limit * loopCount
query.findObjectsInBackground(block: { (objects, error) in
if let objects = objects {
objectArray.append(contentsOf: objects)
if objects.count == limit {
recursiveQuery(loopCount + 1)
} else {
doThis(objectArray, error)
}
} else {
doThis(objects, error)
}
})
}
recursiveQuery()
}

Here's my solution for C# .NET
List<ParseObject> allObjects = new List<ParseObject>();
ParseQuery<ParseObject> query1 = ParseObject.GetQuery("Class");
int totalctr = await query1.CountAsync()
for (int i = 0; i <= totalctr / 1000; i++)
{
ParseQuery<ParseObject> query2 = ParseObject.GetQuery("Class").Skip(i * 1000).Limit(1000);
IEnumerable<ParseObject> ibatch = await query2.FindAsync();
allObjects.AddRange(ibatch);
}

Getting JSONException

I have a json object:
"images":{"1":{"imagename":"image1.gif","url":"image1url"},"2":{"imagename":"image2.gif","url":"image2url"},"3":{"imagename":"image3.gif","url":"image3url"}}
I want to fetch imagename and url from this. I enter into this images. I fetched the values 1,2,3 from images. But I am not able to fetch the json corresponding to this 1,2 and 3.
Its throwing exception stating: No value for 1
or No Value for 2
or No value for 3
What may be the reason for this cause? Please reply..
My present code is:
if(jsonObj.has("images")) {
JSONArray imagesArray = jsonObj.getJSONObject("images").names();
JSONObject imageDetailsObject;
for(int i = 0; i < imagesArray.length(); i++) {
imageDetailsObject = jsonObj.getJSONObject(imagesArray.get(i).toString());
if(imageDetailsObject.has("imagename")) {
//perform some actions
}
if(imageDetailsObject.has("url")) {
//perform some actions
}
}
}

EDITED:
imageDetailsObject = jsonObj.getJSONObject("images").getJSONObject(imagesArray.get(i).toString());

I got it worked by giving:
if (jsonObj.has("images")) {
JSONArray imagesArray = jsonObj.getJSONObject("images").names();
JSONObject imageDetailsObject;
for (int i = 0; i < imagesArray.length(); i++) {
imageDetailsObject = jsonObj.getJSONObject("images").getJSONObject(
imagesArray.getString(i));
if(imageDetailsObject.has("imagename")) {
//perform some actions
}
if (imageDetailsObject.has("url")) {
//perform some actions
}
}
}