Couchbase Lite - Sorting Map/Reduce by values - android

Advertissement: I'm using CouchbaseLite Android 1.4.0 and I'm not ready to switch to 2.0.
To get the context of my issue:
I have 2 doc types: PATIENT & VISIT
A patient is linked to a visit by a field visitId
A visit has a startDateTime and a endDateTime
A visit is considered as 'open' if the endDateTime is null or missing
A visit can be of different types through the field visitType
A visit can be inactive through the field inactive
I want to:
Get all patients (id + name) of all open and active visits of a certain type
The result is paginated using skip and limit
Get the result sorted by name
I managed to get all the correct patients and to paginate the result but the issue is with the sorting.
I can't make a 'post sorting' (getting the result in a random order and then use self-made method to sort them) because of the pagination.
So here is how it works actually without the sorting:
THE MAP/REDUCE
if (isVisit()) {
Object inactive = document.get(FIELDS.INACTIVE);
if(inactive == null)
inactive = false;
Document patientDoc = database.getExistingDocument(FIELDS.PATIENT_ID);
if(patientDoc != null && patientDoc.getProperties() != null) {
Object[] keys = {
document.get(CouchbaseVisitManager.FIELDS.STOP_DATE_TIME),
inactive,
document.get(CouchbaseVisitManager.FIELDS.VISIT_TYPE)
};
Object[] values = {
document.get(FIELDS.PATIENT_ID),
patientDoc.getProperties().get(FIELDS.FAMILY_NAME),
patientDoc.getProperties().get(FIELDS.FIRST_NAME)
}
emitter.emit(keys, values);
}
}
THE QUERY
Query query = getQuery(Views.PATIENTS_CURRENTLY_IN_VISIT); //Helper method to create a query
query.setKeys(new ArrayList<Object>(){{
add(new ArrayList<Object>(){{
add(null);
add(false);
add(visitType);
}});
}});
query.setSkip(skip);
query.setLimit(limit);
And this is working well to get my patients but not for sorting them.
I tried to add patient names inside the view like this:
Object[] keys = {
patientDoc.getProperties().get(FIELDS.FAMILY_NAME),
patientDoc.getProperties().get(FIELDS.FIRST_NAME),
document.get(FIELDS.STOP_DATE_TIME),
inactive,
document.get(FIELDS.VISIT_TYPE)
};
And update my query.keys like this:
query.setStartKey(new ArrayList<Object>(){{
add(null);
add(null);
add(null);
add(false);
add(visitType);
}});
query.setEndKey(new ArrayList<Object>(){{
add(new HashMap<>());
add(new HashMap<>());
add(null);
add(false);
add(visitType);
}});
But this returns no patients (result.size = 0)
So... I don't know how to achieve my goal.
I tought of a way to sort by values but it doesn't seem to exist yet (should wait for 2.0 I think). But is their any workaround to achieve this kind of behavior?
Thank's for reading.

View results are always sorted by key. So you should add the patient name to the key and then you'll get your sorting.
Be careful to put the name into the correct place of the keys array since the sorting is lexicographical, that is if the name is going to be the last component then you'll get all open inactive sorted by name, then all open active sorted by name e.t.c.
In case you need to put your name key in front of the other keys but still be able to filter your results, then you will need to properly construct the start key and the end key so that the full range of every possible patient names would be accepted while the later keys would be filtered.
You will have to provide some values which will definitely be less than or greater than any possible patient name.
Following the key collation defined for Couchbase views, a null will be always less than any string, and an empty map {} can be used to match past the last string. You are basically doing it right in your last sample.
Don't forget to switch the start and end keys logic if you would like to query the view in reverse mode
Another thing to take care of is always incrementing the version of the view after changing its logic or key set, otherwise you will continue getting old results. Incrementing the version will force a refresh of the view, executing the updated logic in the view function over all your data.

Related

How can I store the user number like this (1,2,3...)?

When a new user signs up, there should be a property named userNo. which should be increased by 1 in each document, so that it would be easy to pick random users from db using that userNo. Basically, like each document holds a User number similar to Uid but not like Afhghdfh4hk545, it should be like userNo.23 and so one. If a new user signs up its userNo. should be 24. Here is what I have tried.
Stream dummy =
await FirebaseFirestore.instance.collection('users').snapshots();
var doclength = await dummy.length;
var userNo = (dummy == null || dummy == 0) ? 1 : doclength;
FirebaseFirestore.instance
.collection('users')
.doc(currentUser.uid)
.update({'userNo': userNo});
Alright, you'll need a "meta document" that has a field called something like "next_user_number".
Upon creating a new user, you check the number on that field and use it as the "UserNo" for the newest user.
And after that, you increase the "next_user_number" in the "meta document" by 1. (here you want to use FieldValue increment - search for "firestore increment" for how to do it.)
But... to be absolutely sure this will work even in cases when two users are signing up at the same time or other error-prone cases, make sure you use a "batch write".
A batch write means that both operations are done together, so both incrementing the "next_user_number" and creating the new user with the right "UserNo" number are going to be accurate. (search for "firesotre batch write" to learn more).
Okay, I solved it after drinking mugs of coffee. To all of the future visitors you can get the exact length of documents in that particular collection and then use then keyword and extract the value and store it in int variable. Then you can use extractedValue.size as the userNo.

Using the google sheets api after completing quickstart tutorial

I'm pretty new to Android app development so please forgive my naivety.
I'm currently trying to develop an app that can pull data from a google spreadsheet and write data to it.
I've completed the quickstart tutorial so my code is the same as that right now. It all works correctly.
My issue is I need to be able to read from my own spreadsheet and I don't really understand the code used so I'm struggling to know where to start.
I've looked at this to try and implement the authorisation in fewer steps which I thought might make the code easier to understand - but again my lack of knowledge means I don't know how this fits into all the methods I currently have.
I've looked at the developer documentation and tried to replace the code from the quickstart which retrieves data from the app with this:
ValueRange result = service.spreadsheets().values().get(spreadsheetId, range).execute();
int numRows = result.getValues() != null ? result.getValues().size() : 0;
System.out.printf("%d rows retrieved.", numRows);
But again this is different to the code I already have so doesn't fit in as the getDataFromApi() method requires a return statement. I've tried just changing the spreadsheetId to that of my own spreadsheet and changing the range value to the cells I need,
/**
* Fetch a list of names and majors of students in a sample spreadsheet:
* https://docs.google.com/spreadsheets/d/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/edit
* #return List of names and majors
* #throws IOException
*/
private List<String> getDataFromApi() throws IOException {
String spreadsheetId = "1-hL78Jm9-HijPx9UHthFxcXatkIhA2FR-AQ1lrCUbEg";
String range = "Go Mix 12 0506!B6:D";
List<String> results = new ArrayList<String>();
ValueRange response = this.mService.spreadsheets().values()
.get(spreadsheetId, range)
.execute();
List<List<Object>> values = response.getValues();
if (values != null) {
results.add("Name, Major");
for (List row : values) {
results.add(row.get(1) + ", " + row.get(3));
}
}
return results;
}
but I'm clearly going about it wrong because I get this error:
Invalid index 3, size is 3
I've also followed up to video 15 on this tutorial but it bypasses the need for any authorisation so didn't help.
Basically I've hit a brick wall and need someone to explain to me in simple terms how I can either work with the code from the quickstart tutorial to work with my own spreadsheet and fix that error! OR (more preferable to myself) explain how to do it myself from scratch. NB the spreadsheet cannot be public so I will need authorisation.
I hope that all makes sense!
You are asking to retrieve three columns of data (B through D), but then attempting to read a fourth column (rows.get(3)). In Java (and in most programming languages), lists and arrays are 0-based. So if you want the first item, you call list.get(0), and if you want the third you call list.get(2). This is evident based on the fact that the error message tells you your index of 3 is invalid since the list is size 3 -- you're attempting to read something beyond the end of the list.

How to know the size of ParseRelation (Parse/Android)?

Since a ParseRelation can have a huge number of row, is there a performant way to get the number of these rows?
For a big size I cannot simply query all the list and get the size.
In the Parse SDK documentation, it is not recommended to use query. countInBackground when there is more than 1000 objects.
So how can I query this size?
[EDIT] : potential issue if I increment dedicated counter for the ParseRelation:
Let's say on android I display a list of items with a button to click "addToRelation".
This button should be visible only if the relation is not already done. This means I need first to check on each item if they belong to the relation.
Then, when the user click on several buttons I call for each the backend method to add the relation and increments the counter. (This already make a lot of Parse request).
Now suppose because of some bad cache synchronization the button "addToRelation" is enabled while the relation already exist for this item.
If I call the method:
obj.add("relations",relation);
// increment the relations count by using the increment function
obj.increment("relationsCount");
obj.saveInBackground(. . .)
The method will not crash (I tested that if you add twice the same relation nothing happens)
but the counter will be incremented +1!
To avoid this I need to check twice on each item if they are not already in relation. This create too much redundant remote requests. So how to avoid this ?
Another issue may a
happen when I use saveEventuallyand the method silently fail; so if the User repeat several times the action, the counter will be incremented/decrimented several times in the local cache for the "saveEventually". If for any reason the method save succed finally the counter value will be wrong !!
what you can do is to create additional fields with the name of realtionCount (or something else according to your relation name). This field will be integer and then each time you create a new object you can increment this field using the increment option. So your code should look like the following:
// create new relation
final ParseObject relation = new ParseObject("your_relation_class_name");
relation.put("{RELATION_KEY_FIELD_NAME}","{RELATION_FIELD_VALUE}");
// add the relation to the parent object
obj.add("relations",relation);
// increment the relations count by using the increment function
obj.increment("relationsCount");
obj.saveInBackground(new SaveCallback() {
#Override
public void done(ParseException e) {
if (e == null){
// object saved!
}
}
});

Youtube-api v3: is it possible to get videos on a certain order?

i'm trying to get certain order from youtube videos on my Android application. I want to search, for example:
Fight 1
Fight 2
Fight 3
Fight 4
This is my search list:
YouTube.Search.List search = mYouTubeDataApi.search().list("id,snippet");
search.setKey(ApiKey.YOUTUBE_API_KEY);
search.setQ(mTitle + " " + mSearchQuery);
search.setType("video");
search.setFields("items(id/kind,id/videoId,snippet/title,snippet/thumbnails/default/url)");
search.setMaxResults(YOUTUBE_PLAYLIST_MAX_RESULTS);
searchResponse = search.execute();
So far, i've tried adding to my Search List:
search.setOrder("title");
and also i tried to sort manually the items this way
List<Video> items = videoListResponse.getItems();
Collections.sort(items, new Comparator<Video>() {
#Override
public int compare(Video video, Video t1) {
return video.getSnippet().getTitle().compareToIgnoreCase(t1.getSnippet().getTitle());
}
});
return new Pair(searchResponse.getNextPageToken(), items);
but i still don't get the results as i expect, and my sort is not good (i.e.: fight 1, fight 10, fight 11, fight 2, etc...)
Thanks in advance!
You can use Search: list using the HTTP request:
GET https://www.googleapis.com/youtube/v3/search
with the optional parameter order added to the query.
As discussed in the documentation,
The order parameter specifies the method that will be used to order resources in the API response.
Acceptable values are:
date – sorted in reverse chronological order based on the date they were created.
rating – sorted from highest to lowest rating.
relevance – sorted based on their relevance to the search query. This is the default value for this parameter.
title – sorted alphabetically by title.
videoCount – Channels are sorted in descending order of their number of uploaded videos.
viewCount – sorted from highest to lowest number of views. For live broadcasts, videos are sorted by number of concurrent viewers while the broadcasts are ongoing.

how to retrieve data from firebase in reverse order [duplicate]

I'm trying to test out Firebase to allow users to post comments using push. I want to display the data I retrieve with the following;
fbl.child('sell').limit(20).on("value", function(fbdata) {
// handle data display here
}
The problem is the data is returned in order of oldest to newest - I want it in reversed order. Can Firebase do this?
Since this answer was written, Firebase has added a feature that allows ordering by any child or by value. So there are now four ways to order data: by key, by value, by priority, or by the value of any named child. See this blog post that introduces the new ordering capabilities.
The basic approaches remain the same though:
1. Add a child property with the inverted timestamp and then order on that.
2. Read the children in ascending order and then invert them on the client.
Firebase supports retrieving child nodes of a collection in two ways:
by name
by priority
What you're getting now is by name, which happens to be chronological. That's no coincidence btw: when you push an item into a collection, the name is generated to ensure the children are ordered in this way. To quote the Firebase documentation for push:
The unique name generated by push() is prefixed with a client-generated timestamp so that the resulting list will be chronologically-sorted.
The Firebase guide on ordered data has this to say on the topic:
How Data is Ordered
By default, children at a Firebase node are sorted lexicographically by name. Using push() can generate child names that naturally sort chronologically, but many applications require their data to be sorted in other ways. Firebase lets developers specify the ordering of items in a list by specifying a custom priority for each item.
The simplest way to get the behavior you want is to also specify an always-decreasing priority when you add the item:
var ref = new Firebase('https://your.firebaseio.com/sell');
var item = ref.push();
item.setWithPriority(yourObject, 0 - Date.now());
Update
You'll also have to retrieve the children differently:
fbl.child('sell').startAt().limitToLast(20).on('child_added', function(fbdata) {
console.log(fbdata.exportVal());
})
In my test using on('child_added' ensures that the last few children added are returned in reverse chronological order. Using on('value' on the other hand, returns them in the order of their name.
Be sure to read the section "Reading ordered data", which explains the usage of the child_* events to retrieve (ordered) children.
A bin to demonstrate this: http://jsbin.com/nonawe/3/watch?js,console
Since firebase 2.0.x you can use limitLast() to achieve that:
fbl.child('sell').orderByValue().limitLast(20).on("value", function(fbdataSnapshot) {
// fbdataSnapshot is returned in the ascending order
// you will still need to order these 20 items in
// in a descending order
}
Here's a link to the announcement: More querying capabilities in Firebase
To augment Frank's answer, it's also possible to grab the most recent records--even if you haven't bothered to order them using priorities--by simply using endAt().limit(x) like this demo:
var fb = new Firebase(URL);
// listen for all changes and update
fb.endAt().limit(100).on('value', update);
// print the output of our array
function update(snap) {
var list = [];
snap.forEach(function(ss) {
var data = ss.val();
data['.priority'] = ss.getPriority();
data['.name'] = ss.name();
list.unshift(data);
});
// print/process the results...
}
Note that this is quite performant even up to perhaps a thousand records (assuming the payloads are small). For more robust usages, Frank's answer is authoritative and much more scalable.
This brute force can also be optimized to work with bigger data or more records by doing things like monitoring child_added/child_removed/child_moved events in lieu of value, and using a debounce to apply DOM updates in bulk instead of individually.
DOM updates, naturally, are a stinker regardless of the approach, once you get into the hundreds of elements, so the debounce approach (or a React.js solution, which is essentially an uber debounce) is a great tool to have.
There is really no way but seems we have the recyclerview we can have this
query=mCommentsReference.orderByChild("date_added");
query.keepSynced(true);
// Initialize Views
mRecyclerView = (RecyclerView) view.findViewById(R.id.recyclerView);
mManager = new LinearLayoutManager(getContext());
// mManager.setReverseLayout(false);
mManager.setReverseLayout(true);
mManager.setStackFromEnd(true);
mRecyclerView.setHasFixedSize(true);
mRecyclerView.setLayoutManager(mManager);
I have a date variable (long) and wanted to keep the newest items on top of the list. So what I did was:
Add a new long field 'dateInverse'
Add a new method called 'getDateInverse', which just returns: Long.MAX_VALUE - date;
Create my query with: .orderByChild("dateInverse")
Presto! :p
You are searching limitTolast(Int x) .This will give you the last "x" higher elements of your database (they are in ascending order) but they are the "x" higher elements
if you got in your database {10,300,150,240,2,24,220}
this method:
myFirebaseRef.orderByChild("highScore").limitToLast(4)
will retrive you : {150,220,240,300}
In Android there is a way to actually reverse the data in an Arraylist of objects through the Adapter. In my case I could not use the LayoutManager to reverse the results in descending order since I was using a horizontal Recyclerview to display the data. Setting the following parameters to the recyclerview messed up my UI experience:
llManager.setReverseLayout(true);
llManager.setStackFromEnd(true);
The only working way I found around this was through the BindViewHolder method of the RecyclerView adapter:
#Override
public void onBindViewHolder(final RecyclerView.ViewHolder holder, int position) {
final SuperPost superPost = superList.get(getItemCount() - position - 1);
}
Hope this answer will help all the devs out there who are struggling with this issue in Firebase.
Firebase: How to display a thread of items in reverse order with a limit for each request and an indicator for a "load more" button.
This will get the last 10 items of the list
FBRef.child("childName")
.limitToLast(loadMoreLimit) // loadMoreLimit = 10 for example
This will get the last 10 items. Grab the id of the last record in the list and save for the load more functionality. Next, convert the collection of objects into and an array and do a list.reverse().
LOAD MORE Functionality: The next call will do two things, it will get the next sequence of list items based on the reference id from the first request and give you an indicator if you need to display the "load more" button.
this.FBRef
.child("childName")
.endAt(null, lastThreadId) // Get this from the previous step
.limitToLast(loadMoreLimit+2)
You will need to strip the first and last item of this object collection. The first item is the reference to get this list. The last item is an indicator for the show more button.
I have a bunch of other logic that will keep everything clean. You will need to add this code only for the load more functionality.
list = snapObjectAsArray; // The list is an array from snapObject
lastItemId = key; // get the first key of the list
if (list.length < loadMoreLimit+1) {
lastItemId = false;
}
if (list.length > loadMoreLimit+1) {
list.pop();
}
if (list.length > loadMoreLimit) {
list.shift();
}
// Return the list.reverse() and lastItemId
// If lastItemId is an ID, it will be used for the next reference and a flag to show the "load more" button.
}
I'm using ReactFire for easy Firebase integration.
Basically, it helps me storing the datas into the component state, as an array. Then, all I have to use is the reverse() function (read more)
Here is how I achieve this :
import React, { Component, PropTypes } from 'react';
import ReactMixin from 'react-mixin';
import ReactFireMixin from 'reactfire';
import Firebase from '../../../utils/firebaseUtils'; // Firebase.initializeApp(config);
#ReactMixin.decorate(ReactFireMixin)
export default class Add extends Component {
constructor(args) {
super(args);
this.state = {
articles: []
};
}
componentWillMount() {
let ref = Firebase.database().ref('articles').orderByChild('insertDate').limitToLast(10);
this.bindAsArray(ref, 'articles'); // bind retrieved data to this.state.articles
}
render() {
return (
<div>
{
this.state.articles.reverse().map(function(article) {
return <div>{article.title}</div>
})
}
</div>
);
}
}
There is a better way. You should order by negative server timestamp. How to get negative server timestamp even offline? There is an hidden field which helps. Related snippet from documentation:
var offsetRef = new Firebase("https://<YOUR-FIREBASE-APP>.firebaseio.com/.info/serverTimeOffset");
offsetRef.on("value", function(snap) {
var offset = snap.val();
var estimatedServerTimeMs = new Date().getTime() + offset;
});
To add to Dave Vávra's answer, I use a negative timestamp as my sort_key like so
Setting
const timestamp = new Date().getTime();
const data = {
name: 'John Doe',
city: 'New York',
sort_key: timestamp * -1 // Gets the negative value of the timestamp
}
Getting
const ref = firebase.database().ref('business-images').child(id);
const query = ref.orderByChild('sort_key');
return $firebaseArray(query); // AngularFire function
This fetches all objects from newest to oldest. You can also $indexOn the sortKey to make it run even faster
I had this problem too, I found a very simple solution to this that doesn't involved manipulating the data in anyway. If you are rending the result to the DOM, in a list of some sort. You can use flexbox and setup a class to reverse the elements in their container.
.reverse {
display: flex;
flex-direction: column-reverse;
}
myarray.reverse(); or this.myitems = items.map(item => item).reverse();
I did this by prepend.
query.orderByChild('sell').limitToLast(4).on("value", function(snapshot){
snapshot.forEach(function (childSnapshot) {
// PREPEND
});
});
Someone has pointed out that there are 2 ways to do this:
Manipulate the data client-side
Make a query that will order the data
The easiest way that I have found to do this is to use option 1, but through a LinkedList. I just append each of the objects to the front of the stack. It is flexible enough to still allow the list to be used in a ListView or RecyclerView. This way even though they come in order oldest to newest, you can still view, or retrieve, newest to oldest.
You can add a column named orderColumn where you save time as
Long refrenceTime = "large future time";
Long currentTime = "currentTime";
Long order = refrenceTime - currentTime;
now save Long order in column named orderColumn and when you retrieve data
as orderBy(orderColumn) you will get what you need.
just use reverse() on the array , suppose if you are storing the values to an array items[] then do a this.items.reverse()
ref.subscribe(snapshots => {
this.loading.dismiss();
this.items = [];
snapshots.forEach(snapshot => {
this.items.push(snapshot);
});
**this.items.reverse();**
},
For me it was limitToLast that worked. I also found out that limitLast is NOT a function:)
const query = messagesRef.orderBy('createdAt', 'asc').limitToLast(25);
The above is what worked for me.
PRINT in reverse order
Let's think outside the box... If your information will be printed directly into user's screen (without any content that needs to be modified in a consecutive order, like a sum or something), simply print from bottom to top.
So, instead of inserting each new block of content to the end of the print space (A += B), add that block to the beginning (A = B+A).
If you'll include the elements as a consecutive ordered list, the DOM can put the numbers for you if you insert each element as a List Item (<li>) inside an Ordered Lists (<ol>).
This way you save space from your database, avoiding unnecesary reversed data.

Categories

Resources