for (String[] batch : snappyDB.allKeysIterator().byBatch(0))
What does 'size' param mean in byBatch() method?
Without using byBatch you will only have a KeyIterator which does not implement Iterator or Iterable so you can't use it in a loop.
byBatch(n) creates a BatchIterable which is Iterable and an Iterator. It basically just calls next(n) on the KeyIterator when you call next() on it. (Source)
KeyIterator#next(int max) seems to always attempt to fetch max elements from the Database. So I presume you will most likely have max elements in the batch array from your example on each iteration. So it doesn't make much sense to pass 0 as you're doing (not sure if that even works).
Also just reading the README from the GitHub repo reveals some documentation:
Iterable<String[]> byBatch(int size); // Get an iterable of key batch, each batch of maximum [size] keys.
According to SnappyDB documentation, found here:
Iterable<String[]> byBatch(int size);// Get an iterable of key batch, each batch of maximum [size] keys.
So size specifies the maximum number of keys for each batch. However, according to the documentation:
Please note that you should use the byBatch iterable to process all
keys only on large collections. On reasonably small collections, using
the array based APIs (findKeys and findKeysBetween) with the form for
(String key : db.findKeys("android")) is a lot more efficient.
Iterators should only be used to process large collections or for
collection paging view / access.
So make sure that you really need byBatch in your use case.
Related
I have one Android project where I need to query nearby items & these items should be sorted by time.
Basically, I need docs that are in 100KM. Sorted by time (Field).
So I have checked Firestore docs for this & I got solution (Add geoHash to docs & then query them by geoHasBounds) But there is an issue what if there are 1k docs in 100km then it will load all which is not good, so how can I limit those different queries & gets only 25-30 docs then next 25-30 docs ??
In short, this is what I need-
How can I query the latest 25 docs in 100KM radius & when the user scroll down the next 25 docs?
this is my code of query-
List<GeoQueryBounds> bounds = GeoFireUtils.getGeoHashQueryBounds(center, radiusInM);
final List<Task<QuerySnapshot>> tasks = new ArrayList<>();
for (GeoQueryBounds b : bounds) {
Query newQuery = itemQuery.orderBy("geoHash").startAt(b.startHash).endAt(b.endHash);
tasks.add(newQuery.get());
}
// Collect all the query results together into a single list
Tasks.whenAllComplete(tasks).........
What you are looking for is called pagination. I have answered a question here on Stackoverflow, where I have explained a recommended way in which you can paginate queries by combining query cursors with the "limit() method". I also recommend you take a look at this video for a better understanding.
If you are willing to try Paging 3 library, please see below an article that will help you achieve that.
How to paginate Firestore using Paging 3 on Android?
Edit:
The Tasks.whenAllComplete() method:
Returns a Task with a list of Tasks that completes successfully when all of the specified Tasks complete.
So you can then simply convert each object to a type of object that you need and paginate that list accordingly. Unfortunately, this implies getting all the objects in the first place. Otherwise, you can divide those queries into separate queries and treat them accordingly, by using separate paginantion.
In java a TreeMap is used for dictionaries where keys are kept sorted.
On android, an ArrayMap is used as a "memory-efficient" HashMap where (I infer) keys are kept sorted because
lookups require a binary search and adds and removes require inserting and deleting entries in the array
Memory Usage:
...this container is intended to better balance memory use...
It keeps its mappings in an array data structure -- an integer array of hash codes for each item, and an Object array of the key/value pairs. This allows it to avoid having to create an extra object for every entry put in to the map.
...it will shrink its array as items are removed from it...
We can reasonably conclude that ArrayMap is inappropriate for holding ~1000+ items:
For containers holding up to hundreds of items, the performance difference is not significant, less than 50%.
Note that this implementation is not intended to be appropriate for data structures that may contain large numbers of items.
What are additional costs/benefits of using one over the other generally?
is android's ArrayMap implementation thread-safe (thus slower)?
is there a way to affect the sorting in an ArrayMap?
are there similar data structures in Guava?
etc..
ArrayMap is not thread-safe, as stated in the documentation.
The sorting is always by ascending order of the keys hashcode. You can control the order by overriding hashCode() in the implementation of the key class. Beware of collisions.
ArrayMap allows accessing its keys and values directly by index like an ArrayList and can be iterated on very efficiently.
ArrayMap has the smallest memory consumption of the two. Both have a complexity of O(log n) for retrieval. Insertions and deletions are in theory a bit slower for ArrayMap because they are similar to inserting or deleting an element in the middle of an ArrayList: technically it's O(n) but it's using a single call to System.arrayCopy() to move all values at once very quickly. So if you need to modify the map very often, ArrayMap is probably not the best choice but for all other use cases it's very capable.
According to the previous image
Is it possible to get every child of that matches the value of "11000" in that is inside the array ?
(there might be multiple entries for )
It depends on where you start. From /planes/PaMé7800_..._785/directiones it is definitely possible. But from /planes it is not possible, since you can only query values at a known path under each child.
Essentially your current structure allows you to efficiently find the directiones for each plane, but is does not allow you to efficiently find the planes for a directione. If you want to allow the latter, consider adding an additional data structure for it. For example:
directionesStCPToPlanes
dir11000
PaMé7800_..._785: true
With this additional data structure, you can also look up the inverse relation.
This type of double data storage is quite common and is known as denormalizing the data. For more on this, see:
Many-to-many using Firebase
Many to Many relationship in Firebase
Firebase Query Double Nested
I am developing an app with Firebase, it is a game in which certain scores are shared between players and can grow without a limit. I tried storing them as a String, but then I could not order them with orderByChild for the leaderboard. How can I handle this problem?
You could store the number as a linked list, with each node in the list representing each digit. Be careful with the order; putting the last number (the ones digit) in the list first, makes math with the linked list easier, while the other direction makes it easier to return the nodes in the list to a number on the screen.
To store integers which are big in size java provides a BigInteger Class to handle those numbers. I will suggeat you to use that first read the concept and then try to findout what you exactly want!
Check this one BigInteger
I am wondering how would I be able to run a SQLite order by in this manner
select * from contacts order by jarowinkler(contacts.name,'john smith');
I know Android has a bottleneck with user defined functions, do I have an alternative?
Step #1: Do the query minus the ORDER BY portion
Step #2: Create a CursorWrapper that wraps your Cursor, calculates the Jaro-Winkler distance for each position, sorts the positions, then uses the sorted positions when overriding all methods that require a position (e.g., moveToPosition(), moveToNext()).
Pre calculate string lengths and add them into separate column. Then sort entired table by that that length. Add indexes (if you can). Then add extra filters for example you don't want to compare "Srivastava Brahmaputra" to "John Smith". The length are out of wack by way too much so exclude these kind of comparison by length as a percentage of the total length. So if your word is 10 characters compare it only to words with 10+-2 or 10+-3 characters.
This way you will significantly reduce the number of times this algorithm needs to run.
Typically in the vocalbulary of 100 000 entries such filters reduce the number of comparisons to about 300. Unless you are doing a full blown record linkage and then I would wonder why use Android for that. You would still need to apply probabilistic methods for that and calculate scores and this is not a job for Android (at least not for now).
Also in MS SQL Server Jaro Winkler string distance wrapped into CLR function perform much better, since SQL Server doesn't supprt arays natively and much of the processing is around arrays. So implementation in T-SQL add too much overhead, but SQL-CLR works extremely fast.