I know that pagination is a well discussed topic here but... Is there a way to fetch some amount of nodes(something like 5 at a time) and know when the end has been reached? Firebase Database does not have a way of knowing the total number of nodes. So how will I achieve this?
The only way to know if you have all the data at a location is to perform a single 'value' type query on it. Even then, the data may change after that listener is invoked, so all you really have is a snapshot at a particular moment in tim.
For child events, there is no way to know there is no more data after the current event. In fact, child events may keep firing as more data is added, as long as the listener is still added at that location.
This is why pagination is so hard. There is no generalized way to know if you have a full page of data at any given point, or if there's another page.
Related
I am searching for a method to get notified on my android phone, if the first occurrence of a element of a certain class has a value larger than x on a multiple times a minute updated website.
Though this website is not subject to constant updates, here is an example using amazons "100 Books to Read in a Lifetime":
Currently the first book on the site is sold for $5.99. I would like to be notified on my phone, if it is offered for 5.50 or less. Therefore I would need to implement a method to retrieve the price inside the first occurrence of <span class="a-size-base a-color-price acs_product-price__buying">($5.99</span>), check if the number substring is smaller or equal to 5.5 and notify me on my phone.
Keep in mind, amazon is just an example, I am in search for a general and customizable solution on an often changing website.
What would be a good way of implementing such a notification system?
P.S.: This is my first post on stackoverflow. If you think I should add something or rephrase my question, please let me know!
You can use JSOUP for parsing the html code and then using the selector to get the value of the item, And for the changes in price you have to schedule a job which will run in background and check for updates in a given interval of time, Save the price in device locally on every request and while doing the next request compare the value and if there is any change in value show notification.
I'm working on an app using Firebase and Geofire. On running the Geo query at the current location, let's say I receive 10 keys in the OnKeyEntered override method. Each of these keys is essentially a user node in Firebase. I need to listen to each of the user in the query area for any data change so that I can show updates on the map in realtime.
Currently, I'm adding a ValueEventListener for every key entered but I'm not sure if starting so many listeners at the same time is good idea. The users in the query area can potentially be more than 50. That means I could have 50 open listeners!
Is there a better way to go about it? I was trying to figure out a firebase query to filter on only the geo query keys but was unsuccessful.
Any help would be great!
Listeners are not computationally expensive, unless you have one that's going to be triggered very frequently because the data it's listening to is changing often.
Don't fall into the trap of optimizing your code before you actually observe a need to optimize it. When you see that performance is poor, that's the time to make optimizations. If you need a bunch of listeners to get your work done, go ahead and do that. Just know what your practical upper bound it, and be sure to test that upper bound for problems.
I have an app where I have up to a few thousand entries in a firebase table.
Now someone is connecting to this table and I need to count and sum these items up.
-itemlist
|
--1
|-x:2
--2
|-x:4
and so on. A user can connect at any time (meaning he never saw this table before) and another user can change some value at any given time or add values.
Now if I try and add a ValueEventListener to "itemlist" it works if there is no new data but as soon as I connect to a new database with 1000 entries or so the listener calls for every one of these 1000 entries.
I tried with onChildlistener but this calls the on ChildItem added for every item extra on the first time.
This takes over 2 minutes sometimes.
Is there a way to get the whole database once, calculate it and then only listen for changes?
The Firebase Database has no server-side aggregation operators, since they would inherently be at odds with the scalability and realtime nature of the database.
That leaves you with two options to calculate aggregates:
perform them client side as you already suggested
keep a running aggregate that you update with every change
Performing the aggregation client-side is a good option if the client already needs to display the data anyway. For example: if you're showing the list of users in a specific chat room, you can easily count the number of users client-side from the same data.
But if you don't need the data client-side, then just downloading it to aggregate it is wasteful and will hurt scalability of your app. In such cases, an option is to keep an extra node that keeps the aggregate and update it with every relevant write operation. For example: if you want to show how many users have registered with your app, you could keep a global /user_count that you update whenever a user registers/unregisters. For this update, you'd typically use a transaction.
I have SQLite table contains 1.5M lines, the table contains <lat,lng, info>, what i want to do is, for every time I receive data "lat,lng" from the GPS, I should access the table and display the corresponding info.
my question is, is this operation will be exhaustive and time consuming? if yes, how can I achieve better data retrieval fro mthe table every time i receive GPS data?
You should use Spatialite, it is a Sqlite extension to use with geographical data, it allows you to create spatial indexes and then query data which is included in a rectangle, for example around x meters from your gps coordinates.
You do no want to search for an exact location (GPS is not accurate enough for that), but for nearby entries.
Two-dimensional interval searches are not efficient when using 'normal' indexes.
You would need to use something like an R-tree index, which is not available on Android by default.
If your table never changes, you could prepare the R-tree on another machine, and in the app, search for entries manually.
I don't know exactly what your data looks like but the first thing I'd do (given that GPS position may not evolve much) is to first create a temp table with a smaller range of coordinates so you can probably reduce the 1.5M records into some thousands only. Then everything should be faster.
I didn't know about the R-Tree #CL spoke about but indeed they look very promising if only you could manage to use them under android.
Well if your use case is not as other suggests - means that you are looking for an exact position rather then an area, Then all you can do is make sure you add an index on (lat,lng) and pray for the best...
CREATE INDEX index_lat_lng ON MyTable(lat,lng);
Your question is far to woolly.
what i want to do is, for every time I receive data "lat,lng" from the GPS, I should access the table and display the corresponding info.
You need to define how often you receive data.
Will it be exhaustive and time consuming? Maybe, maybe not, without knowing how often you receive data.
Given the above, have you considered and assuming that the GPS co-ordinates are not coming in from all over the shop, creating an in memory database snapshot of the area in and around where your latitude and longitudes are currently coming from and using that until such time as they stray from the current snapshot area. At that point you will need to write out the existing data create a new snapshot for the new co-ordinates. The snapshot areas could be defined by degrees, minutes or seconds depending how fast the thing, that I suppose you are tracking, is expected to move.
You can try just with a command like:
SELECT info FROM Table WHERE lat = 'lat' AND lng = 'long';
Doesn't that just work well enough for your purpose?
My current situation:
I might have the complete wrong approach on this, so please tell me if I should do this differently.
I send an event from my Android app to track DB load times.
Event Category: 'DB'
Event Label: 'DBLoadTime'
Event Value: 2356 (Or however long the DB took to load in
milliseconds)
In Analytics I would like to get a table view of all the different load times, listed under each other. From the analytics documentation I could see that they seem to accumulate all the event values as they are received. This means I can only get the total value, or an average value. however, they also use a similar example to what I have, where they use the event value to store 'doanload time' for a video.
My question
Is this at all possible, or are values from similar event labels accumulated by default, and there is no way to get them listed as separate values.
If it is not possible, how can I get this info? Maybe by sending the load time as an event label?
Why I need to know:
Highest value does not really serve me any purpose, as there might be one guy with one really slow device experiencing long load times, where the 2nd longest load time is already half of that.
Average value does not tell me much either, as there are many people trying the app out, with just one or two records in the DB, which will always load fast.
It seems like what you want is a histogram to see the distribution of event values for that eventLabel and eventCategory. We've wanted to see the same sort of thing -- because seeing the actual distribution (or even quartiles) is much more helpful than just seeing a mean and a sum.
Unfortunately (d'oh!) GA currently does not support reporting histograms for eventValue. I've spoken with several of the GA developers about this and no one has said that this is going to happen sometime soon (d'oh!).
Thankfully, we've found a workaround that the GA developers have said is ok -- i.e. not in violation of a terms of service.
Instead of reporting the number of milliseconds that the device took to load, you can include this number as a string in EITHER a custom-dimension, or--in this case-- as the string in eventAction. This will allow you to report how many hits there were of each ms number.
If you are using the GA reporting API -- or the very helpful explorer -- then you can re-bin as you like, from which you can generate histograms to see the distribution.