I know this is a somewhat abstract question and that an answer depends on the device, network, user preferences and so forth. Nevertheless I'm really in need of some educated opinion with regards to how "excessive" I can allow my polling to be. Let's say one of my typical polling requests consists of an empty request body (a simple GET) and a couple of hundred kilobytes in response, would a service that polls for this every 4 hours be in the category of excessive?
I'm in the unfortunate situation of not being able to use C2DM, so please no answers suggesting this.
Let's say one of my typical polling requests consists of an empty request body (a simple GET) and a couple of hundred kilobytes in response, would a service that polls for this every 4 hours be in the category of excessive?
I wouldn't think so. Every 4 minutes would be unpleasant from a battery standpoint. Ideally (IMHO), make the polling period be configurable via a preference.
Also, you might wish to watch Reto Meier's Google I|O 2012 presentation, particularly the "Efficiency" section, which gets into lots of good low-level battery consumption stuff.
Also also, with that payload size, IMHO battery is less of an issue than the bandwidth cost for those on pay-as-you-go plans. You're talking ~1MB/day, ~30MB/month. That's probably not unreasonable, but it would be nice if the user understood why you're downloading all that data (vs., say, some sort of diff or delta approach), and it would be nice if the user could throttle your behavior within your app. Otherwise, the user might elect to throttle you from Settings on Android 4.x+ devices.
Related
Short skippable intro:
I work at a rehabilitation hospital that's a couple kilometers from where I live. The hospital pays a bus service that picks the employees up at certain fixed locations. There's this bus driver, that picks us at 7:00. The guy is FREAKISHLY PUNCTUAL. I mean, this guy has to be in the tenth of a second order. The clock turns from 6:59 to 7:00 EXACTLY when he opens the bus door. And I was thinking of recording his punctuality for like 30 days and make a nice Excel spreadsheet for him. Error propagation and everything. He might even get a raise, who knows?
I'll make a simple app to save time in milliseconds, that seems easy enough, there're like half a dozen solutions for that here in Stackoverflow.
So, to the question:
I hit a button on a widget. It gets a time in ms. It saves it on a file (.txt, .csv, whatever)
How would you estimate the mean error of an NTP synchronized Android phone? What's the most precise and exact way to save a timestamp?
Thanks in advance
First thing that comes to mind is that time is relative, in the Einstein kind of way :) So if your reference is an Android phone synchronized with NTP, what is the driver's time reference? What if synchronization or timing somehow gets off, delayed on your phone and you have the impression that he's late one day, when in fact he's precisely on time relative to his reference?
But for the sake of solving the problem i think you can start by assuming he has the same time reference with you: some place in a network. I don't know the details of Android synchronization via time protocols, but i do know i built a so called Network Synchronization API. In your case i think my Java API might be of help. It tries to make a request for the number of milliseconds since the Unix epoch and then reads the response. The feature is that it gives you a framework on which you can estimate your true time compared to the server's by trying to discard network lag.
One thing to bear in mind: the solution above makes a request to my site (a.k.a. GoDaddy servers) which is probably different than the network location Android phones sync with. Feel free to change the request location.
Second thing to bear in mind: this rabbit hole goes deep :) No measurement can be perfect. You can try to achieve human / reasonable precision but it can never be perfect (e.g. there are other unknowns: what if there's a lag in your Android CPU just as you press the button, or what if your own reaction takes a few milliseconds? The moment you press the button is not the same as the moment the doors actually open and my impression is they're at least a few millis apart)
I'm currently writing an Android app that will need to do things such as :
Retrieving sensors data such as
ActivityRecognition data, say every 20-30 seconds
Retrieving GPS data from time to time (e.g. when activity recognition sends that user is using its bike or car), so I'd say a few times per day max for average user. Frequency of GPS data retrieving could be every 5-10 seconds for example.
Of course, this data should be stored somewhere to be, later on, analysed by my app. The analyse part is not the problem here as I do not need a real-time calculation of any kind, so my actual concern is how to store the data efficiently.
So if we consider an average user that will generate about 5000 sensors data + 5000 GPS data :
How to best store this data ? Database ? 1 file per day ? I'd say database for performance issue and simplicity of use, but I'm not sure it's very good practice to open/close a database connection every 10/20 seconds to add just one line of data. Also, a journaled file (one per day) could be a good idea but I think this is pretty bad for a performance point of view, even using serialization ?
Will storing these 10000 data degrade battery life much more than just retrieving sensors (ActivityRecognition, GPS) data without storage ? I mean, it seems to me that it will be a bit overconsuming, but in the same time GPS is already using so much battery...
Is there another way to do that ?
Also thought of in-memory storage then every few minutes it could be put in hard storage (SQLite, files), but I'm not sure this is a good idea in terms of safely keeping the data...
Thanks in advance
Compared to the GPS battery consumption, the reading and writing in the database will consume almost nothing (it is flash storage, after all), so no worries there. The database would be the best option to store this data in my opinion and I don't see a problem in creating a single entry every 10 or 20 seconds.
Also thought of in-memory storage then every few minutes it could be
put in hard storage (SQLite, files), but I'm not sure this is a good
idea in terms of safely keeping the data...
This is a very good idea.
I have it done this way, and most systems that deal with files do it that way (called a buffer).
If your app crashes due a bug, some data will be lost, depending on the buffer size.
In all other cases (device will shut down, user terminates app) you have time to write (flush) the buffer.
Just an short additional note.
Other than shutting down peripherals, not really an option in your case, maximize the length of time the processor is inactive. (This is not the same as minimizing the length of time the process is active.) This allows the processor to drop into lower sleep states (called C-states). The deeper the sleep state, the more power savings.
In a general sense, this means
no polling; use interrupts instead,
if you need to periodically wake up to see if anything needs to
be done, make sure your interrupt period is the maximum allowable.
(Contrary to current practice, waking up every 10 ms does not
improve your responsiveness when the average event happens every 500
ms.)
This also applies to peripherals, as they drop into sleep states also when not active (Called D-states).
Minimize the number of cloud accesses you do.
Maximize the length of time between cloud accesses.
I need my app to receive notifications, but my boss does not want to rely on Google Cloud Messaging so I will use httpPost in a background service instead for periodically check for new messages.
My question is: will that be too demanding for the battery and data consumption? Do you know a better option?
Thank you.
Edit:
This is an app for a delivery store. The messaging starts when you ask for something and ends when you receive the item. The message query will be every minute for about 10 or 20 minutes.
In general, this would not seem like a good idea. The scenario you describe seems like a perfect fit for GCM. I would first try to convince your boss to reconsider. :)
"Polling" a server, even for a brief connection which transmits hardly any content, means the device must be woken (from a low power state) and the radio turned on, and it will stay on for a significant time. The impact on data consumption will probably be insignificant, but battery usage will be a concern.
In case it cannot be avoided, you should check Minimizing the Effect of Regular Updates section of the Efficient Downloads referenced by #CommonsWare in the comments.
Also, Reto Meier gave an excellent talk on this subject at the 2012 Google I/O. His series on Efficient Data Transfers is also very informative. Hope it's helpful.
I'm not sure if this question belongs here, as it is solely based on theory, however I think this fits best in this stackexchange compared to the rest.
I have 500,000 taxis with Android 4 computers inside them. Everyday, after one person or party makes a trip, the computer sends the information about the trip to the Node.js server. There are roughly 35 trips a day, so that means 500,000 taxis * 35 trips = 17,500,000 reports sent a day to the Node.js server. Also, each report has roughly 4000 characters in it, sized around 5KB.
The report that the taxi computers send to the node.js server is just an http post. Node.js will then send back a confirmation to the taxi. If the taxi does not receive the confirmation for report A in an allotted amount of time, it will resend report A.
The node.js server simply receives the report. Sends the confirmation back to the taxi. And then sends the full report to the MongoDB.
One potential problem : Taxi 1 sends report A to node.js. Node.js does not respond within the allotted time, so Taxi 1 resends report A to node.js. Node.js eventually processes everything and sends report A twice to MongoDB.
Thus MongoDB is in charge of checking whether or not it received multiple of the same reports. Then MongoDB inserts the data.
I actually have a couple of questions. Is this too much for NodeJS to handle (I don't think so, but it could be a problem)? Is this too much for MongoDB to handle? I feel like checking for duplicate reports may severely hinder the performance.
How can I make this whole system more efficient? What should I alter or add?
First potential problem is easy to overcome. Calculate a hash of the trip and store them in mongo. Put the key on that field and then compare every next document if the same hash exists. This way checking for duplicate will be extremely easy and really fast. Keep in mind that this document should not have something like time of sending in it.
Second problem: 17,500,000/day is 196/second nontheless sound scary but in reality it is not so much for decent server and for sure is not a problem for Mongodb.
It is hard to tell how to make it more efficient and I highly doubt you should think about it now. Give it a try, do something, check what is not working efficiently and come back with specific questions.
P.S. not to answer all this in the comments. You have to understand that the question is extremely vague. No one knows what do you mean by trip document and how big is it. It can be 1kb, It may be 10Mb, it can be 100Mb (which is bigger then 16 Mb mongodb limit). No one knows. When I told that 196 documents/sec is not a problem, I did not said that exactly this amount of documents is the maximum cap, so even if it will be 2, 3 times more it is still sounds feasible.
You have to try it yourself. Take avarage amazon instance and see how many YOUR documents (create documents which are close to your size and structure) it can save per second. If it can not handle it, try to see how much it can, or can amazon big instance handle it.
I gave you a rough estimate that this is possible, and I have no idea that you want to "include admins using MongoDB, to update, select,". Have you told this in your question?
I'm developing an application that measure the data traffic recived through mobile data interface (no wifi) from all processes. Additionally this counter have to be related to a date range, i.e. betheen March 1 and April 1.
I had read about TrafficStats class, but in the documentation doesn't mentioned any about from when are the stats.
This is my first question and I really appreciate your help.
Thanks
I had read about TrafficStats class, but in the documentation doesn't mentioned any about from when are the stats.
"From when" should not matter to you. Take a reading at a point in time, take another reading at a later point in time, and the difference between the two is the bandwidth consumed between those two points in time.
Additionally this counter have to be related to a date range, i.e. betheen March 1 and April 1.
You would need to handle this yourself, most likely, checking for the amount of bandwidth consumption every so often (e.g., every 4 hours via AlarmManager), storing the results in a database, and then using that information to determine the bandwidth consumed over extended periods of time.