I'm currently writing an Android app that will need to do things such as :
Retrieving sensors data such as
ActivityRecognition data, say every 20-30 seconds
Retrieving GPS data from time to time (e.g. when activity recognition sends that user is using its bike or car), so I'd say a few times per day max for average user. Frequency of GPS data retrieving could be every 5-10 seconds for example.
Of course, this data should be stored somewhere to be, later on, analysed by my app. The analyse part is not the problem here as I do not need a real-time calculation of any kind, so my actual concern is how to store the data efficiently.
So if we consider an average user that will generate about 5000 sensors data + 5000 GPS data :
How to best store this data ? Database ? 1 file per day ? I'd say database for performance issue and simplicity of use, but I'm not sure it's very good practice to open/close a database connection every 10/20 seconds to add just one line of data. Also, a journaled file (one per day) could be a good idea but I think this is pretty bad for a performance point of view, even using serialization ?
Will storing these 10000 data degrade battery life much more than just retrieving sensors (ActivityRecognition, GPS) data without storage ? I mean, it seems to me that it will be a bit overconsuming, but in the same time GPS is already using so much battery...
Is there another way to do that ?
Also thought of in-memory storage then every few minutes it could be put in hard storage (SQLite, files), but I'm not sure this is a good idea in terms of safely keeping the data...
Thanks in advance
Compared to the GPS battery consumption, the reading and writing in the database will consume almost nothing (it is flash storage, after all), so no worries there. The database would be the best option to store this data in my opinion and I don't see a problem in creating a single entry every 10 or 20 seconds.
Also thought of in-memory storage then every few minutes it could be
put in hard storage (SQLite, files), but I'm not sure this is a good
idea in terms of safely keeping the data...
This is a very good idea.
I have it done this way, and most systems that deal with files do it that way (called a buffer).
If your app crashes due a bug, some data will be lost, depending on the buffer size.
In all other cases (device will shut down, user terminates app) you have time to write (flush) the buffer.
Just an short additional note.
Other than shutting down peripherals, not really an option in your case, maximize the length of time the processor is inactive. (This is not the same as minimizing the length of time the process is active.) This allows the processor to drop into lower sleep states (called C-states). The deeper the sleep state, the more power savings.
In a general sense, this means
no polling; use interrupts instead,
if you need to periodically wake up to see if anything needs to
be done, make sure your interrupt period is the maximum allowable.
(Contrary to current practice, waking up every 10 ms does not
improve your responsiveness when the average event happens every 500
ms.)
This also applies to peripherals, as they drop into sleep states also when not active (Called D-states).
Minimize the number of cloud accesses you do.
Maximize the length of time between cloud accesses.
Related
I'm trying to figure out whether or not to use sqlite/spatialite on Android with only 29k rows. I just need to find the nearest locations from the user everytime they move outside 100 meters which could be about every 10 minutes. I feel like running querying a spatial database as opposed to looping a collection and calculating distances could be overkill. When is it overkill to use a database in this case?
It's not overkill. It's actually probably required for you to store that data somewhere if you don't want your users to hate the app.
Running a constant process on your phone constantly consumes the phone's system resources. Consuming resources kills phone batteries. People don't like apps that kill their phone batteries. Repeatedly executing queries for that many records to a web service endpoint doesn't seem like the best idea either, since it would eat up your users' data plans. Users tend not to like that either.
29K records in active memory is probably more phone resources than you should be thinking about thinking about consuming unless you are doing something very, very special.
If your data doesn't change though, a database isn't the only way to store and query your data. There might a better solution somewhere in the middle, but I would not expect good results from consuming an unnecessary allotment of users' data plan and/or battery life.
We are building an application which requires good amount of data exchanges between different users. We are using SQLite to store the info and Rest api to exchange data with server.
To ensure high performance and less CPU /memory hogging but to also maintain good user experience we need following suggestions:
1 We tried running sync at frequency of 30 seconds but it hogs resources.Is there any client side framework which can be used to sync sqlite with MySQL or we have to only plan all possible events for same
2 How does applications like Gmail /twitter work- do they sync only on demand or keep on syncing in background. I feel it is on demand but not sure.
3 Notifications should be server side or client side (based on updates in sqlite). In whatsapp I observed it is client side only. If I do not click a received message I keep on getting the notification about same
4 IF we keep notifications server side and sync on demand basis. then on clicking a new notification when app will open up at that time should we make a sync call
Need an expert opinion that such applications should be designed to manage sync and notifications in such a way that it does not hogs resources and also gives online kind of experience to customer
Your question is pretty broad, but I'll at least give you a direction to start.
I've run local databases in iOS and Android that are over 100 MB without incident. SQLite should never be your problem, if you use it correctly. Even with 100,000 rows of data, it is fast and efficient. Where people get into trouble is by not properly indexing the data or over-normalizing the data. A quick search can tell you how to use indexes to optimize your queries, so I won't go into that any further. Over-normalization seems to be not fully understood, so I'll go into a bit more depth on it.
When designing a server database, it is important to minimize the amount of duplicate data. This often is done by breaking up a single record into multiple tables and using foreign keys. On a 1 GB database, this type of data normalization may save 20%, which is fairly significant. This gain in storage comes at the cost of perform. Sequential lookups and joins are frequently necessary to get complete data. On a server, there are plenty of CPU cycles and memory, and no one really notices if a request takes an extra millisecond or two.
A mobile app is not a full database server. The user is actively staring at the screen waiting for the app to respond. Additionally, the CPU and memory available are minimal, which makes that delay take even longer. To add insult to injury, mobile databases are only a small portion the size of a server database and duplicate data is already pretty minimal. The same data normalization that may have saved 200 MB (20% of 1 GB) server size, may now only save 5% of 10 MB, or 500 KB. That minor gain is not worth the effort or performance hit.
When you sync, you do not need a full data set each time. You only need to get data that has changed since the last sync. In many cases, that will be no change at all. You should find a way to identify what the device has on it now and only get the changes.
It is important that the UI does not stall waiting for the network request. All network activity should be done on a background thread and notify the UI to refresh once the sync completes.
Lastly, I'll mention that SQLite is NOT thread safe. It is important to limit concurrency with your database access.
I'm developing an application that measure the data traffic recived through mobile data interface (no wifi) from all processes. Additionally this counter have to be related to a date range, i.e. betheen March 1 and April 1.
I had read about TrafficStats class, but in the documentation doesn't mentioned any about from when are the stats.
This is my first question and I really appreciate your help.
Thanks
I had read about TrafficStats class, but in the documentation doesn't mentioned any about from when are the stats.
"From when" should not matter to you. Take a reading at a point in time, take another reading at a later point in time, and the difference between the two is the bandwidth consumed between those two points in time.
Additionally this counter have to be related to a date range, i.e. betheen March 1 and April 1.
You would need to handle this yourself, most likely, checking for the amount of bandwidth consumption every so often (e.g., every 4 hours via AlarmManager), storing the results in a database, and then using that information to determine the bandwidth consumed over extended periods of time.
Assume we have a lot of android devices that trying to submit real-time data(a time stamp and some numbers) to an django application.
What we need to do is get the mean value of those data that being submitted per second and constantly update an android device.
How can I achieve these goal?
Your best bet is to memcache the data then store it in the datastore. Your mention of "mean value of all these data per second" doesn't make much sense to me; unless you have a lot of users or are generating a battery-draining amount of data, the mean value per second will probably just be whatever data happened to be submitted that second. Also, "realtime" may have deceived you; it will take time for data to be transmitted to the server and back down to the phone. Especially on EDGE and slower data networks, it could cause a lag of a few seconds.
It may be practical to do per-minute mean calculations; to that end, I would simply do a memcache tuple of the number of values received that minute and the mean they represent. Then when you get a new value, recalculate the mean (multiply the mean by the number of values [not including the increment] and add the new value, then divide by the new number of values). Store this in memcache, and move on. It's unlikely your data will get evicted within that minute if you're using memcache wisely, and if it is evicted, you lost at most a minute of data. If that's unacceptable, back it up to the datastore.
in numerous places, it is mentioned that app widgets should not get updated often, to minimize power consumption.
But, let's consider that an application is doing something important (such as audio recording) for a short period of time, say 30min.
In this case, is it acceptable to update the widget every second from a service?
How can it be that this would consume so much power?
Please consider that this is different from a widget which would update very often during the whole day.
And in my case, such frequent updates would be meant to allow the user to monitor that the operation is being performed continuously and correctly. It's not for fancy visual effects and such.
I don't see a problem with doing this; if you're keeping the phone awake with a long-running background task (audio recording in this case), then the phone can't sleep anyway. I wouldn't expect updating the widget to have a significant impact on battery use in this case.
Of course, the best thing to do is to run some tests on a real device, and compare battery use with and without widget updates, and make widget update interval a user preference.
The main reason widgets shouldn't update constantly is because of the battery consumption used to get the latest data from a server. Since the device will be on anyway, and the update is local to your data, it shouldn't have an impact that is noticeable.
If you were hitting a server instead of local data every second for that long, you would notice a significant draw on the battery.