What is the best way of recording rapidly updating data in Android?
In my specific example I am trying to record various data tidbits whenever (acceleration) onSensorChanged fires, which is often. It's fine for some quick processing and displaying, but anything more (plotting or recording data) makes it rather slow. It's worth noting that I'm storing it in a variable and not to file.
You are advised to do as little as possible in an onSensorChanged, so I tried the following:
In an IntentService: Fetch the rest of the data you want to record and store all of it in a variable. Result: Even though an intent service executes in its own thread, only one can run at a time. Data thus seemed to come in quicker than it could be processed and would end up with a long queue of data to be processed, giving a long and constant stream of GC_FOR_ALLOC. Ultimately making it a bad choice
In an AsyncTask fetch the extra data you want to record and then start an IntentService for recording it to a variable.
Result: Pretty much the same as above.
Overall both of these methods were actually performing significantly worse than just doing the extra data fetching and recording in the onChanged, as well as consuming significantly more power. I haven't been able to find any concrete advice on rapid data storage. Would a buffer or cache perhaps be more efficient?
After some debugging and attempts at using different data structures for storing the rapidly updating data, it turns out that using the IntentService works reasonably well. Changing from an ArrayList with O(n) for adding new elements, to a custom LinkedList that just appended new values to the end with O(1) made a massive difference. As the LL is able to process the data so much quicker, the problem of a huge queue of pending values to record basically disappears.
Related
I am currently developing an RPG game for Android devices and have just implemented a custom method of serialisation I use for saving the player's progress. The saving process can take up to half a second which is enough time for a crash to occur (either caused by the device (i.e. low battery power off), the user (killing the app) or a poorly written firmware/ROM (kernel panics) etc).
While saving the player's data, the old player data is overwritten. This means if a crash was to occur, and if the saving process were to be cancelled/interrupted as a result, the player's data would be lost. This is obviously not ideal and in the future, the game will be saving a lot more data and the save time will be much longer. This will increase the chance of a crash occurring during the save process.
I cannot reduce the save time as I am writing the minimal data the game requires to be able to resume after the app has been restarted.
What foolproof measures, if any, can I take to prevent such data corruption/loss from happening?
You can save your data in a temporary set of files and moving/renaming them when the process is complete, then deleting the previous save files.
If you're not confident with the renaming process, you can add these constraints :
ensure that data is consistent with a checksum
always try to resume from the last consistent saved state, depending on a rule of your own (name of the file, ...)
Another idea would be to cut into pieces your data in order to isolate state that do not change.
If save time is really long, you can try to use remaining CPU time during the game to pre-save parts of the current state that won't probably change (using a lower priority Thread, for instance).
You can save data to a SQLiteDatabase. If changes to the save data fails or is interrupted, the database will automatically be rolled back to a previous known state.
For additional security if you need to perform multiple updates atomically, put all your changes into a transaction. If any of the changes fail, the entire transaction will be rolled back to the pre-transaction state.
For more information about using SQLite, see documentation here. For easier manipulation of your save data in the event you want to share it with other apps or sync it to a backup server, consider interacting with your data via a ContentProvider.
in the App i am developing, i have a a layout with 6 TextViews, these textviews wil display data received at very high rate from a server. so far, i set the
values i receive to the textviews, but when i run the App. the data that textview displays are changing very slow because of the values sent at very high rate.
for an example, the server send almost 100 line in one second, and the datat in this 100 lines should be displayed by the TextViews..but because, as you noticed,
the high rate at which the dtat is being sent, the TextViews cannot show all the data by the time they sent.
What i want to do is, to be able to display these frequent data using the textViews but without lagging or delay in display the data, something like how do you see
the sensor data displayedin a textView.
given the above issue, does using the handler might help?
given the above issue, does using the handler might help?
To answer your question, this is what the docs say in regards to Handler:
There are two main uses for a Handler:
(1) to schedule messages and runnables to be executed as some point in the future; and
(2) to enqueue an action to be performed on a different thread than your own.
Honnestly, I don't see how the Handler could help you achieve what you are looking for.
Do you make your http request in a separate thread? (if not, then you should)
Also, I am not sure I understood the connection between the TextViews and the very high rate from server.
The handler is only a means of enqueuing a Runnable to be run in the Main Thread. You shouldn't plug the output of the server directly to the GUI, because you might be clogging up the main thread.
If you are downloading at high frequency, I'd suggest storing the downloaded data somewhere in memory (or persistence if needed), and have the GUI read this intermediate data at a slower pace.
I'm currently working on an Android application that requires reading from call history and text history. And for further analysis, I've extracted these the huge amount of entries from the corresponding content provider, created and inserted all of them to a SQLite database.
But the problem I've encountered is when this is running on a phone that has been used for years (meaning there's an enormous amount of data generated from calls and texts), the retrieval process and database building process takes too much time and may even cause the app to crash. Even if i tried to put these process in a AsyncTask, the problem still exists. So my question is:
Am i doing it in a good way to just put any time consuming operations away from Main UI, OR What's a better way, if any, to handle very very large amount of data in Android?
Use pagination logic. Fetch only the most recent and relevant data and load older data if the user requests it.
Call history on most android phones is limited to 500 entries CallLog.Calls, while SMS provider has not such limits, you can query the count of the tables and limit your queries to 50 at a time and pass it to a separate thread for processing. Also make sure you run this in a background service with low priority so as to not disturb any other operations ongoing in the device.
My app requires high data throughput. It receives an incoming data stream over bluetooth and has to parse it, scale it, display it, and store the data.
After reading up on different storage methods, I've decided to try to use Sqlite for data storage. I've read up and it seems to get the best write performance, I should use transactions.
But before I even go there, I'm wondering if the DB handler should just be in it's own thread. The BT data processing is already in a separate thread and works well. I see a lot of discussions about accessing sqlite from multiple threads, but I'm thinking I want one thread handling the DB connection and just use intents to get/store data..mostly just to disconnect the display from the data storage.
Will this increase my performance, or is it not worth it?
I'm guessing I'll be writing 20-50 rows a second with up to 19 fields.
Remember that SQLite is totally memory-resident. There is no disk latency time to slow up processing. As such, I would first try to keep the architecture as simple as possible to avoid creating any unnecessary overhead.
I'm trying to make my own gps-tracker, mainly for bike-rides. I managed to make a usable app, at least for personal use, but I wanted to improve it, and have added ContentProviders, Fragments, CursorAdapters and a Service to receive the onLocationChanges from GPS.
However, I have really no idea how to handle the stream of Locations I'm receiving. My old app just saved them to an ArrayList, so right now my Service is sending a Broadcast to my Activity, that saves them to the ArrayList.
Problem is, that when the ride is over, it takes from 5-15 seconds to save the locations to sqlite (yes, I'm using compiledstatement and transaction), and I would like to avoid that, by saving the locations when received. When I tried to do that, my app became unresponsive (as expected), as I was doing the insert in the UI thread, and I do receive location updates often.
This is of course the most important aspect of the app, so what is the solution?
I could do the insert in a thread, but since inserting a single record is expensive, I'm not sure it could keep up.
I could write 25 records (or more) at a time in a transaction, but there will be some logic to keep track of what is saved and what is not.
Is there other options for a better user-experience ?
Use an IntentService to delegate the saving to another thread, then use applyBatch to do inserts.
Personaly I think the gps track would be better in a file than embeded in the database tracks are lickly to be just big. It's the same senario as pictures where embeding in the database is not the recomended solution. The databases would just hold some summary information and a reference to the file. You wouild write out the file as you go so no big pauses at the end. You keep the local ArrayList as well while you are recording if you are using it for displaing a path on a map or a graphical plot etc. That's the way I am doing things in my biking app IpBike.