I'm trying to make my own gps-tracker, mainly for bike-rides. I managed to make a usable app, at least for personal use, but I wanted to improve it, and have added ContentProviders, Fragments, CursorAdapters and a Service to receive the onLocationChanges from GPS.
However, I have really no idea how to handle the stream of Locations I'm receiving. My old app just saved them to an ArrayList, so right now my Service is sending a Broadcast to my Activity, that saves them to the ArrayList.
Problem is, that when the ride is over, it takes from 5-15 seconds to save the locations to sqlite (yes, I'm using compiledstatement and transaction), and I would like to avoid that, by saving the locations when received. When I tried to do that, my app became unresponsive (as expected), as I was doing the insert in the UI thread, and I do receive location updates often.
This is of course the most important aspect of the app, so what is the solution?
I could do the insert in a thread, but since inserting a single record is expensive, I'm not sure it could keep up.
I could write 25 records (or more) at a time in a transaction, but there will be some logic to keep track of what is saved and what is not.
Is there other options for a better user-experience ?
Use an IntentService to delegate the saving to another thread, then use applyBatch to do inserts.
Personaly I think the gps track would be better in a file than embeded in the database tracks are lickly to be just big. It's the same senario as pictures where embeding in the database is not the recomended solution. The databases would just hold some summary information and a reference to the file. You wouild write out the file as you go so no big pauses at the end. You keep the local ArrayList as well while you are recording if you are using it for displaing a path on a map or a graphical plot etc. That's the way I am doing things in my biking app IpBike.
Related
I'm currently working on an Android application that requires reading from call history and text history. And for further analysis, I've extracted these the huge amount of entries from the corresponding content provider, created and inserted all of them to a SQLite database.
But the problem I've encountered is when this is running on a phone that has been used for years (meaning there's an enormous amount of data generated from calls and texts), the retrieval process and database building process takes too much time and may even cause the app to crash. Even if i tried to put these process in a AsyncTask, the problem still exists. So my question is:
Am i doing it in a good way to just put any time consuming operations away from Main UI, OR What's a better way, if any, to handle very very large amount of data in Android?
Use pagination logic. Fetch only the most recent and relevant data and load older data if the user requests it.
Call history on most android phones is limited to 500 entries CallLog.Calls, while SMS provider has not such limits, you can query the count of the tables and limit your queries to 50 at a time and pass it to a separate thread for processing. Also make sure you run this in a background service with low priority so as to not disturb any other operations ongoing in the device.
I am implementing the client-server communication for an app on android. I have some high-level questions before I start to code. Apologize if the question turns out vague or confusing. I am really new to this (2 days of Google-ing is all I have), and I haven't coded anything because this design issue I am asking.
Background: Our android app records users activities. These activities are, for example, the user bookmarks an article. Eventually we want to synchronize these activities with server's database. However this sync is not time-critical, meaning it is not necessary that we send the activity to the server every time there is a new activity.
Current Design: Every time there is an activity, we get the activities and put it in a Java object, mark as "notSync", then make an update call to the database (SQLite) on the mobile. Every 5 minutes or so we make a call into the database, pull the nonSync items into an array of objects, convert it to JSON, and send it to the server.
Question 1: It seems rather inefficient to put an object into database, then every a few minutes pull it out from database as an array of objects, then convert to JSON. Would that be a better design, if we just append all the activities to an array of objects, then convert to JSON directly from that array and clear that array? My dilemma is that, If we implement the latter, there might potentially be a huge array floating around in phone's memory. How do we choose what is the best to implement?
Question 2 When sync mobile SQLite with server MySQL, is our design (using java to pull the rows out as an array of objects, then converting it to JSON) a reasonable practice?
Thanks in advance!
The first thing to keep in mind is where the bottlenecks really are. In my experience communicating with a MySQL server, the connection takes by far the longest, followed by uploading/downloading data, followed by anything I did on the phone. So even without knowing exactly what you're going to be recording, I think it's a safe bet that using SQLite and creating objects will not have a big time impact compared to your network activity.
I would run some basic tests to measure how long various operations take, but to answer your questions:
If you clear the array properly, you won't have to worry about it growing too big and, if your array ever did get that big, you probably have other things to worry about (uploading many MBs of data). However, if your user does something for 4 minutes, then quits the app or something, they'll lose their data. Storing it in the DB is safer.
JSON seems to be the best way to send your data, it's quite efficient for that, and as I said above, I don't thing the SQLite/object creation time will be too great.
What is the best way of recording rapidly updating data in Android?
In my specific example I am trying to record various data tidbits whenever (acceleration) onSensorChanged fires, which is often. It's fine for some quick processing and displaying, but anything more (plotting or recording data) makes it rather slow. It's worth noting that I'm storing it in a variable and not to file.
You are advised to do as little as possible in an onSensorChanged, so I tried the following:
In an IntentService: Fetch the rest of the data you want to record and store all of it in a variable. Result: Even though an intent service executes in its own thread, only one can run at a time. Data thus seemed to come in quicker than it could be processed and would end up with a long queue of data to be processed, giving a long and constant stream of GC_FOR_ALLOC. Ultimately making it a bad choice
In an AsyncTask fetch the extra data you want to record and then start an IntentService for recording it to a variable.
Result: Pretty much the same as above.
Overall both of these methods were actually performing significantly worse than just doing the extra data fetching and recording in the onChanged, as well as consuming significantly more power. I haven't been able to find any concrete advice on rapid data storage. Would a buffer or cache perhaps be more efficient?
After some debugging and attempts at using different data structures for storing the rapidly updating data, it turns out that using the IntentService works reasonably well. Changing from an ArrayList with O(n) for adding new elements, to a custom LinkedList that just appended new values to the end with O(1) made a massive difference. As the LL is able to process the data so much quicker, the problem of a huge queue of pending values to record basically disappears.
Why do we use the sqlite data base in android.I am developing an android application where the data is to be fetched from the server and do some data calculation and show it on the UI.
Is it good for me to fetch the data into the sqlite DB and update the UI on regular interval from the sqlite in evry 20 minutes or will it be good to sent the Http get request to the server and update the data from teh response on the UI.
I wanted to know which one will be better and why?Why to involve sqlite DB?
The data corresponds to some 40X40 table data on which some heavy mathematical processing is to be done and then displayed on the UI(similar to Stock market application) and data needs to be cleared after every 12 hours.
plz advice
Rgds,
Raul
It is good to use database in your case.
Pros:
If your application gets closed the in memory data will be lost, but after that you will be able to restore the state from the database if you have one
Especially for the case of complex calculations it is good to store the result once in the database and not recalculate it multiple times on demand
The database will untie your UI from the internet connection and thus you will be able to display results even if there is not internet connection
Using database you will be able to fetch the updated data from a background service, without impacting your UI
Organizing your data in database usually makes it a lot easier to manage all the application data.
Cons:
Adding database will require a bit of additional effort on your side
As you can see my list proves you SHOULD use database in your case. Maybe I am biased, but at least I provide you with things to consider.
It's really a design decision, SQLite offers a very robust way to organize and persist your data, you're only other options are to write to a file, or to save in SharedPrefs, both methods become a lot harder to manage once the size of your data begins to grow, as you must manually keep a list of objects and manage their names etc etc. 40 x 40 table data is large enough to justify using SQLite, even if you are dropping and recreating the table every 12 hours.
You might want to consider using an ORM library to make fetching and saving data from the DB simpler, ORMLite is good and compatible with Android
http://ormlite.com/
If your application relies heavily on an internet connection you don't need to buffer information in the database. However if you want to use the app where you have bad or no signal you might want to used cached values from the sqlite database.
With slow internet connection your application may be unresponsive so caching may be a good idea - but it doesn't necessarily be in the sqlite database. You should use the sqlite database for data that is required by the device frequently and that is irrelevant to your server component.
If the data is updated frequently but only while the application runs you might want to cache in the devices memory. I assume your app is not running all the time within the 12 hours but is called regularly instead to check something.
12hrs is a long time, so rather than leaving your data wander in RAM, i would suggest you to use database. Because you never know when you may need to read it again.
Otherwise, if your purpose is only to downloaded data, process it and display in activity, then you dont need to involve database because if your app is closed (due to user or low memory), in anyway your app will be downloading fresh data from server... am i right?
> update the UI on regular interval from the sqlite in evry 20 minutes
Dont expect your app to be open for such a long duration.
To precisely suggest to your case
Avoid DB
Fetch Data at app start or at appropriate time when app is opened
and save it in plain java objects.
Define Methods within it that perform operation in it.
Make map or list to save those POJO
Define Seprate Controller Classes within your project to update map of pojo at any
specific change to make fresh data available to UI.
I'm getting hung up on how to handle the data for an app I'm designing. I want to pull a list of items from the net. The data is updated routinely, and I think it would be good to store all the data on the device so the app can load quickly and refresh the data in a background thread rather than have to wait for the network on every start-up.
I think I should make the data available in an XML and have a thread parse and save into a SQLite DB, but I'm not sure if that's the "best practice." Are there other ways that people go about handling this?
The cleanest way (or at least, I think it is the cleanest way) is to implement a custom ContentProvider class that interfaces with your server. You can query the contenprovider and if it doesn't have the data in a local cache (for example a SQLite db as you said) it downloads it from your server and adds it to the local data. Why a content provider? because then you can easily access your data across apps and you have a nice and clean way to get your data when using intents.
Also, I personally prefer not to download data when the app is not running, because it will cost battery life while the user does not actively requests the data.
Sounds reasonable.
If the download of the new data takes more than some seconds, you might want to use a Service. This will allow you to continue the update even when the user has left your app.
Also, think about how you will notify the user that something is going on. Displaying some progress indicator is always a good idea. Otherwise, users might just think the data is not up to date because the app is broken.