I've created a simple app which reads location data (latitude, longitude, speed, altitude) every few seconds using GooglApiClient. I would like to record the data, so that I would be able to calculate and show: distance, average speed, min and max altitude, trace on the map etc. in real-time to the user. Which data storing method would be the best for this purpose? I don't want to see the implementation of this method from you, just want to know which method would be the best. I was thinking about sqLite database, but isn't it too slow and energy consuming for continous read-write operations.
There are four ways to store data in Android that I know of:
SQLite tables.
Shared preferences (key-value pairs essentially).
Creating a File in the storage area.
Serializing an object to a String and storing it as a key-value pair.
For your purpose I'd say that the best way to go is creating SQLite tables. Say that your custom Location object holds latitude and longitude, among other things. So if these Location objects are converted into rows in a table, then its possible for you to easily retrieve the entire columns of these two data types, i.e. all the position data for every one of the stored Location objects.
To answer your question I think you need to know more about how much data you are going to be recording (how long are you tracking) what you need to do with the data (all remote or posting to servers, etc.) and what sort of devices are you targeting. I think your two options are either Sqlite or storing as a file. Both these options should be fine from a power and speed point of view, the main power issue will be you accessing the GPS sensor, not writing to db or file.
If you chose to write to a file, there are a least 3 (and there are many more) useful formats you can stream a file to that you should be aware of. They are:
GPS Exchange Format this is an open standard used by many commercial gps and satnav applications which is used to describe waypoints, tracks, and routes
Geography Markup Language which is an open xml standard for expressing geographical features which is understood by almost all GIS applications
GeoJSON which is a standard for describing geographical features in json, which is easy then to render in a web browser.
If you need to you could easily serialize from a Sqlite data set to these file fomats for data interchange. One thing about streaming to files on Android devices. However you write your file writer, it may get arbitrarily destroyed by the OS due to memory conditions or user priorities. This could well mean you end up with malformed files with streams not closed. This is a lot less likely to happen if you are writing to data tables.
I think sqLite is good enough for you to store location data on every few seconds. You can also buffer data in memory for a short time and write them into sqLite once to reduce I/O operations.
Using a SQLite database can be the easiest and safest way to store location data, but you have to be aware that writing to a database blocks read operations.
Before modifying the database, the writer implicitly acquires an
exclusive lock on the database which prevents readers from accessing
the database until the write is completed.
For this reason, continuous writing to a database can cause problems and have influence on performance of your application.
You can avoid those problems by enabling Write-Ahead Logging which allows reads to proceed concurrently.
Read more on:
https://www.sqlite.org/wal.html
http://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html#enableWriteAheadLogging()
Taking in account the context of your task, I would not stick to Android's SQLite. Since you're using just a bunch of primitives as opposed to a set of fully pledged POJOs, spinning up SQLite's database helper, defining schema and dealing with queries is quite a big overhead.
I'd recommend to take a look at SnappyDb library which is key-value database.
Either might you deal with each of your primitive values separately or wrap them into a POJO, SnappyDB can handle both cases. If you choose the first way, there are handy methods to query for multiple keys: keys search.
Also, SnappyDB is much faster than SQLite (see benchmarks on their page). And easy to set up, just couple of lines of code.
Hope it helps.
Related
I am implementing the client-server communication for an app on android. I have some high-level questions before I start to code. Apologize if the question turns out vague or confusing. I am really new to this (2 days of Google-ing is all I have), and I haven't coded anything because this design issue I am asking.
Background: Our android app records users activities. These activities are, for example, the user bookmarks an article. Eventually we want to synchronize these activities with server's database. However this sync is not time-critical, meaning it is not necessary that we send the activity to the server every time there is a new activity.
Current Design: Every time there is an activity, we get the activities and put it in a Java object, mark as "notSync", then make an update call to the database (SQLite) on the mobile. Every 5 minutes or so we make a call into the database, pull the nonSync items into an array of objects, convert it to JSON, and send it to the server.
Question 1: It seems rather inefficient to put an object into database, then every a few minutes pull it out from database as an array of objects, then convert to JSON. Would that be a better design, if we just append all the activities to an array of objects, then convert to JSON directly from that array and clear that array? My dilemma is that, If we implement the latter, there might potentially be a huge array floating around in phone's memory. How do we choose what is the best to implement?
Question 2 When sync mobile SQLite with server MySQL, is our design (using java to pull the rows out as an array of objects, then converting it to JSON) a reasonable practice?
Thanks in advance!
The first thing to keep in mind is where the bottlenecks really are. In my experience communicating with a MySQL server, the connection takes by far the longest, followed by uploading/downloading data, followed by anything I did on the phone. So even without knowing exactly what you're going to be recording, I think it's a safe bet that using SQLite and creating objects will not have a big time impact compared to your network activity.
I would run some basic tests to measure how long various operations take, but to answer your questions:
If you clear the array properly, you won't have to worry about it growing too big and, if your array ever did get that big, you probably have other things to worry about (uploading many MBs of data). However, if your user does something for 4 minutes, then quits the app or something, they'll lose their data. Storing it in the DB is safer.
JSON seems to be the best way to send your data, it's quite efficient for that, and as I said above, I don't thing the SQLite/object creation time will be too great.
My app requires high data throughput. It receives an incoming data stream over bluetooth and has to parse it, scale it, display it, and store the data.
After reading up on different storage methods, I've decided to try to use Sqlite for data storage. I've read up and it seems to get the best write performance, I should use transactions.
But before I even go there, I'm wondering if the DB handler should just be in it's own thread. The BT data processing is already in a separate thread and works well. I see a lot of discussions about accessing sqlite from multiple threads, but I'm thinking I want one thread handling the DB connection and just use intents to get/store data..mostly just to disconnect the display from the data storage.
Will this increase my performance, or is it not worth it?
I'm guessing I'll be writing 20-50 rows a second with up to 19 fields.
Remember that SQLite is totally memory-resident. There is no disk latency time to slow up processing. As such, I would first try to keep the architecture as simple as possible to avoid creating any unnecessary overhead.
Why do we use the sqlite data base in android.I am developing an android application where the data is to be fetched from the server and do some data calculation and show it on the UI.
Is it good for me to fetch the data into the sqlite DB and update the UI on regular interval from the sqlite in evry 20 minutes or will it be good to sent the Http get request to the server and update the data from teh response on the UI.
I wanted to know which one will be better and why?Why to involve sqlite DB?
The data corresponds to some 40X40 table data on which some heavy mathematical processing is to be done and then displayed on the UI(similar to Stock market application) and data needs to be cleared after every 12 hours.
plz advice
Rgds,
Raul
It is good to use database in your case.
Pros:
If your application gets closed the in memory data will be lost, but after that you will be able to restore the state from the database if you have one
Especially for the case of complex calculations it is good to store the result once in the database and not recalculate it multiple times on demand
The database will untie your UI from the internet connection and thus you will be able to display results even if there is not internet connection
Using database you will be able to fetch the updated data from a background service, without impacting your UI
Organizing your data in database usually makes it a lot easier to manage all the application data.
Cons:
Adding database will require a bit of additional effort on your side
As you can see my list proves you SHOULD use database in your case. Maybe I am biased, but at least I provide you with things to consider.
It's really a design decision, SQLite offers a very robust way to organize and persist your data, you're only other options are to write to a file, or to save in SharedPrefs, both methods become a lot harder to manage once the size of your data begins to grow, as you must manually keep a list of objects and manage their names etc etc. 40 x 40 table data is large enough to justify using SQLite, even if you are dropping and recreating the table every 12 hours.
You might want to consider using an ORM library to make fetching and saving data from the DB simpler, ORMLite is good and compatible with Android
http://ormlite.com/
If your application relies heavily on an internet connection you don't need to buffer information in the database. However if you want to use the app where you have bad or no signal you might want to used cached values from the sqlite database.
With slow internet connection your application may be unresponsive so caching may be a good idea - but it doesn't necessarily be in the sqlite database. You should use the sqlite database for data that is required by the device frequently and that is irrelevant to your server component.
If the data is updated frequently but only while the application runs you might want to cache in the devices memory. I assume your app is not running all the time within the 12 hours but is called regularly instead to check something.
12hrs is a long time, so rather than leaving your data wander in RAM, i would suggest you to use database. Because you never know when you may need to read it again.
Otherwise, if your purpose is only to downloaded data, process it and display in activity, then you dont need to involve database because if your app is closed (due to user or low memory), in anyway your app will be downloading fresh data from server... am i right?
> update the UI on regular interval from the sqlite in evry 20 minutes
Dont expect your app to be open for such a long duration.
To precisely suggest to your case
Avoid DB
Fetch Data at app start or at appropriate time when app is opened
and save it in plain java objects.
Define Methods within it that perform operation in it.
Make map or list to save those POJO
Define Seprate Controller Classes within your project to update map of pojo at any
specific change to make fresh data available to UI.
I am developping an application that retrieves some data from a server.
I have two options:
Get the whole data set from the server and then work with it using the pattern 'singleton' to reduce the number of queries
For each query, get the corresponding data set from the server
What is the best choice?
In my opinion it depends.
It depends on the size of the data and if it even makes sense to return data that your user may not even need.
Personally, in my app that I am building I will be returning only the data that is required at that time. Obviously though, once I have the data I won't be fetching it again if it makes sense to keep hold of it for good or even just temporarily.
I agree with C0deAttack's answer. Your goal should be to minimize network traffic within the constraints of your app being a "good citizen" on the phone. (That means that your app does not negatively impact the user or other applications by using too many resources — including memory and file space.)
From the sound of it, I'm guessing that the data are not that voluminous. If so, I would recommend caching the response and use it locally, thus avoiding repeated queries to the server. Depending on how often the data changes, you might even consider making it persistent, so that the app doesn't have to query the server the next time it starts up. If the response includes an estimated time before it is considered outdated, that would help in establishing an update schedule. (Google's license server uses this idea.)
P.S. I don't see that this has anything (directly) to do with a singleton pattern.
How about storing your data in an sqlite database and do your queries with sql? Since you get sorting and ordering for free, it can help you writing less code. Also you have instant offline functionality if your user has no internet connection. :)
I'm connecting an Android device to an embedded Data Acquisition system via Bluetooth. The DAQ system will take data samples from 50Hz up to potentially 880Hz (possibly more in the future) and push it to the android device either as the data is collected or in bundles at faster sample rates.
There are plenty of examples of how to manage the Bluetooth connection, but not so much on what to do with the data.
I need to persist the data to some kind of long term storage and be able to do this continually at these higher sample rates for an extended period of time.
I know to do this off the UI thread, so no need to harp on that. What storage medium on Android can respond fast enough to keep up with this incoming data? Would the SQLite database be fast enough? Seems like it would bog down fairly quickly.
I know this is a very old question but I thought I'd throw up an answer anyway. SQLite should work. It would be a good idea to buffer the data into an array of bytes of a certain length, depending on what data exactly you are storing. Once that array is full, insert that into the SQL database and accept all new data into a new array. Once that one is full, store it to the database; and on and on like that. In this way you could achieve a sort of double-buffering. Database modification involves disk I/O; you would use the buffer because it's more efficient to write a big chunk of data to the disk than it is to write a lot of smaller chunks.