In terms of storing data in Android, would it be more efficient to use a large ArrayList or setup an SQLite database? I have ~9000 bus stops and stop names (both in String) that I need to store and I was wondering which method would be the quickest or most efficient.
An ArrayList is probably a very bad idea. I assume you want the data to be persistent, so if your user kills your app your data will be lost if you use an ArrayList. A database is persistent(unless the user clears the cache of the app). So I would highly recommend using a database over an ArrayList.
If your data does not change then you could probably have it read on startup and kept in memory while the App runs. For example, having a .json file with all the stops, read it on startup, put the data in a HashMap or something that is easy to use since you will probably lookup stuff. It would probably work fine with ~9000 entries.
If you are going to add or update information about the stops during usage then the SQLite is the better choice though.
1.) Store and retrieve your data from a SQLite DB since you want persistance. And since you say you have 9k+ rows a simple select will give you everything at once and you can easily filter the data as well if you need to
2.) At startup, put all your data into efficient memory structures like HashMaps(or in your case arraylists) and reference them throughout the app. This way you'll only do one time access.
3.) When in doubt build a DB. No harm, more efficient and easier to handle than files
Related
My app will pull some json data when it is started and realistically, once the data has been pulled, I really won't need to pull it again for the duration of the user experience. The backend data may update a few times a day at most, so I think I would like to just grab the data upon app start and then use that same data for the duration and give the user an option to manually refresh the data. So, my question is, where/how should I store that data? I've got all of my data structures set up (classes and sub-classes). And there may be 200 or so instances of some of the classes. If I store everything as member variables in my activity_main class, it won't be available to other activities once the other activities are started. Storing them all in databases could be an option, but it sort of feels like overkill. I really don't need the data to persist between sessions. Is there a way to easily store it in memory and still have it easily accessible to all activities?
You should think about OS killing your app process on low-memory, so backing your data on disk is a good thing to do. Doing so you have an ability to show user data from disk cache while refreshing it in background from server.
Choosing the tool for data storage depends on the way you need to work with data.
Of course, there is an option to use Realm, but you should consider the fact that it is not the relational database. So if you have complex domain model with joins and other relational stuff for your business logic, I'd go with something other. It is thread-safe, also has migrations (but, as for me, migrarations are always pain, you just can not do anythig about it). Realm is supposed to be RxJava-friendly now (support added in v0.87) There are some disadvantages(part of them may already be fixed), but you should consider it before using.
As for more relational approach, there is SQLBrite library:
A lightweight wrapper around SQLiteOpenHelper which introduces reactive stream semantics to SQL operations.
It is not an ORM (but you can add some kind of it on top of SQLBrite if you wish: see this for more info). In fact, being alone this library is doing one thing (and doing it good) - it provides a mechanism for coordinating and composing the notification of updates to tables(Realm also has such ability) such that you can update queries(in SQL fashion) as soon as data changes. And it respects RxJava!
As an alternative to SQLBrite you can look at StorIO.
There are also lots of different ORM solutions, like GreenDAO, ORMLite etc.
But I'm pretty sure, one of the first two libraries (Realm or SQLBrite) will likely help you. So analyze your app, these libs and decide what fits better.
P.S. Great article on how RxJava would help you to work with data from different data sources (in-memory cache + disk cache + network) easily. Might be helpful!
I would still recommend a SQLite Databse, you can easily declare it as a 'in-memory' database, if that is what you want.
However.... I would be rather upset as a user of your application if it downloaded redundant data over and over. I would just recommend making a content provider and being done with it. This gives you access to a SyncAdapter, and defines clear boundaries between where code should go.
The 'trick' with making a good ContentProvider is to make good POJOs, that have methods to convert from POJO -> ContentValues and Cursor -> POJO(s).
Easiest thing is to do is store the json file in Apps data storage and parse the json every time you need.
But this is not recommended as it is costly to parse data every time.
Best option is to implement Realm (Replacement for Sqlite) which is very easy to implement and its amazingly fast.
I'm working on an Android app for homework management. I'm a senior in college, so my experience on larger projects is very limited, but I'd like to design all parts of this app well instead of just throwing something together. This includes the way data is stored.
We have two main objects/entities: Task and Subject. Even if someone uses the app for the whole time they're in college and never deletes anything, I'm guessing there would be a maximum of a few thousand tasks and a couple hundred subjects (not all subjects would be shown at once). The initial version of the app won't sync data with a server, but this is a definite possibility in the future, so I'd like to design with that in mind. We might also have an option for users to send tasks to each other.
Here are my questions:
Would a SQLite database be the best option for storing the amount of data we're likely to have, or would something like serializing it to XML or JSON then loading it into memory when the app starts work?
I'm used to thinking in terms of objects. This means that if I use a database and it has a Task table and a Subject table, my first instinct is to convert each database table row into a corresponding object for viewing/editing. (The objects' setters would contain validation logic.) Is this a good/helpful/necessary way to think? If not, what is the alternative?
Thanks for your help!
This question is broad so may comments below may not be 100% correct as I don't have all the information about your system.
SQLite is better suited for storing thousands of records than files (be it JSON or XML). This is especially true if your data is not static, i.e. will be changed during the usage of your app (which is the case for you, I believe). You can take advantage of existing functionality for records inserts, updates, deletions, using indexes, etc.
Yes, you generally create objects similar to your database. But you don't usually need to convert each and every record from the database into your objects. You usually query the database for a limited number of objects, depending on what you want to show in the UI. Of course, if you need to show all, let's say, tasks, you need to get them all.
1. Would a SQLite database be the best option for storing the amount of data we're likely to have, or would something like serializing it to XML or JSON then loading it into memory when the app starts work?
Yes SQlite will be the option for you.It will give you a structured format and in future if you want to access data from remote end the same structure of tables can be used without much change in the code.
2. I'm used to thinking in terms of objects. This means that if I use a database and it has a Task table and a Subject table, my first instinct is to convert each database table row into a corresponding object for viewing/editing. (The objects' setters would contain validation logic.) Is this a good/helpful/necessary way to think? If not, what is the alternative?
you can simply execute queries to manipulate data.
But dont forget to encryt your database if you storing it in mobile itself.
I will develop an android application with a lot of data (json files with some rows and CSV for graphics data with a lot of rows) , this data change every 5 minutes and replaces all the previous data (or mostly).
What are the best approaches to design this ? I have 2 options:
Save all the data in a sqlite db, and sync this by a IntentService.
save the data in json and csv files and replace this every 5 minutes.
Which approach will the best performance?
This considering the time to parse the files, sorting data, the download time and the consistency of data.
any other ideas?
PD:I need a cache system too, in case if i don't have internet and I need the previous stored data
Advantages of SQLite:
Changes are ACID
You can make complex requests faster (e.g. "give me only fields A,B from items with (C/D)>E")
I would bet more compact for big data (integers are stored as integers instead of a string of individual digit)
Only one file
You can merge old data with new data easily
You can update current data, even while using it
Concurrency can be handled
Advantages for JSON/CSV:
Easier to debug (plain text)
Faster to make a whole update (copy the new file + delete the old one)
For the original question the whole delete/replace of the data makes JSON/CSV the winner.
However, if the application was to retrieve partial data every 10s and merge/update it with the previous one, SQLite would be a better option.
Sqlite is mostly used when you want data to be saved and used in future. In your case data is changing every 5 minutes so its better to have JSON because every time to make the Database connection store and retrieve after 5 minutes will take some time.
UPDATE:
I also had the same Application in which the data was changing every time. In that case I used Map<K, V> and ArrayList to maintain the values, because as the data is changing everytime I think its not feasible to store the data in Sqlite everytime. It needs much time to perform DB Connection store, retrieve, update the data in Sqlite.
I recommend using JSON or some type of object serialisation unless:
You need ACID compliance for write operations
You need to report against the data which may involve copying the data to an external RDBMS
or
You wish to join those complicit in the overuse / abuse of databases, as commonly seen nowadays
Ideally this should depend on whether you need the previous data, for maybe comparing it with current data and so on. As a thumb rule, I use SQLite when you need the data to be stored and retrieved at a later stage. In case the data is only for display, I would rather keep it in program memory. Mind you this does not involve file operation.
Purpose of JSON and SQLite is completely different from each other
JSON = is used send and receive data between server and client.
SQLite= is used to store data.
I have one Application which has more then 5 Activity I want to know how much space is needed to store 10 different values to SharedPreference. or I have to user Sqlitequery ?
Simple in One Activity I have 4 Field like FirstName,LastName, Age, City when user Click on th button then new Activity is called and in this activity Full Information is Display, so I can do that in two way. First Way I Store all the Information in the SharedPreference and in Second Activity I get this Stored Values and another way is i only store record id and store it in intent.putextra and get it to another activity and Fire Sqlite Query. so I want to know which one is best for Application .(As per Memory Uses and Speed )
If you do not need persistent storage then you can simply pass the data using intents from one activity to another or maybe even keep some static variables in a separate class say Const.java
If you need persistent storage and if your data is limited, I would always recommend using SP over SQLite which should be used for relational data. DB operations take more time and can hit performance besides there is always chance of DB getting corrupted.
Sqllite is a really good solution if you want to be able to add easily more preferences.
It's a bit heavy to set-up but really powerfull.
http://developer.android.com/guide/topics/data/data-storage.html
For me, database is for storing data, and sharedpreference just app config.
Shared preferenses apereas to be the choise here, it is very good to store simple settings and very fast and flexible.
A database would IMO be overkill here :)
I have created a database for my application. I will use the data in this database, again and again. I just want to know the preferred way to do this.
One way is to query the database again and again, and the other way in my mind is to have multiple ArrayList for different fields, or something like that, to store data in the start of application in these Lists and then access these Lists whenever I need data(sort of a disconnected mode e.g. DataSets that we have in .Net).
So, which is the efficient way or is there any 3rd better way?
Is your database local, local and external (shipped to your apk) or remote? If it is local then you can query the database everytime you need. Because you rather have operation with your CPU than keep memory occupied. When it comes to remote database than you can implement various ideas about keeping the last query in case of disconnection, then on the next query release the last query results (not good to me, because you might have milion of results, which will occupy so much memory [scenario]), or save the query results (get result - translate into query for your own database = long process). Or you can query again the database again and again, with least effort so many results. I hope you get my idea and will serve you.