My app will pull some json data when it is started and realistically, once the data has been pulled, I really won't need to pull it again for the duration of the user experience. The backend data may update a few times a day at most, so I think I would like to just grab the data upon app start and then use that same data for the duration and give the user an option to manually refresh the data. So, my question is, where/how should I store that data? I've got all of my data structures set up (classes and sub-classes). And there may be 200 or so instances of some of the classes. If I store everything as member variables in my activity_main class, it won't be available to other activities once the other activities are started. Storing them all in databases could be an option, but it sort of feels like overkill. I really don't need the data to persist between sessions. Is there a way to easily store it in memory and still have it easily accessible to all activities?
You should think about OS killing your app process on low-memory, so backing your data on disk is a good thing to do. Doing so you have an ability to show user data from disk cache while refreshing it in background from server.
Choosing the tool for data storage depends on the way you need to work with data.
Of course, there is an option to use Realm, but you should consider the fact that it is not the relational database. So if you have complex domain model with joins and other relational stuff for your business logic, I'd go with something other. It is thread-safe, also has migrations (but, as for me, migrarations are always pain, you just can not do anythig about it). Realm is supposed to be RxJava-friendly now (support added in v0.87) There are some disadvantages(part of them may already be fixed), but you should consider it before using.
As for more relational approach, there is SQLBrite library:
A lightweight wrapper around SQLiteOpenHelper which introduces reactive stream semantics to SQL operations.
It is not an ORM (but you can add some kind of it on top of SQLBrite if you wish: see this for more info). In fact, being alone this library is doing one thing (and doing it good) - it provides a mechanism for coordinating and composing the notification of updates to tables(Realm also has such ability) such that you can update queries(in SQL fashion) as soon as data changes. And it respects RxJava!
As an alternative to SQLBrite you can look at StorIO.
There are also lots of different ORM solutions, like GreenDAO, ORMLite etc.
But I'm pretty sure, one of the first two libraries (Realm or SQLBrite) will likely help you. So analyze your app, these libs and decide what fits better.
P.S. Great article on how RxJava would help you to work with data from different data sources (in-memory cache + disk cache + network) easily. Might be helpful!
I would still recommend a SQLite Databse, you can easily declare it as a 'in-memory' database, if that is what you want.
However.... I would be rather upset as a user of your application if it downloaded redundant data over and over. I would just recommend making a content provider and being done with it. This gives you access to a SyncAdapter, and defines clear boundaries between where code should go.
The 'trick' with making a good ContentProvider is to make good POJOs, that have methods to convert from POJO -> ContentValues and Cursor -> POJO(s).
Easiest thing is to do is store the json file in Apps data storage and parse the json every time you need.
But this is not recommended as it is costly to parse data every time.
Best option is to implement Realm (Replacement for Sqlite) which is very easy to implement and its amazingly fast.
Related
In my android app i am performing sql data extensively where i need to download the data from server , store it into database and populating it on the ListView.
moreover, i need to perform the database search and filter the data in a ListView too.
so far in my past projects i have used simple method to manipulate database like manually opening & closing database , getting data from database using cursor and storing it in a array list and populating over ListView.
after exploring i came across the ContentResolver and LoaderManager and tried few samples too but i am not able to decide what to use where i need to perform more database operation with search in a ListView.
I would even love to know in which scenario what approach is preferable.
i am waiting for commonsware comments too ;)
ContentResolver/ContentProvider is meant for providing data to other applications. If you aren't going to do that, then using it is clunky and overkill. It really doesn't provide any value, and even Google's own docs suggest not using it for inside of your app.
LoaderManager- meh. The idea of a Loader is useful, and you're probably already using it- a loader is just the idea of reading large amounts of data from your DB on an AsyncTask. LoaderManager will provide some efficiency gains if you're being restarted due to configuration changes, but there's other ways to achieve them. It isn't a bad thing to use, but it will save you minimal to no work- you still have to write a custom CursorLoader to make your db call, and you'll have some code to manage the loader manager. You aren't wrong to use it, I'm just not convinced of its value.
I'm working on an Android app for homework management. I'm a senior in college, so my experience on larger projects is very limited, but I'd like to design all parts of this app well instead of just throwing something together. This includes the way data is stored.
We have two main objects/entities: Task and Subject. Even if someone uses the app for the whole time they're in college and never deletes anything, I'm guessing there would be a maximum of a few thousand tasks and a couple hundred subjects (not all subjects would be shown at once). The initial version of the app won't sync data with a server, but this is a definite possibility in the future, so I'd like to design with that in mind. We might also have an option for users to send tasks to each other.
Here are my questions:
Would a SQLite database be the best option for storing the amount of data we're likely to have, or would something like serializing it to XML or JSON then loading it into memory when the app starts work?
I'm used to thinking in terms of objects. This means that if I use a database and it has a Task table and a Subject table, my first instinct is to convert each database table row into a corresponding object for viewing/editing. (The objects' setters would contain validation logic.) Is this a good/helpful/necessary way to think? If not, what is the alternative?
Thanks for your help!
This question is broad so may comments below may not be 100% correct as I don't have all the information about your system.
SQLite is better suited for storing thousands of records than files (be it JSON or XML). This is especially true if your data is not static, i.e. will be changed during the usage of your app (which is the case for you, I believe). You can take advantage of existing functionality for records inserts, updates, deletions, using indexes, etc.
Yes, you generally create objects similar to your database. But you don't usually need to convert each and every record from the database into your objects. You usually query the database for a limited number of objects, depending on what you want to show in the UI. Of course, if you need to show all, let's say, tasks, you need to get them all.
1. Would a SQLite database be the best option for storing the amount of data we're likely to have, or would something like serializing it to XML or JSON then loading it into memory when the app starts work?
Yes SQlite will be the option for you.It will give you a structured format and in future if you want to access data from remote end the same structure of tables can be used without much change in the code.
2. I'm used to thinking in terms of objects. This means that if I use a database and it has a Task table and a Subject table, my first instinct is to convert each database table row into a corresponding object for viewing/editing. (The objects' setters would contain validation logic.) Is this a good/helpful/necessary way to think? If not, what is the alternative?
you can simply execute queries to manipulate data.
But dont forget to encryt your database if you storing it in mobile itself.
In terms of storing data in Android, would it be more efficient to use a large ArrayList or setup an SQLite database? I have ~9000 bus stops and stop names (both in String) that I need to store and I was wondering which method would be the quickest or most efficient.
An ArrayList is probably a very bad idea. I assume you want the data to be persistent, so if your user kills your app your data will be lost if you use an ArrayList. A database is persistent(unless the user clears the cache of the app). So I would highly recommend using a database over an ArrayList.
If your data does not change then you could probably have it read on startup and kept in memory while the App runs. For example, having a .json file with all the stops, read it on startup, put the data in a HashMap or something that is easy to use since you will probably lookup stuff. It would probably work fine with ~9000 entries.
If you are going to add or update information about the stops during usage then the SQLite is the better choice though.
1.) Store and retrieve your data from a SQLite DB since you want persistance. And since you say you have 9k+ rows a simple select will give you everything at once and you can easily filter the data as well if you need to
2.) At startup, put all your data into efficient memory structures like HashMaps(or in your case arraylists) and reference them throughout the app. This way you'll only do one time access.
3.) When in doubt build a DB. No harm, more efficient and easier to handle than files
I understand that Content Providers are made to allow publicly sharing data between applications. However, I'm wondering if anyone has thoughts about making a Content Provider to use just within your own app. Would there be any advantages to doing this? Any disadvantages?
In the past I've just implemented the SQliteOpenHelper to access data from my database, but I'm considering creating a Content Provider. I feel like the URI approach to requesting data is clear and concise. On the other hand, will using a Content Provider just for my application be redundant ( since within it I will have a SQliteOpenHelper class ) and more work than I need?
I would argue it is definitely a good idea to use a ContentProvider even if you don't intend to make it public.
It's good practice to provide the extra level of abstraction over your data to make it easier to change internally. What if you decide to change the underlying database structure at a later time? If you use a ContentProvider you can contain all the structural changes within it, where as if you don't use one, you are forced to change all areas of the code that are affected by the structural changes. Besides, it's nice to be able to re-use the same standard API for accessing data rather than littering your code with low-level access to the database.
Also, there is always the chance that you might want to expose your data in the future. If you don't use a ContentProvider up front, it will be much harder to retrofit it in at a later date.
Then, there's the other parts of the Android where ContentProvider's are required/recommended such as when using SyncAdapters and if you want an App Widget that involves data access for instance.
In summary, there is very little overhead involved in writing a ContentProvider up front (once you have learned the API which is a good idea anyway) so it makes sense to do so, even for private data.
If you are not planning to share data, don't think about Content Providers. They are powerful but hard to write and it will be just silly to implement them if you are going to use them internally.
However, I'm wondering if anyone has thoughts about making a Content Provider to use just within your own app.
Of course... for instance, for an old TODO list app I wrote, I had to write a content provider to allow other apps retrieve and access the tasks states. It was part of the requirements, but more than that it made sense and made the app nicer.
Take a look at the MOTODEV Studio for Eclipse. It is a development environment that extends Eclipse. They have a tool where you can automatically generate a content provider for a database. If a content provider makes it easier to access your data and it doesn't have a significant impact on performance go ahead and use it. In most scenarios this will be the case.
In short,Content Providers helps in managing your data effectively. I would suggest to use them for the following reasons.
It acts as an abstraction layer between your UI and database. You can implement data validation in ContentProviders to validate the data entered by the user. It also lets you to modify the structure of the database without touching the UI and other parts.
They play along nicely with other android framework classes like SyncAdapter. For eg., you can automatically refresh a list, when a value in a database changes using ContentProviders along with CursorLoader. Without ContentProviders you have to implement a lot of functionalities like these on your own.
We can safely expose our private data to other apps. Using ContentProviders will allow us to share our data easily and safely with other apps.
So even if you don't need any of these functionalities now, you might need them in future and its good to go the extra mile and implement them right now.
I agree ContentProviders are a little difficult to grasp but they are definitely helpful, even if you want to use them internally for you own app. The best thing about it is that you can customize the contentproviders for suitable URIs.
Here's a scenario where you may have 5 tables in your database, but you need to join a few of them in certain orders before using them. And make a content URI for each of these joins. You could then each use these URIs as a table :)
I suggest you go ahead with Content Provider, you'll be amazed to see how powerful it is.
In my view point, the content-provider comes with plenty of advantages leave alone just sharing data with other apps. If you need to synchronize with the server using a Sync-Adapter, use google cloud messaging, auto update the UI when the underlying data in the DB changes using Loaders, implement search, use widgets... then the content provider is for you.
I prefer you follow the guideline on because one day you may need to implement some of the above features attached to the content-provider
By the way, you can quickly build you database and CP in less than 5 minutes using content provider generator
As said in documentation:
Creating a Content provider
You don't need a provider to use an SQLite database if the use is
entirely within your own application.
So why bother developing this overhead? You want easier and faster development, right? So one layer of abstraction (SQLiteOpenHelper descendent) is enough.
See Occam's Razor
Do not make an entities without very good reason.
Using a Content Provider can help in an additional level of abstraction - Putting it within your own application make add a significant development time to your project. However if you are using it to share data, application settings or configurations across multiple applications then the Content Provider is your choice.
Watch your security levels and I would recommend using SQLcipher to encrypt data-at-reset (DAR) if your Content Provider is writing to SQLite. (I've used a content provider in a few solutions and provided the ability to take a live "snap shot" of the operational values for debugging and testing.)
Do not use content provider if do not wish to share data with other apps. Use simple sqlitedatabase to perform database operations. Be careful while using content providers for storing confidential data because your confidential information may be accessed by other apps
i have done SQLite database programming for Android, but i dont know anything about
Content-Provider except this: "As i have referred Android Developer page , Android SDK explained about "Content-provider" as it is used to store and retrieve data."
But then,
What is the exact difference between "Content-Provider" and "SQLite Database"?
Which is best to store data, when ?
Any example or helps !!
I found one major difference, as follows:
Storing your data in a database is one good way to persist your data, but there's a caveat in Android-databases created in Android are visible only to the application that created them. That is to say, a SQLite database created on Android by one application is usable only by that application, not by other applications.
So, if you need to share data between applications, you need to use the content provider model as recommended in Android. This article presents the basics of content providers and how you can implement one.
I found this article at this link
Really nice information provided.
What is the exact difference between
"Content-Provider" and "SQLite
Database"?
ContentProvider is a facade -- an API you can implement that exposes databases to other processes. It can be implemented in a way where the data is stored in a SQLite database, but it does not have to be.
Which is best to store data, when ?
That is impossible to answer in the abstract. Generally speaking, unless something is requiring you to use a ContentProvider, just use a database.
I have made many good apps with thousands of users using them which simply used SQLite methods. But that was a while ago and I had to manually write lots of code which now can easily be taken care of by ContentProvider. Back then I was not in favour of using Content Providers because it seemed to only add complexity in the code.
However for last couple of years, as Android has evolved, I have moved to ContentProvider as it saves time and allows you do to more. I now use it extensively. Once you have a Content Provider class written, your life becomes much easier. With ContentProvider I can much easily deal with Cursor Loaders, Loader Callbacks and Bulk Inserts for which I had to write everything manually in the past and still it didn't work as efficiently. Especially when updating the list view, which is now automatically updated thanks to just one notifychange() method. This means now I don't have to type my own listeners and manually updating the content in list views and adapters. Plus, I don't need to worry about opening and closing of databases or worry about memory leaks. That's all handled by the Content Provider. The only problem which once in a while I face is that that you cannot do some complex queries in ContentProviders. In this case you can still use raw queries and use the old fashioned manual interaction with sqlite.
If you have previously written your own DbAdapter, Helper and Observer, you can safely carry them on to your new apps without spending time to convert everything to ContentProvider. But based on my experience, I would highly recommend to move to ContentProvider. It'll take some time to get used to it, but once you have got experience with it, you'll stay with it.
UPDATE 2017
I have now switched to Realm, a much better way to use databases on any platform. Spend a few hours learning it, and save countless hours in your app development career.
1. Content Providers are not Thread Safe
By default content providers are not thread safe. If you have multiple threads using a content provider you can see many different exceptions being thrown and other data inconsistencies. The easiest way to fix this is to use the synchronized keyword on each of the public methods exposed by the content provider.
In this way only one thread at a time can access these methods.
2. Play nice when doing lots of writes
I have the need in the new Serval Maps application to import data from binary files into the database used internally by the application. In order to do this and play nice with the rest of the application it is best to:
Spawn a new thread to undertake the import so other threads are not adversely impacted, in particularly the thread in charge of updating the UI; and
Pause briefly at the end of the each import to give other threads which need to use the synchronized methods more of a chance.
3. Content providers force you to think laterally sometimes
The way that content providers in Android work is to provide a layer of abstraction between the rest of your code and the underlying database. This is mainly due to the fact, as far as I can tell, that content providers can access data from places other than databases.
This means that you can’t execute raw SQL queries on the underlying database and you need to specify the various components of a SQL query using variables passed to the various methods such as the query method. If you have a task that doesn’t fit into the way that SQL is handled by a content provider you have two options:
Think laterally about the query, maybe you can get the data that you need by alternative queries and accessing the results from the cursor; and
Use a URI for accessing the data normally and a special URI that is matched to a specific query for those tasks that don’t have alternatives.
Content Providers are used when you want to share your data across applications.
If you have a database attached with an application and you want another application to use some data, you can implement a content provider that exposes the data
The main difference is: when your app needs to share information to another apps, use Content-Provider. SQLite only storage data for the app who creates it
I read this answer while looking for same doubt, so thought of sharing it.
it states -
It's good practice to provide the extra level of abstraction over your data to make it easier to change internally. What if you decide to change the underlying database structure at a later time? If you use a ContentProvider you can contain all the structural changes within it, where as if you don't use one, you are forced to change all areas of the code that are affected by the structural changes. Besides, it's nice to be able to re-use the same standard API for accessing data rather than littering your code with low-level access to the database.
So, using a content provider would be a good idea.
Think of advanced Content Management Systems. Each object (page, image, news article, event item, etc.) has a content, an address, user permissions, and ways to interact with it from different parts of the system. Content Providers do that for Android. You can now share files or images you may have stored in your application. You can also create custom sharable objects, like bussiness contacts, editable notes, etc. And specify security and the default application to deal with such object when you open them from any other application.
One difference is that Content Providers have platform support for Content Observers. Your going to need to implement your own Observable pattern for a SQLite database.
How to automatically re-query with LoaderManager
ContentObserver for SQLite?