I am thinking about how to ideally implement a cache layer in my Android app.
Currently I have generic Activities which display data coming from a remote server. The data is represented by a DTO TemplateInstance. Each TemplateInstance has a Map with Components in it and each of the components can have child components. The components themselves can be Text (String), Image (ByteArray) or Time (or whatever by sub-classing Component).
Currently my app loads a TemplateInstance from the server each time an Activity is started.
I would now like to implement a cache layer in the app, so that
the time to display data is reduced to a minimum,
the data is refreshed when it is changed on the server.
My strategy for this looks like this:
The started Activity loads the TemplateInstance from a local storage by an ID (if exists)
A UpdateService checks in the background if the TemplateInstance has changed on the server (using a version field in the database)
If the server version is greater than the local one or there is no local TemplateInstance then retrieve the data from the server, update the local store and update the view
I implemented this already successfully with db4o. There are just two problems with this solution:
db4o is under GPL (I cannot use it)
db4o is really slow when I load TemplateInstances which have many images (4 seconds for a query)
Now I am looking for the best replacement for db4o. My ideas about that are until now:
SQLite is not suitable because of the structure of the data
I don´t need database functionality - retrieving objects by ID would be enough
Holding the objects in memory would be significantly faster
The memory state should be saved to disk when application exits, so the objects can be reinstantiated at startup
What do you think is the best solution for this?
My research on this brought me to EHCache and JCS, which I have never used. Do you think they are appropriate, also in respect of resources on an Android phone? Or do you have other suggestions?
If I understand your situation correctly, I think you should implement your own caching solution.
I would use an HashMap<id, TemplateInstance>. The HashMap is serializable and you could store/load it using ObjectOutputStream and ObjectInputStream, respectively.
db4o is not limited to GPL, via its dOCL you can opt for other open source licenses and, if you can't go open source at all, it's also totally free for Android apps
You can use my fork of
simple-disk-cache This is an easy to use disk cache which uses DiskLruCache under the hood.
I have replace Apache Commons IO dependence with google guava. And have add new public methods:
put(String key, String value, Object[] array) - the value I have use to put the timestamp for max TTL (after this time in ms the cache expired)
and: T getArray(String key, Class type)
You can put array from Serializable objects like this:
cache.put("key", String.valueOf(new Date().getTime() + 60000), Groups[] arrayGroups);
and get it: Groups[] cacheArray = getCacheArray("key", Groups[].class);
you can put in cache Bitmap images too.
Related
I am creating a data repository layer in my application which in turn serves data from two sources, either network API calls or getting data from local storage. When there is a request to the data repository layer, I first need to check if the data is present in the local storage or not. If not, I then make a call to my API to get the data. To check if the data is in local storage or not, I was thinking of two ways to implement this:
On app startup, I load all the data in the database into the memory (eg. lists, maps etc.) and then whenever I have to check of existence of data in the local storage, I check from the memory. This has a possible problem that I have faced earlier as well. When the app is in background, Android system, might clear up this allocated memory to serve memory to other apps. This makes things complicated for me.
Whenever I want to check the existence of data in the local storage, I directly make queries to SQL tables and decide based upon that. This is much more leaner and cleaner solution than the above mentioned case. The only worry for me here is the performance hit. I wanted to know if the SQLite database runs after being loaded into the memory or not? If yes, then the memory layer of data that I had created above is useless. If not, is it safe to keep on querying the SQLite database for each and every small data request?
SQLite caches some data, and the Android OS will keep the database file in its file cache, as long as there is enough memory.
So when using SQLite, your app's performance is automatically optimized, depending on available memory.
My app will pull some json data when it is started and realistically, once the data has been pulled, I really won't need to pull it again for the duration of the user experience. The backend data may update a few times a day at most, so I think I would like to just grab the data upon app start and then use that same data for the duration and give the user an option to manually refresh the data. So, my question is, where/how should I store that data? I've got all of my data structures set up (classes and sub-classes). And there may be 200 or so instances of some of the classes. If I store everything as member variables in my activity_main class, it won't be available to other activities once the other activities are started. Storing them all in databases could be an option, but it sort of feels like overkill. I really don't need the data to persist between sessions. Is there a way to easily store it in memory and still have it easily accessible to all activities?
You should think about OS killing your app process on low-memory, so backing your data on disk is a good thing to do. Doing so you have an ability to show user data from disk cache while refreshing it in background from server.
Choosing the tool for data storage depends on the way you need to work with data.
Of course, there is an option to use Realm, but you should consider the fact that it is not the relational database. So if you have complex domain model with joins and other relational stuff for your business logic, I'd go with something other. It is thread-safe, also has migrations (but, as for me, migrarations are always pain, you just can not do anythig about it). Realm is supposed to be RxJava-friendly now (support added in v0.87) There are some disadvantages(part of them may already be fixed), but you should consider it before using.
As for more relational approach, there is SQLBrite library:
A lightweight wrapper around SQLiteOpenHelper which introduces reactive stream semantics to SQL operations.
It is not an ORM (but you can add some kind of it on top of SQLBrite if you wish: see this for more info). In fact, being alone this library is doing one thing (and doing it good) - it provides a mechanism for coordinating and composing the notification of updates to tables(Realm also has such ability) such that you can update queries(in SQL fashion) as soon as data changes. And it respects RxJava!
As an alternative to SQLBrite you can look at StorIO.
There are also lots of different ORM solutions, like GreenDAO, ORMLite etc.
But I'm pretty sure, one of the first two libraries (Realm or SQLBrite) will likely help you. So analyze your app, these libs and decide what fits better.
P.S. Great article on how RxJava would help you to work with data from different data sources (in-memory cache + disk cache + network) easily. Might be helpful!
I would still recommend a SQLite Databse, you can easily declare it as a 'in-memory' database, if that is what you want.
However.... I would be rather upset as a user of your application if it downloaded redundant data over and over. I would just recommend making a content provider and being done with it. This gives you access to a SyncAdapter, and defines clear boundaries between where code should go.
The 'trick' with making a good ContentProvider is to make good POJOs, that have methods to convert from POJO -> ContentValues and Cursor -> POJO(s).
Easiest thing is to do is store the json file in Apps data storage and parse the json every time you need.
But this is not recommended as it is costly to parse data every time.
Best option is to implement Realm (Replacement for Sqlite) which is very easy to implement and its amazingly fast.
Im wondering whats the best way to cache objects in an Android application? I am working with a REST application and have written a notification class that fetches number of new messages, events, guestbookposts etc from our API in JSON-format.
I use these numbers to display a badge on the icons so users can see if they have any new undread messages, events etc...
I would however like to cache this object for about 5 minutes or so to save performance, but its not as easy as just to set an expire date on the sharedpreferences class...so how do i do it i ask you?
You have a lot of options, your choice depends on application needs:
caching using SQLite database
caching to a file (internal/external storage)
LruCache util from support package
If you're going to cache images, you can use one of 'all-in-one' libraries:
UniversalImageLoader
Picasso
If you use HttpUrlConnection class to interact with server, take a look at HttpResponseCache class.
But none of these options provide built-in support of setting cache lifetime, you have to implement it by yourself if needed.
I will develop an android application with a lot of data (json files with some rows and CSV for graphics data with a lot of rows) , this data change every 5 minutes and replaces all the previous data (or mostly).
What are the best approaches to design this ? I have 2 options:
Save all the data in a sqlite db, and sync this by a IntentService.
save the data in json and csv files and replace this every 5 minutes.
Which approach will the best performance?
This considering the time to parse the files, sorting data, the download time and the consistency of data.
any other ideas?
PD:I need a cache system too, in case if i don't have internet and I need the previous stored data
Advantages of SQLite:
Changes are ACID
You can make complex requests faster (e.g. "give me only fields A,B from items with (C/D)>E")
I would bet more compact for big data (integers are stored as integers instead of a string of individual digit)
Only one file
You can merge old data with new data easily
You can update current data, even while using it
Concurrency can be handled
Advantages for JSON/CSV:
Easier to debug (plain text)
Faster to make a whole update (copy the new file + delete the old one)
For the original question the whole delete/replace of the data makes JSON/CSV the winner.
However, if the application was to retrieve partial data every 10s and merge/update it with the previous one, SQLite would be a better option.
Sqlite is mostly used when you want data to be saved and used in future. In your case data is changing every 5 minutes so its better to have JSON because every time to make the Database connection store and retrieve after 5 minutes will take some time.
UPDATE:
I also had the same Application in which the data was changing every time. In that case I used Map<K, V> and ArrayList to maintain the values, because as the data is changing everytime I think its not feasible to store the data in Sqlite everytime. It needs much time to perform DB Connection store, retrieve, update the data in Sqlite.
I recommend using JSON or some type of object serialisation unless:
You need ACID compliance for write operations
You need to report against the data which may involve copying the data to an external RDBMS
or
You wish to join those complicit in the overuse / abuse of databases, as commonly seen nowadays
Ideally this should depend on whether you need the previous data, for maybe comparing it with current data and so on. As a thumb rule, I use SQLite when you need the data to be stored and retrieved at a later stage. In case the data is only for display, I would rather keep it in program memory. Mind you this does not involve file operation.
Purpose of JSON and SQLite is completely different from each other
JSON = is used send and receive data between server and client.
SQLite= is used to store data.
I have an Android app, where a part of the app is a list of data which is currently contained in a string-array (in an xml resource). I currently release updates every so often to the actual app, which do nothing more than update this list of data. (yes, in hindsight this method was a bad idea to start with).
My goal now is to change this so that I will be able to only update that one part of the app that needs to be changed. I have a webserver, and am now serving a JSON version of the data off a URL. So all the app has to do is hit that URL, check if it changed (perhaps using a version number), and then update.
My problem lies in the actual implementation:
Where/how should I store this data? As a raw file? SharedPrefs? Database? [i.e. what are the pros and cons of each]
How can I preform a seamless upgrade where even if something devastating happens during the update [such as a user pulling a battery...], it still won't break the app?
Should the updating code live in a service?
I would separate your data crud into a small background service. Use the provided SQLLite. To verify data consistency you could use md5 checks, database rollback features and most importantly design a small set of tests. One only sending a partial file, i.e. "the broken transmission test", garbage file, etc. Keep it a separate and testable component of your app.