I am creating a data repository layer in my application which in turn serves data from two sources, either network API calls or getting data from local storage. When there is a request to the data repository layer, I first need to check if the data is present in the local storage or not. If not, I then make a call to my API to get the data. To check if the data is in local storage or not, I was thinking of two ways to implement this:
On app startup, I load all the data in the database into the memory (eg. lists, maps etc.) and then whenever I have to check of existence of data in the local storage, I check from the memory. This has a possible problem that I have faced earlier as well. When the app is in background, Android system, might clear up this allocated memory to serve memory to other apps. This makes things complicated for me.
Whenever I want to check the existence of data in the local storage, I directly make queries to SQL tables and decide based upon that. This is much more leaner and cleaner solution than the above mentioned case. The only worry for me here is the performance hit. I wanted to know if the SQLite database runs after being loaded into the memory or not? If yes, then the memory layer of data that I had created above is useless. If not, is it safe to keep on querying the SQLite database for each and every small data request?
SQLite caches some data, and the Android OS will keep the database file in its file cache, as long as there is enough memory.
So when using SQLite, your app's performance is automatically optimized, depending on available memory.
Related
The android documentation has the following options below but does not explain what circumstances each is best for. What are the pros and cons of each method? e.g. Under what conditions would SQL be better than Shared Preferences?
Shared Preferences
Internal Storage
External Storage
SQLite Databases
Network Connection
Different Storage options in Android
Content Providers
Consider the structured data added to the device from application1 is
not accessible to another application2 present in the same device but
the profile photo added to the device by application1 is available to
the application2 running in the same device
Consider android device as a city, the applications in it are the
houses in the city, people in the houses(application) are the data.
Now content provider is like an broker in the city(android device).
This broker provide access for the people in the city for finding
different houses referring as the content provider in the android
device provide access for the data in the device for different
applications.
Shared Preferences
Consider I have an App say a Face book App which I use to log in to
my account.
Now the very first time I enter my username and password to get
access to my account. Say I log out of the application an hour later
again I use the same Face book App to login again to my application.
I have to enter username and password again to login to my account
and I set a theme to my application and other settings on how my app
looks in my current phone
This is un-necessary because consider I am using my phone to login to
the application. So I will always use my phone to login again and
again, thus entering my credentials again and again is more work
shows it’s not a user friendly app
Shared Preferences is very handy in such scenarios where I can use
its feature to share my data in a xml file Which physically exists in
the Android app installed in my phone which is not destroyed even if
the app is closed. Here we can save user preferences data of the
current application.
As a result next time I open my app in my phone I can see the data
automatically filled in the necessary fields and the settings are
File Storage
In Android we can use the device storage space to store the data in
it for the applications. The type of data involves things such as a
text file, image file, video file, audio file etc.
As seen in the figure as we can see that there are two places we can
do this. One way is to write the raw files into primary /secondary
storage. Another way is to write the cache files into the
primary/secondary storage.
There is also difference between storing raw data and the cache data,
the raw data once stored in memory by user has to be explicitly
deleted by the user explicitly otherwise it would exist till then.
Cache data stored in memory is not a permanent data because the
system automatically deletes it if it feels there is shortage of
memory.
Internal Storage:
Consider a user in an application has stored data in internal
storage, then only that user of that application has access to that
data on the mobile and that data is automatically deleted when the
user uninstalls the application. Speaking of which internal memory is
private.
The apps internal storage directory is stored using the name package
name in a special place in the android file system.
Other apps or users of current app have no access to the file set by
a particular user and a particular app unless it is explicitly made
available to the user for readable/writable access.
SQLite
Sqlite is used to store more structured data locally in a mobile
where the android app is running. Structured data involves as of
which shown in the figure like a student’s information in the form of
rows and columns.
Sqlite offers similar functionality like Mysql and oracle but with
limited functional features. Some of the things involve performing
query operations on tables. There are features though like creating
views but also some features are not available like stored procedure.
Sqlite is very helpful in storing complex and large data which can be
downloaded once and can be used again and again until the application
is running. When the application is closed the sqlite database is
also destroyed.
Putting all the pieces together
Shared preferences are good for storing ... an application's preferences, and other small bits of data. It's a just really simple persistent string key store for a few data types: boolean, float, int, long and string. So for instance if my app had a login, I might consider storing the session key as string within SharedPreferences.
Internal storage is good for storing application data that the user doesn't need access to, because the user cannot easily access internal storage. Possibly good for caching, logs, other things. Anything that only the app intends to Create Read Update or Delete.
External storage. Great for the opposite of what I just said. The dropbox app probably uses external storage to store the user's dropbox folder, so that the user has easy access to these files outside the dropbox application, for instance, using the file manager.
SQLite databases are great whenever you are going to use a lot of structured data and a relatively rigid schema for managing it. Put in layman's terms, SQLite is like MySQL or PostgreSQL except instead of the database acting as a server daemon which then takes queries from the CGI scripts like php, it is simply stored in a .db file, and accessed and queried through a simple library within the application. While SQLite cannot scale nearly as big as the dedicated databases, it is very quick and convenient for smaller applications, like Android apps. I would use an SQLite db if I were making an app for aggregating and downloading recipes, since that kind of data is relatively structured and a database would allow for it to scale well. Databases are nice because writing all of your data to a file, then parsing it back in your own proprietary format it no fun. Then again, storing data in XML or JSON wouldn't be so bad.
Network connection refers to storing data on the cloud. HTTP or FTP file and content transfers through the java.net.* packages makes this happen.
SharedPreferences is mainly for application-specific settings that you can access via your Settings menu - like application settings. It's a good idea to keep everything simple here - mostly boolean flags, short strings, or integers. SharedPreferences data persist on device reboot, and are removed along with app uninstallation. Data is saved as a key-value pair.
Internal Storage is mostly used for larger non-persistent data storage. You utilize internal storage if you want to process an image, a short video clip, a large text file, etc. But you don't store the processed data in the internal storage - its function is more like a CPU's RAM. The amount of available internal storage for your application depends on the device, but it's always a good idea to keep anything under 1MB. Data is referenced via its file path.
External Storage does not only refer to the SDCard storage, but for higher-end phones, this can mean internal mountable storage (like in the Galaxy Nexus or S2). This is where you store the large video files, the high-resolution images, and the 20-megabyte text file you want to parse in your application. This is also a place to store data that you want shared across devices if you swap sd cards. Data is also referenced via its file path.
SQLite Databases is where you'd store pretty much anything you want in a regular database - with the advantage of organizing things into tables, rows, and columns. It works best with things that you want displayed in the UI as lists - the best example would be the great invention knows as the CursorAdapter. Data stored here also persist on device reboot, and are removed with app uninstallation. You can also share data across applications with sqlite db if you hook it up to a ContentProvider. Data is accessed using a Cursor, where you can call methods as if you're executing sql statements.
Network Connection is not really a data storage technique, but can be a way of persisting data for a specific user provided the device is connected to the internet, using some sort of authentication. You have to balance out between downloading data every time the app needs it, or having a one-time data sync, which would ultimately lead to another of the storage options mentioned above.
Shared preferences are key/value pairs, nothing more. So if you want to keep track of say, students and their test score, it really won't work well for that.
A database is just that, a database. You can define as many columns (and tables) as you need to get the job done.
If it's preferences for your app, use shared preferences (almost any preference I can think of can be done that way), if it's anything else more complicated, use a database.
I am developing a chat app that uses firebase database to store data. The usual approach while developing a chat app is to keep the database nodes synced so that you access the messages offline. So the problem rises when I implement the firebase offline capabilities to keep the data nodes synced. Firebase suggests two required steps for accessing data offline:
Enabling disk persistance
this is enabled according to the documentation by using this line of code (in my case I add it in application class):
FirebaseDatabase.getInstance().setPersistanceEnabled(true);
and
Keeping a node synced
this is enabled by simply adding keepsynced(true) to any databasereference that you wish to keep synced, like this:
ChatNode.keepSynced(true);
What is the difference between the two?
According to the firebase team answers on this site, I deduced that:
1) (Disk persistance) stores the data on the device disk to use them when needed, and data is stored wether you write data or read data.
a) If you write data offline: data is stored on disk and is sent to database when you go online again.
b) If you read data offline: the listener that was read online and was kept in disk and stored, you will be able to read it offline from disk.
2) (keep synced true) will keep a database reference synced in 2 ways:
a) If you are also using (disk persistence) with (keep synced) you will be able to keep data synced on disk ... which seems to be the default behavior of (disk persistence).
b) If you are using (keep synced) alone then you only store to what is known as the app memory.
The problem
I did set both of the methods, but my app is now very laggy and slow and sometimes stops on its own.
The question
If all the things that I said above are true, then would this method of offline capability be a heavy load on my app?
If I kept many listeners synced and set persistence enabled, then would the disk become full of data? Should I clean the data? Is the data on disk cleaned by itself in both methods? Is data cleaned by itself from memory?
I want to avoid the lagging and slow response in my app, thanks for your help.
You are right about your assumptions. If you are using FirebaseDatabase.getInstance().setPersistenceEnabled(true); means that Firebase will create a local copy of your database which also means that every change that is made while you are offline, will be added to a queue. So, as this queue grows, local operations and application startup will slow down. So the speed depends on the dimension of that queue. But rememeber, Firebase is designed as an online database that can work for short to intermediate periods of being disconnected and not as an offline database.
Second, if are using many listeners, don't forget to remove the listener accordingly to the life-cycle of your activity like this:
databaseReference.removeEventListener(valueEventListener);
My Android app is fetching data from the web (node.js server).
The user create a list of items (usually 20-30 but it can be up to 60+). For each item I query the server to get information for this item. Once this info is fetched (per item), it won't change anymore but new records will be added as time go by (another server call not related to the previous one).
My question is about either storing this info locally (sqlite?) or fetching this info from the server every time the user asks for it (I remind you the amount of calls).
What should be my guidelines whether to store it locally or not other than "speed"?
You should read about the "offline first" principles.
To summarize, mobile users won't always have a stable internet connection (even no connection at all) and the use of your application should not be dependant on a fulltime internet access.
You should decide which data is elligible for offline storage.
It will mainly depend on what the user is supposed to access most often.
If your Items don't vary, you should persist them locally to act as a cache. Despite the fact that the data mayn't be really big, users will welcome it, as your app will need less Internet usage, which may lead to long waits, timeouts, etc.
You could make use of Retrofit to make the calls to the web service.
When it comes to persisting data locally within an Android application, you can store it in several ways.
First one, the easiest, is to use Shared Preferences. I wouldn't suggest you this time, as you're using some objects.
The second one is to use a raw SQLite database.
However, I'd avoid making SQL queries and give a try to ORM frameworks. In Android, you can find several, such as GreenDAO, ORMLite, and so on. This is the choice you should take. And believe me, initially you might find ORMs quite difficult to understand but, when you learn how do they work and the benefits they provide us, you'll love them.
My app tracks school grades, calculates averages, etc. and stores all of this in a SQLite database. If a user has to reinstall or gets a new phone, I'd like to be able to restore their data.
It looks like most developers do this either by backing up to SD card or by using Android Backup Service through Google. I'm not sure which is the better method. I'd like restoring to be simple but reliable. I welcome any comments on this.
One thing I'm trying to understand is why Google says to extend BackupAgent instead of BackupAgentHelper if using a database.
If you have an SQLite database that you want to restore when the user re-installs your application, you need to build a custom BackupAgent that reads the appropriate data during a backup operation, then create your table and insert the data during a restore operation.
Why can't I just back up the database as a file and then restore the file? My SQLiteOpenHelper class already handles upgrades if db versions are different. I guess I could just abort on a downgrade.
Why can't I just back up the database as a file and then restore the
file? My SQLiteOpenHelper class already handles upgrades if db
versions are different. I guess I could just abort on a downgrade.
Reason: same database file may not work on different device models(even though most of the cases, it should work, there are cases where it will fail). It depends on parameters like page size etc set at sqlite engine level. Ideal way is to backup the data rather than copying the whole file
It's suggested that you avoid backing up the whole db file all the time mostly because that's a lot of redundant data traffic, especially if you've only changed one record in a large db. Being able to write per-record updates to the backup system is much more efficient (though of course is not nearly as simple to implement).
I'm starting with Android development, and I'm trying to do an application that will help users to find events around where they live. Most of those events are static, and therefore could be stored in an internal database. However, there are other events that might have to be added, or existing events that might have to be modified.
My questions are the following:
Which aspects should I take into consideration in order to decide whether should I use an internal database or external database?
If I decide to use an internal database, what are the approaches to update the user database with the new events or whatever changes that might have to be done?
Thanks :)
To extend a bit Kevin's answer I would add that one of the approach could be to use an external database that just return through a webservice a timestamp of the last database update. If this timestamp changes, your local database should be update (maybe by sending only differential changes).
Internal database is an interesting solution if : your app means to be running without connection or the data volume is too important to be requested at each launch.
Without too much detail about your app, I'd keep an internal database that you periodically update from a remote server (which I assume you're calling the "external database").