I am working on an android app that is storing very important data in local sqlite database. Is there some possibility that data will be lost? I mean beside uninstalling the app or clearing its data
TLDR: there's no 100% safe storage. In practice the actual answer is no.
Like with all storage, there are risks (ordered from more likely to less likely):
Malicious users exploit a vulnerability on the system to mess with your data.
Physical support damage, by either external factors (e.g. radiation, strong electromagnetic fields) or by physical wearing (entropy will eventually win)
Filesystem corruption (e.g. bugs in the filesystem driver)
The only where the data is lost:
1. User clears the appData
2. User Reset's the phone.
So you can store the db file in an external storage with encryption. So the db remains even after the user clears the app data.
The best way to protect the data is to use GDrive integration to backup the db file. So there's always a backup available in the cloud.
Related
The android documentation has the following options below but does not explain what circumstances each is best for. What are the pros and cons of each method? e.g. Under what conditions would SQL be better than Shared Preferences?
Shared Preferences
Internal Storage
External Storage
SQLite Databases
Network Connection
Different Storage options in Android
Content Providers
Consider the structured data added to the device from application1 is
not accessible to another application2 present in the same device but
the profile photo added to the device by application1 is available to
the application2 running in the same device
Consider android device as a city, the applications in it are the
houses in the city, people in the houses(application) are the data.
Now content provider is like an broker in the city(android device).
This broker provide access for the people in the city for finding
different houses referring as the content provider in the android
device provide access for the data in the device for different
applications.
Shared Preferences
Consider I have an App say a Face book App which I use to log in to
my account.
Now the very first time I enter my username and password to get
access to my account. Say I log out of the application an hour later
again I use the same Face book App to login again to my application.
I have to enter username and password again to login to my account
and I set a theme to my application and other settings on how my app
looks in my current phone
This is un-necessary because consider I am using my phone to login to
the application. So I will always use my phone to login again and
again, thus entering my credentials again and again is more work
shows it’s not a user friendly app
Shared Preferences is very handy in such scenarios where I can use
its feature to share my data in a xml file Which physically exists in
the Android app installed in my phone which is not destroyed even if
the app is closed. Here we can save user preferences data of the
current application.
As a result next time I open my app in my phone I can see the data
automatically filled in the necessary fields and the settings are
File Storage
In Android we can use the device storage space to store the data in
it for the applications. The type of data involves things such as a
text file, image file, video file, audio file etc.
As seen in the figure as we can see that there are two places we can
do this. One way is to write the raw files into primary /secondary
storage. Another way is to write the cache files into the
primary/secondary storage.
There is also difference between storing raw data and the cache data,
the raw data once stored in memory by user has to be explicitly
deleted by the user explicitly otherwise it would exist till then.
Cache data stored in memory is not a permanent data because the
system automatically deletes it if it feels there is shortage of
memory.
Internal Storage:
Consider a user in an application has stored data in internal
storage, then only that user of that application has access to that
data on the mobile and that data is automatically deleted when the
user uninstalls the application. Speaking of which internal memory is
private.
The apps internal storage directory is stored using the name package
name in a special place in the android file system.
Other apps or users of current app have no access to the file set by
a particular user and a particular app unless it is explicitly made
available to the user for readable/writable access.
SQLite
Sqlite is used to store more structured data locally in a mobile
where the android app is running. Structured data involves as of
which shown in the figure like a student’s information in the form of
rows and columns.
Sqlite offers similar functionality like Mysql and oracle but with
limited functional features. Some of the things involve performing
query operations on tables. There are features though like creating
views but also some features are not available like stored procedure.
Sqlite is very helpful in storing complex and large data which can be
downloaded once and can be used again and again until the application
is running. When the application is closed the sqlite database is
also destroyed.
Putting all the pieces together
Shared preferences are good for storing ... an application's preferences, and other small bits of data. It's a just really simple persistent string key store for a few data types: boolean, float, int, long and string. So for instance if my app had a login, I might consider storing the session key as string within SharedPreferences.
Internal storage is good for storing application data that the user doesn't need access to, because the user cannot easily access internal storage. Possibly good for caching, logs, other things. Anything that only the app intends to Create Read Update or Delete.
External storage. Great for the opposite of what I just said. The dropbox app probably uses external storage to store the user's dropbox folder, so that the user has easy access to these files outside the dropbox application, for instance, using the file manager.
SQLite databases are great whenever you are going to use a lot of structured data and a relatively rigid schema for managing it. Put in layman's terms, SQLite is like MySQL or PostgreSQL except instead of the database acting as a server daemon which then takes queries from the CGI scripts like php, it is simply stored in a .db file, and accessed and queried through a simple library within the application. While SQLite cannot scale nearly as big as the dedicated databases, it is very quick and convenient for smaller applications, like Android apps. I would use an SQLite db if I were making an app for aggregating and downloading recipes, since that kind of data is relatively structured and a database would allow for it to scale well. Databases are nice because writing all of your data to a file, then parsing it back in your own proprietary format it no fun. Then again, storing data in XML or JSON wouldn't be so bad.
Network connection refers to storing data on the cloud. HTTP or FTP file and content transfers through the java.net.* packages makes this happen.
SharedPreferences is mainly for application-specific settings that you can access via your Settings menu - like application settings. It's a good idea to keep everything simple here - mostly boolean flags, short strings, or integers. SharedPreferences data persist on device reboot, and are removed along with app uninstallation. Data is saved as a key-value pair.
Internal Storage is mostly used for larger non-persistent data storage. You utilize internal storage if you want to process an image, a short video clip, a large text file, etc. But you don't store the processed data in the internal storage - its function is more like a CPU's RAM. The amount of available internal storage for your application depends on the device, but it's always a good idea to keep anything under 1MB. Data is referenced via its file path.
External Storage does not only refer to the SDCard storage, but for higher-end phones, this can mean internal mountable storage (like in the Galaxy Nexus or S2). This is where you store the large video files, the high-resolution images, and the 20-megabyte text file you want to parse in your application. This is also a place to store data that you want shared across devices if you swap sd cards. Data is also referenced via its file path.
SQLite Databases is where you'd store pretty much anything you want in a regular database - with the advantage of organizing things into tables, rows, and columns. It works best with things that you want displayed in the UI as lists - the best example would be the great invention knows as the CursorAdapter. Data stored here also persist on device reboot, and are removed with app uninstallation. You can also share data across applications with sqlite db if you hook it up to a ContentProvider. Data is accessed using a Cursor, where you can call methods as if you're executing sql statements.
Network Connection is not really a data storage technique, but can be a way of persisting data for a specific user provided the device is connected to the internet, using some sort of authentication. You have to balance out between downloading data every time the app needs it, or having a one-time data sync, which would ultimately lead to another of the storage options mentioned above.
Shared preferences are key/value pairs, nothing more. So if you want to keep track of say, students and their test score, it really won't work well for that.
A database is just that, a database. You can define as many columns (and tables) as you need to get the job done.
If it's preferences for your app, use shared preferences (almost any preference I can think of can be done that way), if it's anything else more complicated, use a database.
The android documentation has the following options below but does not explain what circumstances each is best for. What are the pros and cons of each method? e.g. Under what conditions would SQL be better than Shared Preferences?
Shared Preferences
Internal Storage
External Storage
SQLite Databases
Network Connection
Different Storage options in Android
Content Providers
Consider the structured data added to the device from application1 is
not accessible to another application2 present in the same device but
the profile photo added to the device by application1 is available to
the application2 running in the same device
Consider android device as a city, the applications in it are the
houses in the city, people in the houses(application) are the data.
Now content provider is like an broker in the city(android device).
This broker provide access for the people in the city for finding
different houses referring as the content provider in the android
device provide access for the data in the device for different
applications.
Shared Preferences
Consider I have an App say a Face book App which I use to log in to
my account.
Now the very first time I enter my username and password to get
access to my account. Say I log out of the application an hour later
again I use the same Face book App to login again to my application.
I have to enter username and password again to login to my account
and I set a theme to my application and other settings on how my app
looks in my current phone
This is un-necessary because consider I am using my phone to login to
the application. So I will always use my phone to login again and
again, thus entering my credentials again and again is more work
shows it’s not a user friendly app
Shared Preferences is very handy in such scenarios where I can use
its feature to share my data in a xml file Which physically exists in
the Android app installed in my phone which is not destroyed even if
the app is closed. Here we can save user preferences data of the
current application.
As a result next time I open my app in my phone I can see the data
automatically filled in the necessary fields and the settings are
File Storage
In Android we can use the device storage space to store the data in
it for the applications. The type of data involves things such as a
text file, image file, video file, audio file etc.
As seen in the figure as we can see that there are two places we can
do this. One way is to write the raw files into primary /secondary
storage. Another way is to write the cache files into the
primary/secondary storage.
There is also difference between storing raw data and the cache data,
the raw data once stored in memory by user has to be explicitly
deleted by the user explicitly otherwise it would exist till then.
Cache data stored in memory is not a permanent data because the
system automatically deletes it if it feels there is shortage of
memory.
Internal Storage:
Consider a user in an application has stored data in internal
storage, then only that user of that application has access to that
data on the mobile and that data is automatically deleted when the
user uninstalls the application. Speaking of which internal memory is
private.
The apps internal storage directory is stored using the name package
name in a special place in the android file system.
Other apps or users of current app have no access to the file set by
a particular user and a particular app unless it is explicitly made
available to the user for readable/writable access.
SQLite
Sqlite is used to store more structured data locally in a mobile
where the android app is running. Structured data involves as of
which shown in the figure like a student’s information in the form of
rows and columns.
Sqlite offers similar functionality like Mysql and oracle but with
limited functional features. Some of the things involve performing
query operations on tables. There are features though like creating
views but also some features are not available like stored procedure.
Sqlite is very helpful in storing complex and large data which can be
downloaded once and can be used again and again until the application
is running. When the application is closed the sqlite database is
also destroyed.
Putting all the pieces together
Shared preferences are good for storing ... an application's preferences, and other small bits of data. It's a just really simple persistent string key store for a few data types: boolean, float, int, long and string. So for instance if my app had a login, I might consider storing the session key as string within SharedPreferences.
Internal storage is good for storing application data that the user doesn't need access to, because the user cannot easily access internal storage. Possibly good for caching, logs, other things. Anything that only the app intends to Create Read Update or Delete.
External storage. Great for the opposite of what I just said. The dropbox app probably uses external storage to store the user's dropbox folder, so that the user has easy access to these files outside the dropbox application, for instance, using the file manager.
SQLite databases are great whenever you are going to use a lot of structured data and a relatively rigid schema for managing it. Put in layman's terms, SQLite is like MySQL or PostgreSQL except instead of the database acting as a server daemon which then takes queries from the CGI scripts like php, it is simply stored in a .db file, and accessed and queried through a simple library within the application. While SQLite cannot scale nearly as big as the dedicated databases, it is very quick and convenient for smaller applications, like Android apps. I would use an SQLite db if I were making an app for aggregating and downloading recipes, since that kind of data is relatively structured and a database would allow for it to scale well. Databases are nice because writing all of your data to a file, then parsing it back in your own proprietary format it no fun. Then again, storing data in XML or JSON wouldn't be so bad.
Network connection refers to storing data on the cloud. HTTP or FTP file and content transfers through the java.net.* packages makes this happen.
SharedPreferences is mainly for application-specific settings that you can access via your Settings menu - like application settings. It's a good idea to keep everything simple here - mostly boolean flags, short strings, or integers. SharedPreferences data persist on device reboot, and are removed along with app uninstallation. Data is saved as a key-value pair.
Internal Storage is mostly used for larger non-persistent data storage. You utilize internal storage if you want to process an image, a short video clip, a large text file, etc. But you don't store the processed data in the internal storage - its function is more like a CPU's RAM. The amount of available internal storage for your application depends on the device, but it's always a good idea to keep anything under 1MB. Data is referenced via its file path.
External Storage does not only refer to the SDCard storage, but for higher-end phones, this can mean internal mountable storage (like in the Galaxy Nexus or S2). This is where you store the large video files, the high-resolution images, and the 20-megabyte text file you want to parse in your application. This is also a place to store data that you want shared across devices if you swap sd cards. Data is also referenced via its file path.
SQLite Databases is where you'd store pretty much anything you want in a regular database - with the advantage of organizing things into tables, rows, and columns. It works best with things that you want displayed in the UI as lists - the best example would be the great invention knows as the CursorAdapter. Data stored here also persist on device reboot, and are removed with app uninstallation. You can also share data across applications with sqlite db if you hook it up to a ContentProvider. Data is accessed using a Cursor, where you can call methods as if you're executing sql statements.
Network Connection is not really a data storage technique, but can be a way of persisting data for a specific user provided the device is connected to the internet, using some sort of authentication. You have to balance out between downloading data every time the app needs it, or having a one-time data sync, which would ultimately lead to another of the storage options mentioned above.
Shared preferences are key/value pairs, nothing more. So if you want to keep track of say, students and their test score, it really won't work well for that.
A database is just that, a database. You can define as many columns (and tables) as you need to get the job done.
If it's preferences for your app, use shared preferences (almost any preference I can think of can be done that way), if it's anything else more complicated, use a database.
I have a problem. I am using xyz.db file and which is stored in asset folder. I am copying all data from xyz.db to application db which is stored in data/data/com.xyz/abc.sqlite in storage folder. Now I want to secure asset's xyz.db file. Because It can be easily extract from apk by reverse engineering. Please help me to secure my asset folder's database file.
You can perform the following to make it relatively difficult to access data in DB.
Password protected zip file to contain db which at runtime should be extracted.
Encrypt the file with symmetric key and again at runtime decrypt it.
Utilize sqlcipher that performs encryption for Data at Rest.
In both the above cases you will need to worry about storing the password or key. There is no sure shot way to protect the file but the above would require more effort and should be added as basic protection.
There's no final solution to your problem.
Any technique you'll use can be beaten by a determined skilled attacker.
You have to accept that if you want to store database xyz.sql in your apk file and you later want your app to use it, then it will be also possible for someone that reverse your app to retrieve it. Basically just because the plain text information at a certain moment will be available on the phone.
Hope i've been clean enough
Keep security in mind
As usual in Android the access rights of the database file determine who can use your database. If you follow the standard way presented in the following posts of this series, your database file will be located within the private directory of your app. This means that your app owns the database file and no one else can access it. Even using the other less common ways to create the database you can only grant access to the file. Thus others can access all of your database or nothing. There is no middle ground.
Still: You should never rely on data being safe from prying eyes in the database. Any sensitive data should be encrypted. Very sensitive data should not be stored on the device at all. Keep in mind that if the device gets lost, any misbehaving finder of the device can gain access to the database file as well as to your app. On a rooted device all files can be read. Apps like SQLite Editor make it easy to read even sensitive data – if they are not encrypted:
In cases where data privacy is of utmost importance, you have to revert to secured services or force the user to enter a secret every time before encrypting and storing the data or reading and decrypting them respectively.
source
Does Android do an encryption to the sqlite database owned by the application ?
Reading the documentation and various forums I could only gather that the database is protected by the application owning it and it is not visible to the other apps .
Can this mean that it is a very viable security hole ? Why doesn't Android advice developers to encrypt the sqlite databases ?
Android does not encrypt sqlite databases (or any files created by the application, no matter their location). Files (including a database) created in internal storage are protected by uid, which means they cannot be easily read by other apps. However, if the device is rooted, they can be easily copied and/or read.
It's only insecure if you store data that shouldn't be stored there. Your options if you want a higher level of security are (a) to store encrypted data (b) use some other encrypted database service.
Android provides following options for data storage:
Shared Preferences - Store private primitive data in key-value pairs.
Internal Storage - Store private data on the device memory.
External Storage - Store public data on the shared external storage.
SQLite Databases - Store structured data in a private database.
Cache - Cache data during application is running.(May be clear on shortage of space)
What I was wondering is the life time of these storage.
Which storage(s) clears on application close (Obviously cache)
Which storage(s) clears on application re-install
Which storage(s) clears on application update
Which storage(s) clears on application un-install
Which storage(s) clears on application clear data
Which storage(s) clears on mobile factory reset
Which storage(s) clears on Rom upgrade/change
An extension to this question is, using which storage my data is more secure in terms of loss and in terms of access by unauthorized resources (users, applications in case of root).
Which storage(s) clears on application close (Obviously cache)
Cache
Which storage(s) clears on application re-install
Cache
Which storage(s) clears on application update
Cache
Which storage(s) clears on application un-install
Cache, SQLite, Shared Preferences
Which storage(s) clears on application clear data
Cache, SQLite, Shared Preferences
Which storage(s) clears on mobile factory reset
It clears all, but not External Storage data
An extension to this question is, using which storage my data is more secure in terms of loss and in terms of access by unauthorized resources (users, applications in case of root).
It is best to store the data in the SQLite in the encrypted form. If your device is rooted then it means you can access even the SQLite.
Application Close : Cache files wont be deleted after application close.
As per android docs,
When the device is low on internal storage space, Android may delete these cache files to recover space. However, you should not rely on the system to clean up these files for you. You should always maintain the cache files yourself and stay within a reasonable limit of space consumed, such as 1MB. When the user uninstalls your application, these files are removed.
Application re-install and un-install : Cache, Shared Preferences, Internal Storage and Databases will be removed when un-installing the app.
Application Update: Usually everything you had on previous version will be restored. Docs are also not clear about this.
Application Clear data : Everything except External Storage will be deleted permanently.
Factory Reset: Doing a Factory Reset will erase all the apps and its data except preinstalled ones. You can restore your apps with google account but not data (If data not backed up with BackUp Api.
And for your final question,
There is NO Secure data storage if you store data in device. Even External data storage can be removed with USB file options. For secure data, you should maintain user data on your server and get it on demand.
Hope I was clear.
In answer to your question, persistent storage (SQL, Internal\External storage, SharedPreferences) acts mostly the same by default, and Cache acts different.
Persistent storage - will be NOT be cleared in following scenarios: 1, 2, 3 - will be cleared in other scenarios.
Internal\External storage - can survive application removal (if configured appropriately), so, has possibility to NOT be deleted on 4 & 5, if you specifically handle this.
Cache - will be cleared in all scenarios (you may be able to access cache after application close in certain instances, but not reliably).
7* (ROM updates can retain application data using a backup solution, otherwise, would act same as 6)