We have an Android application which stores its data in a local SQLite database; primarily for performance, but also to allow for working off-line (as we are often in areas with low signal).
At the moment, the data is stored in encrypted format (passed down from our web servers), but this in itself causes a performance issue, where for example, if we want to search records for a particular "surname", we need to decrypt ALL of the data, rather than using a straight SQL query, to include where surname='Smith'
We cannot (as it currently stands) store the data in a more friendly 'open-format', as it's possible to 'root' the device, take a copy of the MySQL database, open it and read the data.
Is there the means (perhaps someone can provide an example) to either password protect the local SQLite database or somehow apply encryption, so that we can (from an application perspective) have the database available in open format, but so that if any would-be hacker got hold of the device and rooted it ... they would have a hard time reading our data?
I have searched for a suitable solution and cannot find any options for the SQLite database, any 3rd party software or any examples of code that do this.
SqlCipher, this will might work in your case
Remote Storage:
Your data is sensitive and needs to be accessed by the user on the go from different devices. If your app is a good one then the above line will hold true.
Security + Remote access from any device says you maintain your dB on a remote server.
Your flow can be :
User login --> Token --> Auth Token in every call --> Process request and get/put data in/from dB
Local Storage:
Let's say that you only want to store data locally and don't want to store it on the server. For this you can use public-key cryptography
You can use a public Key in your app to encrypt the data and store it. Now, you want to access the data. Request the private key from the server and decrypt it.
Again, to get access to private key you should use some form of authorization (or anyone can access your key).
Without the private key, even if a hacker roots the phone and gets his/her hands on the dB, the data would be useless.
Related
The android documentation has the following options below but does not explain what circumstances each is best for. What are the pros and cons of each method? e.g. Under what conditions would SQL be better than Shared Preferences?
Shared Preferences
Internal Storage
External Storage
SQLite Databases
Network Connection
Different Storage options in Android
Content Providers
Consider the structured data added to the device from application1 is
not accessible to another application2 present in the same device but
the profile photo added to the device by application1 is available to
the application2 running in the same device
Consider android device as a city, the applications in it are the
houses in the city, people in the houses(application) are the data.
Now content provider is like an broker in the city(android device).
This broker provide access for the people in the city for finding
different houses referring as the content provider in the android
device provide access for the data in the device for different
applications.
Shared Preferences
Consider I have an App say a Face book App which I use to log in to
my account.
Now the very first time I enter my username and password to get
access to my account. Say I log out of the application an hour later
again I use the same Face book App to login again to my application.
I have to enter username and password again to login to my account
and I set a theme to my application and other settings on how my app
looks in my current phone
This is un-necessary because consider I am using my phone to login to
the application. So I will always use my phone to login again and
again, thus entering my credentials again and again is more work
shows it’s not a user friendly app
Shared Preferences is very handy in such scenarios where I can use
its feature to share my data in a xml file Which physically exists in
the Android app installed in my phone which is not destroyed even if
the app is closed. Here we can save user preferences data of the
current application.
As a result next time I open my app in my phone I can see the data
automatically filled in the necessary fields and the settings are
File Storage
In Android we can use the device storage space to store the data in
it for the applications. The type of data involves things such as a
text file, image file, video file, audio file etc.
As seen in the figure as we can see that there are two places we can
do this. One way is to write the raw files into primary /secondary
storage. Another way is to write the cache files into the
primary/secondary storage.
There is also difference between storing raw data and the cache data,
the raw data once stored in memory by user has to be explicitly
deleted by the user explicitly otherwise it would exist till then.
Cache data stored in memory is not a permanent data because the
system automatically deletes it if it feels there is shortage of
memory.
Internal Storage:
Consider a user in an application has stored data in internal
storage, then only that user of that application has access to that
data on the mobile and that data is automatically deleted when the
user uninstalls the application. Speaking of which internal memory is
private.
The apps internal storage directory is stored using the name package
name in a special place in the android file system.
Other apps or users of current app have no access to the file set by
a particular user and a particular app unless it is explicitly made
available to the user for readable/writable access.
SQLite
Sqlite is used to store more structured data locally in a mobile
where the android app is running. Structured data involves as of
which shown in the figure like a student’s information in the form of
rows and columns.
Sqlite offers similar functionality like Mysql and oracle but with
limited functional features. Some of the things involve performing
query operations on tables. There are features though like creating
views but also some features are not available like stored procedure.
Sqlite is very helpful in storing complex and large data which can be
downloaded once and can be used again and again until the application
is running. When the application is closed the sqlite database is
also destroyed.
Putting all the pieces together
Shared preferences are good for storing ... an application's preferences, and other small bits of data. It's a just really simple persistent string key store for a few data types: boolean, float, int, long and string. So for instance if my app had a login, I might consider storing the session key as string within SharedPreferences.
Internal storage is good for storing application data that the user doesn't need access to, because the user cannot easily access internal storage. Possibly good for caching, logs, other things. Anything that only the app intends to Create Read Update or Delete.
External storage. Great for the opposite of what I just said. The dropbox app probably uses external storage to store the user's dropbox folder, so that the user has easy access to these files outside the dropbox application, for instance, using the file manager.
SQLite databases are great whenever you are going to use a lot of structured data and a relatively rigid schema for managing it. Put in layman's terms, SQLite is like MySQL or PostgreSQL except instead of the database acting as a server daemon which then takes queries from the CGI scripts like php, it is simply stored in a .db file, and accessed and queried through a simple library within the application. While SQLite cannot scale nearly as big as the dedicated databases, it is very quick and convenient for smaller applications, like Android apps. I would use an SQLite db if I were making an app for aggregating and downloading recipes, since that kind of data is relatively structured and a database would allow for it to scale well. Databases are nice because writing all of your data to a file, then parsing it back in your own proprietary format it no fun. Then again, storing data in XML or JSON wouldn't be so bad.
Network connection refers to storing data on the cloud. HTTP or FTP file and content transfers through the java.net.* packages makes this happen.
SharedPreferences is mainly for application-specific settings that you can access via your Settings menu - like application settings. It's a good idea to keep everything simple here - mostly boolean flags, short strings, or integers. SharedPreferences data persist on device reboot, and are removed along with app uninstallation. Data is saved as a key-value pair.
Internal Storage is mostly used for larger non-persistent data storage. You utilize internal storage if you want to process an image, a short video clip, a large text file, etc. But you don't store the processed data in the internal storage - its function is more like a CPU's RAM. The amount of available internal storage for your application depends on the device, but it's always a good idea to keep anything under 1MB. Data is referenced via its file path.
External Storage does not only refer to the SDCard storage, but for higher-end phones, this can mean internal mountable storage (like in the Galaxy Nexus or S2). This is where you store the large video files, the high-resolution images, and the 20-megabyte text file you want to parse in your application. This is also a place to store data that you want shared across devices if you swap sd cards. Data is also referenced via its file path.
SQLite Databases is where you'd store pretty much anything you want in a regular database - with the advantage of organizing things into tables, rows, and columns. It works best with things that you want displayed in the UI as lists - the best example would be the great invention knows as the CursorAdapter. Data stored here also persist on device reboot, and are removed with app uninstallation. You can also share data across applications with sqlite db if you hook it up to a ContentProvider. Data is accessed using a Cursor, where you can call methods as if you're executing sql statements.
Network Connection is not really a data storage technique, but can be a way of persisting data for a specific user provided the device is connected to the internet, using some sort of authentication. You have to balance out between downloading data every time the app needs it, or having a one-time data sync, which would ultimately lead to another of the storage options mentioned above.
Shared preferences are key/value pairs, nothing more. So if you want to keep track of say, students and their test score, it really won't work well for that.
A database is just that, a database. You can define as many columns (and tables) as you need to get the job done.
If it's preferences for your app, use shared preferences (almost any preference I can think of can be done that way), if it's anything else more complicated, use a database.
The android documentation has the following options below but does not explain what circumstances each is best for. What are the pros and cons of each method? e.g. Under what conditions would SQL be better than Shared Preferences?
Shared Preferences
Internal Storage
External Storage
SQLite Databases
Network Connection
Different Storage options in Android
Content Providers
Consider the structured data added to the device from application1 is
not accessible to another application2 present in the same device but
the profile photo added to the device by application1 is available to
the application2 running in the same device
Consider android device as a city, the applications in it are the
houses in the city, people in the houses(application) are the data.
Now content provider is like an broker in the city(android device).
This broker provide access for the people in the city for finding
different houses referring as the content provider in the android
device provide access for the data in the device for different
applications.
Shared Preferences
Consider I have an App say a Face book App which I use to log in to
my account.
Now the very first time I enter my username and password to get
access to my account. Say I log out of the application an hour later
again I use the same Face book App to login again to my application.
I have to enter username and password again to login to my account
and I set a theme to my application and other settings on how my app
looks in my current phone
This is un-necessary because consider I am using my phone to login to
the application. So I will always use my phone to login again and
again, thus entering my credentials again and again is more work
shows it’s not a user friendly app
Shared Preferences is very handy in such scenarios where I can use
its feature to share my data in a xml file Which physically exists in
the Android app installed in my phone which is not destroyed even if
the app is closed. Here we can save user preferences data of the
current application.
As a result next time I open my app in my phone I can see the data
automatically filled in the necessary fields and the settings are
File Storage
In Android we can use the device storage space to store the data in
it for the applications. The type of data involves things such as a
text file, image file, video file, audio file etc.
As seen in the figure as we can see that there are two places we can
do this. One way is to write the raw files into primary /secondary
storage. Another way is to write the cache files into the
primary/secondary storage.
There is also difference between storing raw data and the cache data,
the raw data once stored in memory by user has to be explicitly
deleted by the user explicitly otherwise it would exist till then.
Cache data stored in memory is not a permanent data because the
system automatically deletes it if it feels there is shortage of
memory.
Internal Storage:
Consider a user in an application has stored data in internal
storage, then only that user of that application has access to that
data on the mobile and that data is automatically deleted when the
user uninstalls the application. Speaking of which internal memory is
private.
The apps internal storage directory is stored using the name package
name in a special place in the android file system.
Other apps or users of current app have no access to the file set by
a particular user and a particular app unless it is explicitly made
available to the user for readable/writable access.
SQLite
Sqlite is used to store more structured data locally in a mobile
where the android app is running. Structured data involves as of
which shown in the figure like a student’s information in the form of
rows and columns.
Sqlite offers similar functionality like Mysql and oracle but with
limited functional features. Some of the things involve performing
query operations on tables. There are features though like creating
views but also some features are not available like stored procedure.
Sqlite is very helpful in storing complex and large data which can be
downloaded once and can be used again and again until the application
is running. When the application is closed the sqlite database is
also destroyed.
Putting all the pieces together
Shared preferences are good for storing ... an application's preferences, and other small bits of data. It's a just really simple persistent string key store for a few data types: boolean, float, int, long and string. So for instance if my app had a login, I might consider storing the session key as string within SharedPreferences.
Internal storage is good for storing application data that the user doesn't need access to, because the user cannot easily access internal storage. Possibly good for caching, logs, other things. Anything that only the app intends to Create Read Update or Delete.
External storage. Great for the opposite of what I just said. The dropbox app probably uses external storage to store the user's dropbox folder, so that the user has easy access to these files outside the dropbox application, for instance, using the file manager.
SQLite databases are great whenever you are going to use a lot of structured data and a relatively rigid schema for managing it. Put in layman's terms, SQLite is like MySQL or PostgreSQL except instead of the database acting as a server daemon which then takes queries from the CGI scripts like php, it is simply stored in a .db file, and accessed and queried through a simple library within the application. While SQLite cannot scale nearly as big as the dedicated databases, it is very quick and convenient for smaller applications, like Android apps. I would use an SQLite db if I were making an app for aggregating and downloading recipes, since that kind of data is relatively structured and a database would allow for it to scale well. Databases are nice because writing all of your data to a file, then parsing it back in your own proprietary format it no fun. Then again, storing data in XML or JSON wouldn't be so bad.
Network connection refers to storing data on the cloud. HTTP or FTP file and content transfers through the java.net.* packages makes this happen.
SharedPreferences is mainly for application-specific settings that you can access via your Settings menu - like application settings. It's a good idea to keep everything simple here - mostly boolean flags, short strings, or integers. SharedPreferences data persist on device reboot, and are removed along with app uninstallation. Data is saved as a key-value pair.
Internal Storage is mostly used for larger non-persistent data storage. You utilize internal storage if you want to process an image, a short video clip, a large text file, etc. But you don't store the processed data in the internal storage - its function is more like a CPU's RAM. The amount of available internal storage for your application depends on the device, but it's always a good idea to keep anything under 1MB. Data is referenced via its file path.
External Storage does not only refer to the SDCard storage, but for higher-end phones, this can mean internal mountable storage (like in the Galaxy Nexus or S2). This is where you store the large video files, the high-resolution images, and the 20-megabyte text file you want to parse in your application. This is also a place to store data that you want shared across devices if you swap sd cards. Data is also referenced via its file path.
SQLite Databases is where you'd store pretty much anything you want in a regular database - with the advantage of organizing things into tables, rows, and columns. It works best with things that you want displayed in the UI as lists - the best example would be the great invention knows as the CursorAdapter. Data stored here also persist on device reboot, and are removed with app uninstallation. You can also share data across applications with sqlite db if you hook it up to a ContentProvider. Data is accessed using a Cursor, where you can call methods as if you're executing sql statements.
Network Connection is not really a data storage technique, but can be a way of persisting data for a specific user provided the device is connected to the internet, using some sort of authentication. You have to balance out between downloading data every time the app needs it, or having a one-time data sync, which would ultimately lead to another of the storage options mentioned above.
Shared preferences are key/value pairs, nothing more. So if you want to keep track of say, students and their test score, it really won't work well for that.
A database is just that, a database. You can define as many columns (and tables) as you need to get the job done.
If it's preferences for your app, use shared preferences (almost any preference I can think of can be done that way), if it's anything else more complicated, use a database.
I have to develop an app both for iOS and Android. This app uses a small sqlite database with sensitive data, and depending on the user's input, it makes some process with this data. After this process is made, the database is no longer accessed unless the user has to enter a new input (it's not probable that this happens).
Since the database has sensitive data I have to protect it. I know is impossible to get a complete protection but I would like to make life harder for a possible attacker.
And since I don't know too much about security I'm not sure what would be the best alternative taking into account "difficulty / security".
I've thought in two alternatives:
1) Include the database in the apps and encrypt or obfuscate it.
But I guess it wouldn't be very difficult to get the database with a rooted Android.
2) Have the database stored in a server, each time the user enters the input the app downloads the database, the process is made and after that I remove the database from the app.
But the process takes 5-10 minutes and maybe this time is enough to get the database from the app, so I would have to add the same protection as in 1) plus a protection in the server.
Which would be the best option (difficult / security)? There would be any other options?
I've tried several Google searches but I'm confuse with all that information and I would like to try with the huge acknowledge of this community.
UPDATED:
Well, reading your answers I'm inclined to have a backend and not download the database to the app. It's not the preferred solution of the client, but it seem is the best by far so I'll try to convince it.
Anyway, due to the type of process the app has to do I'm not sure if it's possible with a backend.
Since it's a complete different question I would like to ask it in other thread: https://stackoverflow.com/questions/29942688/remote-sqlite-queries-from-app
NEW UPDATE:
I keep thinking in this and I don't find a good solution because the process is too complex. I've thought in something I would like to know if it's possible or if it has no sense (sorry but I don't know too much about backend development).
SERVER -> my sqlite database & PHP web services
DEVICE -> a txt file with user inputs (20MB)
Device: In the device the txt file is compressed (10MB)
Device: The app send the txt file to the server via POST web service.
Server: This txt file is stored in the server.
Server: This txt file is decompressed.
Server: This txt file is loaded into a new table InputTable inside my sqlite database.
Server: A new table UserFinalDataTable (almost 10000 rows) is created making lots of queries using InputTable and the rest of my database.
Server: UserFinalDataTable is converted into JSON and sended back to the device as the response to the web service called in step 2.
Device: The app receive this JSON and converts it to something useful for the app.
Is this possible or any (if not all) of the steps is impossible to achieve?
The most secure and reliable way is to store the database on your server encrypted with a symmetric encryption algorithm. Better yet if you also use an asymmetric encryption algorithm on top of that, if possible with some sort of forward secrecy, in case your server gets hacked. It all depends on what degree of security you want to achieve.
Another way would be to store the database on the device with the server's encryption key, when needed send the database to your server for decryption and return the decrypted output. Although this doesn't make sense in most situations, especially yours.
Neither. The most secure way is to keep the database on the server and access it only by webservice, with proper account authentication. If you send any data at all to a client, a sufficiently motivated attacker will get it. In the case of a whole database file, it would be trivial to root the device and read it. And if you encrpyt it, SQLite won't be able to read it (not to mention the decryption key would be local so easily reverse engineered).
The most secure strategy you say....backend server with a db and web service.
Use SQLCipher to encrypt your db.
It works either on iOS and Android
I currently work on a web message application , and I am storing the chat log on the database and some user data on share preferences.
The problem is, I wonder are there any way to make the data not accessible/ not readable when I open them in file explorer? Or how can I implment higher protection measures on the data ? I have searched for some approach but it somehow does not fit for the web messaging application.
For example:
Store data on server. If the data is chat log then there is a huge set of data and the network traffic to the server is very high
Encryption. If I use the SHA or else to encryt the data , it is not possible to read them
Thanks for helping
Database can be accessed if the phone is rooted, so you should encrypt the data that is stored in the database and decode it when reading from the database.
You probably could use one way RSA encryption, storing a key to decrypt the content (when required) on the server and the one way encryption key can be on the device. Reading the key alone should not create a lot of load on the server.
The key will be in the device memory at the time of decryption but intercepting in there may already be complex.
I want to know how safe it is to pack the database with the application in android. Can the database be easily accessed by the users? As the database that I have will have data which I dont want to be hacked by users for misuse, what is the best way to protect the database in mobile apps?
Also my application would use web service(contacting my own website) e.g. http:\www.mysite.com/services/xxx
My site will in turn return some data to the mobile app. If someone decompiles the java code(in apk), he will easily get access to the URL i am using for web service. How can i protect my data on website to be attacked by malicious users. If anyone gets to know the URL, he can simply type that URL in browser and get all data in json format which i dont want as that data can be quite sensitive. Even if I keep it encoded, then the user can get to know the encoding from the java code(which he gets after decompiling apk).
How to keep my DB safe from being misused?
If my application is to show the local places like restaurants, bars etc on mobile should i always fetch them from the website using web service or provide a local database with these details so that information can be fetched quickly. In this case , I can provide a UPDATE web servcie which will update the local database. But security of local DB is of great concern to me.
Can anyone please suggest where to keep the DB and how to safeguard it?
Rgds,
Sapan
Local databases and your apk file can be read by any rooted device easily. This tool can even decompile your resources as explained in this youtube tutorial (I never tried that myself actually).
So you would have to store your data encrypted in your database and decrypt it form your application code to be sure that noone can access it by simply getting the database form the data directory of his device.
You shouldn't put your sensitive data (like passwords etc) in the resource folder, because it can be decompiled, put it in your code.
Now some words to your JSON API. Hiding the URL is not enough, since the user can track your requests easily by a sniffer and get that anyway. You should provide a authentication mechanism to protect unauthorized access and also protect your communication by SSL. (E.g. using HTTP authentication - makes only sense when your server provides SSL.)
This are the things you should think about and decide yourself how sensitive your data actually is.
As far as I understand you're going to:
Pack initial DB in your APK file (say with res/asset folder)
During first run explode DB file from res/asset to application data folder
Then from to time fetch data into DB from website/webservice
In this case there are basically 2 vulnerabilities (stored data I mean):
Initial DB image, since it's packed with APK (which is in real life just ZIP archive), so anyone can unpack and see what's packed in your DB
DB file stored in application data folder (usually /data/data/MY_APPLICATION_PACKAGE/databases). This folder is accessible on rooted device, so again your data can easily be screened
The only option to be secured is to encrypt your database content. Easiest way to do it to store sensitive data in BLOBs (in form of XML of JSON) and encrypt/decrypt those BLOBs after/before actual usage of certain records.
Myself personally did it in my app - and it works well.
check this links for protecting your apk file for decompile
How to make apk Secure. Protecting from Decompile
Protecting Android apk to prevent decompilation, network sniffing etc
decompiling DEX into Java sourcecode