In Android applications, what is the best strategy to generate reports that can be viewed later by user? I'm talking about normal running of the application, not necessarily errors & exceptions.
I've a few options in mind but each one seems to have issues:
Logcat (use a specific tag “MyAppsLog”, provide user a functionality to read all statements logged using that tag)
Note: I believe the primary purpose of Logcat is for providing debugging info for the developer, not for the end user.
Manually open a flat file and append your logging statements to it.
Note: Can go this route if there is no standard mechanism for logging and report generation. Since this is such a standard requirement, I'm hoping not to re-invent the wheel. Also, if the application is re-installed the file can get lost. If written in sdcard/external storage to avoid this possibility, might not be private.
Use a third party tool like ACRA
or android-remote-stacktrace
Note: I think the purpose of these tools is crash-reporting, I don't think they are the best bet for standard report generation.
Many enterprise applications need to have a way to generate reports (normal running of app, not error), that the enterprise-user can view later. I'm hoping the answers here would be useful to more people than just me.
Well how about using an SQLite database, and dumping the log data in a table? Beats the flat file option I think. You can even aggregate dumped data or start queries on it.
Sqlite is heavy and taking lots of memory and process.
Advantages of flat file:
single point of all log info,
easy to manage,
easy to delete (clear cache)
performance (when file size grow up) performance of apps remain same
Anyone can read log file and can know which process is running on
if data is not big/complex, not that much important (if any one read it) then use flat file only.
Related
After analyzing my Android application with a security tool, it has detected a high level vulnerability "File unsafe delete check". I have investigated about this, and it seems that the problem is that the application uses "file.delete()".
That function is considered unsafe because data could theoretically be retrieved with a tool that scans all the storage device. So, if that way of deleting is "unsafe"... what is the "safe" way to delete files in Android? (to avoid getting that "security error" that is supposedly a "high level" one). What is the proper way to delete files in Android Development?
I am getting the same security warning in 2 different applications, one made with native Java and the other one with Xamarin Forms. Thank you very much!
what is the "safe" way to delete files in Android?
There is none for the vast majority of Android devices. You use delete() on File.
That function is considered unsafe because data could theoretically be retrieved with a tool that scans all the storage device
If the Android device happens to use a classic hard drive (spinning magnetic media), you can overwrite the data before deleting it. On any sort of flash media, that will be ineffective, as the physical location where the data is written can vary with each write operation ("wear leveling").
So, this really boils down to your objective:
If you feel that the user will be harmed if this data is available to be read, store it encrypted with a user-supplied passphrase.
If you are simply trying to avoid this warning, ask the developers of this "security tool" what they are expecting you to do. Or, find a better tool.
This is not an Android specific issue.
It has to do with how file systems work, and the physical storage media it self.
When you delete a file, regardless of API, what is actually deleted is the record in the files table.
The actual data on disk or flash storage remains.
There is a method for secure deletion:
Before deleting the file, overwrite its contents with garbage or zeros several times.
But, this method only works for magnetic media such as hard disks.
Android devices use NAND flash for storage.
Because the number of writes a NAND chip can take before it fails is a lot less than that of magnetic memory, these chips usually come with a mechanism that spreads out the write commands.
What this means is that even if you try to write random data or zeros over your file, there is no guarantee the actual data will be overwritten.
The writes may go to a different sector to avoid wear.
So, on one hand, for flash storage it is enough to overwrite the file once, but on the other hand, it is impossible to do correctly at application level.
If you want to make your application secure, you must make sure to store sensitive data encrypted.
Then, even if someone tries to read the raw storage, they wouldn't be able to recover the data.
Don't store user credentials (like passwords) in regular files on Android.
Use Android accounts API and let the OS manage security.
If you still need file storage but want to protect the data, encrypt it in memory and then write to file.
As said by the other answers the first thing to consider under a theoretical point of view is if there is really a need to store any sensitive information in files to be kept on customer side
If this is really the case, encryption is the real way to guarantee proper security. Files would be protected not only against the recovery after deletion but also during their known life on the device
That said, in the case of a vulnerability assessment - i.e. a static analysis of the code - it would not be immediate to detect that you are calling for a deletion [via file.delete()] of encrypted files. Or maybe you are just calling the deletion of files with nothing to hide
In both these cases the found vulnerability would just be a false positive. Which is part of the game because you can guess that it's quite complicated for an automated tool to understand if something really "deserves" protection or not
What you can do to get rid of the vulnerability is adding the logic to empty the files before calling file.delete(). You have a sample here for this purpose. This will solve the vulnerability detection you are experiencing
My app are sometime needed syncing with web servers and pull the data in mobile sqlite database for offline usages, so database size is keep growing exponentially.
I want to know how the professional app like whatsapp,hike,evernote etc manage their offline sqlite database.
Please suggest me the steps to solve this problem.
PS: I am asking about offline database (i.e growing in the size after syncing) management do not confuse with database syncing with web servers.
I do not know how large is your data size is. However, I think it should not be a problem storing reasonably large data into the internal memory of an application. The internal memory is shared among all applications and hence it can grow until the storage getting filled.
In my opinion, the main problem here is the query time if you do not have the proper indexing to your database tables. Otherwise, keeping the databases in your internal storage is completely fine and I think you do not have to be worried about the amount of data which can be stored in the internal storage of an application as the newer Android devices provide better storage capability.
Hence, if your database is really big, which does not fit into the internal memory, you might consider having the data only which is being used frequently and delete otherwise. This highly depends on the use case of your application.
In one of the applications that I developed, I stored some large databases in the external memory and copied them into the internal memory whenever it was necessary. Copying the database from external storage into internal storage took some time (few seconds) though. However, once the database got copied I could run queries efficiently.
Let me know if you need any help or clarification for some points. I hope that helps you.
For max size databases. AFAIK You don't want to loose what's on the device and force a reload.
Ensure you don't drop the database with each new release of your app when a simple alter table add column will work.
What you do archive and remove from the device give the user a way to load it in the background.
There might be some Apps / databases where you can find a documentation, but probably this case is limited and an exception.
So to know exactly what's going on you need to create some snapshots of the databases. You can start with that of one app only, or do it directly with several, but without analyzing you won't get a reliable statement.
The reasons might be even different for each app as databases and app-features differ naturally too.
Faster growth in size than amount of incoming content might be related to cache-tables or indexing for searches, but perhaps there exist other reasons too. Without verification and some important basic-info about it, it's impossible to tell you a detailed reason.
It's possible that table-names of a database give already some hints, but if tablenames or even fields just use meaningless strings, then you've to analyze the data inside including the changes between snapshots.
The following link will help in understanding what exactly Whatsapp is using,
https://www.quora.com/How-is-the-Whatsapp-database-structured
Not really sure if you have to keep all the data all the time stored on the device, but if you have a choice you can always use cloud services (like FCM, AWS) to store or backup most of the data. If you need to keep all the data on the device, then perhaps one way is to use Caching mechanisms in your app.
For Example - Using LRU (Least Recently Used) to cache/store the data that you need on the device, while storing the rest on the cloud, and deleting whats unneeded from the device. If needed you can always retrieve the data on demand (i.e. if the user tries to pull to refresh or on a different action) and delete it whenever its not being used.
I am currently working on a (commercial) logistics project. We build a (partially) automated storage system in which the goods are stored randomly (think of nano-amazon). The positions of the objects are stored on the main computer and we are at the moment implementing the offsite backup via WAL (any objections?). One of our problems is that we have to operate during a power blackout and we can't produce enough energy for our computers for the worst case duration of the blackout which could be several hours. [This probably will never happen as we are in Germany, but there are some regulations we need to fulfill].
So my idea is to use a tablet [cheaper than a laptop], send the WAL-files to it so that the user can access the data during the blackout. But so far, I have seen no server implementation for tablets (either android or ios). Isn't there any or did I just not find it?
But maybe I'm also moving into the wrong direction. The Database is rather small (<50000 objects in the warehouse with each < 1kb) and the information we need during blackout is just one table (object_id -> position_in_warehouse) so that I even think about writing this information into a file and using git to copy the changes to the tablet. We also only need to know which objects have been removed during the blackout so that this information can easily be migrated back to the original db.
Or do you have other ideas?
Does your time have any value to you? Discard the Android + PostgreSQL option right now.
Keep it simple. You can get a cheap laptop for practically nothing, especially second hand. Since you clearly don't care about it actually working as a backup option, that seems like a no-brainer. You can run a streaming replica with WAL archiving for fallback.
For your real fallback option, you're on the right track with writing out the data you require to a flat file and syncing just that. Remember to actually test it - you should actually use it occasionally and make sure it works.
BTW, for your WAL-streaming backups, I suggest PgBarman, which will manage retention and rotation for you. You should also do logical dumps, and remember to test your backups.
I don't think there's a port of Postgres to Android - to use WAL files you'd need a working server. Even if it was ported, then you can't ship WAL files from x86 server to Android tablet - master and slave have to be the same major version, OS and architecture.
You really should just periodically export your data from Postgres to a simple file (I'd recommend SQLite) and just download it from a server. I suppose your tablets use WiFi and this file would be like 10MB zip-compressed.
Alternatively you could use rsync to keep this file updated. Don't use git - it will keep all previous versions of this file on your tablet - it would grow rather fast.
I'm currently developing an Android game which saves data into a SQLite database. This is not really "sensitive" data, but I don't want users to be able to modify it (for obvious reasons of game balance, as it would be cheating). And it's quite easy to access and modify a SQLite db when your phone is rooted (there are plenty of applications for that in the market).
So should I even worry about that, or consider users with a rooted phone can do whatever they want including cheating and that's their choice? Or could I somehow encrypt the data I don't want them to modify, or add a MD5 checksum or something similar?
Another approach would be to abandon SQLite altogether and use some kind of binary files with game data.
Please let me know if some of you already encountered similar issues, and what are the approaches you followed, as well as the "good practices" in the Android gaming development community.
Thanks.
Root access for everybody and security are mutually exclusive.
Any application or user with root permissions can read and modify each and every file on your system, as well as all of the main memory. That doesn't leave many places to store a potential encryption key for the database.
You could hide parts of the key in the executables, configuration files etc, but everything you could come up with would be nothing more than obfuscation and security by obscurity.
If a user opts to grant root access to everybody, that's their decision, and it's not your job as an app developer to prevent any harm that might be caused.
Update:
Storing API keys in Android, is obfustication enough? is a pretty similar issue - it's about protecting API keys, but it's the same situation with regards to your options.
sqlcipher for Android might help here.
https://guardianproject.info/code/sqlcipher/
I think based on your requirement the best method is using consistency of data,
for example MD5 the score and time, then put score and time and MD5 in to the table, then every time wanting to use that row of DB check the MD5 of the score and time if the one in DB and the one which calculated are same, the row is consistent otherwise it was hacked!
You may find your happiness on Preferences Files Look here
1) I am wondering if the Android SDK has already predefined encryption functions or if one would have to write this all from scratch?
2) Suppose you have classes for encryption, either written by yourself or provided by the SDK, how to make sure, that the content of the RAM (that contains the key in plaintext) is never written to the permanent storage of the android device? I guess if too much RAM is used, there's some kind of swapping, besides that there are techniques like HTC's fastboot that write the whole content to the storage if I am not wrong? Any chance to prevent the RAM content for your software being saved on a permanent storage? Or is there maybe some kind of event before the RAM content is written to the permanenent storage, so one could wipe the key before that?
Thanks you very much for any hint!
1) I am wondering if the Android SDK
has already predefined encryption
functions or if one would have to
write this all from scratch?
You can do it with Java Cryptography Architecture (JCA). Mainly it is the javax.crypto.* package. Here is the JCA Reference Guide.
Also, this example may be helpful.
As to the second question - I just have not much to say. The only idea is to not keep a handle on the sensetive objects for long. Create per each need, use and nullify it as soon as used. Don't ever put into a static context.