Is it safe to always use "getFilesDir()" as the only destination to cache your app's data?
Will that directory always be available?
What happens when the destination has no free space? does Android handle that?
Will that directory always be available?
Yes.
What happens when the destination has no free space? does Android handle that?
Android has no ability to magically cause a device's on-board storage to increase in size. Android takes various steps to try to help the user clean up used storage space. However, eventually the user can still run out of space. Depending on how you are working with internal storage, you may get an IOException, or you may get a silent failure (e.g., write a zero-byte file).
Related
I am trying to create an Android application which attempts to retrieve deleted/lost data in Android devices. Is there anyway to do this? I prefer to use Flutter, but willing to use any other tech stack. I think, we might need to use low level languages. Can anyone help me out :)
The thing is, we need to access the segment of memory/storage which is not in the directory and not visible in the file explorer. And we plan to use some algorithms to make sense of the data/bits we get and show it to the user
No this is impossible. You can never do it with an Android application.
Accessing the free space with an android kernel is not possible.
You can try to use memory of the phone pretend to be a hard disk and use EaseUS Data Recovery on Windows
After analyzing my Android application with a security tool, it has detected a high level vulnerability "File unsafe delete check". I have investigated about this, and it seems that the problem is that the application uses "file.delete()".
That function is considered unsafe because data could theoretically be retrieved with a tool that scans all the storage device. So, if that way of deleting is "unsafe"... what is the "safe" way to delete files in Android? (to avoid getting that "security error" that is supposedly a "high level" one). What is the proper way to delete files in Android Development?
I am getting the same security warning in 2 different applications, one made with native Java and the other one with Xamarin Forms. Thank you very much!
what is the "safe" way to delete files in Android?
There is none for the vast majority of Android devices. You use delete() on File.
That function is considered unsafe because data could theoretically be retrieved with a tool that scans all the storage device
If the Android device happens to use a classic hard drive (spinning magnetic media), you can overwrite the data before deleting it. On any sort of flash media, that will be ineffective, as the physical location where the data is written can vary with each write operation ("wear leveling").
So, this really boils down to your objective:
If you feel that the user will be harmed if this data is available to be read, store it encrypted with a user-supplied passphrase.
If you are simply trying to avoid this warning, ask the developers of this "security tool" what they are expecting you to do. Or, find a better tool.
This is not an Android specific issue.
It has to do with how file systems work, and the physical storage media it self.
When you delete a file, regardless of API, what is actually deleted is the record in the files table.
The actual data on disk or flash storage remains.
There is a method for secure deletion:
Before deleting the file, overwrite its contents with garbage or zeros several times.
But, this method only works for magnetic media such as hard disks.
Android devices use NAND flash for storage.
Because the number of writes a NAND chip can take before it fails is a lot less than that of magnetic memory, these chips usually come with a mechanism that spreads out the write commands.
What this means is that even if you try to write random data or zeros over your file, there is no guarantee the actual data will be overwritten.
The writes may go to a different sector to avoid wear.
So, on one hand, for flash storage it is enough to overwrite the file once, but on the other hand, it is impossible to do correctly at application level.
If you want to make your application secure, you must make sure to store sensitive data encrypted.
Then, even if someone tries to read the raw storage, they wouldn't be able to recover the data.
Don't store user credentials (like passwords) in regular files on Android.
Use Android accounts API and let the OS manage security.
If you still need file storage but want to protect the data, encrypt it in memory and then write to file.
As said by the other answers the first thing to consider under a theoretical point of view is if there is really a need to store any sensitive information in files to be kept on customer side
If this is really the case, encryption is the real way to guarantee proper security. Files would be protected not only against the recovery after deletion but also during their known life on the device
That said, in the case of a vulnerability assessment - i.e. a static analysis of the code - it would not be immediate to detect that you are calling for a deletion [via file.delete()] of encrypted files. Or maybe you are just calling the deletion of files with nothing to hide
In both these cases the found vulnerability would just be a false positive. Which is part of the game because you can guess that it's quite complicated for an automated tool to understand if something really "deserves" protection or not
What you can do to get rid of the vulnerability is adding the logic to empty the files before calling file.delete(). You have a sample here for this purpose. This will solve the vulnerability detection you are experiencing
I am currently storing bitmap images inside my cache, each bitmap can be upto 3mb in size each. I am using getCacheDir() however, after reading androids documentation, I found this:
Note: you should not rely on the system deleting these files for you; you should always have a reasonable maximum, such as 1 MB, for the amount of space you consume with cache files, and prune those files when exceeding that space.
So I am considering switching my cache to using getExternalCacheDir(), but I am abit uncertain about this:
The platform does not always monitor the space available in shared storage, and thus may not automatically delete these files. Apps should always manage the maximum space used in this location. Currently the only time files here will be deleted by the platform is when running on JELLY_BEAN_MR1 or later and isExternalStorageEmulated(File) returns true.
I would like to have full control of the cache dir, because if files are randomly deleted this could really affect the running of my application. I have already set an upper limit for the size of the cache so it will never exceed a certain amount of space. How can I now prevent the system from monitoring and deleting from this cache as it pleases?
thanks
Don't think that cache is permanent storage. You mustn't store there files that you rely on. Cache used for storing data that you don't really want to reload or recreate, but you can do it if needed (as with loading pictures from web. Much faster to load from disk, if already done, but can be done without it). If you really depend on this files use getFilesDir() this will route you to specific app directory, that wouldn't be erased if no memory (but still can be erased by user in settings). Also, if you have data that is static and you need it always, you must store it in the assets folder, all other data must be considered as temporary (any time user or system can erase it) and checked for existence.
I did some testing and, with an Android emulator, outputted a few pathways:
Internal Storage context.getFilesDir() has path /data/data/package_name/files
Private External Storage with context.getExternalFilesDir(null) has path /storage/emulated/0/Android/data/packagename/files/
Public External Storage with Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES) has path /storage/emulated/0/Pictures/
In my app, I've defined an object that has a timestamp string and an array for images (more specifically, paths to those images), and this information is stored in a sqlite database.
But is this a bad idea? Because if someone wanted to move this app to another device, wouldn't the paths change? I'm a little confused as to how files can be moved around, how different devices impact this situation, etc. I am confused as to what the correct approach is for storing image references in the sqlite table. Do you store absolute paths like above? Or do you store a number representing whether the pic is internal/private external/public external and pull that way regardless of device/file structure?
I'm just trying to make sure I don't publish an app that lacks some critical feature simply because I misunderstood something. Like if I start using an app that can take pictures/store them internally, it's all on internal storage, all on this phone. But if I want to start saving pics to an SD card or something else, I have to change where I store the pics and how I reference them. And what if I upgrade my phone? I'd have to somehow carry/transfer all that stuff to a new infrastructure, etc. Or the "cloud", even.
In other words I am just trying to understand all the possible use cases, here, in terms of how Android app data is stored and transferred.
Your questions are a little broad. I will try to answer them in turns:
But is this a bad idea?
Not in itself. The issue here is how are you managing changes and the Android alterations that may occur in the future (as well as the versions you are currently supporting)
Because if someone wanted to move this app to another device, wouldn't the paths change?
Relative paths would never change. They were implemented (I am guessing around API 2~6) to ensure that hard disk management would not be necessary in apps. By getting the Enviroment.getExternalFilesDir(stuffs..) you would garantee a path to the same file, regardless of readl disk allocation.
how files can be moved around, how different devices impact this situation, etc.
Files from Desktops (PC, Mac, Linux, Ubuntu, etc) follow a contracted with Android devices, that simulate the same disk system. This way, a PC can save a file (png, mp3, custom, etc) in a "root folder" that becomes the same root for Enviroment.getExternalFilesDir(stuffs..) BUT data private to your application is never accessible (on non-rooted devices), to the outside system.
Do you store absolute paths like above?
You can. But I believe this goes against the system best-practices. Saving a relative path, then building a File object, in regards to that path is the correct approach. Do note that the File class in Android expects you to know a bit about the file you are refferencing.... either its extension, name regex, physical location, relational location, etc... you need to know at least one of those.
Finally, what I think you asked:
Can the user replicate the same content in different devices?
For this, you must ensure that the user is "know" for instance, using the Play Store account, you can then have a repository, and have devices syncronize with that repository. Or even add a "export" function, that copies files, and imports them somewhere else.
I currently have my app caching image files in the cache sub-directory for the application. The images are used in a ListView and stored in a HashMap of SoftReferences to Bitmaps.
So my question is this, what is the best way to cache these image files without inflating the space my application uses AND remains responsive from a user standpoint.
Things I am concerned about:
I know the user can clear the cache and that it is done automatically when space is low on internal memory, but I feel most users will see a several MB app and uninstall it. Also if the space is constantly low, my app will just keep downloading the images, making it appear slower.
Most devices have an SD card pre-installed, but what should I do when it is not inserted? The SD card may also be slower compared to internal storage, affecting my app's performance.
Should I include an option to choose the location of the cache?
Should I attempt to manage the size of my cache (be it in the /cache or /sdcard) or just forget about it?
Thank you for your time (its a long one I know) and please post any relevant experience.
I can't offer you a comprehensive set of best practices, but I can offer what I've learned so far:
Managing your cache is a good idea. My app's cache is such that I know that I'll never need more than a certain number of cached files, so whenever I insert a new file into the cache, I delete the oldest files until I'm under the limit I have set. You could do something similar based on size, or simply age.
Caching to the SD card, if it's available, is a good idea if your cache needs to take up a lot of space. You'll need to manage that space just as carefully, since it won't automatically clear that space for you. If you're caching image files, be sure to put them in a directory that begins with a dot, like "/yourAppHere/.cache". This will keep the images from showing up in the gallery, which is really annoying.
Letting the user choose the location of the cache seems like overkill to me, but if your audience is very geeky, it might be appreciated.
I haven't noticed a much of a penalty when caching to the SD, but I don't know how your app uses that information.
Everyone has good ideas. I like the idea of using SoftReference's, although I'm not sure how often those get cleaned up, as this varies so much from VM to VM. You might want to combine that with regular HashMap to prevent you entire cache getting cleared every few minutes.
EclipseLink has a few different cache implementations and pretty good documentation on them. You could probably take advantage of a few ideas from the implementation (e.g., LRU, MRU, etc.). e.g.,
hard cache
soft cache
combined hard/soft cache
Since you're tuning a cache down to the nitty-gritty, I would recommend tuning it to different devices based on the hard specs. This is normally bad design, but the scope of the hardware that your software runs on mandates it, IMHO. e.g.,
Detect the amount of available memory on the SD card. Most new smart phones come with multi-GB SD cards, and those are pretty hard to fill up with regular usage for most users. Use away! You can also detect the amount of space available on the SD card on startup, and increase/decrease the size of your cache on startup.
Detect the amount of available memory and configure your caches with that in mind. If a user is using a hardware-intensive application, I don't think they'll mind that it makes up 200MB of RAM and provides a very fast user experience, especially since they spent a lot of money to have a phone that has 1-2GB RAM.
Good luck!
Should I include an option to choose the location of the cache?
IMO: No, let make it more simplest as possible (Except you can include advance setting for expert user)
Should I attempt to manage the size of my cache (be it in the /cache
or /sdcard) or just forget about it?
IMO: This is optional, it is double sword: your more work on background will help user more convenience but also more bug prone
Use 3rd libs:
IMO using 3rd library as Picasso is better, it handle cache automatically by order: Memory cache -> Disk cache -> Network