I'm writing an android app that makes use of many images(dozens). These images, like most of the other data in the app, are updated from a remote database. For the data I am going to have a local database and sync it with the remote database every time the remote db is updated.
The remote database will store the images as URLs, and my app will download the images from these urls to display them in the app. So I could just have my local database sync with the remote one and I'll have the URL of all the image files I'm using and I can re download them from the remote server every time the app is run, but this is obviously slow and wastes a lot of data.
What I want to do is, everytime the database is updated and needs to be synced, the app will sync its local db and download the new images from their URLs as usual, but then it will save the image files somewhere on the device, so next time the app is run it can just grab the images from the device.
I can't seem to find an effective way of doing this, perhaps it's because it's a bad idea to do it this way in the first place? Sharepreferences probably won't have enough room, external storage isn't available all the time, and I hear it's a bad idea to just store a big chunk of binary data(such as an image) in the local database.
What are my options here?
Start with Android Storage Options.
Further:
external storage isn't available all the time
That's true in theory, but in practice, you'll have external storage 99% of the time, especially if you're developing for newer devices. Note that "external storage" and SD Card aren't the same thing -- the terminology here is confusing. For this, use getExternalFilesDir().
Regardless, if your image storage is a reasonable amount (<100MB is reasonable IMHO), then you can just use internal storage. As of Android 3.x, this won't cause a device to run out of space like early devices did. In practice, you'll be fine 99% of the time. Here you might want to use the cache dir, returned by getCacheDir().
In one of my apps I have a large number of files that are synced with a CDN. I use the sync process and database to retrieve download URLs and MD5 hashes of the files. If the MD5 hash has changed, then I download the file again in the background. I simply store the file using the hash as the file-name, so I can easily resolve duplicate files. Periodically, I also walk the cacheDir to see whether there are any files that are no longer referenced in the database.
Related
Is there a place where I can store and manage my own images outside of internal storage? I don't want other apps to be able to see or access these images. Should I use external storage? Does such a place exist in the new MediaStore? It's fine if they're deleted when the app is deleted.
This solution needs to support API 21 or higher.
I know there are a lot of questions like this, but they're 10+ years old and a lot has changed since then.
Use case / background
I have an app where all data is stored locally on the device (no external servers).
Users can choose custom background images for journal entries. A user could choose to use a different image for each journal entry they create. They can create as many journal entries as they want. They may revisit those journal entries. So, I need to store an unknown amount of images for the lifetime of the app. I've been saving a copy of the images the user picks from the gallery in my local app storage via context.filesDir.
I noticed a crash Fatal Exception: android.database.sqlite.SQLiteDiskIOException disk I/O error (code 4874 SQLITE_IOERR_SHMSIZE) and after googling, I found This error may indicate that the underlying filesystem volume is out of space.
My concern is that my app is running out of internal storage space because of the user images I'm storing.
Where should I be storing these images? I originally chose internal storage because I wanted my users' images to be reasonably private (since I don't know if they're storing sensitive images or not). I also wanted to make sure the images would always be available even if the source image (chosen from user's media) is deleted. However, I hadn't considered the limits imposed on internal storage. Silly me!
Darshil's answer is correct. Using the recommended Storage Access Framework for your use case, you should use getFilesDir() which will return your app's internal storage, which is private to your app.
Where should I be storing these images?
problem is due to limit of resource which is out of our control. However if you really want to store all the images, you can take some approaches:
1. Online: Use some cloud servers for storing user data. This might cost you a lot.
2. Offline: Tell user that you have only the limited amount of storage and storing more images will require to delete some older ones.
3. Both: Store in device. When internal storage is running low, tell user to buy some type of premium subscription to store unlimited cloud photos.
Android does not provide a place for you to store private photos (can't be accessed by user or other apps) outside of app's internal storage. I know you can contradict me by saying that one can store them in external storage by using getExternalStoragePublicDirectory() but the problem is that it is a shared directory accessible to users and other apps and remains when the app is deleted.
So I suggest you to use getFilesDir() as the directory returned by it is hidden from users and is deleted when the app is deleted. And also implement a image compressing tool in your app which automatically compresses images when uploaded by the user and then save it to internal storage. This won't solve the problem completely but I guess it's a start.
I would recommend checking out AWS Amplify which allows you to integrate your application with AWS services. For what you have described, you could use amplify to give your app the ability to authenticate users and set up cloud storage on AWS S3 which you can configure to only allow users to access and edit files that they have uploaded. One nice thing about AWS Amplify and AWS S3 cloud storage is that there is a free tier which allows you to develop your application for little to no cost. Depending on the amount of data that you will be uploading, it may be quite a while before you surpass the free tier limits.
A guide like this one may help you learn more.
I am not sure if I fully understand your question, but writing a quick tip.
try cloud services to store your images like AWS s3, cloudinary. Cloudinary is much cheaper if you want to try like 25gb/month free.
With limited storage space appearing to be the root cause, I recommend intentionally using the UI to encourage users to use existing images over new images. (This can work in addition to the technical options provided in other answers)
For example, present the user with a list of available background images and a link to add a new image. In this case, using an existing image is 1 click with a preview, where getting a new image requires opening a new view and searching for the new image.
Using the UI in this way doesn't directly get you more space, but it can help you to more efficiently use the space you have by guiding users to use existing images. It also gives you a place to warn about (or limit) adding more images when there is not more space available.
*If you still must have more storage on the device outside of the app, you could try using the public space with encryption for "privacy".
I am creating an app where the user will find a topic of interest, and download audio and images related to this topic stored using NoSQL database. I don't want the user to have direct access to the file after downloading, since the content should belong to the creator. The data should be downloaded, not streaming, as in some cases the user may not have internet access.
I wasn't sure how to go about this. I thought the files could be downloaded to the resources folder for each instance of the app. First, I could not find how to do this, and I also wasn't sure if I would reach a maximum memory limit.
An alternative I thought of was to encrypt the files then decrypt them within the app.
Thanks!
Alex
I am downloading files from a remote server to be saved for use on an android device. These files are mostly images and audio but I would like it so that a user cannot access this data using methods other than the app.
The issue is that these files could be large and that's why I use external storage. The problem here is that users can access the data stored on external storage by hooking the phone up to a computer.
Apart from using internal storage, do I have any options to secure the files written to disk?
If I do have to use internal storage, are there any other disadvantages other than the fact that some users might have a small amount of internal storage?
Thanks for your time! Any help will be much appreciated! :)
EDIT 1 :
The data being downloaded is paid content (via In App Billing). This is why I'm concerned about restricting access.
EDIT 2 :
I am already adding a "." before the folder I'm saving my data to. I'm more concerned about 'power' users who would know to look in hidden folders.
Seems like you are populating app's cache from web. It is always recommended to use external storage for such purposes because some users may not have enough internal memory to accommodate this.
Since you downloading includes images and audios, i wouldn't suggest you to use encryption/decryption as it will slow down the app's processing. So there's nothing much you can do to secure your data if you want to put on external memory and skip its encryption.
I personally don't think any harm/disadvantage to leave such data as it is (not encrypted). But if you have some of the files important, perhaps you may perform encryption on those particular files only.
I am trying to decide where to store images that are sent as part of instant messages coming in to an app. These messages are viewable in a conversation history view for sometimes a significant period of time after their original receipt. You can imagine any number of other use cases that would have a similar requirement, so the question here is on the "best practice for storing an indeterminate quantity and size of images"
Assumptions
SQLite storage is clearly a bad option since the image size is not
bounded.
It is neither desirable nor undesirable that these images be publicly available to other apps or discoverable by MediaScanner. We are assumed to be perfectly neutral on this point...
This leaves two parts to this question:
1. External Storage
It seems like external storage is to be preferred when available because it is likely to have more room than anything else:
The documentation says the following:
...use getExternalCacheDir() to
open a File that represents the external storage directory where you
should save cache files. If the user uninstalls your application,
these files will be automatically deleted. However, during the life of
your application, you should manage these cache files and remove those
that aren't needed in order to preserve file space.
Unlike internal storage cache, there is no statement made about the automatic reclamation of space on external storage by Android. Still the word "cache" makes me nervous.
Question 1: Do these files remain until explicitly deleted regardless?
Question 2: Is there any other external storage other than the cache that is automatically deleted upon app uninstall AND is preferable to the external cache for some specific reason?
2. Internal Storage
Clearly not every device has external storage, so there needs to be a provision for internal storage.
Question 3: Is the only practical difference between the internal cache retrieved through getCacheDir() and files created with openFileOutput(FILENAME, Context.MODE_PRIVATE) that Android may delete files in the cache directory when under pressure for storage space?
Do these files remain until explicitly deleted regardless?
I haven't read the code, but the javadoc explicitely says
The platform does not monitor the space available in external storage, and thus will not automatically delete these files. Note that you should be managing the maximum space you will use for these anyway, just like with getCacheDir().
Is there any other external storage other than the cache that is automatically deleted upon app uninstall AND is preferable?
None that I know of.
practical difference between the internal cache retrieved through getCacheDir() and files created with openFileOutput?
It's just a facility method, AFAIK
I have a large data that I have to use in my first android application, could any one guide me the best way to achieve this?
Is there something like embedded database in android framework?
Can i keep my database on my custom server and provide web service to my application?
Which one of the above is better, since I can offer the user to purchase the application and keep the data locally on his device in compressed format.
Thanks
You can ship an android application with the database file in the res/raw folder, and then programmatically copy the file onto SD Card. I personally compress my database file into a zip and uncompress it on first run.
Android has support for SQLite databases which is a very efficient, embedded, crossplatform database. Would recommend this.
Using a remote server might not be a good idea, since signal on wireless networks is often slow and unreliable.
See this answer for me details.
Ship an application with a database
But if the file size of the compressed database is >1MB then you'll need to download it from a webserver and store it locally (Android has a 1MB file size limit on internal files/resources)
Here are some brief thoughts.
Android has SQLite, you could use that for storing and accessing data.
But there are restrictions on such file sizes of 1mb.
I'd recommend setting up a simple web server that can accept GET requests.