As we all know starting Android Q (API=29) introduced new Scoped Storage concept, which basically prevents access to external files/dirs, except media files. Namely:
Programmer can use internal/app specific storage (no limits) - getFilesDir()
The only external storage can be accessed is via getExternalFilesDir(), but during app uninstall data will be deleted
There's option to access external storage via MediaStore, but it works only for media files
App still can read any external file via: getContentResolver().openInputStream(uri)
In my case I have app, which stores user data in SQLite database in app specific directory, database size can be up-to several gigs.
In all previous versions of my app users used to backup data to external storage/SD/USB and transfer those data to their new device, so they could continue to use data (it's encrypted database containing any kind of private data, including audio/video/documents and so on).
Question: how can I backup this database? I couldn't manage to find appropriate way to save/backup data to any external storage...
P.S. Android autobackup service is unworkable due to the size of database.
Related
We are implementing a backup/restore system for the app. We used Google Drive API as documented in the Android Guide
We also store image URIs in the database. In the restoring process, we get URIs but lost permission to reach images when we re-install the app. We have the following exception.
Failed query: java.lang.SecurityException: Permission Denial: opening
provider ... requires that you obtain access using
ACTION_OPEN_DOCUMENT or related APIs
We get image URI via "ACTION_OPEN_DOCUMENT" but can't persist it after deleting/install the application. What are the related APIs? Do we need to move images in an app-specific folder and backup images itself also?
Thank you for reading this, any help is appreciated
What are the related APIs?
ACTION_OPEN_DOCUMENT_TREE and ACTION_CREATE_DOCUMENT are probably what they are referring to.
Do we need to move images in an app-specific folder and backup images itself also?
We cannot really answer that — that is a business decision as much as a technical one. However, if your app is uninstalled and reinstalled:
You lose access to content identified by Uri values that you obtained from the Storage Access Framework
On Android 10+, you lose access to any files that your older app installation created that survived the uninstall process (e.g., they were in a shared collection)
How you work around that is up to you. In addition to your proposal, you could:
Have all the images be stored in a single directory tree (obtained via ACTION_OPEN_DOCUMENT_TREE), store relative paths in your database, and request access to that tree again after your app is restored
Remove all of the Uri values from the database before backing it up and live without access to those images if the database is restored
Store the image data in the database itself (typically not recommended but technically possible)
Android documentation mentions this:
External storage directories: These directories include both a dedicated location for storing persistent files, and another location for storing cache data. Although it's possible for another app to access these directories if that app has the proper permissions, the files stored in these directories are meant for use only by your app. If you specifically intend to create files that other apps should be able to access, your app should store these files in the shared storage part of external storage instead
How do I make sure that other apps don't access the external storage? Also, is it true across all Android versions?
You need to create an abstraction using Content Provider.
"A content provider presents data to external applications as one or more tables that are similar to the tables found in a relational database. A row represents an instance of some type of data the provider collects, and each column in the row represents an individual piece of data collected for an instance."
Refer to the link above for documentation.
I have a SQLite database on my sdcard (/storage/sdcard/MyDir/mydb.sqlite) and try to write some data in it. Due to the Storage Access Framework I can not access the file directly and I need to get write access like it is described here. This grants me write access via a DocumentFile.
However, the database needs to accessed via JDBC. JDBC requires a String to the database file to establish a database connection. This leads to the following problems:
If I use the Java-Path I can access the database, but I have no writing permission. Even if the access was granted like described above.
If I use the Uri of the DocumentFile (having writing permission) JDBC does not find the database file.
I can place my database in the private directory of the application on the sdcard (/storage/sdcard/Android/data/my.application.package/) where I have writing permission. However, I would like to store the database outside that directory, since the database should be accessed by multiple applications.
I tried this suggestion, which works on the device storage, however does not work on the sdcard. On the sdcard I get this stacktrace:
android.database.sqlite.SQLiteCantOpenDatabaseException: unknown error (code 1294 SQLITE_CANTOPEN_ENOENT[1294]): Could not open database
at android.database.sqlite.SQLiteConnection.nativeOpen(Native Method)
at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:284)
at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:215)
at android.database.sqlite.SQLiteConnectionPool.openConnectionLocked(SQLiteConnectionPool.java:705)
at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:272)
at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:239)
at android.database.sqlite.SQLiteDatabase.openInner(SQLiteDatabase.java:1292)
at android.database.sqlite.SQLiteDatabase.open(SQLiteDatabase.java:1247)
at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:930)
at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:947)
When I access the database on the device storage (internal storage and external storage) I can write the database. Only if I access the database on the sdcard I have no writing permission.
Is there any possibility to get write access to specific files on the sdcard based on java.io.File like it is possible for DocumentFile?
Does anybody have a different idea how I can access the data on the sdcard with writing permission?
I have a SQLite database on my sdcard (/storage/sdcard/MyDir/mydb.sqlite)
Using removable storage as a transfer location for a SQLite database is OK. For example, using removable storage for backup and restore operations is fine.
However, for a live database, removable storage is very risky. On many devices, removable storage is removable, and you will suffer catastrophic problems if the user removes the storage while you are trying to write to your database.
And, as you are discovering, using removable storage for a live data is impractical.
the database needs to accessed via JDBC
Android has a built-in SQLite database API. Attempting to use Android's half-baked JDBC classes for accessing a SQLite database is very strange.
When I access the database on the device storage (internal storage and external storage) I can write the database
You will encounter problems on Android 10 and higher, as you no longer have direct filesystem access to most of external storage.
Is there any possibility to get write access to specific files on the sdcard based on java.io.File like it is possible for DocumentFile?
No, sorry. You do not have filesystem access to arbitrary locations on removable storage as of Android 4.4. Android 10 extends that to arbitrary locations on external storage. Either:
Limit yourself to the directories that you can access, or
Use removable storage only for backup/restore operations, with the live database in a more conventional location (getDatabasePath() on Context)
I want to create SQL db file on USB (otg) device. The problem is that on newer devices I access the USB using SAF so no real file path or handle any more.
All android sqlite openOrCreateDatabase methods need file or path which is not longer possible with SAF/DocumentFile.
The other options I was thinking about using sqlite alternatives but seems Realm doesn't support this either?
Any other ideas?
So I assume sqlite API will soon be updated to support DocumentFile or Uri?
To make it clear app works with large data on an usb stick, app also needs db to store some info on the usb. Of course I can alwasy create on device and latter copy on usb, but if the process stops in the middle data will be lost?
You can not store SQLite databases using SAF because SQLite does not support opening databases on (SAF) Uris, and probably never will for reasons explained in the linked answer.
You need to create the database in an accessible location, perform any read/write operations and then copy the database onto the USB drive. To avoid data loss, don't overwrite the target file (if it exists) and instead use a temporary file and rename that only after the complete operation is successful.
If you look at the all available storage access methods, the only other option is Context#getExternalFilesDirs which should not work because...
The returned paths do not include transient devices, such as USB flash drives connected to handheld devices.
However, for the devices I tested on this does not seem to be true, e.g. you actually can access USB devices using getExternalFilesDirs(). E.g. if you have a volume uuid 00A1-CC00 you can access the SAF Uri content://com.android.externalstorage.documents/tree/00A1-CC00%3A/document/00A1-CC00%3AAndroid%2Fdata%2Fcom.example%2Ffiles as /storage/00A1-CC00/Android/data/com.example/files.
If you absolutely need to do database operations without a temporary internal file, you may be able to use an h2 in memory database. This would require a modified BACKUP TO statement to allow writing to a SAF OutputStream.
I am working on logs inside android, I thought two ways for storing logs, and one is on external directory as a text file or a log file while other is to store in database. I found database method more useful in my case. My question is if I UN-install and reinstall the app will the database will be affected? In case of yes what should I do? I cannot place the logs online. How to take the backup or safe that database so it won’t be affected in case of UN- installation.
My question is if I uninstall and reinstall the app will the database will be affected?
Yes, your database will by default be stored in the application's data directory, which is deleted along with your application on uninstallation.
You can instead write a file of a filetype of your choosing (whether that's a simple text file, or a database file) to the external storage. You can obtain the directory path using:
Environment.getExternalStoragePublicDirectory(type) for obtaining a directory path of files of a specific type, such as images or videos;
Environment.getExternalStorageDirectory() for obtaining the primary external storage directory, under which you could create a new path.
I would nevertheless discourage you from doing this, because it would require your users to manually dispose of any files after uninstalling your application. Perhaps you should reconsider the justification of choosing for this solution.
If you uninstall the app, then the database (and all other data stored in the apps private storage for that matter) will be gone.
You could store your logs in the public external storage, but this will expose your logs to other applications as well as the user.
One possible approach could be to use application private storage for your 'live' logs, and make periodic backups to the public storage. In case of a new installation, you can check your designated backup location and attempt to restore previous logs from the backup.