I'm trying to read a given directory recursively and find all folders and files. The directory and subdirectory is found, but no files.
The storage structure on my virtual device:
The code used to "walk through the given directory"
import android.util.Log
import java.io.File
// path = "/storage/1B10-1D17/ReadTest/"
class FileHelper {
fun walkTest(path: String){
File(path).walkTopDown().forEach {
// println(it)
Log.e("walkTest extension", it.extension)
if(it.isDirectory){
Log.e("walkTest", "Directory: ${it.name}")
}else{
Log.e("walkTest", "File: ${it.name}")
}
}
}
}
This will output the following:
E/walkTest extension:
E/walkTest: Directory: ReadTest
E/walkTest extension:
E/walkTest: Directory: SubFolder
In my android manifest file I have the following line and the permission is granted:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
What is the flaw in my code, how can I find the three pdf files stored in these directories?
Update 1/2
As can be seen in the screenshot below, the media access is granted:
If I run the following command (as suggested by dan1st), I get the following output, which shows that pdf file in place:
> adb shell ls /storage/1B10-1D17/ReadTest/
File_1.pdf
SubFolder
You generally have no access to arbitrary files on removable storage. This is not changed by READ_EXTERNAL_STORAGE.
Your proposed solution is to use MANAGE_EXTERNAL_STORAGE. Technically, this works, and it is a fine solution for personal-use apps. If your intention is to distribute this app, you will need to file paperwork with Google (and perhaps other app stores) justifying your use of this permission. Stores that disagree with your justification may elect to not distribute your app.
Alternatively, you can use the Storage Access Framework. Use ACTION_OPEN_DOCUMENT_TREE / ActivityResultContracts.OpenDocumentTree and let the user decide where on the user's device (or in the user's cloud storage) the user wants your app to access the user's content. You can then use DocumentFile to walk the document tree to find sub-trees and documents.
Related
I'll have an Android App currently compiled against the API29 which I'm working on an update for API31.
The App is using an huge ultralite database which is located in the directory
/sdcard/Documents/myApp/db
Because I now have to apply scoped storage', I'll use the App-Directory on the external sdcard by using context.getExternalFilesDir(null).
I'll have to fulfill an important requirement, that the User needs to download this big database by PC only once
and store this db- file in a dedicated directory on the Android Device.
This will be done from a person (in a staging role) that prepares dozens of devices for dozens of users in a staging process,
so not alle devices needs to download the DB for themselfes.
When the App is starting it will have a look on this staging-directory and would then move the DB-file to the internal Working storage of the APP.
So my question is:
How can I get the permission for myApp to grant myApp access to a dedicated directory on the SDCard e.g. /sdcard/Documents/myApp/db in a way
that the user can provide there a db for myAPP, which myApp will move to internal storage in case of no DB, is present?
It would be also important to just set this permissions once on a device and having then the app confirmed to have access to this directory as long as installed on the device.
(/sdcard/Documents/myApp/db -> app-internal-dir)
Constraints:
It's no option to use via API29 in the AndroidManifest android:requestLegacyExternalStorage="true". Reason:
I anyway need to provide up a mechanism, where the user can copy a db file to the SD-Card which the app will use in case of no DB is present.
The app should then move (not copy) this huge file into its app directory.
I would also like to have a dedicated directory on the sdcard, where the application can move the current DB into it. (Do the opposite: app-internal-dir -> /sdcard/Documents/myApp/db-export)
It is no option to use the MANAGE_EXTERNAL_STORAGE permission, due this app is not a classified App that would get the permission from Google to do that (like a filebrowser or a antivirus App)
Can anybody give me some recommendation which approach to use?
The whole Logic is currently used by File Objects and Paths to working directories with the java.io.File-API
Thanks in advance!
EDITED:
I'm able to open the folder and query the DocumentFiles when it returns from the picker:
Here is my code that starts the picker:
// Choose a directory using the system's file picker.
val intent = Intent(Intent.ACTION_OPEN_DOCUMENT_TREE).apply {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
// Optionally, specify a URI for the directory that should be opened in
// the system file picker when it loads.
val pathAfterPrimaryIndicator = bootstrapViewModel.getDbMigrationDirectoryName()?.substringAfter("/0/")
val uri =DocumentsContract.buildDocumentUri("com.android.externalstorage.documents", "primary:$pathAfterPrimaryIndicator")
putExtra(DocumentsContract.EXTRA_INITIAL_URI, uri)
}
}
startActivityForResult(intent, DOCUMENT_TREE_ACTIVITY_REQUEST)
Here is some code that handles the return:
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
super.onActivityResult(requestCode, resultCode, data)
if (resultCode == RESULT_OK && requestCode == DOCUMENT_TREE_ACTIVITY_REQUEST) {
val treeUri: Uri? = data!!.data
val pickedDir = DocumentFile.fromTreeUri(this, treeUri!!)
pickedDir?.listFiles()?.forEach {
// test if its a Db file and move it to the internal storage
if (it.isFile) {
// QUESTION: --> How can I MOVE those files (some files are huge) found here to the app-internal directory?
}
}
The question is now still open how to MOVE the directory content into the app internal directory. Because some of the files are huge so copy them is not really an Option. How can this be done?
Android 11+:
If the file comes from a PC then the officer can directly place the file in getExternalFilesDir(null) and you are done.
If the officer places the file in a public directory of external storage then your app should use ACTION_OPEN_DOCUMENT to let the user choose the file.
As Android is very inconsistent between different major Versions regarding File access, I feel a bit lost.
I try to describe the problem as easy as possible:
My Company uses a commercial native DRM to protect other native library's we provide. We have a Licensing App, which invoked some Voodoo to end up with Licensing files in say /sdcard/companyname/LicenseContainer/. Other protected Apps looked at this directory in native code, checking if the user has a valid License.
The Android 10 update however, invalidated this workflow completely as it only provides scoped storage access. We could do a workaround using Storage Manager to grant access, which is unfortunately also deprecated now.
So my Question is now:
How can one App save files to a location on /sdcard/FOLDER which are
not deleted on App deletion
Accessible in native code by other apps
I'm a bit overwhelmed with all the possible solutions (SAF, FileProvider, etc), which invoke often that one app grants permissions to the other. But the files should be accessible without an installed first app who put it there.
I know there must be a solution, as recent FileManagers (i.e. Files by Google) get access to the whole /sdcard/ directory.
Whats the easiest, future-proof route to go here without invoking "hacks" like android:requestLegacyExternalStorage="true"
You may ask the user to give you access to any file or directory, including the root of internal storage or external SD card. You can make this access permanent for your app, be able to read/write files anywhere with the Scoped Storage API afterwards, until the app is uninstalled or reset.
Then, if you need to read or write a file in native C/C++ code, you may get Linux file descriptor (int number) of the file and pass it to native code to use with fdopen() call for example.
Here is a Java code snippet to get a file descriptor form a single file Uri (which in string form is like content://...)
ParcelFileDescriptor parcelFileDescriptor =
getContentResolver().openFileDescriptor(uri, "r"); // gets FileNotFoundException here, if file we used to have was deleted
int fd = parcelFileDescriptor.getFd(); // detachFd() if we want to close in native code
If you have source code for your native libraries, or can call them with C FILE* - it will work fine. The only problem is when you don't have the source code and they expect a file path/name. * UPDATE *: it is still possible to use the path/file name strings to pass to C/C++ functions that expect a file name. Simply instead of the "real path/file name", create a name to symbolic link like this:
// fd is file descriptor obtained in Java code above
char fileName[32];
sprintf(fileName, "/proc/self/fd/%d", fd);
// The above fileName can be passed to C/C++ functions that expect a file name.
// They can read or write to it, depending on permissions to fd given in Java,
// but I guess C/C++ code can not create a new file. Someone correct me,
// if I'm mistaken here.
However, at this time I'm not sure that when you create a file in a directory beyond the app "sandbox" in that way, if the system will delete this file too after uninstall... Would need to write a quick test on Android 10 to find out, and then we still won't know if Google won't change this behavior in future.
If you want to save files in shared storage (where it can be accessed by users & other apps) you need to use
for Media Files (Images, Videos, Audio, Downloads) use MediaStore
for Documents and Other Files use Storage Access Framework (this is simply a system file picker)
For instance you can use the following snippet to save a pdf file using Storage Access Framework
const val CREATE_FILE = 1
private fun createFile(pickerInitialUri: Uri) {
val intent = Intent(Intent.ACTION_CREATE_DOCUMENT).apply {
addCategory(Intent.CATEGORY_OPENABLE)
type = "application/pdf"
putExtra(Intent.EXTRA_TITLE, "invoice.pdf")
// Optionally, specify a URI for the directory that should be opened in
// the system file picker before your app creates the document.
putExtra(DocumentsContract.EXTRA_INITIAL_URI, pickerInitialUri)
}
startActivityForResult(intent, CREATE_FILE)
}
After the user has picked a directory we still need to handle the result Uri in on onActivityResult method.
override fun onActivityResult(
requestCode: Int, resultCode: Int, resultData: Intent?) {
if (requestCode == CREATE_FILE && resultCode == Activity.RESULT_OK) {
// The result data contains a URI for directory that
// the user selected.
resultData?.data?.also { uri ->
// save your data using the `uri` path
}
}
}
You can read about this in more detail on the following blogpost
https://androidexplained.github.io/android/android11/scoped-storage/2020/09/29/file-saving-android-11.html
Background
Up until Android Q, if we wanted to get information about an APK file, we could use WRITE_EXTERNAL_STORAGE and READ_EXTERNAL_STORAGE to get access to the storage, and then use PackageManager.getPackageArchiveInfo function on the file-path.
Similar cases exist, such as using ZipFile class on a compressed file, and probably countless framework APIs and third party libraries.
The problem
Google announced a huge amount of restrictions recently on Android Q.
One of them is called Scoped Storage, which ruins storage permission when it comes to accessing all files the device has. It lets you either handle media files, or use the very restricted Storage-Access-Framework (SAF) which can't allow apps to reach and use files using File API and file-paths.
When Android Q Beta 2 was published, it broke a lot of apps because of it, including of Google. The reason was that it was turned on by default, affecting all apps, whether they target Android Q or not.
The reason is that many apps, SDKs and Android framework itself - all use File API quite often. On many cases, they also don't support InputStream or SAF-related solutions. An example for this is exactly the APK parsing example I wrote about (PackageManager.getPackageArchiveInfo).
On Q beta 3, however, things changed a bit, so that app that target Q will have the scoped storage, and there is a flag to disable it and still use the normal storage permissions and File API as usual. Sadly the flag is only temporary (read here), so it's delaying the inevitable .
What I've tried
I've tried and found the next things:
Using the storage permission indeed didn't let me read any file that's not media file (I wanted to find APK files). It's as if the files don't exist.
Using SAF, I could find the APK file, and with some workaround to find its real path (link here), I've noticed that File API can tell me that indeed the file exist, but it couldn't get its size, and the framework failed to use its path using getPackageArchiveInfo . Wrote about this here
I tried to make a symlink to the file (link here), and then read from the symlink. It didn't help.
For the case of parsing APK files, I tried to search for alternative solutions. I've found 2 github repositories that handle the APK using a File class (here and here), and one that uses InputStream instead ( here). Sadly the one that uses InputStream is very old, missing various features (such as getting the app's name and icon) and isn't going to be updated anytime soon. Besides, having a library requires maintenance to keep up with future versions of Android, otherwise it might have issues in the future, or even crash.
The questions
Generally, is there a way to still use File API when using SAF ? I'm not talking about root solutions or just copying the file to somewhere else. I'm talking about a more solid solution.
For the case of APK parsing, is there a way to overcome this issue that the framework only provides file-path as a parameter? Any workaround or a way to use InputStream perhaps?
How to handle SAF when I can only handle File or file-path? It is possible, even if you can send only a Java File object, or path string to a library function which you cannot modify:
First, obtain a Uri to a file you need to handle (in String form it would be like "content://..."), then:
try {
ParcelFileDescriptor parcelFileDescriptor =
getContentResolver().openFileDescriptor(uri, "r"); // may get FileNotFoundException here
// Obtain file descriptor:
int fd = parcelFileDescriptor.getFd(); // or detachFd() if we want to close file in native code
String linkFileName = "/proc/self/fd/" + fd;
// Call library function with path/file string:
someFunc(/*file name*/ linkFileName);
// or with File parameter
otherFunc(new File(linkFileName));
// Finally, if you did not call detachFd() to obtain the file descriptor, call:
parcelFileDescriptor.close();
// Otherwise your library function should close file/stream...
} catch (FileNotFoundException fnf) {
fnf.printStackTrace(); // or whatever
}
Posting another answer just to have more room and let me insert code snipes. Given the file descriptor as explained in my previous answer, I tried using net.dongliu:apk-parser package mentioned by #androiddeveloper in the original question, as follows (Lt.d is my shorthand to using Log.d(SOME_TAG, string...)):
String linkFileName = "/proc/self/fd/" + fd;
try (ApkFile apkFile = new ApkFile(new File(linkFileName))) {
ApkMeta apkMeta = apkFile.getApkMeta();
Lt.d("ApkFile Label: ", apkMeta.getLabel());
Lt.d("ApkFile pkg name: ", apkMeta.getPackageName());
Lt.d("ApkFile version code: ", apkMeta.getVersionCode());
String iconStr = apkMeta.getIcon();
Lt.d("ApkFile icon str: ", iconStr);
for (UseFeature feature : apkMeta.getUsesFeatures()) {
Lt.d(feature.getName());
}
}
catch (Exception ex) {
Lt.e("Exception in ApkFile code: ", ex);
ex.printStackTrace();
}
}
It gives me the correct app label, for the icon it gives me only a string to the resource directory (like "res/drawable-mdpi-v4/fex.png"), so again raw ZIP reading functions would have to be applied to read the actual icon bits. Specifically I was testing ES File Explorer Pro APK (bought this product and saved APK for my own backup, got the following output:
I/StorageTest: ApkFile Label: ES File Explorer Pro
I/StorageTest: ApkFile pkg name: com.estrongs.android.pop.pro
I/StorageTest: ApkFile version code: 1010
I/StorageTest: ApkFile icon str: res/drawable-mdpi-v4/fex.png
I/StorageTest: android.hardware.bluetooth
I/StorageTest: android.hardware.touchscreen
I/StorageTest: android.hardware.wifi
I/StorageTest: android.software.leanback
I/StorageTest: android.hardware.screen.portrait
I'm on an Android 5.0 device where a usb mass storage is mounted at "/storage/usbotg" via USB OTG cable.
Unfortunately from my app I only have read access to that folder. Please note that I set the write external storage permission as I'm able to write to device storage.
I post the following code as reference:
string Root = "/storage/usbotg";
string[] Dirs = Directory.GetDirectories(Root);
string NewFolder = Path.Combine(Root, "NewFolder");
Directory.CreateDirectory(NewFolder);
This gives me an exception on the last line (but I'm able to list subdirectories in Dirs)
Exception:
System.UnauthorizedAccessException
Exception Message:
Access to the path "/storage/usbotg/NewFolder" is denied.
If I use:
string Root = "/storage/emulated/0";
everything is working fine and the "NewFolder" is created.
What I'm missing? How can I write to that folder?
I'm using Xamarin.Forms 2.5.0
Thanks for your help
Accessing data outside your application's private storage using the file system is more restricted with each Android version.
The recommended options are :
Use getExternalFilesDirs(), getExternalCacheDirs, ... : this gives you one or more directories specific to you application (usually directories named after the package name). This does not work for removable media, unless they're adopted.
Use the Storage Access Framework : ask the user (using ACTION_OPEN_DOCUMENT_TREE) to choose the storage root. You can then manage the content of the picked directory through the SAF API. You can persist the permission, so you only need to ask the user once. It seems that it is what ES File explore does to get write permission.
A (much) more detailed explanation by Mark Murphy.
Is there such thing on any Android based device as shared internal storage? I know you can use the SDCard as a shared file location between applications, but for a project I'm working on we dont want the ability for a SD Card to go missing (sensitive material).
The problem is as follows. App1 allows a user to browse (for example) some word documents and download them to the proposed shared storage, the user can then pass this file to Documents 2 Go editing them and saving the change. App 1 then allows an upload of the changed file.
I don't fancy building a document editor word/excel directly into app, unless thats easy?
EDIT:
The second app is "Documents 2 Go" I won't be able to edit its AndroidManifest
I faced a similar situation now for txt files and did this.
File downloadedFile= new File( context.getFilesDir(), "simple.txt" );
downloadedFile.setReadable( true, false );
downloadedFile.setWritable( true, false ); //set read/write for others
Uri downloadFileUri = Uri.fromFile( downloadedFile );
Intent intentToEditFile = new Intent( Intent.ACTION_EDIT );
intentToEditFile.setDataAndType( downloadFileUri, "text/plain" );
context.startActivity( intentToEditFile );
Now the 'Document 2 Go' editor will be launched to edit the file and
will be able to edit simple.txt
Note 1: Uri should be created from same file object that was set with
setReadable()/setWritable.
Note 2: Read/Write permission for other users might not be reflected in file
system. Some times I cannot see rw-rw-rw- in adb shell
I believe ContentProviders is your solution. Its the Android recommended method for sharing application data between different apps.
https://stackoverflow.com/a/6708681/804447
Sharing data between apps is what ContentProviders are for. Assuming that you know how to write a ContentProvider and access it, you can access files via ParcelFileDescriptor, which includes constants for the mode in which you create the files.
What you need now is to limit access so that not everybody can read the files through the content provider, and you do that via android permissions. In the manifest of one your apps, the one that will host the files and the content provider, write something like this:
<permission android:name="com.example.android.provider.ACCESS" android:protectionLevel="signature"/>
and in both apps add this:
<uses-permission android:name="com.example.android.provider.ACCESS" />
by using protectionLevel="signature", only apps signed by you can access your content provider, and thus your files.