Develop Reference

Kotlin File(path).walkTopDown() does not find files

I'm trying to read a given directory recursively and find all folders and files. The directory and subdirectory is found, but no files.
The storage structure on my virtual device:
The code used to "walk through the given directory"
import android.util.Log
import java.io.File
// path = "/storage/1B10-1D17/ReadTest/"
class FileHelper {
fun walkTest(path: String){
File(path).walkTopDown().forEach {
// println(it)
Log.e("walkTest extension", it.extension)
if(it.isDirectory){
Log.e("walkTest", "Directory: ${it.name}")
}else{
Log.e("walkTest", "File: ${it.name}")
}
}
}
}
This will output the following:
E/walkTest extension:
E/walkTest: Directory: ReadTest
E/walkTest extension:
E/walkTest: Directory: SubFolder
In my android manifest file I have the following line and the permission is granted:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
What is the flaw in my code, how can I find the three pdf files stored in these directories?
Update 1/2
As can be seen in the screenshot below, the media access is granted:
If I run the following command (as suggested by dan1st), I get the following output, which shows that pdf file in place:
> adb shell ls /storage/1B10-1D17/ReadTest/
File_1.pdf
SubFolder

You generally have no access to arbitrary files on removable storage. This is not changed by READ_EXTERNAL_STORAGE.
Your proposed solution is to use MANAGE_EXTERNAL_STORAGE. Technically, this works, and it is a fine solution for personal-use apps. If your intention is to distribute this app, you will need to file paperwork with Google (and perhaps other app stores) justifying your use of this permission. Stores that disagree with your justification may elect to not distribute your app.
Alternatively, you can use the Storage Access Framework. Use ACTION_OPEN_DOCUMENT_TREE / ActivityResultContracts.OpenDocumentTree and let the user decide where on the user's device (or in the user's cloud storage) the user wants your app to access the user's content. You can then use DocumentFile to walk the document tree to find sub-trees and documents.

Nearby Connections API: Cannot access received file on Android 10

I am following this guide with the goal of transferring an image file which on the receiver side should be uploaded via a REST API.
Android 10 blocks access to public folders like the 'Downloads' folder in which all received files from Nearby Connections API are stored in, in a 'Nearby' folder.
File payloadFile = filePayload.asFile().asJavaFile();
I have a Payload object, and as the guide suggests the code above should get the file. Unfortunately the above code returns null when targeting api 29 and running on an Android 10 device. Works fine on earlier Android versions.
I can get the ParcelFileDescriptor by doing File payloadFile = filePayload.asFile().asParcelFileDescriptor(); but not sure how to access the file then?
Via the ParcelFileDescriptor I have tried reading the file in the following ways but I always get some kind of permission or bad access exception:
BitmapFactory.decodeStream(FileInputStream(payloadFileDescriptor.fileDescriptor))
or
BitmapFactory.decodeFileDescriptor(payloadFileDescriptor.fileDescriptor)
The file is stored correctly in the Downloads folder as I can see and open it via a file browser app.
Also tried accessing via a content resolver (MediaStore.Downloads.EXTERNAL_CONTENT_URI and MediaStore.Images.EXTERNAL_CONTENT_URI) but no luck.
A note here is that the files are save with no extension, so maybe that's why Mediastore can't find anything?
I use "com.google.android.gms:play-services-nearby:17.0.0".
As mentioned I really want to receive a file and upload it. Is this totally impossible with Nearby Connections API on Android 10?

Nearby connections 18.0.0 has been released. You can now do this -
if (VERSION.SDK_INT >= VERSION_CODES.Q) {
// Because of https://developer.android.com/preview/privacy/scoped-storage, we are not
// allowed to access filepaths from another process directly. Instead, we must open the
// uri using our ContentResolver.
Uri uri = filePayload.asFile().asUri();
try {
// Copy the file to a new location.
InputStream in = context.getContentResolver().openInputStream(uri);
copyStream(in, new FileOutputStream(new File(context.getCacheDir(), filename)));
} catch (IOException e) {
// Log the error.
} finally {
// Delete the original file.
context.getContentResolver().delete(uri, null, null);
}

How to resolve Android 10's Storage Mess

As Android is very inconsistent between different major Versions regarding File access, I feel a bit lost.
I try to describe the problem as easy as possible:
My Company uses a commercial native DRM to protect other native library's we provide. We have a Licensing App, which invoked some Voodoo to end up with Licensing files in say /sdcard/companyname/LicenseContainer/. Other protected Apps looked at this directory in native code, checking if the user has a valid License.
The Android 10 update however, invalidated this workflow completely as it only provides scoped storage access. We could do a workaround using Storage Manager to grant access, which is unfortunately also deprecated now.
So my Question is now:
How can one App save files to a location on /sdcard/FOLDER which are
not deleted on App deletion
Accessible in native code by other apps
I'm a bit overwhelmed with all the possible solutions (SAF, FileProvider, etc), which invoke often that one app grants permissions to the other. But the files should be accessible without an installed first app who put it there.
I know there must be a solution, as recent FileManagers (i.e. Files by Google) get access to the whole /sdcard/ directory.
Whats the easiest, future-proof route to go here without invoking "hacks" like android:requestLegacyExternalStorage="true"

You may ask the user to give you access to any file or directory, including the root of internal storage or external SD card. You can make this access permanent for your app, be able to read/write files anywhere with the Scoped Storage API afterwards, until the app is uninstalled or reset.
Then, if you need to read or write a file in native C/C++ code, you may get Linux file descriptor (int number) of the file and pass it to native code to use with fdopen() call for example.
Here is a Java code snippet to get a file descriptor form a single file Uri (which in string form is like content://...)
ParcelFileDescriptor parcelFileDescriptor =
getContentResolver().openFileDescriptor(uri, "r"); // gets FileNotFoundException here, if file we used to have was deleted
int fd = parcelFileDescriptor.getFd(); // detachFd() if we want to close in native code
If you have source code for your native libraries, or can call them with C FILE* - it will work fine. The only problem is when you don't have the source code and they expect a file path/name. * UPDATE *: it is still possible to use the path/file name strings to pass to C/C++ functions that expect a file name. Simply instead of the "real path/file name", create a name to symbolic link like this:
// fd is file descriptor obtained in Java code above
char fileName[32];
sprintf(fileName, "/proc/self/fd/%d", fd);
// The above fileName can be passed to C/C++ functions that expect a file name.
// They can read or write to it, depending on permissions to fd given in Java,
// but I guess C/C++ code can not create a new file. Someone correct me,
// if I'm mistaken here.
However, at this time I'm not sure that when you create a file in a directory beyond the app "sandbox" in that way, if the system will delete this file too after uninstall... Would need to write a quick test on Android 10 to find out, and then we still won't know if Google won't change this behavior in future.

If you want to save files in shared storage (where it can be accessed by users & other apps) you need to use
for Media Files (Images, Videos, Audio, Downloads) use MediaStore
for Documents and Other Files use Storage Access Framework (this is simply a system file picker)
For instance you can use the following snippet to save a pdf file using Storage Access Framework
const val CREATE_FILE = 1
private fun createFile(pickerInitialUri: Uri) {
val intent = Intent(Intent.ACTION_CREATE_DOCUMENT).apply {
addCategory(Intent.CATEGORY_OPENABLE)
type = "application/pdf"
putExtra(Intent.EXTRA_TITLE, "invoice.pdf")
// Optionally, specify a URI for the directory that should be opened in
// the system file picker before your app creates the document.
putExtra(DocumentsContract.EXTRA_INITIAL_URI, pickerInitialUri)
}
startActivityForResult(intent, CREATE_FILE)
}
After the user has picked a directory we still need to handle the result Uri in on onActivityResult method.
override fun onActivityResult(
requestCode: Int, resultCode: Int, resultData: Intent?) {
if (requestCode == CREATE_FILE && resultCode == Activity.RESULT_OK) {
// The result data contains a URI for directory that
// the user selected.
resultData?.data?.also { uri ->
// save your data using the `uri` path
}
}
}
You can read about this in more detail on the following blogpost
https://androidexplained.github.io/android/android11/scoped-storage/2020/09/29/file-saving-android-11.html

How to handle SAF when I can only handle File or file-path?

Background
Up until Android Q, if we wanted to get information about an APK file, we could use WRITE_EXTERNAL_STORAGE and READ_EXTERNAL_STORAGE to get access to the storage, and then use PackageManager.getPackageArchiveInfo function on the file-path.
Similar cases exist, such as using ZipFile class on a compressed file, and probably countless framework APIs and third party libraries.
The problem
Google announced a huge amount of restrictions recently on Android Q.
One of them is called Scoped Storage, which ruins storage permission when it comes to accessing all files the device has. It lets you either handle media files, or use the very restricted Storage-Access-Framework (SAF) which can't allow apps to reach and use files using File API and file-paths.
When Android Q Beta 2 was published, it broke a lot of apps because of it, including of Google. The reason was that it was turned on by default, affecting all apps, whether they target Android Q or not.
The reason is that many apps, SDKs and Android framework itself - all use File API quite often. On many cases, they also don't support InputStream or SAF-related solutions. An example for this is exactly the APK parsing example I wrote about (PackageManager.getPackageArchiveInfo).
On Q beta 3, however, things changed a bit, so that app that target Q will have the scoped storage, and there is a flag to disable it and still use the normal storage permissions and File API as usual. Sadly the flag is only temporary (read here), so it's delaying the inevitable .
What I've tried
I've tried and found the next things:
Using the storage permission indeed didn't let me read any file that's not media file (I wanted to find APK files). It's as if the files don't exist.
Using SAF, I could find the APK file, and with some workaround to find its real path (link here), I've noticed that File API can tell me that indeed the file exist, but it couldn't get its size, and the framework failed to use its path using getPackageArchiveInfo . Wrote about this here
I tried to make a symlink to the file (link here), and then read from the symlink. It didn't help.
For the case of parsing APK files, I tried to search for alternative solutions. I've found 2 github repositories that handle the APK using a File class (here and here), and one that uses InputStream instead ( here). Sadly the one that uses InputStream is very old, missing various features (such as getting the app's name and icon) and isn't going to be updated anytime soon. Besides, having a library requires maintenance to keep up with future versions of Android, otherwise it might have issues in the future, or even crash.
The questions
Generally, is there a way to still use File API when using SAF ? I'm not talking about root solutions or just copying the file to somewhere else. I'm talking about a more solid solution.
For the case of APK parsing, is there a way to overcome this issue that the framework only provides file-path as a parameter? Any workaround or a way to use InputStream perhaps?

How to handle SAF when I can only handle File or file-path? It is possible, even if you can send only a Java File object, or path string to a library function which you cannot modify:
First, obtain a Uri to a file you need to handle (in String form it would be like "content://..."), then:
try {
ParcelFileDescriptor parcelFileDescriptor =
getContentResolver().openFileDescriptor(uri, "r"); // may get FileNotFoundException here
// Obtain file descriptor:
int fd = parcelFileDescriptor.getFd(); // or detachFd() if we want to close file in native code
String linkFileName = "/proc/self/fd/" + fd;
// Call library function with path/file string:
someFunc(/*file name*/ linkFileName);
// or with File parameter
otherFunc(new File(linkFileName));
// Finally, if you did not call detachFd() to obtain the file descriptor, call:
parcelFileDescriptor.close();
// Otherwise your library function should close file/stream...
} catch (FileNotFoundException fnf) {
fnf.printStackTrace(); // or whatever
}

Posting another answer just to have more room and let me insert code snipes. Given the file descriptor as explained in my previous answer, I tried using net.dongliu:apk-parser package mentioned by #androiddeveloper in the original question, as follows (Lt.d is my shorthand to using Log.d(SOME_TAG, string...)):
String linkFileName = "/proc/self/fd/" + fd;
try (ApkFile apkFile = new ApkFile(new File(linkFileName))) {
ApkMeta apkMeta = apkFile.getApkMeta();
Lt.d("ApkFile Label: ", apkMeta.getLabel());
Lt.d("ApkFile pkg name: ", apkMeta.getPackageName());
Lt.d("ApkFile version code: ", apkMeta.getVersionCode());
String iconStr = apkMeta.getIcon();
Lt.d("ApkFile icon str: ", iconStr);
for (UseFeature feature : apkMeta.getUsesFeatures()) {
Lt.d(feature.getName());
}
}
catch (Exception ex) {
Lt.e("Exception in ApkFile code: ", ex);
ex.printStackTrace();
}
}
It gives me the correct app label, for the icon it gives me only a string to the resource directory (like "res/drawable-mdpi-v4/fex.png"), so again raw ZIP reading functions would have to be applied to read the actual icon bits. Specifically I was testing ES File Explorer Pro APK (bought this product and saved APK for my own backup, got the following output:
I/StorageTest: ApkFile Label: ES File Explorer Pro
I/StorageTest: ApkFile pkg name: com.estrongs.android.pop.pro
I/StorageTest: ApkFile version code: 1010
I/StorageTest: ApkFile icon str: res/drawable-mdpi-v4/fex.png
I/StorageTest: android.hardware.bluetooth
I/StorageTest: android.hardware.touchscreen
I/StorageTest: android.hardware.wifi
I/StorageTest: android.software.leanback
I/StorageTest: android.hardware.screen.portrait

Check existance of a file to unlock app

I wish to create a file on my device somewhere (I don't mind where or what the file contains, my only concern is the name of the file), then have my app look for the existence of that file. The idea is to have a mechanism to secretly unlock some features of my app to anyone I tell about the file.
My only problem is that I have never written any code to read/write files outside of my own app's space before, so I'm not sure how to do it. Can someone give me some pointers.

You can use File.exists() method to determine whether the file exists or not.
File testfile = new File(Environment.getExternalStorageDirectory()
+ File.separator ,
"myfile");
if (testfile.exists()) {
//file exists. enable your features
}