Develop Reference

Cannot find files created using QStandardPath on android device

I'm developing an app on android using Qt 5.15.0. My app creates a database and some other files using QStandardPaths::AppDataLocation. everything works fine from the code side: debug output makes me think the files are there and that the app is using them. The problem is that when I search from the PC inside the smartphone folders I cannot find the files created. So this makes me think I am not writing to the folder I'm expecting to.
This is my code:
appDataPath = QStandardPaths::writableLocation(QStandardPaths::AppDataLocation);
QDir directory;
if (!directory.exists(appDataPath + QStringLiteral("/DATABASE")))
directory.mkpath(appDataPath + QStringLiteral("/DATABASE"));
if (!directory.exists(appDataPath + QStringLiteral("/DATA")))
directory.mkpath(appDataPath + QStringLiteral("/DATA"));
qDebug() << "Percorso: " << appDataPath << " dir exists:" << directory.exists(appDataPath + QStringLiteral("/DATABASE"));
QFile f( appDataPath + "/DATABASE/Prova.txt" );
if( f.open( QIODevice::WriteOnly ) )
{
f.write( "Ciao" );
f.close();
}
if( f.error() != QFileDevice::NoError )
qDebug() << QString("Error writing file '%1':").arg(appDataPath + "/DATABASE/Prova.txt") << f.errorString();
I also write a file that I read at startup and I can read from it so I assume the file is present and is not saved as temporary.
The output is:
D/MyApp(17849): Percorso: "/data/data/org.qtproject.example/files" dir exists: true
Now I suppose I can find my files in this directory inside the phone in this folder:
Questo PC\Samsung Galaxy J3 (2016)\Phone\Android\data\org.qtproject.example\files
But there are no directories or files inside. I bet I am missing some trivial things.
I am giving the app these permission:
android.permission.WRITE_EXTERNAL_STORAGE
android.permission.READ_EXTERNAL_STORAGE
Can someone help me?
Thanks in advance!

It seems that
QStandardPaths::AppLocalDataLocation, QStandardPaths::AppDataLocation
don't give the correct path to the app's file folder. I encountered the same issue with Qt 5.14.1. While QStandardPaths gave me
/data/user/0/com.mycompany.myapp/files
a call via a QAndroidJniObject to the java side gave me
/storage/emulated/0/Android/data/com.mycompany.myapp/files
Using the latter all directories and files were created correctly. I suggest to either try my work-around with
QtNative.activity().getExternalFilesDir(null).getAbsolutePath();
in an added java class and call it through a QAndroidJniObject or report this as a bug to Qt.
Nevertheless, there might be another solution to the problem I'm not aware of.

Install APK using root, handling new limitations of "/data/local/tmp/" folder

Background
So far, I was able to install APK files using root (within the app), via this code:
pm install -t -f fullPathToApkFile
and if I want to (try to) install to sd-card :
pm install -t -s fullPathToApkFile
The problem
Recently, not sure from which Android version (issue exists on Android P beta, at least), the above method fails, showing me this message:
avc: denied { read } for scontext=u:r:system_server:s0 tcontext=u:object_r:sdcardfs:s0 tclass=file permissive=0
System server has no access to read file context u:object_r:sdcardfs:s0 (from path /storage/emulated/0/Download/FDroid.apk, context u:r:system_server:s0)
Error: Unable to open file: /storage/emulated/0/Download/FDroid.apk
Consider using a file under /data/local/tmp/
Error: Can't open file: /storage/emulated/0/Download/FDroid.apk
Exception occurred while executing:
java.lang.IllegalArgumentException: Error: Can't open file: /storage/emulated/0/Download/FDroid.apk
at com.android.server.pm.PackageManagerShellCommand.setParamsSize(PackageManagerShellCommand.java:306)
at com.android.server.pm.PackageManagerShellCommand.runInstall(PackageManagerShellCommand.java:884)
at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:138)
at android.os.ShellCommand.exec(ShellCommand.java:103)
at com.android.server.pm.PackageManagerService.onShellCommand(PackageManagerService.java:21125)
at android.os.Binder.shellCommand(Binder.java:634)
at android.os.Binder.onTransact(Binder.java:532)
at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:2806)
at com.android.server.pm.PackageManagerService.onTransact(PackageManagerService.java:3841)
at android.os.Binder.execTransact(Binder.java:731)
This seems to also affect popular apps such as "Titanium backup (pro)", which fails to restore apps.
What I've tried
Looking at what's written, it appears it lacks permission to install APK files that are not in /data/local/tmp/.
So I tried the next things, to see if I can overcome it:
set the access to the file (chmod 777) - didn't help.
grant permissions to my app, of both storage and REQUEST_INSTALL_PACKAGES (using ACTION_MANAGE_UNKNOWN_APP_SOURCES Intent) - didn't help.
create a symlink to the file, so that it will be inside the /data/local/tmp/, using official API:
Os.symlink(fullPathToApkFile, symLinkFilePath)
This didn't do anything.
create a symlink using this :
ln -sf $fullPathToApkFile $symLinkFilePath
This partially worked. The file is there, as I can see it in Total Commander app, but when I try to check if it exists there, and when I try to install the APK from there, it fails.
Copy/move (using cp or mv) the file to the /data/local/tmp/ path, and then install from there. This worked, but it has disadvantages: moving is risky because it temporarily hides the original file, and it changes the timestamp of the original file. Copying is bad because of using extra space just for installing (even temporarily) and because it wastes time in doing so.
Copy the APK file, telling it to avoid actual copy (meaning hard link), using this command (taken from here) :
cp -p -r -l $fullPathToApkFile $tempFileParentPath"
This didn't work. It got me this error:
cp: /data/local/tmp/test.apk: Cross-device link
Checking what happens in other cases of installing apps. When you install via via the IDE, it actually does create the APK file in this special path, but if you install via the Play Store, simple APK install (via Intent) or adb (via PC), it doesn't.
Wrote about this here too: https://issuetracker.google.com/issues/80270303
The questions
Is there any way to overcome the disadvantages of installing the APK using root on this special path? Maybe even avoid handling this path at all?
Why does the OS suddenly require to use this path? Why not use the original path instead, just like in the other methods of installing apps? What do the other methods of installing apps do, that somehow avoids using the spacial path?

One solution, in case you don't mind the moving procedure, is to also save&restore the timestamp of the original file, as such:
val tempFileParentPath = "/data/local/tmp/"
val tempFilePath = tempFileParentPath + File(fullPathToApkFile).name
val apkTimestampTempFile = File(context.cacheDir, "apkTimestamp")
apkTimestampTempFile.delete()
apkTimestampTempFile.mkdirs()
apkTimestampTempFile.createNewFile()
root.runCommands("touch -r $fullPathToApkFile ${apkTimestampTempFile.absolutePath}")
root.runCommands("mv $fullPathToApkFile $tempFileParentPath")
root.runCommands("pm install -t -f $tempFilePath")
root.runCommands("mv $tempFilePath $fullPathToApkFile")
root.runCommands("touch -r ${apkTimestampTempFile.absolutePath} $fullPathToApkFile")
apkTimestampTempFile.delete()
It's still a bit dangerous, but better than copying files...
EDIT: Google has shown me a nice workaround for this (here) :
We don't support installation of APKs from random directories on the device. They either need to be installed directly from the host using 'adb install' or you have to stream the contents to install --
$ cat foo.apk | pm install -S APK_SIZE
While I think this is incorrect that they don't support installing of APK files from random paths (always worked before), the workaround does seem to work. All I needed to change in the code of installing an APK file is as such:
val length = File(fullPathToApkFile ).length()
commands.add("cat $fullPathToApkFile | pm install -S $length")
Thing is, now I have some other questions about it :
Does this workaround avoid the moving/copying of the APK into storage, and without affecting the original file ? - seems it does
Will this support any APK file, even large ones? - seems it succeeds in doing it for an APK that takes 433MB, so I think it's safe to use for all sizes.
This is needed only from Android P, right? - so far seems so.
Why does it need the file size as a parameter ? - No idea, but if I remove it, it won't work

Thanks for the answers! I looked everywhere else as well to get a whole setup for OTA to work for Android 10 and so on. It 100% works on Samsung Galaxy Tab 10.1 running Android 10.
Here is a medium article with the code:
https://medium.com/#jnishu1996/over-the-air-ota-updates-for-android-apps-download-apk-silent-apk-installation-auto-launch-8ee6f342197c
The magic is running this command with root access:
process = Runtime.getRuntime().exec("su");
out = process.getOutputStream();
DataOutputStream dataOutputStream = new DataOutputStream(out);
// Get all file permissions
dataOutputStream.writeBytes("chmod 777 " + file.getPath() + "\n");
// Perform silent installation command, all flags are necessary for some reason, only this works reliably post Android 10
String installCommand = "cat " + file.getAbsolutePath() + "| pm install -d -t -S " + file.length();
// Data to send to the LaunchActivity to the app knows it got updated and performs necessary functions to notify backend
// es stands for extraString
// In LaunchActivity onCreate(), you can get this data by running -> if (getIntent().getStringExtra("OTA").equals("true"))
String launchCommandIntentArguments = "--es OTA true --es messageId " + MyApplication.mLastSQSMessage.receiptHandle();
// Start a background thread to wait for 8 seconds before reopening the app's LaunchActivity, and pass necessary arguments
String launchCommand = "(sleep 8; am start -n co.getpresso.Presso/.activities.LaunchActivity " + launchCommandIntentArguments + ")&";
// The entire command is deployed with a ";" in the middle to launchCommand run after installCommand
String installAndLaunchCommand = installCommand + "; " + launchCommand;
// begins the installation
dataOutputStream.writeBytes(installAndLaunchCommand);
dataOutputStream.flush();
// Close the stream operation
dataOutputStream.close();
out.close();
int value = process.waitFor();

How to set executable permissions on a file in Android to execute using "Runtime.getRuntime().exec(..)"

I'm currently trying to natively call on an executable on the shell from my Android Project which is in my application home directory. I can see via the command line when using ADB that my file is not getting executable file permissions, however. I've tried the following:
File file = new File(myFileLocation);
if(file.exists()){
boolean executable = file.setExecutable(true);
}
'executable' remains false.
I've also tried the following:
Process processChmod = Runtime.getRuntime().exec("/system/bin/chmod u+x " + myExecutableLocation);
processChmod.waitFor();
When I try to issue a command on the process, I get the following IOException:
java.io.IOException: Permission denied java.io.IOException: Error
running exec(). Command: [/storage/emulated/0/myApp/iperf, -c,
iperf.eltel.net] Working Directory: null Environment: null
The command I'm trying to issue is:
Runtime.getRuntime().exec("/storage/emulated/0/myApp/iperf -c iperf.eltel.net")
Again, at this stage, I can see that the file permissions on the process I wish to use is simply r/w and not executable. Any help appreciated! By the way, I'm trying to use the 'iPerf' C library and there is a compiled armeabi module as well as the original sources/JNI code. There are no compilation issues and this looks like a file permissions issue more than anything.

Files located on SD card aren't executable. Move the file to internal storage (e.g. to /data/local or, if you get permission denied, to data/local/tmp).
That is how I did it :
String filePath = getFilesDir().getAbsolutePath() + File.separator + "myFileName";
File myFile = new File(filePath);
if (!myFile.canExecute()) {
Log.d("File is not executable, trying to make it executable ...");
if (myFile .setExecutable(true)) {
Log.d("File is executable");
} else {
Log.d("Failed to make the File executable");
}
} else {
Log.d("File is executable");
}

I think Android's chmod does not understand u+x syntax, try using the numeric notation like 744.

Native Subprocess Can't Create File in Application Folder or on SDcard

Since this turned out to be a long read, I'll start with a very truncated version:
Are native processes not allowed to create/edit files created by their spawning java process regardless of having appropriate location/ permission/ file ownership?
The full question:
My application consists of a set of native binaries compiled with the NDK toolchain and a typical java Android interface. The native programs are packaged in the /assets folder and copied by the java application out into a /data/data/com.domain.app/nativeApplication folder.
There are two native application in question. One which modifies a jpeg created by the java layer, and a second which analyzes that jpeg and returns an int as output. The issue is this: When called by the java layer, the jpeg modifier doesn't work.
I've done a good amount of troubleshooting on it already so I'll try to dissuade you from some of the obvious potential answers starting with the most obvious.
Correct Compilation:
Through adb shell I can call both of the programs in question, as is, from the phone's disk. The second image analyzer program runs without a hitch when called either via adb or as designed in the software IF I do the intermediary step manually from adb.
Permissions:
Jpg Permission - The application has permission to write to external storage. I've tried to have the file read in from both the SDcard and /data/data/com.domain.app/nativeApp/img/name.jpg; no joy. As stated the second program operates as designed when the image is in either location, if I manually run the first program through adb.
File/Folder Permissions: All of the folders in ~/nativeApplication/* have been created BY the android application itself, it is the owner and it has chmodded all of the sub-directories and files to 777.
Supporting Code:
Calling the jpg converter process:
NativeSetup.changePermission("/data/data/com.domain.program/nativeApp/img/img1.jpg");
try {
Process jpegConvert = Runtime.getRuntime().exec("/data/data/com.domain.program/nativeApp/bin/jpegtransformer -opts /data/data/com.domain.program/nativeApp/img/img1.jpg > /data/data/com.domain.program/nativeApp/img/img2.jpg");
jpegConvert.waitFor();
} catch (IOException e) {
throw new RuntimeException(e);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
Call used to create the files in ~/nativeApp/*
private static void copyFile(String assetPath, String localPath, Context context) {
try {
InputStream in = context.getAssets().open(assetPath);
FileOutputStream out = new FileOutputStream(localPath);
int read;
byte[] buffer = new byte[4096];
while ((read = in.read(buffer)) > 0) {
out.write(buffer, 0, read);
}
chmod call: (it works according to ls -l through adb)
static void changePermission(String localPath) {
try {
Process chmod = Runtime.getRuntime().exec("/system/bin/chmod 777 " +localPath);
chmod.waitFor();
} catch (IOException e) {
throw new RuntimeException(e);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
out.close();
in.close();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Permissions of the file the program can't edit and then output a changed copy:
~/nativeApp/img $ ls -l
-rwxrwxrwx app_89 app_89 93061 2011-07-23 18:38 img1.jpg
Permission of the application that can't edit the above jpg:
~/nativeApp/bin $ ls -l jpegtransformer
-rwxrwxrwx app_89 app_89 95268 2011-07-23 18:01 jpegtransformer
Permissions of folders in ~/nativeApp/
~/nativeApp $ ls -l
drwxrwxrwx app_89 app_89 2011-07-23 17:57 bin
drwxrwxrwx app_89 app_89 2011-07-23 18:27 img
Theories:
At this point my only viable theory is that there is some android policy not governed by chmod which is keeping the jpegtransformer from editing the image when called by the java layer. Since the image analyzer doesn't change the image I imagine it skirts this? Are native processes not allowed to create files? Any ideas would be great. Sorry for the long read.
-SS

When you spawn a process from your app, it runs under the same uid as you do, so all of the same Android and filesystem permissions apply. I think you need to look for something else going on, such as using bad paths.
If you want to verify, just use "adb shell ps" to see the running processes and you should see that your process has the same uid as your main app.
That said... Android really does not support spawning processes like this, and it is strongly discouraged. Please consider doing this just by loading the native code as a shared library and using JNI to call it. This will also be a lot more efficient, and communication with your native code potentially simpler since JNI is a much richer and more direct way to call in to it.

I think your problem is not related to permissions - you try to redirect the output using >, while this is a shell operator and does not work from java. grab the process output stream and write it to a file using Java standard file I/O API.
Though I'm not sure this is the problem, you shouldn't have problem to modify files in your app folder and the issue I described is an issue I met more than once.

Android Pre-installing NDK Application

We are trying to pre-install a NDK Application into the /system/app directory. If I open the apk file in a ZIP file manager, the .so file is inside the lib directory. However, when we preinstall the apk file, the apk's .so file is not copied to system/lib directory, causing for the application to fail when we launched it in the device.
Can anyone please tell me what should be set in the Android.mk for the APK file so that the .so file will be extracted from the APK file and copied to system/lib directory? We need to include the application in the system image.
Any feedback will be greatly appreciated.
Thanks,
artsylar

I had the same need and after 2 days of heavy research, I came up with a solution to this problem. It is not simple and requires you to be able to modify the Android System code as well.
Basically PackageManagerService prevents system applications to unpack their native binaries (.so files), unless they have been updated. So the only way to fix this is by modifying PMS.java (aptly named since trying to solve this problem put me in a terrible mood).
On the system's first boot, I check every system package for native binaries by writing a isPackageNative(PackageParser.Package pkg) function:
private boolean isPackageNative(PackageParser.Package pkg) throws IOException {
final ZipFile zipFile = new ZipFile(pkg.mPath);
final Enumeration<? extends ZipEntry> privateZipEntries = zipFile.entries();
while (privateZipEntries.hasMoreElements()) {
final ZipEntry zipEntry = privateZipEntries.nextElement();
final String zipEntryName = zipEntry.getName();
if(true) Log.e(TAG, " Zipfile entry:"+zipEntryName);
if (zipEntryName.endsWith(".so")) {
zipFile.close();
return true;
}
}
zipFile.close();
return false;
}
This function checks every package for a native library and if it has one, I unpack it. PMS does this check in scanPackageLI(....). Search for the following code in the method:
if (isSystemApp(pkg) && !isUpdatedSystemApp(pkg))
and add the isPackageNative(pkg) check. There are other small modifications required but you'll probably figure it out once you have this direction. Hope it helps!

I think you cannot do it by default as Android's /system partition is mounted as read-only! You need a rooted phone so as to mount the /system with write privileges through this command:
mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system.
So, if you have a rooted phone you can add in your application this code:
Process p;
try {
// Preform su to get root privledges
p = Runtime.getRuntime().exec("su");
// Attempt to write a file to a root-only
DataOutputStream os = new DataOutputStream(p.getOutputStream());
// gain root privileges
os.writeBytes("mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system\n");
// do here the copy operation you want in /system/lib file, for example:
os.writeBytes("mv /sdcard/mylib.so /system/lib/\n");
// Close the terminal
os.writeBytes("exit\n");
os.flush();
} catch (IOException e) {
toastMessage("could not get root access");
}
Otherwise, you have to follow the solution that digitalmouse12 gave..

You will have to "adb push" the .so file yourself. Also, you don't necessarily have to push your library into system/lib (the folder might deny you permission anyway). Most push it to data/app and then load by issuing
System.load("/data/app/<libName>.so");

There's probably documentation somewhere, but if you cannot find that, I would suggest identifying a pre-installed app with an associated jni library .so and examining the android sources or corresponding system image or update.zip to see how it's handled.
In other words, programming by example...