Develop Reference

Install APK using root, handling new limitations of "/data/local/tmp/" folder

Background
So far, I was able to install APK files using root (within the app), via this code:
pm install -t -f fullPathToApkFile
and if I want to (try to) install to sd-card :
pm install -t -s fullPathToApkFile
The problem
Recently, not sure from which Android version (issue exists on Android P beta, at least), the above method fails, showing me this message:
avc: denied { read } for scontext=u:r:system_server:s0 tcontext=u:object_r:sdcardfs:s0 tclass=file permissive=0
System server has no access to read file context u:object_r:sdcardfs:s0 (from path /storage/emulated/0/Download/FDroid.apk, context u:r:system_server:s0)
Error: Unable to open file: /storage/emulated/0/Download/FDroid.apk
Consider using a file under /data/local/tmp/
Error: Can't open file: /storage/emulated/0/Download/FDroid.apk
Exception occurred while executing:
java.lang.IllegalArgumentException: Error: Can't open file: /storage/emulated/0/Download/FDroid.apk
at com.android.server.pm.PackageManagerShellCommand.setParamsSize(PackageManagerShellCommand.java:306)
at com.android.server.pm.PackageManagerShellCommand.runInstall(PackageManagerShellCommand.java:884)
at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:138)
at android.os.ShellCommand.exec(ShellCommand.java:103)
at com.android.server.pm.PackageManagerService.onShellCommand(PackageManagerService.java:21125)
at android.os.Binder.shellCommand(Binder.java:634)
at android.os.Binder.onTransact(Binder.java:532)
at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:2806)
at com.android.server.pm.PackageManagerService.onTransact(PackageManagerService.java:3841)
at android.os.Binder.execTransact(Binder.java:731)
This seems to also affect popular apps such as "Titanium backup (pro)", which fails to restore apps.
What I've tried
Looking at what's written, it appears it lacks permission to install APK files that are not in /data/local/tmp/.
So I tried the next things, to see if I can overcome it:
set the access to the file (chmod 777) - didn't help.
grant permissions to my app, of both storage and REQUEST_INSTALL_PACKAGES (using ACTION_MANAGE_UNKNOWN_APP_SOURCES Intent) - didn't help.
create a symlink to the file, so that it will be inside the /data/local/tmp/, using official API:
Os.symlink(fullPathToApkFile, symLinkFilePath)
This didn't do anything.
create a symlink using this :
ln -sf $fullPathToApkFile $symLinkFilePath
This partially worked. The file is there, as I can see it in Total Commander app, but when I try to check if it exists there, and when I try to install the APK from there, it fails.
Copy/move (using cp or mv) the file to the /data/local/tmp/ path, and then install from there. This worked, but it has disadvantages: moving is risky because it temporarily hides the original file, and it changes the timestamp of the original file. Copying is bad because of using extra space just for installing (even temporarily) and because it wastes time in doing so.
Copy the APK file, telling it to avoid actual copy (meaning hard link), using this command (taken from here) :
cp -p -r -l $fullPathToApkFile $tempFileParentPath"
This didn't work. It got me this error:
cp: /data/local/tmp/test.apk: Cross-device link
Checking what happens in other cases of installing apps. When you install via via the IDE, it actually does create the APK file in this special path, but if you install via the Play Store, simple APK install (via Intent) or adb (via PC), it doesn't.
Wrote about this here too: https://issuetracker.google.com/issues/80270303
The questions
Is there any way to overcome the disadvantages of installing the APK using root on this special path? Maybe even avoid handling this path at all?
Why does the OS suddenly require to use this path? Why not use the original path instead, just like in the other methods of installing apps? What do the other methods of installing apps do, that somehow avoids using the spacial path?

One solution, in case you don't mind the moving procedure, is to also save&restore the timestamp of the original file, as such:
val tempFileParentPath = "/data/local/tmp/"
val tempFilePath = tempFileParentPath + File(fullPathToApkFile).name
val apkTimestampTempFile = File(context.cacheDir, "apkTimestamp")
apkTimestampTempFile.delete()
apkTimestampTempFile.mkdirs()
apkTimestampTempFile.createNewFile()
root.runCommands("touch -r $fullPathToApkFile ${apkTimestampTempFile.absolutePath}")
root.runCommands("mv $fullPathToApkFile $tempFileParentPath")
root.runCommands("pm install -t -f $tempFilePath")
root.runCommands("mv $tempFilePath $fullPathToApkFile")
root.runCommands("touch -r ${apkTimestampTempFile.absolutePath} $fullPathToApkFile")
apkTimestampTempFile.delete()
It's still a bit dangerous, but better than copying files...
EDIT: Google has shown me a nice workaround for this (here) :
We don't support installation of APKs from random directories on the device. They either need to be installed directly from the host using 'adb install' or you have to stream the contents to install --
$ cat foo.apk | pm install -S APK_SIZE
While I think this is incorrect that they don't support installing of APK files from random paths (always worked before), the workaround does seem to work. All I needed to change in the code of installing an APK file is as such:
val length = File(fullPathToApkFile ).length()
commands.add("cat $fullPathToApkFile | pm install -S $length")
Thing is, now I have some other questions about it :
Does this workaround avoid the moving/copying of the APK into storage, and without affecting the original file ? - seems it does
Will this support any APK file, even large ones? - seems it succeeds in doing it for an APK that takes 433MB, so I think it's safe to use for all sizes.
This is needed only from Android P, right? - so far seems so.
Why does it need the file size as a parameter ? - No idea, but if I remove it, it won't work

Thanks for the answers! I looked everywhere else as well to get a whole setup for OTA to work for Android 10 and so on. It 100% works on Samsung Galaxy Tab 10.1 running Android 10.
Here is a medium article with the code:
https://medium.com/#jnishu1996/over-the-air-ota-updates-for-android-apps-download-apk-silent-apk-installation-auto-launch-8ee6f342197c
The magic is running this command with root access:
process = Runtime.getRuntime().exec("su");
out = process.getOutputStream();
DataOutputStream dataOutputStream = new DataOutputStream(out);
// Get all file permissions
dataOutputStream.writeBytes("chmod 777 " + file.getPath() + "\n");
// Perform silent installation command, all flags are necessary for some reason, only this works reliably post Android 10
String installCommand = "cat " + file.getAbsolutePath() + "| pm install -d -t -S " + file.length();
// Data to send to the LaunchActivity to the app knows it got updated and performs necessary functions to notify backend
// es stands for extraString
// In LaunchActivity onCreate(), you can get this data by running -> if (getIntent().getStringExtra("OTA").equals("true"))
String launchCommandIntentArguments = "--es OTA true --es messageId " + MyApplication.mLastSQSMessage.receiptHandle();
// Start a background thread to wait for 8 seconds before reopening the app's LaunchActivity, and pass necessary arguments
String launchCommand = "(sleep 8; am start -n co.getpresso.Presso/.activities.LaunchActivity " + launchCommandIntentArguments + ")&";
// The entire command is deployed with a ";" in the middle to launchCommand run after installCommand
String installAndLaunchCommand = installCommand + "; " + launchCommand;
// begins the installation
dataOutputStream.writeBytes(installAndLaunchCommand);
dataOutputStream.flush();
// Close the stream operation
dataOutputStream.close();
out.close();
int value = process.waitFor();

batch scripting - if exist ./sdcard/file.any using adb

I'd like to check whether a file exists on my Android device, and if not - to push it. What is the syntax for doing so using adb on batch?
Something like:
if exist ./sdcard/file.any do echo "exists"
else adb push file.any ./sdcard/

Running in a command prompt window if /? or help if outputs the help pages for internal command if.
The command processor is not very tolerant on syntax. An else branch is only possible with using parentheses. The true branch must have ( which must be on same line as command if separated with a space from last argument of the condition.
The closing ) for true branch can be on same line as opening ( or on a different line.
The keyword else must be on same line as closing ) of the true branch after a space.
If round brackets are used also for the else branch, opening ( must be on same line as else separated with a space from the keyword.
The keyword do is completely wrong here as it is a keyword for command FOR.
Some of the working variants.
Everything on one line with as less spaces as possible (hard to read):
if exist "directory\file" (echo exists) else (adb push "file" "directory")
Everything on one line with more spaces for easier reading:
if exist "directory\file" ( echo exists ) else ( adb push "file" "directory" )
Condition on first line, true branch on second line, else branch without parentheses on third line:
if exist "directory\file" (
echo exists
) else adb push "file" "directory"
Each part of the entire condition on separate lines:
if exist "directory\file" (
echo exists
) else (
adb push "file" "directory"
)
There are more variants possible, but they are all horrible to read.
It is best to use second or fourth variant as those two are the best for reading.
I recommend to use always fourth variant if an else branch is used, too.
.\ at beginning of file name / directory name could be omitted.
And the directory separator on Windows is the backslash character.
NOTE: I don't know if if exist ./sdcard/file.any or .\sdcard\file.any works at all. The question does not contain any information about
Android version,
the Android device itself,
how the device is configured regarding to accessing files and directories on its storage medias – as removable storage(s) making it possible to access the files and directories from Windows command line environment or just via Media Transfer Protocol which is not supported from Windows command line environment,
in which environment this batch file is running, i.e. what is the current directory, what are the values of the environment variables PATH and PATHEXT, etc.
So the answer covers only the IF syntax mistake, not how to check for existence of a file on any Android device with any Android version connected with whatever device drivers and data storage accessing protocols are used for accessing the device.

I know this is kinda old but how about something around the lines of:
set cmd="adb shell ls | find /c "theFile" "
FOR /F %%K IN (' !cmd! ') DO SET TEST=%%K
if !TEST! GTR 0 (
echo the file exists
) else (
echo the file does not exist
)
I am using this for a very similar scenario which works for me. Thought I'd share.

Libgit2 running on Android unable to perform clone operation - returns "failed to set permissions" error

I have just built Libgit2 (v0.20.0) for Android (target SDK version 18, debugging on a rooted device running Cyanogenmod 10.1.2, Android 4.2.2) and a simple function like getting the version number of Libgit2 works fine through the JNI. But when I use the git_clone function it stops right after the objects/info folder is created and returns this error:
Error -1 cloning repository - Failed to set permissions on '/storage/sdcard0/it/ptt/.git/objects/info': Operation not permitted
I have given the application the WRITE_EXTERNAL_STORAGE permission but I guess it still can't chmod unless owner of the file. When I use adb shell to check out the permission mode of the info folder I get:
d---rwxr-x system sdcard_rw 2014-05-15 09:31 info
And by using pwd.h functions I get the username that the c code (that is calling git_clone) is under to be u0_a92. How am I suppose to get pass this I suppose very Android related issue? Is there a simple way to stop Libgit2 from calling p_chmod or can I give it permissions to do so?

I ended up defining p_chmod as a method always returning true to get passed the error. In the bash script I use to build libgit2 I inserted the following lines that leaves the source files in an unmodified condition after building for android:
LIBGIT2_POSIX_PATH="$LIBGIT2_SOURCE_PATH/src/posix.h"
LIBGIT2_POSIX_BACKUP_PATH="$LIBGIT2_SOURCE_PATH/src/posix_original.h"
printf "#include \"always_success.h\"\nint always_success() { return 0; }" > "$LIBGIT2_SOURCE_PATH/src/always_success.c"
printf "int always_success();" > "$LIBGIT2_SOURCE_PATH/src/always_success.h"
cp $LIBGIT2_POSIX_PATH "$LIBGIT2_POSIX_BACKUP_PATH"
sed -i "s/^#define\sp_chmod(p, m).*$/#include \"always_success.h\"\n#define p_chmod(p, m) always_success()\nextern int always_success();\n/" $LIBGIT2_POSIX_PATH
# run the build process with cmake ...
# restore chmod manipulated source header
mv $LIBGIT2_POSIX_BACKUP_PATH $LIBGIT2_POSIX_PATH
There is probably a cleaner way to solve this but at least now I dont get that error anymore. Thanks to Carlos for his help!
UPDATE
Running adb shell mount | grep sdcard I could see that the sdcard which I am trying to clone the repository into uses the vfat file system which according to this forum thread doesn't support unix-style permissions.

How to order test cases for Spoon Automated Testing in Android?

I have added a suite() method to order my tests the way I want them and thus when I run it through Android JUnit they are executed accordingly. But then I noticed that when I use the Spoon execution, the one using cmd, my test cases are executed alphabetically which is the default order.
Why does this happen and how would you counter it without renaming my test cases?

I have the same issue as you; I require a specific order that my test need to be ran in. The app I am testing is too complicated to run in an unpredictable order. My solution was this:
Add this to your build.gradle:
spoon {
if (project.hasProperty('spoonClassName')){
className = project.spoonClassName
}
}
Now, you can execute a specific class with a command like so:
gradle spoon -PspoonClassName=< com.your.pakage.ClassName>
Next, create a file at the root of your Android project: runAllTests.sh
Edit your .sh to look like this:
#!/bin/sh
date +%b-%dT%H.%M > timestamp.out
sites="$HOME"/path/to/project/root
timestamp="$(cat "$sites"/timestamp.out)"
result_folder="$sites"/results
destdir="$result_folder/Results-$timestamp"
mkdir -p "$destdir"
echo "Directory created: ${destdir##*/}"
<---------- Here you start running the test --------------->
echo "Starting Master Setup"
gradle spoon -PspoonClassName=com.espresso.test.MasterSetup
cp -r "$sites"/app/build/spoon "$destdir"/MasterSetup
echo "Results saved to MasterSetup"
echo "Starting WorkoutSchedule"
gradle spoon -PspoonClassName=com.espresso.test.WorkoutSchedule
cp -f "$sites"/app/build/spoon "$destdir"/WorkoutSchedule
echo "Results saved to WorkoutSchedule"
echo "Starting Setting.test"
gradle spoon -PspoonClassName=com.espresso.test.Settings
cp -r "$sites"/app/build/spoon "$destdir"/Settings
echo "Results saved to Settings"
Then, give the script permissions
cd to the script
type chmod u+x runAllTest.sh
You're set. Now just cd to your root, then to execute your test, type . runAllTest.sh.
So, what this does:
First, it creates a timestamp.out. I use this so I can save my results to a file over and over without previous results being overwritten. You do not need this part.
Next, it creates a result folder in the root of your project if it is not already there.
Then, it will make a folder inside the results folder named Results-SOME-DATE.
Lastly, each test will run, saving the results to the normal spot on your project. (Inside build/spoon) Once test are complete it will copy the results to the results folder, and name each test result appropriately so it is easy to see all your tests ran.
NOTE: This script was wrote for MAC. If your on windows or anything else, this script may need modifications.
Additionally: You will find it is inconvenient to open in to each folder to get the index.html opened. So I wrote this script to add to your bash_profile:
function open-results () {
# the browser to open up `index.html' in.
browser='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
# let the user know what directory we're looking in
printf "looking in %s" "$(pwd)"
echo ...
for paths in $(find ./ -name 'debug' -type d); do
for files in $(find "$paths" -name 'index.html'); do
open -a "$browser" "$files"
done
done
echo done
}
Now, cd in terminal to the Results-SOME-DATE, and type open-results. Again, this was written for terminal. You may need to modify depending on your OS. But the structure should be the same
I hope this helps.

The jUnit testing philosophy is that test cases should not depend on each other so order shouldn't be important. That's why you're finding it hard to do. You might want to consider using the "setUp" method to create initial conditions for your test cases rather than having them build on each other.

How can I copy the contentsof data/ to sdcard/ without using adb?

Hi I need to copy/move the contents of data/tombstones to sdcard/tombstones
I'm using the command below:
mv data/tombstones /sdcard/tombstones
"failed on 'tombstones' - Cross-device link"
but I'm getting above error.

You have a SANE VERSION of the mv command
paraphrasing a few bits from lbcoder from xda and darkxuser from androidforums
"failed on 'tombstones' - Cross-device link"
It means that you can't create a hard link on one device (filesystem) that refers to a file on a different filesystem.
This is an age-old unix thing. You can NOT move a file across a filesystem using most implementations of mv. mv is not made to copy data from device to device, it simply changes a file's location within a partition. Since /data and /sdcard are different partitions, it's failing.
Consider yourself fortunate that you have a SANE VERSION of the mv command that doesn't try anyway -- some old versions will actually TRY to do this, which will result in a hard link that points to NOTHING, and the original data being INACCESSIBLE.
The mv command does NOT MOVE THE DATA!!! It moves the HARDLINK TO
THE DATA.
If you want to move the file to a different filesystem, you need to use the "cp" command. Copy the file to create a SECOND COPY of it on a different filesystem, and then DELETE the OLD one with the "rm" command.
A simple move command:
#!/bin/bash
dd if="$1" of="$2"
rm -f "$1"
You will note that the "cp" command returns true or false depending on the successful completion of the copy, therefore the original will only be removed IF the file copied successfully.
OR
#!/bin/bash
cat data/tombstones > sdcard/tombstones
rm data/tombstones
These script can be copied into some place referenced by the PATH variable and set executable.
Different Interface
If you need a different interface from adb you may move files using the FileExplorer in DDMS View.
Side note:
You can move a file into a folder when:
You're root;
It is your app directory;
You've used chmod from adb or elsewhere to change permissions

Basically you don't have permission to access /data/tombstones in a production version .
It seems we have to 'root' the device first.
But I failed to root my Samsung S4 which is using Android 4.3