I am writing a gradle task to install an apk unto emulators before espresso tests are run.
This is the task I have so far.
task installButlerApk {
doLast {
println "Verifying test-butler installation in emulators"
final adb = "$android.sdkDirectory.absolutePath/platform-tools/adb"
final String[] split = ["$adb", "devices", "-l"].execute().text.split("\\r?\\n")
split.each {
if (it.isEmpty())
return;
println "Emulator: $it"
final emu = it.split("\\s")[0]
checks whether the APK is already installed
if (["$adb", "-s", "$emu", "shell", "pm", "list", "packages"].execute().text.contains(butlerPackage))
return;
final installResult = ["$adb", "-s", "$emu", "install", "$butlerApkPath"].execute().text
if (!installResult.contains("Success"))
println "Could not install APK. Install output:\n$installResult"
else
println "Installed $butlerApkPath in $emu successfully"
}
}
}
However when I run it via the terminal the task ends up freezing. I am not sure why. I did some research about it and at one point I thought the command that was being passed to ProcessGroovyMethods' execute was failing because it was being passed as a string (execute(String self)) so I then used the array representation of execute (execute(String[] commandArray)) to see if that would work but I am still ending up with the same result so I am just asking for someone who has experience writing these tasks to give me some assistance. So far, I am printing the result of the command and it hasn't shown any errors. It's just stuck at the building process for hours.
Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.
C:\Users\Joel\Documents\Projects\Forms>gradlew installButlerApk
Picked up _JAVA_OPTIONS: -XX:ParallelGCThreads=2
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon:
https://docs.gradle.org/2.14.1/userguide/gradle_daemon.html.
Incremental java compilation is an incubating feature.
:app:installButlerApk
Verifying test-butler installation in emulators
Emulator: List of devices attached
> Building 0% > :app:installButlerApk
Well, this is expected behavior.
If you look at your output closely, you see
Emulator: List of devices attached
So following your code:
println "Emulator: $it"
outputs that line I quoted
final emu = it.split("\\s")[0]
takes the first space separated token which is List
checks whether the APK is already installed
this will not even compile, but I guess you just forgot the comment characters that you added in the question as explanation
if (["$adb", "-s", "$emu", "shell", "pm", "list", "packages"].execute().text.contains(butlerPackage))
return;
Now you execute adb -s List shell pm list
Executed manually this two times prints error: device not found for me and then exits, so your contains condition is false and the return is not done.
final installResult = ["$adb", "-s", "$emu", "install", "$butlerApkPath"].execute().text
Now you execute adb -s List install butler.apk
Executed manually this three times prints out error: device not found, then one time - waiting for device - and then sits there waiting until you cancel it, or a device with serial number List becomes available which of course will never happen and thus your task hangs until you kill it.
You have to skip the header line when you work through the list of devices, as this is of course not a device.
Besides this, you can of course use the Groovy standard ways to execute external commands. Yet while in Gradle, I'd rather use the Gradle variants. If you only want to execute one thing it would be a task of type Exec, if you want to execute multiple things like in your case, it is the Project.exec() or Script.exec() methods, so you would do something like
def output
new ByteArrayOutputStream().withStream { baos ->
exec {
executable adb
args "-s", emu, "shell", "pm", "list", "packages"
standardOutput os
}.assertNormalExitValue()
output = baos.toString()
}
if (output.contains(butlerPackage)) {
return
}
exec {
executable adb
args "-s", emu, "install", butlerApkPath
}.assertNormalExitValue()
Related
I am able to run a shell script on the Mac command line, but when I call it in Android Studio java it fails with this error:
GenerateActivity::THE COMMAND=[./go.top-level, Sweden]
java.io.IOException: Cannot run program "./go.top-level" (in directory "/data/user/0/com.example.frontpage/files/scripts"): error=13, Permission denied
My java code to call the script is as follows:
dir = lcontext.getFilesDir(); //lcontext is passed into this procedure
File workingDirectory = new File(dir + "/scripts/");//go.top-level is in /scripts dir
List<String> command = new ArrayList<String>();
command.add("./go.top-level"); // command
command.add(country); // command
System.out.println(TAG + "THE COMMAND=" + command.toString());
// creating the process
ProcessBuilder pb = new ProcessBuilder(command);
pb.directory(workingDirectory);
try {
pb.start(); //pb.wait() errors out indicating need a lock and a thread.....
} catch (Exception e) {
e.printStackTrace();
}
The process is failing immediately in the java. But, when I run it at the command line, i.e., via:
%./go.top-level Sweden (in /scripts dir)
it works, although it takes a couple of minutes. (note: I do "run-as com.example.frontpage" before running). The java seems logical to me since I set the workingDirectory appropriately. Also the /scripts folder and all of the files in it as well as the other folders used in this code are set to 777 permissions dynamically in the code. Why doesn't the error say, "./go.top-level Sweden" since the script call takes one argument? What am I missing here as far as this permission error? Is pb.start() a bad idea since the execution takes on the order of minutes? But when I try pb.wait() I get a different error indicating I need a lock and a thread. If that is best can someone indicate how best to do that?
TIA
My current setup for instrumentation tests cannot handle the screenshots saved to each device during test sharding. It doesn't know what to do because it detects multiple devices, and so those tasks all fail. I'm trying to figure out how to scale it so regardless of how many devices are being used to run my tests, whether it's one or a dozen, screenshots are grabbed from each device and moved to a single folder in my HTML report. I'm guessing I need to somehow pass the device name when I run ./gradlew cAT, so it can slap it onto all the other tasks I run as a prefix.
So that's my question and you don't have to read anymore if you already know how to answer it.
I have several assumptions, including:
1. My screenshot grabbing code (see below) is set up properly (and should not be scrapped entirely)
2. I'm running shards properly. So far I open a bunch of emulators, grab their names with "adb devices" in the terminal which returns something like:
List of devices attached
emulator-5554 device
emulator-5556 device
emulator-5558 device
Then I open three separate terminals and run one of the below commands in each terminal:
ANDROID_SERIAL=emulator-5554 ./gradlew cAT -Pandroid.testInstrumentationRunnerArguments.numShards=3 -Pandroid.testInstrumentationRunnerArguments.shardIndex=0
ANDROID_SERIAL=emulator-5556 ./gradlew cAT -Pandroid.testInstrumentationRunnerArguments.numShards=3 -Pandroid.testInstrumentationRunnerArguments.shardIndex=1
ANDROID_SERIAL=emulator-5558 ./gradlew cAT -Pandroid.testInstrumentationRunnerArguments.numShards=3 -Pandroid.testInstrumentationRunnerArguments.shardIndex=2
Then the tests run on the emulators and gradle creates a single HTML test report, mashing all of their results together by what I assume is magic.
Previous setup
As of now, I just run it without sharding. I take a screenshot when a test fails. This is saved to the device. I run the tests with the standard ./gradlew connectedAndroidTest (or ./gradlew cAT) and have tasks that follow it to pull the screenshots off the device and store them in the HTML report. It's kind of a mess and could use some work, but that code looks like this:
TestCase file that takes a screenshot when a test fails:
#Rule
public TestName name = new TestName();
#Rule
public TestWatcher watchman = new TestWatcher() {
#Override
public void failed(Throwable e, Description description) {
screenshotFailedTest(name.getMethodName());
}};
public void screenshotFailedTest(String testName) {
File path = new File(getScreenshotPath());
if (!path.exists()) {
path.mkdirs(); }
UiDevice device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation());
device.takeScreenshot(new File(path, testName));
}
private String getScreenshotPath() {
return Environment.getExternalStorageDirectory().getAbsolutePath() + "/screenshots/" + getTargetContext().getPackageName();
}
My nightmare gradle file's tasks that handle the screenshots on the device:
tasks.whenTaskAdded { task ->
if (task.name.startsWith('connectedAndroidTest') || (task.name.startsWith('connected') && task.name.contains('AndroidTest'))) {
println "Starting Connected Android Tests"
task.finalizedBy screenshotFailedTests
}}
task createScreenshotFolder(type: Exec) {
commandLine './create_screenshot_folder.sh'
// see script below
}
task copyScreenshots(type: Exec) {
def adb = android.sdkDirectory.path + "/platform-tools/adb"
executable "$adb"
args "pull",
"/storage/emulated/0/screenshots/<app.name>/",
"${buildDir}/reports/androidTests/connected/flavors/INTERNAL/"
}
task removeScreenshotsFromDevice(type: Exec) {
def adb = android.sdkDirectory.path + "/platform-tools/adb"
executable "$adb"
args "shell",
"rm",
"-r",
"/storage/emulated/0/screenshots/<app.name>/"
}
task renameScreenshotFolder {
doLast {
file("${buildDir}/reports/androidTests/connected/flavors/INTERNAL/<app.name>").renameTo(file("${buildDir}/reports/androidTests/connected/flavors/INTERNAL/Screenshots"))
}}
task screenshotFailedTests {
dependsOn 'createScreenshotFolder'
dependsOn 'copyScreenshots'
dependsOn 'removeScreenshotsFromDevice'
dependsOn 'renameScreenshotFolder'
tasks.findByName('copyScreenshots').mustRunAfter 'createScreenshotFolder'
tasks.findByName('removeScreenshotsFromDevice').mustRunAfter 'copyScreenshots'
tasks.findByName('renameScreenshotFolder').mustRunAfter 'removeScreenshotsFromDevice'
}
And then the create_screenshot_folder.sh script referenced above:
# Call adb and check if folder exists, create if it doesn't
cd $ANDROID_HOME/platform-tools/
./adb shell "if [ -e /storage/emulated/0/screenshots/ ]; then echo Screenshot folder already exists; else mkdir /storage/emulated/0/screenshots/ && echo Creating screenshot folder; fi"
./adb shell "if [ -e /storage/emulated/0/screenshots/<app.name>/ ]; then echo The screenshot folder already exists; else mkdir /storage/emulated/0/screenshots/<app.name>/ && echo Creating screenshot folder; fi"
It's rather messy and maybe there's a much simpler way to do all of that without external libraries, but it seems to do just fine for a single device.
have you tried running the tests via
./gradlew spoonTaskname
That takes care of the sharding automatically and you can eliminate that part from your framework. Also, it should also take care of pulling all the screenshots from the emulators/devices and merge them to the final report.
Background
So far, I was able to install APK files using root (within the app), via this code:
pm install -t -f fullPathToApkFile
and if I want to (try to) install to sd-card :
pm install -t -s fullPathToApkFile
The problem
Recently, not sure from which Android version (issue exists on Android P beta, at least), the above method fails, showing me this message:
avc: denied { read } for scontext=u:r:system_server:s0 tcontext=u:object_r:sdcardfs:s0 tclass=file permissive=0
System server has no access to read file context u:object_r:sdcardfs:s0 (from path /storage/emulated/0/Download/FDroid.apk, context u:r:system_server:s0)
Error: Unable to open file: /storage/emulated/0/Download/FDroid.apk
Consider using a file under /data/local/tmp/
Error: Can't open file: /storage/emulated/0/Download/FDroid.apk
Exception occurred while executing:
java.lang.IllegalArgumentException: Error: Can't open file: /storage/emulated/0/Download/FDroid.apk
at com.android.server.pm.PackageManagerShellCommand.setParamsSize(PackageManagerShellCommand.java:306)
at com.android.server.pm.PackageManagerShellCommand.runInstall(PackageManagerShellCommand.java:884)
at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:138)
at android.os.ShellCommand.exec(ShellCommand.java:103)
at com.android.server.pm.PackageManagerService.onShellCommand(PackageManagerService.java:21125)
at android.os.Binder.shellCommand(Binder.java:634)
at android.os.Binder.onTransact(Binder.java:532)
at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:2806)
at com.android.server.pm.PackageManagerService.onTransact(PackageManagerService.java:3841)
at android.os.Binder.execTransact(Binder.java:731)
This seems to also affect popular apps such as "Titanium backup (pro)", which fails to restore apps.
What I've tried
Looking at what's written, it appears it lacks permission to install APK files that are not in /data/local/tmp/.
So I tried the next things, to see if I can overcome it:
set the access to the file (chmod 777) - didn't help.
grant permissions to my app, of both storage and REQUEST_INSTALL_PACKAGES (using ACTION_MANAGE_UNKNOWN_APP_SOURCES Intent) - didn't help.
create a symlink to the file, so that it will be inside the /data/local/tmp/, using official API:
Os.symlink(fullPathToApkFile, symLinkFilePath)
This didn't do anything.
create a symlink using this :
ln -sf $fullPathToApkFile $symLinkFilePath
This partially worked. The file is there, as I can see it in Total Commander app, but when I try to check if it exists there, and when I try to install the APK from there, it fails.
Copy/move (using cp or mv) the file to the /data/local/tmp/ path, and then install from there. This worked, but it has disadvantages: moving is risky because it temporarily hides the original file, and it changes the timestamp of the original file. Copying is bad because of using extra space just for installing (even temporarily) and because it wastes time in doing so.
Copy the APK file, telling it to avoid actual copy (meaning hard link), using this command (taken from here) :
cp -p -r -l $fullPathToApkFile $tempFileParentPath"
This didn't work. It got me this error:
cp: /data/local/tmp/test.apk: Cross-device link
Checking what happens in other cases of installing apps. When you install via via the IDE, it actually does create the APK file in this special path, but if you install via the Play Store, simple APK install (via Intent) or adb (via PC), it doesn't.
Wrote about this here too: https://issuetracker.google.com/issues/80270303
The questions
Is there any way to overcome the disadvantages of installing the APK using root on this special path? Maybe even avoid handling this path at all?
Why does the OS suddenly require to use this path? Why not use the original path instead, just like in the other methods of installing apps? What do the other methods of installing apps do, that somehow avoids using the spacial path?
One solution, in case you don't mind the moving procedure, is to also save&restore the timestamp of the original file, as such:
val tempFileParentPath = "/data/local/tmp/"
val tempFilePath = tempFileParentPath + File(fullPathToApkFile).name
val apkTimestampTempFile = File(context.cacheDir, "apkTimestamp")
apkTimestampTempFile.delete()
apkTimestampTempFile.mkdirs()
apkTimestampTempFile.createNewFile()
root.runCommands("touch -r $fullPathToApkFile ${apkTimestampTempFile.absolutePath}")
root.runCommands("mv $fullPathToApkFile $tempFileParentPath")
root.runCommands("pm install -t -f $tempFilePath")
root.runCommands("mv $tempFilePath $fullPathToApkFile")
root.runCommands("touch -r ${apkTimestampTempFile.absolutePath} $fullPathToApkFile")
apkTimestampTempFile.delete()
It's still a bit dangerous, but better than copying files...
EDIT: Google has shown me a nice workaround for this (here) :
We don't support installation of APKs from random directories on the device. They either need to be installed directly from the host using 'adb install' or you have to stream the contents to install --
$ cat foo.apk | pm install -S APK_SIZE
While I think this is incorrect that they don't support installing of APK files from random paths (always worked before), the workaround does seem to work. All I needed to change in the code of installing an APK file is as such:
val length = File(fullPathToApkFile ).length()
commands.add("cat $fullPathToApkFile | pm install -S $length")
Thing is, now I have some other questions about it :
Does this workaround avoid the moving/copying of the APK into storage, and without affecting the original file ? - seems it does
Will this support any APK file, even large ones? - seems it succeeds in doing it for an APK that takes 433MB, so I think it's safe to use for all sizes.
This is needed only from Android P, right? - so far seems so.
Why does it need the file size as a parameter ? - No idea, but if I remove it, it won't work
Thanks for the answers! I looked everywhere else as well to get a whole setup for OTA to work for Android 10 and so on. It 100% works on Samsung Galaxy Tab 10.1 running Android 10.
Here is a medium article with the code:
https://medium.com/#jnishu1996/over-the-air-ota-updates-for-android-apps-download-apk-silent-apk-installation-auto-launch-8ee6f342197c
The magic is running this command with root access:
process = Runtime.getRuntime().exec("su");
out = process.getOutputStream();
DataOutputStream dataOutputStream = new DataOutputStream(out);
// Get all file permissions
dataOutputStream.writeBytes("chmod 777 " + file.getPath() + "\n");
// Perform silent installation command, all flags are necessary for some reason, only this works reliably post Android 10
String installCommand = "cat " + file.getAbsolutePath() + "| pm install -d -t -S " + file.length();
// Data to send to the LaunchActivity to the app knows it got updated and performs necessary functions to notify backend
// es stands for extraString
// In LaunchActivity onCreate(), you can get this data by running -> if (getIntent().getStringExtra("OTA").equals("true"))
String launchCommandIntentArguments = "--es OTA true --es messageId " + MyApplication.mLastSQSMessage.receiptHandle();
// Start a background thread to wait for 8 seconds before reopening the app's LaunchActivity, and pass necessary arguments
String launchCommand = "(sleep 8; am start -n co.getpresso.Presso/.activities.LaunchActivity " + launchCommandIntentArguments + ")&";
// The entire command is deployed with a ";" in the middle to launchCommand run after installCommand
String installAndLaunchCommand = installCommand + "; " + launchCommand;
// begins the installation
dataOutputStream.writeBytes(installAndLaunchCommand);
dataOutputStream.flush();
// Close the stream operation
dataOutputStream.close();
out.close();
int value = process.waitFor();
I have an android project and trying to start the emulator. It works fine locally, but not on the build server, so I'm trying to figure out why, but I'm not able to see any of the error output (even locally).
task startEmulator << {
def process = "emulator -avd nexus4 -no-boot-anim -no-window".execute()
//...consumption code here
}
Here's what I tried for reading the error output:
process.waitFor()
print process.err.text
and
process.waitForProcessOutput(System.out, System.err)
and
process.consumeProcessOutput(System.out, System.err)
and
process.consumeProcessErrorStream(System.err)
The error code gets set properly to 1 or 0 depending on whether the run was successful. It's not a matter of the code exiting too soon. For example when I try the code with an AVD name that doesn't exist, the error stream is empty.
What am I missing?
PS: Can't use an exec task since I need this execute call to be asynchronous/in the background.
Make sure the tomcat user (if you are using a tomcat) has the privileges to run the command emulator. If tomcat user doesn´t have the privileges, you are not going to be able to run the command.
My workaround is to use a string buffer.
def bout = new StringBuffer()
// Not sure if a common buffer is always a good idea.
"ls".execute().waitForProcessOutput(bout, bout)
println bout.toString()
I'd like to run some android tests but I want to scedule or delay the execution of these tests. I'm trying to do this from SL4A. The idea is to install SL4A start the server from my laptop, run a special python script that will sleep for about 20 seconds then wakeup and launch the tests. I am performing some rather involved automation that requires my tests be run after I install them and disconnect the USB cable. I know I can issue intents from SL4A but I'm looking for the equivalent of:
adb shell am instrument \
-e class MyInstrumentationTestCase \
-w MyInstrumentationTestRunner
Can this be done via intent? Should I send a broadcast or use the start activity functions?
I've tried running a system command directly on the device from Python with this script but I get a "permission denied" error:
from subprocess import call
call(["am", "instrument", "-e", "class", "com.example.android.app.test.TestContactList", "-w", "com.example.app.test/com.zutubi.android.junitreport.JUnitReportTestRunner"])
UPDATE
I've also tried using os.system to run the command (see below modifications) and this gives a different error: soinfo_link_image(linker.cpp:1635): could not load library "libanroid_runtime.so" needed by "app_process"; caused by so info_relocate(linker.cpp:975): cannot locate symbol " sqlite3_stmt_readonly" referenced by "lib android_runtime.so"...CANNOT LINK EXECUTABLE
import os
#from subprocess import call
#call(["am", "instrument", "-e", "class", "com.example.android.app.test.TestContactList", "-w", "com.example.android.app.test/com.zutubi.android.junitreport.JUnitReportTestRunner"])
os.system('echo "Running tests"')
os.system('am instrument -e class com.example.android.app.test.TestContactList -w com.example.android.app.test/com.zutubi.android.junitreport.JUnitReportTestRunner')
What are any other options?
I have the following function, which will launch any SL4A script from within a Python SL4A script, as a separate process. You can not use subprocess for this type of thing.
You may be able to deduce what you need from this.
from android import Android
droid = Android()
def launch_script(path, visible=False):
visibilty = 'FORE' if visible else 'BACK'
activity = 'com.googlecode.android_scripting.action.LAUNCH_{0}GROUND_SCRIPT'.format(visibilty)
extras = {'com.googlecode.android_scripting.extra.SCRIPT_PATH': path}
packagename = 'com.googlecode.android_scripting'
classname = 'com.googlecode.android_scripting.activity.ScriptingLayerServiceLauncher'
intent = droid.makeIntent(activity, None, None, extras, None, packagename, classname).result
droid.startActivityIntent(intent)