Develop Reference

AWS Device Farm shows multiple test cases as one with custom testspec.yml

I'm trying to run automated TestNG tests for Android in AWS Device Farm. Currently all test cases are somehow transformed into one test case in report (test spec output shows that all test cases were invoked). Now report looks like this: Suites tab, Tests tab.
Custom test spec:
version: 0.1
phases:
install:
commands:
- export APPIUM_VERSION=1.14.2
- avm $APPIUM_VERSION
- ln -s /usr/local/avm/versions/$APPIUM_VERSION/node_modules/.bin/appium /usr/local/avm/versions/$APPIUM_VERSION/node_modules/appium/bin/appium.js
pre_test:
commands:
- export CLASSPATH=$CLASSPATH:$DEVICEFARM_TESTNG_JAR
- export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/*
- export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/dependency-jars/*
- echo "Start appium server"
- >-
appium --log-timestamp
--default-capabilities "{\"deviceName\": \"$DEVICEFARM_DEVICE_NAME\", \"platformName\":\"$DEVICEFARM_DEVICE_PLATFORM_NAME\",
\"app\":\"$DEVICEFARM_APP_PATH\", \"udid\":\"$DEVICEFARM_DEVICE_UDID\", \"platformVersion\":\"$DEVICEFARM_DEVICE_OS_VERSION\",
\"chromedriverExecutable\":\"$DEVICEFARM_CHROMEDRIVER_EXECUTABLE\",
\"autoAcceptAlerts\": false , \"fullReset\": false, \"noReset\": true
}"
>> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 &
- >-
start_appium_timeout=0;
while [ true ];
do
if [ $start_appium_timeout -gt 60 ];
then
echo "appium server never started in 60 seconds. Exiting";
exit 1;
fi;
grep -i "Appium REST http interface listener started on 0.0.0.0:4723" $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>&1;
if [ $? -eq 0 ];
then
echo "Appium REST http interface listener started on 0.0.0.0:4723";
break;
else
echo "Waiting for appium server to start. Sleeping for 1 second";
sleep 1;
start_appium_timeout=$((start_appium_timeout+1));
fi;
done;
test:
commands:
- echo "Navigate to test package directory"
- cd $DEVICEFARM_TEST_PACKAGE_PATH
- echo "Unzipping TestNG tests jar"
- unzip *-tests.jar
- echo "Start Appium TestNG test"
- java -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH org.testng.TestNG -d $DEVICEFARM_LOG_DIR/test-output testng.xml -verbose 10
post_test:
commands:
artifacts:
- $DEVICEFARM_LOG_DIR
I need reports that looks like this: Expected report

How to log to the fastlane screengrab output

I am using fastlanes screengrab to capture screenshots of an app. screengrab is utilizing androids ui testing framework espresso to navigate through the app. if one of my ui tests fails I want the other tests to continue capturing screenshots but at the same time I want to be notified about the failed test.
To prevent the test to fail and stopping the whole program I surrounded all my tests with try / catch. When a test fails I want the the exception to be logged to the screengrab output:
[10:24:37]: Running tests for locale: bs-BA
[10:24:37]: $ adb -s emulator-5554 shell am instrument --no-window-animation -w \
-e testLocale bs_BA \
-e endingLocale en_US \
-e class my.app.tests.MainLocationScreenshots \
-e isScreengrab true \
my.app.test/my.app.runners.RxTestRunner
[10:24:40]: ▸ my.app.tests.MainLocationScreenshots:
[10:24:41]: ▸ my.app.tests.MainLocationScreenshots:....
[10:25:37]: ▸ Time: 42.311
[10:25:37]: ▸ OK (7 tests)
I tried the following without success:
Log.d("", "Can you read this?????????")
Log.d("Screengrab", "Can you read this?")
Log.println(Log.ERROR,"Screengrab","What about this?")
System.out.print("But I bet this works")
All the Log calls appear in Logcat but not in the screengrab output, the System.out.print does not appear in either.
Is there a way to log the the screengrab output? Is it documented somewhere I haven't looked?
Thanks!

subprocess.check_output fails to execute a command but the same works in windows

I am trying to connect two devices to my pc and run some commands on them using python and adb.
when I run the command from the command prompt, it goes through fine, but when i put those in the python script, they give me errors.
this is causing the errors all the time:
from subprocess import check_output, CalledProcessError
try:
adb_ouput = check_output(["adb","devices","-l","|", "grep", "\"model\""])
print adb_ouput
except CalledProcessError as e:
print e
The error message I get is this:
Usage: adb devices [-l]
Command '['adb', 'devices', '-l', '|', 'grep', '"model"']' returned non-zero exit status 1
When I try the same code without the grep command, it works
adb_ouput = check_output(["adb","devices","-l"])
It gives me the right output.
When I try the same in windows command prompt, It works fine (I am replacing grep with FINDSTR because i'm using it in windows, and I have tried doing the same in the python script as well, with 'shell = True' and also without.)
eg:
adb devices -l | FINDSTR "model"
This gives me an ouptut without any problems.
The output I get is
123ab6ef device product:xxxxxxxxx model:xxxxxxxxx device:xxxxxxxxx
bd00051a4 device product:yyyyyyyyyy model:yyyyyyyyyy device:yyyyyyyyy
I am trying to understand where I am going wrong here, but can't figure it out.
So far I have checked the docs: https://docs.python.org/3/library/subprocess.html
https://docs.python.org/3/library/subprocess.html#subprocess.CalledProcessError
These just give me the error codes.
I have also looked at these answers:
Python, adb and shell execution query
I took some error checking from here and added to my code.
Python subprocess.check_output(args) fails, while args executed via Windows command line work OK
python check_output fails with exit status 1 but Popen works for same command
I think i am close but just can't put my finger on it.
Any help would be appreciated.

First
adb_ouput = check_output(["adb","devices","-l","|", "grep", "\"model\""])
certainly requires shell=True, but even with that it's not equivalent to
adb devices -l | FINDSTR "model"
When using check_output, you're passing "model" as grep argument literally but you should pass just model. "model" is not in your output (with quotes) so grep fails to find it, and returns exitcode 1 which is not really an error for grep but makes check_output trigger an exception because it expects 0.
So I would to this as a quickfix:
adb_ouput = check_output(["adb","devices","-l","|", "grep", "model"],shell=True)
And as a longfix I'd perform the grep command directly with python.
adb_output = check_output(["adb","devices","-l"])
for l in adb_output.splitlines():
if "model" in l:
print(l)

Calabash Android: Asset path error when trying to run a test on an APK

Calabash Android was working fine on my Mac OS X then quite suddenly and spontaneously, whenever I ran the following command...
calabash-android run
I received the following error:
2014-03-03 17:48:38 - JDK found on PATH.
2014-03-03 17:48:38 - Android SDK found at: /Applications/AndroidStudio.app/sdk/
2014-03-03 17:48:38 - /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby -S cucumber -v
Code:
* features/support/env.rb
* features/support/app_installation_hooks.rb
* features/support/app_life_cycle_hooks.rb
* features/support/hooks.rb
* features/step_definitions/calabash_steps.rb
Features:
* features/my_first.feature
Parsing feature files took 0m0.003s
Feature: Login feature
Scenario: As a valid user I can log into my app # features/my_first.feature:3
2014-03-03 17:48:38 - First scenario in feature - reinstalling apps
2014-03-03 17:48:38 - connected_devices: ["6b5140a7"]
2014-03-03 17:48:38 - "/Applications/AndroidStudio.app/sdk//platform-tools/adb" -s 6b5140a7 forward tcp:34777 tcp:7102
2014-03-03 17:48:38 -
W/asset ( 4263): Asset path is neither a directory nor file (type=1).
ERROR: dump failed because assets could not be loaded
'package' not found in aapt output (RuntimeError)
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/helpers.rb:10:in `package_name'
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/operations.rb:72:in `uninstall_apps'
/Users/Nauman/Desktop/APKs/features/support/app_installation_hooks.rb:22:in `Before'
W/asset ( 4265): Asset path is neither a directory nor file (type=1).
ERROR: dump failed because assets could not be loaded
'package' not found in aapt output (RuntimeError)
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/helpers.rb:10:in `package_name'
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/operations.rb:448:in `wake_up'
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/operations.rb:477:in `start_test_server_in_background'
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/operations.rb:93:in `start_test_server_in_background'
/Users/Nauman/Desktop/APKs/features/support/app_life_cycle_hooks.rb:5:in `Before'
When I press "Login" # calabash-android-0.4.20/lib/calabash-android/steps/press_button_steps.rb:17
Then I see "Welcome to coolest app ever" # calabash-android-0.4.20/lib/calabash-android/steps/assert_steps.rb:5
2014-03-03 17:48:39 - java -jar /Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/lib/screenshotTaker.jar 6b5140a7 screenshot_0.png
2014-03-03 17:48:40 - It looks like your app is no longer running.
It could be because of a crash or because your test script shut it down.
HTTPClient::KeepAliveDisconnected (HTTPClient::KeepAliveDisconnected)
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:882:in `block in parse_header'
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/timeout.rb:66:in `timeout'
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/timeout.rb:97:in `timeout'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:875:in `parse_header'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:858:in `read_header'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:667:in `get_header'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient.rb:1137:in `do_get_header'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient.rb:1086:in `do_get_block'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient.rb:887:in `block in do_request'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient.rb:986:in `rescue in protect_keep_alive_disconnected'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient.rb:980:in `protect_keep_alive_disconnected'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient.rb:886:in `do_request'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient.rb:774:in `request'
/Library/Ruby/Gems/2.0.0/gems/httpclient-2.3.4.1/lib/httpclient.rb:684:in `post'
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/operations.rb:334:in `make_http_request'
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/operations.rb:290:in `http'
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/operations.rb:556:in `shutdown_test_server'
/Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.20/lib/calabash-android/operations.rb:97:in `shutdown_test_server'
/Users/Nauman/Desktop/APKs/features/support/app_life_cycle_hooks.rb:12:in `After'
Failing Scenarios:
cucumber features/my_first.feature:3 # Scenario: As a valid user I can log into my app
1 scenario (1 failed)
2 steps (2 skipped)
0m2.432s
I would appreciate any help; I am stumped and at my wits end over this.

1. As your first line of error states JDK found on PATH.
Kindly install JDK & give the proper path in CMD.
2. You have to give absolute path for APK like calabash-android run path/to/apk.apk
3. Also, ensure you execute setup, build & resign this command before run.
To create proper .keystore file
calabash-android setup file.apk
To instrument your app
calabash-android resign file.apk
To build test server
calabash-android build file.apk

Android/adb redirect console app output

I am trying to get the output of a WLAN driver control interface via adb shell on a motorola droid 2. The command is called wlan_cu. To run the command,I am using:
% wlan_cu -itiwlan0 -s /data/jay/stat.sh
Connection established with supplicant .../Connection> Bssid_list,
Connect, Disassociate, Status, Full_bssid_list, wPs/
Status : DISCONNECT MAC : f8.7b.7a.7b.b7.9b SSID : <empty>
BSSID : 00.00.00.00.00.00
Channel : <empty>
Driver/, Connection/, Management/, Show/, Privacy/, scAn/, roaminG/, qOs/, poWer/, eVents/, Bt coexsistance/, Report/, dEbug/, biT/, aboUt, Quit
[1] Segmentation fault wlan_cu -itiwlan0 -s /data/jay/stat.sh
The -s option is to read wlan_cu commands from a script.
% cat /data/jay/stat.sh
c s
/
q
If I try to redirect the output i.e.
% wlan_cu -itiwlan0 -s /data/jay/stat.sh &> out.txt
out.txt is created, but empty and the output still goes to screen. Anyone have an idea? been stuck on this for quite a while.

you can try to redirect both stdout and stderr to the out.txt. Try this,
# wlan_cu -itiwlan0 -s /data/jay/stat.sh > out.txt 2>&1