I am having problems with timeouts running my UI tests in circle ci since the command connectedAndroidTest is taking more than 10 minutes to run.
So I am trying to split them into test suits and run each suite at a time.
I found how to create suites for my android tests here: https://developer.android.com/reference/junit/framework/TestSuite.html
But I can't find how to run them with the connectedAndroidTest command.
I have not found any way to execute the TestSuite, but I found two other options to solve my timeout problems with the circle ci tests:
Run tests by package:
./gradlew app:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.package=<package>
Run tests by type:
./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.size=<small|medium|large>
You need to add #SmallTest, #MediumTest, #LargeTest to your test classes or methods to split them into the groups and avoid hitting a timeout.
Another option that I found was to change the timeout of the command:
- ./gradlew app:connectedCheck -PdisablePreDex:
timeout: 1800
To add to #jonathanrz answer:
Run tests by class:
./gradlew app:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class= com.example.app.MyTestClass
Run tests by method:
./gradlew app:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class= com.example.app.MyTestClass#myTestMethod
Btw: You can add further arguments to your tests by adding:
-Pandroid.testInstrumentationRunnerArguments.foo="My\ Extra\ Info
Just note how you have to escape spaces here if they exist
Related
Do instrumentation tests in android run in parallel or sequentially?
I want to run my android tests sequentially can someone help me how to configure it.
#FixMethodOrder(MethodSorters.JVM)
#FixMethodOrder(MethodSorters.NAME_ASCENDING)
#FixMethodOrder(MethodSorters.DEFAULT)
test01() test02() test03()
A0_test() A1_test() A2_test()
When I ran "run vts -m VtsHalWifiSupplicantV1_0Target", I get 0 result from it.
vts-tf > run vts -m VtsHalWifiSupplicantV1_0Target
...
10-15 09:56:06 I/ResultReporter: Invocation finished in 48s. PASSED: 0, FAILED: 0, MODULES: 2 of 2
Do I need to write some code to run the vts test case?
I expected to get something passed or failed. How can I get results?
Thanks in advance.
There are at least two reasons why a VTS test run would have that result.
The HAL Testability Checker decided to skip the test. This could be because the HAL you want to test is not running on the target or you try to run an x86 test on an x64 platform.
vts-tradefed will call the test binary with the --gtest_list_tests option in order to get the amount of tests. If this call fails vts-tradefed will assume you have zero test cases.
In any case you can get more information by checking the host logs.
zcat $ANDROID_HOST_OUT/vts/android-vts/logs/latest/inv_*/host_log*.txt.gz | less
Why Circleci doesn't see my test?
I have test called MyAppTest.java. But when I run build on circleci it only shows the following results:
Your build ran 2 tests in testDebugUnitTest, testReleaseUnitTest with 0 failures
Slowest test: com.myapp.android.ExampleUnitTest addition_isCorrect (took 0.00 seconds).
Why it doesn't show result for MyAppTest?
The reason for this that configuration of my config.yml file was only for Unit tests. In the end I've found proper configuration for Instrumentation tests, but still faced error of not enough memory.
I've set my tests for my android projects to run automatically in team city. Tests are running and all is fine, but TeamCity doesnt seem to auto discover the test output. It doesn't catch that tests are being run (doesnt say x/x successfull for example). I've tried adding --info to the gradle connectedCheck target, but no luck. Any ideas how to achieve this?
I'm using the Gradle Teamcity-runner with the default instrumentation-config in Android Gradle.
Example output from build log for a couple of tests:
[10:11:20][:aproject:connectedInstrumentAFlavorDebug] com.test.sync.worker.SyncResponseValidatorTests > testPermanentErrorResponse[Galaxy Nexus - 4.2.1] [32mSUCCESS [0m
[10:11:20][:aproject:connectedInstrumentTestAFlavorDebug] com.test.sync.worker.SyncResponseValidatorTests > testSuccess[Galaxy Nexus - 4.2.1] [32mSUCCESS [0m
Compile the chromium for Android
Build every test:
$ ninja -C out/Release
Running the layout Tests
$ webkit/tools/layout_tests/run_webkit_tests.sh
I get following errors:
Using port 'chromium-linux-x86_64' Test configuration: Placing test results in
/host/chromium/src/webkit/Release/layout-test-results Baseline search
path: chromium-linux -> chromium-win -> generic Using Release build
Pixel tests enabled Regular timeout: 6000, slow test timeout: 30000
Command line:
/host/chromium/src/third_party/WebKit/out/Release/DumpRenderTree -
Found 29487 tests; running 28395, skipping 1092. Unable to find test
driver
at /host/chromium/src/third_party/WebKit/out/Release/DumpRenderTree
For complete Linux build requirements, please see:
http://code.google.com/p/chromium/wiki/LinuxBuildInstructions
Build check failed
Support for layout tests on Android is being worked on at the moment. See https://code.google.com/p/chromium/issues/detail?id=232044