I am building an Android Studio/Gradle project on TeamCity server. I am somewhat new to TeamCity. Currently, the unit tests auto-generated by Android Studio are run automatically when the project is built on TeamCity and are displayed under "Tests" . I also have an instrumented test (a test which runs on the connected android device), but it does not get run automatically like the unit tests do.
My solution was to add a Gradle build step in TeamCity to run the instrumented test. So far, I've had little success. I used the gradle tasks uninstallAll connectedAndroidTest, which runs the instrumented test, but the test result does not show up under "Tests" along with the unit tests. If the instrumented test fails, the build fails, but the failed test still does not show up under "Tests".
What am I doing wrong? Is there a correct way to run instrumented tests on TeamCity?
The connectedAndroidTest will output files specifying test results according to this pattern:
HTML test result files:
path_to_your_project/module_name/build/outputs/reports/androidTests/connected/
directory.
XML test result files:
path_to_your_project/module_name/build/outputs/androidTest-results/connected/ directory.
(from here)
Using that output file you can use the XML Report Processing feature of TeamCity. In your Build Configuration just go to the Build Features tab and add the XML Report Processing feature. Use the Google Test option and point it to the report output directory like so:
After that you should see your instrumented test results show up in your builds just like regular JUnit tests:
You can tweak test task:
test.dependsOn uninstallAll, connectedAndroidTest
Another way would be to include your test (though not sure if this will work as I am not sure what uninstallAll and connectedAndroidTest tasks are doing):
test {include 'org/foo/**'}
Related
I have a project with a single module app and then lots of other modules like search and detail. app has MainActivity and almost nothing else. The other modules have lots of #Composable functions that make up the actual completed application. The individual modules also have tests in androidTest for automated testing.
How can I run the tests declared in each submodule but against the app build apk?
Directory structures may make this clearer:
/project/app/src/main/.../MainActivity.kt
/project/search/src/main/.../SearchComposable.kt
/project/search/src/androidTest/.../SearchTests.kt
/project/detail/src/main/.../DetailComposable.kt
/project/detail/src/androidTest/.../DetailTests.kt
If I run
./gradlew :search:connectedDebugAndroidTest
it will try to run those tests but they don't have an Activity, they only really run from app. Unit tests of course work fine, but automated tests do not.
If I run
./gradlew :app:connectedDebugAndroidTest
Then it will only run those declared within app. I haven't figured out a way to add a dependency from app's androidTest configuration to search/detail's androidTest output (lots of searching, nothing promising yet).
If I run
./gradlew connectedDebugAndroidTest
It will try to run all and fail to run search and detail since they don't have an activity.
In my project I am using the following commands to generate my app APK and instrumented test APK.
./gradlew :app:assembleDebug
./gradlew :app:assembleDebugAndroidTest
Using the generated files I am triggering a test in Test Lab using gcloud, which works without problems.
Now, I want to have annotations for the test size (small, medium, large), so only a subset of tests can be triggered in Test Lab, based on the test annotations.
Can it be done or is there a different approach to handle it?
Yes, by using the --test-targets flag. It supports the same options as the -e flag for AndroidJUnitRunner.
If you use the SmallTest, MediumTest, LargeTest annotations from android.support.test.filters package you can e.g. only run the small test the following way:
gcloud firebase test android run --test-targets "size small" <your other flags>
Custom annotations are supported too:
gcloud firebase test android run --test-targets "annotation com.example.MyAnnotation
For more options, take a look at the gcloud documentation for this command.
We have a project, that has some instrumented tests that run part of CI
Now we want to add some integration tests (of type instrumentation tests), but they should not run as part of pipeline, but on demand through a separate Jenkins job or command
I am trying to create a separate module for integration tests that depend on "app" module, but that's throwing a lot of errors like below
errors showing up in android studio while it tries to resolve the app module
I wanted to understand if its even possible?
It is possible. There is probably an easier way than what I am about to suggest but you can implement a rule that conditionally ignores test and change the value of this variable in your CI job.
There are different approaches.
1)Determine which tests to run via a boolean stored in server. So mark your normal ui tests with the condition shouldUITest and the others shouldInstrumentationTest and change their value in server
2)Change variables in app.gradle with buildflavors. (Basically create 2 different variants and run whichever one you want
3)Detect the build is a CI build, and act accordingly in app.gradle
def ciBuild= System.getenv("CI") == "true"
if (ciBuild) {
//Do stuff
}
I have Espresso instrumentaion test cases in following form in Android studio.
Now, I want to run few of the classes separately.
E.g : I want to run Only CrashersTest and EM3AppUtil classes using gradlew command and rest classes using another command.
How can i achieve it?
I am using below command
./gradlew :app:connectedLocalDebugAndroidTest
It runs all the classes for LocalDebug variant
I want to run only few classes
There isn't a great way to do it for instrumented tests right now that I know of.
You can create a test suite to run just certain test you want, like documented here
https://developer.android.com/training/testing/unit-testing/instrumented-unit-tests.html#test-suites
Or use the #SmallTest, #MediumTest or #LargeTest annotations on your tests and then pass in on command line like so
./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.size=small
Place the cursor on the class name in java file or select the file in file view then press ctrl+shift+F10
Refer this How to run only one test class on gradle for running tests from gradlew command
I have been working on a module, whenever it is build using "gradlew assemble"it will generate an aar file in build/output folder. I have created few tests extending the InstrumentationTestCase in android app module, I want to run those tests whenever I build the library using gradle command.If compilation of tests "pass" the build(aar) should happen else it should fail. Could any one please help me on this?
Running gradlew clean connectedCheck should run through all of your tests on all connected devices and will also build out your aar file. That's the easiest way.