I am implementing CI in my android project, but I am not able to put my Android espresso tests to run in the pipeline. Can anybody help me?
You need to have devices to run espresso tests against. You can use services like a Firebase Test Lab and send your build to be tested there, and after the tests are finished, your results will be returned back to the bitbucket pipeline so you can continue with your CI process.
You can explore google cloud CLI, with which you can run bash scripts from your CI to send the build to the TestLab.
Related
Android Studio when I create Firebase Test Lab -> Record Robot Script so errors is Espresso Testing Framework does not support Compose projects.
do you know how to solve this issue?
You cannot record a Robo script for a Compose project in Android Studio, because of the limitations of the underlying framework. However, you can create a Robo script manually following the Robo scripts reference guide and then run this Robo script in Firebase Test Lab.
I'm working on an SDK that uploads real-time Android test results to a backend. I'd like to be able to attach a test listener to the instrumented tests that run when the connectedAndroidTest gradle command is invoked.
I can annotate the tests with #RunWith and have them use a custom runner with its own listener, but this solution is not ideal, as users implementing this SDK will have to annotate each and every test. We're looking for something that works at a global level.
For unit tests, we can hook into the gradle test API and listen for test results. But, instrumented tests use different gradle APIs, provided by the Android Gradle Plugin (this is where the connectedAndroidTest task comes from). There doesn't seem to be an equivalent gradle-based test listener.
I can set a JUnit test listener via the testInstrumentationRunner in build.gradle - but this listener runs on the Android device under test - I'm looking for something that can listen to test results on the local machine, rather than the Android device.
Android Studio seems to have this capability - when running Instrumented Tests via a run configuration, the IDE can report on instrumented test results in real-time. I've had a dig around in the source code, but it's not clear to me how this works.
Any help would be greatly appreciated!
I am working on an Android project. I am trying to build a CICD pipeline for my application on the Bitrise. I am following an example on the Bitrise official page, https://devcenter.bitrise.io/testing/device-testing-for-android/. So I added Android Build and [Beta] Virtual Device Testing steps into the workflow in my Bitrise Dashboard.
But when I build the app, it is only running the tests within the app/src/test folder. I have also written UI Tests using the Expresso framework in the app/src/androidTest folder. But the ones in the app/src/androidTest are not run. What changes do I have to make in order to run those tests?
There are three types of Android tests that can be run with the Virtual Device Testing step:
Robo, the testing monkey that can be configured for end-to-end testing.
Game Loop, which I suppose you would use if your app was a game.
Instrumentation, this will run tests written in androidTest/
In order to run instrumentation tests you will need to build both an app.apk and appTest.apk, conveniently Bitrise has a step called Android Build for UI Testing
Make sure to change the Test type input variable.
I've got my travis CI set up with espresso instrumental testing for my android library but some test fail on Travis CI while on my local machine (emulator and physical device) everything is okay and all test are passed..
I don't understand why on the server side someone fail, while on the local machine in my ide, with the same script command everything is okay.
Can someone help me out?
Thanks
Instrumented UI tests using Espresso Testing Framework depend on the emulator that you created.
We normally use Android Studio locally, and a different one in CI build servers.
The chosen server is not the reason of the failure, probably the screen size of the emulator.
I have seen several post regarding making a build in Jenkins fail if the unit test execution fail (e.g. this one). It turns out that by default Jenkins reports builds with failing tests as unstable and some people do not like that. This, however, will be perfectly fine for me. I just want to be able to easily differentiate builds with passing tests from such with failing tests.
And here is the catch: I am developing for Android so my build is configured following this page. Basically the tests are run with the following command:
ant all clean emma debug install test
As result coverage report is generated and published in Jenkins.
All posts I have read about configuring the Jenkins result according to tests results
were dealing with ant task manipulation. However, if we look at the android build.xml the Android tests are run with adb command: adb shell am instrument .... I don't know how to configure this command to print the tests results. It can be configured to print the coverage report. I have already done that, but was never able to make the build fail according to the coverage report.
I hope somebody else also faced the same problem and managed to solve it. Any guidance will be most appreciated.
Consider using 'the-missing-android-xml-junit-test-runner' - the idea being to produce jUnit reports that Jenkins understands. Disclaimer: I do have no personal experience with adb. However, I do know from experience that Jenkins deals with properly generated jUnit reports very gracefully.
This is possible, configure the server using the following documentation:
https://wiki.jenkins-ci.org/display/JENKINS/Android+Emulator+Plugin
See "Parsing monkey output" in the link.
Addition, Jenkins can test instrumentation, below have a link of a good place to start in the learning of instrumentation:
http://developer.android.com/resources/tutorials/testing/helloandroid_test.html
I hope it is useful.