I added unit tests (that run on bare JVM) to a Kotlin library, and I'd like to also run the code in ART (Android Runtime) and Dalvik VM against the Android API, reusing the defined tests in the test sourceSet, and running them with tests in the androidTest sourceSet, on a real device or in an emulator.
Is this possible, and if so, how?
The reason why I want to do this is that I run some code on the main thread when on a real device, falling back to a custom thread if an error/exception is thrown because not running on a real device, but I also want to test the code when it doesn't fallback because it is running on a real device which has access to the main thread.
Yes, move the JVM tests from 'test' directory to androidTest target. It runs the JVM tests along with instrumentation tests.
Related
In this code lab, it says that “Compose tests are instrumented tests. This means they require a device (physical device or emulator) to run on.” This implies they must be in the androidTest package.
Can anyone explain why this is? I am able to run my compose tests as unit tests using Robolectric (test package) so not sure why the documentation explicitly calls this out.
I am working on an Android project. I am trying to build a CICD pipeline for my application on the Bitrise. I am following an example on the Bitrise official page, https://devcenter.bitrise.io/testing/device-testing-for-android/. So I added Android Build and [Beta] Virtual Device Testing steps into the workflow in my Bitrise Dashboard.
But when I build the app, it is only running the tests within the app/src/test folder. I have also written UI Tests using the Expresso framework in the app/src/androidTest folder. But the ones in the app/src/androidTest are not run. What changes do I have to make in order to run those tests?
There are three types of Android tests that can be run with the Virtual Device Testing step:
Robo, the testing monkey that can be configured for end-to-end testing.
Game Loop, which I suppose you would use if your app was a game.
Instrumentation, this will run tests written in androidTest/
In order to run instrumentation tests you will need to build both an app.apk and appTest.apk, conveniently Bitrise has a step called Android Build for UI Testing
Make sure to change the Test type input variable.
Are unit test cases provided for any app in AOSP(Android Open Source Project)in android? Suppose take Gallery App, which is a system app. If I download the AOSP, Is complete test suite available for Gallery App. I meant to say both unit tests and Instrumentation test suites?
From my experience in working with AOSP, there is a python script called runtest.py in platform development/testrunner/runtest.py to run every available unit tests on each module in AOSP. It is a good starting point to look for what you want, but beware some unit tests are outdated and obsoleted.
You can add tests and then be include into runtest.py to form a test suite. runtest.py will help you compile, deploy then run the unit test suite on the target device.
I'm planning to port my JUnit tests to Android Testing framework.
Some of the tests only involve the JVM but not the Android system, is it still necessary to port them? Android is using Dalvik and will replace it with ART(Android Runtime) in Lollipop, both of which are different from the JVM (Please correct if I'm wrong.). In this sense, it seems necessary to port all the JUnit tests to the Android Testing framework.
However, some tutorial argues that
"If possible, you should prefer to run your unit tests directly on the
JVM as the test execution is much faster compared to the time required
to deploy and run the test on an Android device."
http://www.vogella.com/tutorials/AndroidTesting/article.html#androidtesting
I'm not an expert on JVM, Dalvik nor ART. Can someone clarify this issue? Is it necessary to port the tests that only involve JVM to Android Testing Framework?
Thanks!
This depends on what you would like to test and the environment in which you run your tests in.
In theory, pure non-Android-specific Java code (aka. POJOs) you write will work on whatever virtual machine you run it on--JVM, Dalvik, or ART.
If you agree with that theory, then if you are already running your tests on a JVM, there is no need to run it on Dalvik or ART, especially if you consider the speed at which JVM will execute the tests. (Running it on Dalvik involves building the apk, and installing it, and running an adb shell command which can take some time...)
Of course, if you feel that the behavior of your Java code may change depending on the runtime, then you should probably port those tests over to your Android tests.
Recently, I checked out Android Testing Support provided in Android SDK.
As far as I know, you can write Unit Tests, which can be executed directly on JVM in your computer or CI Server (e.g. Jenkins) without Android device or emulator. The only requirement is the fact that these tests and code, which is tested have to be written in pure Java without any dependencies to Android SDK and Android Context. These tests have to be located at src/test/java directory. You can switch test artifact in Build Variants in Android Studio to "Unit Tests" and run your tests with IDE. you can also run them via Gradle wrapper with the command: ./gradlew test
If your tests or code, which is going to be tested depend on Android SDK and Android Context, you should use Android Instrumentation tests, which have to be located in src/androidTest/java directory. These tests have to be executed on Android device or emulator. You can switch test artifact in Android Studio to "Android Instrumentation Tests" and run your tests with IDE or use Gradle wrapper command ./gradlew connectedCheck
You can check official article about Unit Testing at: http://tools.android.com/tech-docs/unit-testing-support . It's worth reading. It's a bit outdated, because now you can use build tools: 1.1.0 instead of 1.1.0-rc1, but rest of the information is up to date. After reading this article, you should understand this topic better.
I have a Robolectric test project setup, but I'd like to also run these tests on my device to check that I don't get bit by JVM vs Dalvik implementation differences.
Unlike robolectric tests, I won't run these tests frequently. My concern is that there's little effort to maintain the test suite and that they verify actual device functionality.
What's the best way to do that?
What I've currently got:
My robolectric test project as a test case TestPackage. I created an Android Test project with a test case TestRoboOnAndroid. It creates a TestPackage and has a test for each test in TestPackage.
Right now, every time I add a test to my robolectric suite, I need to manually add it to my device suite. Is there some way to do that automatically with reflection?
Also, Robolectric uses JUnit 4 (by default) and Android uses JUnit 3. So I have to write all of my Robolectric tests using JUnit 3 style (importing from junit.framework instead of org.junit).
The whole point of Robolectric is NOT to run it on the device and the design is based on that. If you want to run something on the device look at the default instrumentation tests from the SDK or Robotium.
It is totally feasible to run your Robolectric tests on the JVM and in addition create Robotium tests that rely on device specific interaction (e.g. creating screenshots, hardware differences...) and run on devices and emulators all combined into one build.
The easiest way to do that is to use the Android Maven Plugin.
I use a tiered system, where I prefer earlier tiers where possible:
Pure unit tests. I try to make as much code as possible fully independent of Android APIs, and then use "pure" unit tests which can run on any JVM. These tests are the fastest, and it helps keep code that has no need to be Android-specific portable.
Robolectric-supported unit tests. Where my code has only small dependencies on Android APIs, that can be satisfied by Robolectric shadows, I test it with Robolectric. There is a little more setup time for Robolectric compared to pure tests, but it's still faster than starting/running on an emulator.
Android framework tests. Where Robolectric doesn't cut it - either because the shadows don't exist, or because I'm heavily using Android APIs (and therefore want to test against the Real Thing) - I write test that run on the emulator/device with the default framework.
The point of the tiers is to keep things as simple as possible, which keeps the full suite faster and helps promote cleaner code.
We use Robolectric for unit testing against the JVM and Calabash-Android for system testing against Dalvik. Both can be integrated into our Jenkins CI build and between the two tools I feel that we cover all the bases.