I'm planning to port my JUnit tests to Android Testing framework.
Some of the tests only involve the JVM but not the Android system, is it still necessary to port them? Android is using Dalvik and will replace it with ART(Android Runtime) in Lollipop, both of which are different from the JVM (Please correct if I'm wrong.). In this sense, it seems necessary to port all the JUnit tests to the Android Testing framework.
However, some tutorial argues that
"If possible, you should prefer to run your unit tests directly on the
JVM as the test execution is much faster compared to the time required
to deploy and run the test on an Android device."
http://www.vogella.com/tutorials/AndroidTesting/article.html#androidtesting
I'm not an expert on JVM, Dalvik nor ART. Can someone clarify this issue? Is it necessary to port the tests that only involve JVM to Android Testing Framework?
Thanks!
This depends on what you would like to test and the environment in which you run your tests in.
In theory, pure non-Android-specific Java code (aka. POJOs) you write will work on whatever virtual machine you run it on--JVM, Dalvik, or ART.
If you agree with that theory, then if you are already running your tests on a JVM, there is no need to run it on Dalvik or ART, especially if you consider the speed at which JVM will execute the tests. (Running it on Dalvik involves building the apk, and installing it, and running an adb shell command which can take some time...)
Of course, if you feel that the behavior of your Java code may change depending on the runtime, then you should probably port those tests over to your Android tests.
Recently, I checked out Android Testing Support provided in Android SDK.
As far as I know, you can write Unit Tests, which can be executed directly on JVM in your computer or CI Server (e.g. Jenkins) without Android device or emulator. The only requirement is the fact that these tests and code, which is tested have to be written in pure Java without any dependencies to Android SDK and Android Context. These tests have to be located at src/test/java directory. You can switch test artifact in Build Variants in Android Studio to "Unit Tests" and run your tests with IDE. you can also run them via Gradle wrapper with the command: ./gradlew test
If your tests or code, which is going to be tested depend on Android SDK and Android Context, you should use Android Instrumentation tests, which have to be located in src/androidTest/java directory. These tests have to be executed on Android device or emulator. You can switch test artifact in Android Studio to "Android Instrumentation Tests" and run your tests with IDE or use Gradle wrapper command ./gradlew connectedCheck
You can check official article about Unit Testing at: http://tools.android.com/tech-docs/unit-testing-support . It's worth reading. It's a bit outdated, because now you can use build tools: 1.1.0 instead of 1.1.0-rc1, but rest of the information is up to date. After reading this article, you should understand this topic better.
Related
I am working on an Android project. I am trying to build a CICD pipeline for my application on the Bitrise. I am following an example on the Bitrise official page, https://devcenter.bitrise.io/testing/device-testing-for-android/. So I added Android Build and [Beta] Virtual Device Testing steps into the workflow in my Bitrise Dashboard.
But when I build the app, it is only running the tests within the app/src/test folder. I have also written UI Tests using the Expresso framework in the app/src/androidTest folder. But the ones in the app/src/androidTest are not run. What changes do I have to make in order to run those tests?
There are three types of Android tests that can be run with the Virtual Device Testing step:
Robo, the testing monkey that can be configured for end-to-end testing.
Game Loop, which I suppose you would use if your app was a game.
Instrumentation, this will run tests written in androidTest/
In order to run instrumentation tests you will need to build both an app.apk and appTest.apk, conveniently Bitrise has a step called Android Build for UI Testing
Make sure to change the Test type input variable.
I'm building my Android ap with Microsoft App Center.
Is there any way to run JUnit JVM unit tests in App Center?
At the moment App Center is running only the Android instrumented tests, this is, the tests that run on one emulator or real device (located in androidTest folder)
What I want to do is to run the tests that are pure Java/Kotlin and do not depend in the Android framework to work, the ones that run just on the JVM (located in test folder).
All I've found regarding tests in App Center is https://learn.microsoft.com/en-us/appcenter/build/build-test-integration.
Thanks
I contacted App Center and this is not supported yet. I created a feature request for this.
I am developing an automation framework using Appium and Webdriver. I have a bunch of classes for the framework and JUnit tests. Everything is working in a separated repository in which the appium server is initialized, the tests run and then the server is terminated.
My question is if I should move those tests into the Android app repo, in order for the developers to have a more confortable access to them and run the tests against a dev or debug apk.
Some questions:
1) Should these appium tests be located on the test folder (unit tests) or androidTest (instrumented tests)?
2) How is the best way to execute test tasks in gradle in order to generate an apk (with different flavors/environments), install it in a phone or emulator and run the tests?
I haven't found a lot of examples answering these questions in the web.
Thanks!
We're trying to get code coverage analysis of a set of manual tests run on an Android application running on a device. We build our application using maven, so the cool emma and ant solution isn't viable.
Does anybody have explicit experience getting emma or cobertura (or any other open source coverage tool) working for manual testing of Android applications? We can't afford to purchase a solution (so no Clover, or Java Test Coverage Tool, etc.).
I have a Robolectric test project setup, but I'd like to also run these tests on my device to check that I don't get bit by JVM vs Dalvik implementation differences.
Unlike robolectric tests, I won't run these tests frequently. My concern is that there's little effort to maintain the test suite and that they verify actual device functionality.
What's the best way to do that?
What I've currently got:
My robolectric test project as a test case TestPackage. I created an Android Test project with a test case TestRoboOnAndroid. It creates a TestPackage and has a test for each test in TestPackage.
Right now, every time I add a test to my robolectric suite, I need to manually add it to my device suite. Is there some way to do that automatically with reflection?
Also, Robolectric uses JUnit 4 (by default) and Android uses JUnit 3. So I have to write all of my Robolectric tests using JUnit 3 style (importing from junit.framework instead of org.junit).
The whole point of Robolectric is NOT to run it on the device and the design is based on that. If you want to run something on the device look at the default instrumentation tests from the SDK or Robotium.
It is totally feasible to run your Robolectric tests on the JVM and in addition create Robotium tests that rely on device specific interaction (e.g. creating screenshots, hardware differences...) and run on devices and emulators all combined into one build.
The easiest way to do that is to use the Android Maven Plugin.
I use a tiered system, where I prefer earlier tiers where possible:
Pure unit tests. I try to make as much code as possible fully independent of Android APIs, and then use "pure" unit tests which can run on any JVM. These tests are the fastest, and it helps keep code that has no need to be Android-specific portable.
Robolectric-supported unit tests. Where my code has only small dependencies on Android APIs, that can be satisfied by Robolectric shadows, I test it with Robolectric. There is a little more setup time for Robolectric compared to pure tests, but it's still faster than starting/running on an emulator.
Android framework tests. Where Robolectric doesn't cut it - either because the shadows don't exist, or because I'm heavily using Android APIs (and therefore want to test against the Real Thing) - I write test that run on the emulator/device with the default framework.
The point of the tiers is to keep things as simple as possible, which keeps the full suite faster and helps promote cleaner code.
We use Robolectric for unit testing against the JVM and Calabash-Android for system testing against Dalvik. Both can be integrated into our Jenkins CI build and between the two tools I feel that we cover all the bases.