I'm a fanboy of Test-pyramid application in software development and want the same to be applied in Mobile software development as well.
For this, I'm desiring to leverage both Appium (for functional tests) and Espresso (for Integration Tests), and have the following questions:
The structure of my tests in Android Studio with Gradle has only 2 folders: app/src/test and app/src/androidTest.
The former test is meant for writing Unit tests with JUnit. The later for functional tests; and I'm using espresso tests for integration-testing making use of the latter directory.
I know I can create a separate project for Appium and write functional tests. But prefer to have those as part of the project in separate test source directory, so that it can be run at different intervals in my CI pipeline. How do I go about doing this? Is it possible to create another source test directory like app/src/functionalTest? Pointers to sample projects in github or resources is well appreciated.
Related
I will want to run a functional tests on firebase on various devices. Does firebase support functional testing in android?
Their testlab has three types of tests:
Robo test
Instrumentation test
Gameloop
Robo test is ruled out as it is just a stress test and a load test like a money runner
When it comes to instrumentation test, first thing I want to be clear on whether it is a functional test written in expresso, UIautomator and ranorex?
If yes, I see that I have to upload two .apk's in there, one is AppApk and TestApk. I would like to know if AppApk is the application under test and TestApk is the apk that is created to test the application?
If yes, I found uiautomator tests are bundled inside the src folder and is a java file. I also found that, there is a python wrapper for uiautomator that can be coded to write tests for android.
Now there is kivy.org to bundle python file into an .apk since android studio does not support python.
Now, How do i run Android uiautomator tests in python? Is there a way, to run python scripts inside the android studio?
And How to run functional tests in firebase? Do they support?
Android studio is creating 2 folder androidTest and test. can anyone give example of testcases for both folders.
The test/ folder will contain unit testing code that runs on your system's JVM. This is the same as plain java unit testing, so any classes that don't touch the Android framework would work great here.
The androidTest/ folder will contain instrumentation tests instead - these require a device or emulator to run. Things you'd test with it are Activities, Services, Providers, etc.
For examples on both, you can refer to the documentation:
https://github.com/googlesamples/android-testing
https://google.github.io/android-testing-support-library/samples/index.html
I have some plain old Java classes and simple JUnit tests in my Android Studio project. They currently live inside my Android Module. If I run them as Android tests they pass fine, but they're really slow as it requires launching or connecting to the emulator.
If I try to run one as a JUnit test, I hit the !!! JUnit version 3.8 or later expected JUnit version problem. This answer explains how to workaround that issue. Unfortunately, I then hit the java.lang.NoClassDefFoundError: junit/textui/ResultPrinter, which I can't figure out how to fix.
I then tried to move my JUnit tests into a separate Java module that depends on the Android Module. Everything compiles fine. I added a new configuration to launch the JUnit tests (:MyJUnitTests:test), but the tests fail with package com.myproject.util does not exist. It looks like maybe the classpath isn't including the classes from the dependent Android Module.
Does anybody know how to fix this classpath issue? I've looked at lots of related answers and none of them seem to work for me. Or is it a just a bad idea to try and keep my JUnit tests in their own Module?
I can get the JUnit tests to run fast and pass if I move my plain Java classes and their JUnit tests into a completely separate Module (e.g. here), and reverse the dependency so the Android Module depends on the Java Module. Is that the preferred solution?
The basic problem is that Android Framework classes don't work well outside the context of Android OS, so any code that has dependencies on Framework classes doesn't run in a regular JUnit environment, as you've found.
Your solution of trying to move the JUnit tests into a separate module won't work because you can't have a plain Java module depend on an Android module. Android Gradle modules don't act like Java modules, because Android builds are much more complex, and because the end result of an Android module build will be an APK or an AAR, which other module types won't understand.
If you can move your plain Java classes and unit tests into a plain Java module and have the Android modules depend on that, it will be the best approach that will get the most support from official features in the IDE. It will also have an architectural benefit in that it will keep those classes free from Android abstractions, making it more portable, and enforcing a separation of business logic from UI or storage or other things more device-specific.
If that's difficult for you to do, then a lot of developers in your shoes go with Robolectric, which is a test harness allowing code depending on many parts of the Android Framework to run in a JUnit environment without an emulator or device. It's not officially supported by Google, but Google is aware that it's widely used and tries hard to not wantonly break it.
I have a Robolectric test project setup, but I'd like to also run these tests on my device to check that I don't get bit by JVM vs Dalvik implementation differences.
Unlike robolectric tests, I won't run these tests frequently. My concern is that there's little effort to maintain the test suite and that they verify actual device functionality.
What's the best way to do that?
What I've currently got:
My robolectric test project as a test case TestPackage. I created an Android Test project with a test case TestRoboOnAndroid. It creates a TestPackage and has a test for each test in TestPackage.
Right now, every time I add a test to my robolectric suite, I need to manually add it to my device suite. Is there some way to do that automatically with reflection?
Also, Robolectric uses JUnit 4 (by default) and Android uses JUnit 3. So I have to write all of my Robolectric tests using JUnit 3 style (importing from junit.framework instead of org.junit).
The whole point of Robolectric is NOT to run it on the device and the design is based on that. If you want to run something on the device look at the default instrumentation tests from the SDK or Robotium.
It is totally feasible to run your Robolectric tests on the JVM and in addition create Robotium tests that rely on device specific interaction (e.g. creating screenshots, hardware differences...) and run on devices and emulators all combined into one build.
The easiest way to do that is to use the Android Maven Plugin.
I use a tiered system, where I prefer earlier tiers where possible:
Pure unit tests. I try to make as much code as possible fully independent of Android APIs, and then use "pure" unit tests which can run on any JVM. These tests are the fastest, and it helps keep code that has no need to be Android-specific portable.
Robolectric-supported unit tests. Where my code has only small dependencies on Android APIs, that can be satisfied by Robolectric shadows, I test it with Robolectric. There is a little more setup time for Robolectric compared to pure tests, but it's still faster than starting/running on an emulator.
Android framework tests. Where Robolectric doesn't cut it - either because the shadows don't exist, or because I'm heavily using Android APIs (and therefore want to test against the Real Thing) - I write test that run on the emulator/device with the default framework.
The point of the tiers is to keep things as simple as possible, which keeps the full suite faster and helps promote cleaner code.
We use Robolectric for unit testing against the JVM and Calabash-Android for system testing against Dalvik. Both can be integrated into our Jenkins CI build and between the two tools I feel that we cover all the bases.
Is anyone using Junit and Android? Or is that just a worthy hope? Is there a tutorial anywhere?
Android has great support for JUnit 3
From Testing Fundamentals on Android Developers:
The Android testing framework, an integral part of the development environment, provides an architecture and powerful tools that help you test every aspect of your application at every level from unit to framework.
The testing framework has these key features:
Android test suites are based on JUnit. You can use plain JUnit to test a class that doesn't call the Android API, or Android's JUnit extensions to test Android components. If you're new to Android testing, you can start with general-purpose test case classes such as AndroidTestCase and then go on to use more sophisticated classes.
The Android JUnit extensions provide component-specific test case classes. These classes provide helper methods for creating mock objects and methods that help you control the lifecycle of a component.
Test suites are contained in test packages that are similar to main application packages, so you don't need to learn a new set of tools or techniques for designing and building tests.
The SDK tools for building and tests are available in Eclipse with ADT, and also in command-line form for use with other IDES. These tools get information from the project of the application under test and use this information to automatically create the build files, manifest file, and directory structure for the test package.
The SDK also provides monkeyrunner, an API testing devices with Python programs, and UI/Application Exerciser Monkey, a command-line tool for stress-testing UIs by sending pseudo-random events to a device.
This document describes the fundamentals of the Android testing framework, including the structure of tests, the APIs that you use to develop tests, and the tools that you use to run tests and view results. The document assumes you have a basic knowledge of Android application programming and JUnit testing methodology.
I have been using Roboelectric which is awesome because it does not launch the simulator making the run time for the tests very very quick. A sample project is provided as an example and can be found here at github
You can also use Robotium to drive the UI from within JUnit for more functional style testing.
"Note that the Android testing API supports JUnit 3 code style, but not JUnit 4." (Source)
If you want to use JUnit4 or have existing JUnit4 tests you can use JUnit4Android.