Recently I started to observe a very strange situation:
a whole bunch of unit tests (~160 different unit tests) fails because of one empty class in directory module-name/src/test/.
This class looks in the following way:
package com.myapp.android.common.extensions
class MyExtensionsTest {
}
When I remove this class, all unit tests pass. When I remove class declaration and leave only package in the file, all unit tests pass. But as soon as I add declaration
class MyExtensionsTest {
}
to the package, this makes ~160 different unit tests fail.
And the failing tests cover code of classes that seem to be not connected to the extension file MyExtensions at all.
And all these ~160 different unit tests fail
the same way if MyExtensionsTest class contains any test methods.
What can be a reason of this behaviour?
Is there anything wrong with my unit test setup?
I need to run a unit test based on whether this asset exists at runtime. The reason is, I am downloading the file in react native and in android, I am running some unit tests that requires this file.
I would like to run the unit test only if this file exists. Does anyone have any suggestions or code samples on how this can be achieved? Or is there another way these unit tests can be accomplished?
You shouldn't do that in a unit test because you want to have the file locally in your testing environment or have a mock that provides it. Otherwise you're not Eradicating Non-Determinism in Tests.
Let's assume you need to do something like that anyway. So, I would add a condition in the assert expression:
#Test
fun `Given a variable When has a value Then assert it has a value`(){
var myVar = null
myVar = getAValue()
myVar?.let {
assertNotEquals(null, myVar)
}
}
To me, eradicating non determinism in this particular context means that test scope has always specific expected values, the conditional expression in the let or an if enclosing an assert expression violates that. Hence the code above shouldn't be part of your tests. Instead, you have to write a test for the case myVar is null, you write another test for the case myVar is non null.
That's why I use Given, When, Then the conditional state would make the Given/When very messy.
I'm getting android.content.res.Resources$NotFoundException: String resource ID when trying to access resources defined in the androidTest/res during runtime on my tests. I've tried a couple of things and no luck. Isn't this supposed to work? My code sample:
ExampleInstrumentedTest.kt
#RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {
#Test
fun test() {
val result = InstrumentationRegistry.getInstrumentation().targetContext.getString(
com.myproject.test.R.string.my_test
)
assertEquals("my test", result)
}
}
strings.xml
<resources>
<string name="my_test">my test</string>
</resources>
Has anyone faced this issue?
Found the solution for this. The issue at hand is that InstrumentationRegistry.getInstrumentation().targetContext will only be able to access what's included in the application being tested, so in this case I should use InstrumentationRegistry.getInstrumentation().context, which will be able to access test resources. For reference:
getTargetContext
Return a Context for the target application being instrumented. Note that this is often different than the Context of the instrumentation code, since the instrumentation code often lives is a different package than that of the application it is running against. See getContext() to retrieve a Context for the instrumentation code.
getContext
Return the Context of this instrumentation's package. Note that this is often different than the Context of the application being instrumentated, since the instrumentation code often lives is a different package than that of the application it is running against. See getTargetContext() to retrieve a Context for the target application.
from doc: https://developer.android.com/reference/android/app/Instrumentation
I want to combine both Robolectric and Cucumber (JVM).
Currently I have two classes ActivityStepdefs where two step definitions for activity management are defined.
My second class is RoActivity Where for example an activity is created from it's class name, and where Robolectric will be used.
When I run RoActivityTest using RobolectricTestRunner the test in this class passes, but when I run RunCukesTest (class for running features as junit test) the code from RoActivity is not running as part of Robolectric, i.e. RunCukesTest search for features on my project and match it with a method inside ActivityStepdefs and finally this class will call a method from RoActivity
Is possible to run test with both junit both* runners?
I'm not sure but perhaps it's possible to do something like powermock, using junit rules.
In that case for which one should I have to define the rule?
*Cucumber and Robolectric
My small 5 cents.
Cucumber is mostly used for acceptance tests (correct me if you use it for unit testing) and Robolectric is mostly used for unit testing.
As for me, it is overkill to write cucumber during TDD. And Robolectric is still not android and I would run acceptance tests on real device or at least emulator.
I'am facing the same problem, after some google work, I got a solution:
#RunWith(ParameterizedRobolectricTestRunner::class)
#CucumberOptions( features = ["src/test/features/test.feature","src/test/features/others.feature"], plugin = ["pretty"])
class RunFeatures(val index: Int, val name:String) {
companion object {
#Parameters(name = "{1}")
#JvmStatic
fun features(): Collection<Array<Any>> {
val runner = Cucumber(RunFeatures::class.java)
Cucumber()
val children = runner.children
return children.mapIndexed{index, feature ->
arrayOf(index,feature.name)
}
}
}
#Test
fun runTest() {
val core = JUnitCore()
val feature = Cucumber(RunFeatures::class.java).children[index]!!
core.addListener(object: RunListener() {
override fun testFailure(failure: Failure?) {
super.testFailure(failure)
fail("$name failed:\n"+failure?.exception)
}
})
val runner = Request.runner(feature)
core.run(runner)
}
}
but seems not an pretty solution for me, can somebody help me out these problem:
must explicitly list all feature file path. but cannot use pattern such as *.feature
when failed cannot know which step failed.
parameter can only pass primitive type data,
I've get into cucumber source , but seems CucumberOptions inline Cucumber , I cannot pass it programmatically but can only use annotation .
There are several methods of unit testing in Android, what's the best one for testing a custom view I've written?
I'm currently testing it as part of my activity in an instrumentation test case, but I'd rather test just the view, isolated.
A simple solution for the lack of a View-focused TestCase implementation would be to create a simple Activity within your test project that includes your view. This will allow you to write tests against the view using a simple Activity. Information on Activity testing:
http://developer.android.com/reference/android/test/ActivityUnitTestCase.html
As mentioned in wikibooks:
unit testing is a method by which individual units of source code are tested to determine if they are fit for use.
So when you say you want to test your custom view, you can check various methods of your custom views like "onTouchEvent", "onDown", "onFling", "onLongPress", "onScroll", "onShowPress", "onSingleTapUp", "onDraw" and various others depending on your business logic. You can provide mock values and test it. I would suggest two methods of testing your custom view.
1) Monkey Testing
Monkey testing is random testing performed by automated testing tools.
G.D.S. Prasad on geekinterview.com
and:
A monkey test is a unit test that runs with no specific test in mind. The monkey in this case is the producer of any input. For example, a monkey test can enter random strings into text boxes to ensure handling of all possible user input or provide garbage files to check for loading routines that have blind faith in their data.
sridharrganesan on geekinterview.com
This is a black box testing technique and it can check your custom view in so many unique conditions that you will get astonished :) .
2) Unit Testing
2a) Use Robotium Unit Testing Framwork
Go to Robotium.org or http://code.google.com/p/robotium/ and download the example test project. Robotium is a really easy to use framework that makes testing of android applications easy and fast. I created it to make testing of advanced android applications possible with minimum effort. Its used in conjunction with ActivityInstrumentationTestCase2.
2b) Use Android Testing Framework
Here are the links to the reference:
http://developer.android.com/reference/android/test/ActivityInstrumentationTestCase2.html
and
http://developer.android.com/reference/android/test/ActivityUnitTestCase.html
For starters:
http://developer.android.com/guide/topics/testing/testing_android.html
According to one user : Aside from easily testing non platform
dependent logic I haven't found a
clever way to run tests, so far (at
least for me) any actual platform
logic testing is cumbersome. It's
almost non trivial anyway because I've
found differences in implementation
between the emulator and my actual
device and I hate to run a unit test
implementation on my device just to
remove the application afterwards.
My strategy has been: Try to be
concise and make the logic well
thought out and then test
implementation piece by piece (less
then desirable).
Also Stephen Ng provides good aproach for real Unit Test for Android projects solution: https://sites.google.com/site/androiddevtesting/
One user has made a screencast.
Here's a ScreenCast I made on how I got Unit Tests to work. Simple Unit
Tests and more complex unit tests that
depend on having a reference to
Context or Activity objects.
http://www.gubatron.com/blog/2010/05/02/how-to-do-unit-testing-on-android-with-eclipse/
Hope it helps you testing your custom view in all possible conditions :)
Comment (futlib) All your suggestions seem to involve testing the ACTIVITY, while I really want to test just the VIEW. I might want to use this view in other activities, so it doesn't make much sense for me to test it with a specific one. – futlib
Answer: To implement a custom view,
you will usually begin by providing
overrides for some of the standard
methods that the framework calls on
all views. For example "onDraw",
"onKeyDown(int, KeyEvent)",
"onKeyUp(int, KeyEvent)",
"onTrackballEvent(MotionEvent)" etc of
your custom view. So when you want to
do unit testing for your custom you'll
have to test these methods, and
provide mock values to it so that you
can test your custom view on all
possible cases. Testing these methods
doesn't mean that you are testing your
ACTIVITY, but it means testing your
custom view (methods/functions) which
is within an activity. Also you'll
have to put your custom view in an
Activity eventually for your target
users to experience it. Once
thoroughly tested , your custom view
can be placed in many projects and
many activities.
Here's a different suggestion which works fine in many cases: Assuming you are referencing your custom view from within a layout file, you can use an AndroidTestCase, inflate the view, and then perform tests against it in isolation. Here's some example code:
my_custom_layout.xml:
<?xml version="1.0" encoding="utf-8"?>
<de.mypackage.MyCustomView ...
MyCustomView.java:
public class MyCustomView extends LinearLayout {
public MyCustomView(Context context, AttributeSet attrs) {
super(context, attrs);
}
public void setTitle(CharSequence title) {
((TextView) findViewById(R.id.mylayout_title_textView)).setText(title);
}
...
MyCustomViewTest.java:
public class MyCustomViewTest extends AndroidTestCase {
private MyCustomView customView;
#SuppressLint("InflateParams")
#Override
protected void setUp() throws Exception {
super.setUp();
customView = (MyCustomView) LayoutInflater.from(getContext())
.inflate(R.layout.my_custom_layout, null);
}
public void testSetTitle_SomeValue_TextViewHasValue() {
customView.setTitle("Some value");
TextView titleTextView = (TextView) valueSelection.findViewById(R.id.mylayout_title_textView);
assertEquals("Some value", titleTextView.getText().toString());
}
...
I struggled a lot to set up screenshot tests for my custom view.
Here is how I managed to do that and everything I learned in the process.
It may not be the most convenient method, but I put it here anyway.
And of course, screenshot testing is now a little bit easier in Jetpack Compose.
⚠ Caution #1
You can use JUnit 4 if you want. I'm using JUnit 5. Because JUnit 5 is built on Java 8 from the ground up, its instrumentation tests will only run on devices running Android 8.0 (API 26) or newer. Older phones/emulators will skip the execution of these tests completely, marking them as ignored.
If you want to run JUnit 5 tests on Android, refer to this answer for how to set it up.
⚠ Caution #2
The screenshot tests may not work on other devices even if they have the same screen DPI (they may not work at all on devices with different screen DPIs). For example, even when I use the same device in my local machine and on GitHub Actions to run the tests, they do not produce the same result (GitHub Actions assertions fail). So, I had to disable them on GitHub Actions.
If you want to disable the screenshot tests on GitHub Actions (or other CI), see this answer.
⚠ Caution #3
If you have resources in instrumented tests (in androidTest source set) and you want to reference their id, you should use them like this (note the package name followed by .test):
com.example.test.R.id.an_id
For example, if your package name is my.package.name then to access the layout file in src/androidTest/res/layout/my_layout.xml in your tests, you use my.package.name.test.R.layout.my_layout.
⚠ Caution #4
Since we are saving our test screenshots on the external storage of the device/emulator, we need to make sure that we have both WRITE_EXTERNAL_STORAGE permission added in the manifest and adb install options -g and -r configured in the build script. When running on Marshmallow+, we also need to have those permissions granted before running a test. -g is for granting permissions when installing the app (works on Marshmallow+ only) while -r is to allow reinstalling of the app.
These correspond to adb shell pm install options.
Just be aware that this does not work with Android Studio yet.
So, create an AndroidManifest.xml file in src/androidTest/ directory and add the following to it:
<manifest package="my.package.name">
<!-- For saving screenshots in tests -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"
tools:ignore="ScopedStorage"
tools:remove="android:maxSdkVersion"/>
<application android:requestLegacyExternalStorage="true">
<activity android:name=".MyActivityThatContainsTheView"/>
</application>
</manifest>
and add the adb install options in your library Gradle build file:
android {
// Note that adbOptions block is deprecated in Android Gradle Plugin 7.0.0;
// replace adbOptions block with installation block
adbOptions {
installOptions("-g", "-r")
}
}
⚠ Caution #5
I save the reference screenshot (the one I want to compare with the current screenshot) in src/androidTest/assets directory. So, specify that directory as an assets entry in the library build file:
android {
sourceSets {
// This is Kotlin DSL; see https://stackoverflow.com/a/59920318 for groovy DSL
get("debug").assets.srcDirs("src/androidTest/assets")
}
⚠ Caution #6
To pass instrumentation arguments when running the tests (like shouldSave in my code), do this:
For a Gradle task:
Running the task from command line: pass your arguments after the task name
./gradlew myTask -Pandroid.testInstrumentationRunnerArguments.shouldSave=true
Running the task with Studio: pass your arguments in run config Arguments: field
-Pandroid.testInstrumentationRunnerArguments.shouldSave=true
For an Android Studio Android Instrumented Tests run configuration:
Select Edit Configurations... from run configuration popup, then select your run configuration, click ... in front of Instrumentation arguments: field and then add a name-value entry like Name shouldSave Value true.
See this article and this post.
⚠ Caution #7
The first time you want to run the screenshot tests and also whenever you update your custom view that might change its visuals, you should run the tests passing true for shouldSave argument so the new screenshots are saved in the device (see comments above save method in code below for the location of the images) and then manually copy the new screenshots to your src/androidTest/assets/ directory so they will be the new reference ones.
⚠ Caution #8
Make sure to use -ktx versions of androidx libraries (like AndroidX Core library) for Kotlin.
The -ktx variants contain useful Kotlin extension functions. Example:
implementation("androidx.core:core-ktx:1.6.0")
⚠ Caution #9
Make sure the device screen is on and unlocked for the activity to go to resumed state.
The code
This is my test activity in src/androidTest/java/com/example/ directory that exposes the view that I want to take its screenshot as a property:
class MyActivityThatContainsTheView : AppCompatActivity() {
lateinit var myView: MyView
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(my.package.name.test.R.layout.my_layout_that_contains_the_view)
myView = findViewById(my.package.name.test.R.id.my_view_id_in_the_layout_file)
}
}
And finally, this is my tests and how I save, load, and compare the screenshots:
#DisabledIfBuildConfigValue(named = "CI", matches = "true")
class ScreenshotTestView {
#JvmField
#RegisterExtension
val scenarioExtension = ActivityScenarioExtension.launch<MyActivityThatContainsTheView>()
lateinit var scenario: ActivityScenario<MyActivityThatContainsTheView>
// See ⚠ Caution #6 above in the post
val shouldSave = InstrumentationRegistry.getArguments().getString("shouldSave", "false").toBoolean()
val shouldAssert = InstrumentationRegistry.getArguments().getString("shouldAssert", "true").toBoolean()
#BeforeEach fun setUp() {
scenario = scenarioExtension.scenario
scenario.moveToState(Lifecycle.State.RESUMED)
}
#Test fun test1() {
val screenshotName = "screenshot-1"
scenario.onActivity { activity ->
val view = activity.myView
view.drawToBitmap()
.saveIfNeeded(shouldSave, screenshotName)
.assertIfNeeded(shouldAssert, screenshotName)
}
}
fun Bitmap.saveIfNeeded(shouldSave: Boolean, name: String): Bitmap {
if (shouldSave) save(name)
return this
}
fun Bitmap.assertIfNeeded(shouldCompare: Boolean, screenshotName: String) {
if (shouldCompare) assert(screenshotName)
}
/**
* The screenshots are saved in /Android/data/my.package.name.test/files/Pictures
* on the external storage of the device.
*/
private fun Bitmap.save(name: String) {
val context = InstrumentationRegistry.getInstrumentation().targetContext
val path = context.getExternalFilesDir(Environment.DIRECTORY_PICTURES)
val file = File(path, "$name.png")
file.outputStream().use { stream ->
compress(Bitmap.CompressFormat.PNG, 100, stream)
}
}
private fun Bitmap.assert(screenshotName: String) {
val reference = loadReferenceScreenshot(screenshotName)
// I'm using AssertJ library; you can simply use assertTrue(this.sameAs(reference))
assertThat(this.sameAs(reference))
.withFailMessage { "Screenshots are not the same: $screenshotName.png" }
.isTrue()
}
private fun loadReferenceScreenshot(name: String): Bitmap {
val context = InstrumentationRegistry.getInstrumentation().context
val assets = context.resources.assets
val reference = assets.open("$name.png").use { stream ->
BitmapFactory.decodeStream(stream)
}
return reference
}
}