I'm currently building an app in Android, and using Robotium to do functional tests (By the way, don't use Robotium on anything less that Android 1.6, it is way too buggy).
Some of these tests have a random tendency to fail, mainly Robotium missing a text field, or timing out, not reading text. I am trying to use the #FlakyTest annotation, so they will run two or three times before throwing out a failed test error. However, the annotation is not working, the tests do not re-run after a failure.
Here is how I am using the annotation:
public class ClassName extends ActivityInstrumentationTestCase2<HomeActivity>{
#LargeTest
#FlakyTest(tolerance=3)
public void testMethod(){
//Here I run my roboitium scripts.
}
}
Then I run it from the command line:
adb shell am instrument -w com.jayway.test/android.test.InstrumentationTestRunner
Neither eclipse nor the command line execution of the tests takes into account the flaky test annotation. Does anyone see an error with how I am trying to apply #FlakyTest?
I can't see any issue with your use of the #FlakyTest annotation.
I put together a quick test case to test #FlakyTest and Robotium (v2.2):
public class FlakyTestCase extends ActivityInstrumentationTestCase2<Main> {
private static int count = 0;
private Solo solo;
public FlakyTestCase() {
super("com.stackoverflow.example", Main.class);
}
#Override
public void setUp() throws Exception {
solo = new Solo(getInstrumentation(), getActivity());
}
#LargeTest
#FlakyTest(tolerance=3)
public void testFlaky(){
Log.e("FlakeyTestCase", "Execution Count:" + ++count);
solo.assertCurrentActivity(null,Main.class);
solo.clickOnText("Doesn't Exist");
Log.e("FlakeyTestCase", "Shouldn't make it here");
}
}
LogCat showed the following messages:
Execution Count: 1
Execution Count: 2
Execution Count: 3
So the #FlakyTest annotation was definitely being invoked. The (final) failure of the test was shown as:
junit.framework.AssertionFailedError: The text: Doesn't Exist is not found!
And the message "Shouldn't make it here" was never logged.
So as far as I can see, there is no issue with how you've declared your annotation or any problems with #FlakyTest and Robotium, v2.2 anyway.
Perhaps there is an issue with another part of your test code?
In general, when writing tests for Android (with or without Robotium) you have to be much more careful. You can't just say "is this visible". You need to wrap everything in a "wait for" cycle, so would say "wait for this to be visible". This is particularly a problem when running in the emulators, because sometimes things take long without any good reason. Without the waiting cycles, you will never have a consistent run. We have a few hundred tests and we have never needed to use the FlakyTest annotation.
Robotium missing a text field, or timing out, not reading text means
We have to check clearly if the text or any existed on the screen then only need to perform the actions like
if(solo.searchText("Doesn't Exist", true){
solo.clickOnText("Doesn't Exist");
}
Similar if any components like button or others we can achieve this by above logic.
Add this to your code:
import android.util.Log;
Related
I'm trying to implement a simple test that uses android.os.Process.myPid (actually I have a large class that uses myPid, but to avoid posting large source I simplified everything down to this):
import org.junit.Test;
import static org.junit.Assert.assertTrue;
public class ProcessTest {
#Test
public void testTest() {
assertTrue(true);
}
#Test
public void testPid() {
int pid = android.os.Process.myPid();
//assertTrue(true);
}
}
First test passes ok, but second fails with this error message:
java.lang.UnsatisfiedLinkError: android.os.Process.myPid()I
What's wrong? How to test classes that use android.os.Process?
You can't do that in unit tests. You can't access Android OS packages in unit tests. You can mock them! You want a process id, but there's no OS running (apart from the machine you're developing on).
After a while I found solution for my problem -- instrumentation tests. Well, yes, as Krzysztof Kubicki pointed out, it can't be done in unit tests. But while it's possible to mock things, is difficult in real case. I was able to mock pid for my simple test, but the real thing was AesCbcWithIntegrity by Tozny with all bells and whistles, including things like this:
int bytesRead = (Integer) Class
.forName("org.apache.harmony.xnet.provider.jsse.NativeCrypto")
.getMethod("RAND_load_file", String.class, long.class)
.invoke(null, "/dev/urandom", 1024);
This is kinda difficult to mock. So, instead of mocking I created instrumentation test and ran tests on emulator.
And that solved my problem without much effort. Yes, instrumentation tests are slow compared to regular unit testing, but at least it works.
I have Android instrumentation tests with Espresso. Some of my tests must be run on an emulator - due to using LinkedIn's TestButler (https://github.com/linkedin/test-butler) library. This library toggles wifi/gsm for specific test runs, and that is why these tests must be run on an emulator.
My question is - can I annotate any specific tests to run on an emulator, while having the other tests run on a real device?
Thanks
Yes, you can use a #ConditionalIgnore annotation as described in http://www.codeaffine.com/2013/11/18/a-junit-rule-to-conditionally-ignore-tests/.
You will have something like
public class SomeTest {
#Rule
public ConditionalIgnoreRule rule = new ConditionalIgnoreRule();
#Test
#ConditionalIgnore( condition = NotRunningOnEmulator.class )
public void testSomething() {
// ...
}
}
public class NotRunningOnEmulator implements IgnoreCondition {
public boolean isSatisfied() {
return !Build.PRODUCT.startsWith("sdk_google");
}
}
EDIT
For this specific case of detecting a device or emulator you can also use #RequiresDevice.
The most straightforward solution I found is to use JUnit Assume API: http://junit.org/junit4/javadoc/4.12/org/junit/Assume.html
So, inside the test methods that can only be run on an emulator, I put this code:
Assume.assumeTrue("This test must be run in an emulator!", Build.PRODUCT.startsWith("sdk_google"));
This results in the said test being ignored when it isn't run on an emulator, and a handy error msg in the run window:
As you can see, the other two tests passed fine (in the green), and the entire test suite was able to run.
I have a problem setting up Robotium tests to run on Travis without random false posivities.
Every couple of builds I get
pl.mg6.agrtt.TestActivityTests > testCanEnterTextAndPressButton[test(AVD) - 4.4.2] FAILED
junit.framework.AssertionFailedError: EditText is not found!
at com.robotium.solo.Waiter.waitForAndGetView(Waiter.java:540)
on all my tests.
I have created a simple project on GitHub to show the issue.
You may see how it builds on Travis. Note build #7 failed after modyfing unrelated file.
I'm suspecting this to be caused by emulator being locked or its sceeen dimmed. I could reproduce this issue on local machine by turning connected device's screen off and then running
./gradlew connectedAndroidTest
After modyfing tests I got a different error message, which is somewhat more informative, so I'm adding it just in case someone tries to find a solution:
pl.mg6.agrtt.TestActivityTests > testCanFindViewsEnterTextAndPressButton[test(AVD) - 4.4.2] FAILED
junit.framework.AssertionFailedError: Click at (160.0, 264.0) can not be completed! (java.lang.SecurityException: Injecting to another application requires INJECT_EVENTS permission)
at com.robotium.solo.Clicker.clickOnScreen(Clicker.java:106)
While the root cause of this problem is still unknown to me, after some investigation and with a help from Robotium's author Renas Reda I could confirm what I initially suspected that emulator indeed locks itself.
A workaround I'm using now is this code put in setUp method:
getInstrumentation().runOnMainSync(new Runnable() {
#Override
public void run() {
getActivity().getWindow().addFlags(WindowManager.LayoutParams.FLAG_DISMISS_KEYGUARD);
}
});
Robotium discards invisible views when using enterText(int, String). Instead use getView(int) of Solo to use resulting view in enterText(View, String).
Like this:
public void testCanEnterTextAndPressButton() {
solo.enterText(((EditText) solo.getView(R.id.editText1)), "my login");
solo.enterText(((EditText) solo.getView(R.id.editText2)), "my password");
solo.clickOnView(solo.getView(R.id.button));
}
And if the device screen is locked Robotium fails to run those instructions you gave. You might want to disable screen locking.
By code above my tests pass.
Your guess is probably right. One way to be sure that it is is to catch the exception that is thrown and call
solo.takeScreenshot("screenshotFileName");
and then take a look at the screenshot that is saved to your phone's SD card to see what your phone was doing at the time of the error.
I solved this problem by turning on the device's "Stay Awake" setting so it won't sleep while recharging.
after each test execution junit goes to the next test. And activity closes automatically after each test execution.
Its OK when I test functions, but NOT OK when I test views. I want to see execution results on the screen before junit will execute next test.
So, I want to make small delay between tests.
There is one way I can do it:
public void testMyView() throws InterruptedException {
getActivity().runOnUiThread(new Runnable() {
#Override
public void run() {
// viewOperations...
}
});
Thread.sleep(3000);
}
But I dont like this way because this code is dirty (especially when I have a lot of such tests).
Another way seems much better:
#UiThreadTest
public void testHandleEmptyAlphabetList() throws InterruptedException {
// view operations
Thread.sleep(3000);
}
BUT: It will block UI thread but NOT test thread. I need to block a test thread.
Is there any way to run Thread.sleep(3000); from this code separately in a test thread?
So, what is the best way to make thread delays between tests?
p.s. Please don't recommend me additional test frameworks, I want to solve my problem using embed tools. Thank you.
I suggest ditching this cumbersome way of unit testing and using Robolectric, so your tests run in the JVM and you don't have to deal with all this nasty setup. You'll be pleasantly surprised on how quickly your View testing runs, and that you don't need to run an emulator!
Try to use Robotium for the tests that need a real android running.
solo = new Solo(getInstrumentation(), getActivity());
[...]
if (failure) {
solo.takeScreenshot();
}
This will save a screenshot to your SD card (needs WRITE_EXTERNAL_STORAGE permission)
Or if you prefer not to use another testing framework (though I would recommend you take a look at it), just steal the code that is used there for taking screenshots
Im having a problem starting or running any activity unit tests from within eclipse.
Even i start a clean project and make a simple test class it always prints to the console:
[2010-10-05 13:10:24 - testAndroid] Collecting test information
[2010-10-05 13:10:25 - testAndroid] Test run failed: Test run incomplete. Expected 2 tests, received 0
Any ideas ?
Just for testing, I have created a fresh Android project called Demo with a test project called DemoTest
The main activity to test is called Main and I have created a simple testclass MainTest that looks like this:
package net.demo.test;
import android.test.ActivityInstrumentationTestCase2;
import net.demo.Main;
public class MainTest extends ActivityInstrumentationTestCase2<Main>
{
public MainTest()
{
super("net.demo", Main.class);
// TODO Auto-generated constructor stub
}
}
My tests used to run fine before, but suddenly I cant run any of them, they all fail with the same error, even I create new a project. It seems like it something to do with Eclipse or and not with the Code.
Update:
Seems like extending SingleLaunchActivityTestCase<Main> is working, but still got no clue about how to make ActivityInstrumentationTestCase2<Main> working.
I had no regression problems. I just couldn't get the example to work. I finally fixed it by defining two constructors:
public MainActivityTest(String pkg, Class<MainActivity> activityClass) {
super("com.myapp", MainActivity.class);
}
public MainActivityTest() {
super("com.myapp", MainActivity.class);
}
It turned out that most emulators before 2.3.3 were silently swallowing the error generated when construction went wrong.
You must put at least 2 methods (i.e 2 test cases) into the Test class. even methods without definition inside can do the trick