We are building a complex Android application consisting of many screens and workflows spread across many Activities. Our workflows are similar to what you might see on a Bank's ATM machine, for example, there is an Activity to login in that transitions to a main menu Activity which can transition to other activities based on the user's choices.
Since we have so many workflows we need to create automated tests that span multiple activities so we can test a workflow from end to end. For example, using the ATM example, we would want to enter a valid PIN, verify that sends us to the main menu, choose withdraw cash, verify that we are on the withdraw cash screen, etc., etc., and eventually find ourselves back on the main menu or "logged" out.
We've toyed with the test APIs that come with Android (e.g. ActivityInstrumentationTestCase2) and also with Positron, but neither seem capable of testing beyond the bounds of a single Activity, and while we can find some utility in these tools for some unit testing, they won't meet our needs for testing scenarios that cut across multiple Activities.
We are open to an xUnit framework, scripting, GUI recorders/playbacks, etc. and would appreciate any advice.
I feel a bit awkward about answering my own bounty question, but here it is...
I've searched high and low on this and can't believe there is no answer published anywhere. I have come very close. I can definitely run tests that span activities now, but my implementation seems to have some timing issues where the tests don't always pass reliably. This is the only example that I know of that tests across multiple activities successfully. Hopefully my extraction and anonymizing of it did not introduce errors. This is a simplistic test where I type a username and password into a login activity, and then observe a proper welcome message is shown on a different "welcome" activity:
package com.mycompany;
import android.app.*;
import android.content.*;
import android.test.*;
import android.test.suitebuilder.annotation.*;
import android.util.*;
import android.view.*;
import android.widget.*;
import static org.hamcrest.core.Is.*;
import static org.hamcrest.core.IsNull.*;
import static org.hamcrest.core.IsInstanceOf.instanceOf;
import static org.junit.Assert.*;
import static com.mycompany.R.id.*;
public class LoginTests extends InstrumentationTestCase {
#MediumTest
public void testAValidUserCanLogIn() {
Instrumentation instrumentation = getInstrumentation();
// Register we are interested in the authentication activiry...
Instrumentation.ActivityMonitor monitor = instrumentation.addMonitor(AuthenticateActivity.class.getName(), null, false);
// Start the authentication activity as the first activity...
Intent intent = new Intent(Intent.ACTION_MAIN);
intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
intent.setClassName(instrumentation.getTargetContext(), AuthenticateActivity.class.getName());
instrumentation.startActivitySync(intent);
// Wait for it to start...
Activity currentActivity = getInstrumentation().waitForMonitorWithTimeout(monitor, 5);
assertThat(currentActivity, is(notNullValue()));
// Type into the username field...
View currentView = currentActivity.findViewById(username_field);
assertThat(currentView, is(notNullValue()));
assertThat(currentView, instanceOf(EditText.class));
TouchUtils.clickView(this, currentView);
instrumentation.sendStringSync("MyUsername");
// Type into the password field...
currentView = currentActivity.findViewById(password_field);
assertThat(currentView, is(notNullValue()));
assertThat(currentView, instanceOf(EditText.class));
TouchUtils.clickView(this, currentView);
instrumentation.sendStringSync("MyPassword");
// Register we are interested in the welcome activity...
// this has to be done before we do something that will send us to that
// activity...
instrumentation.removeMonitor(monitor);
monitor = instrumentation.addMonitor(WelcomeActivity.class.getName(), null, false);
// Click the login button...
currentView = currentActivity.findViewById(login_button;
assertThat(currentView, is(notNullValue()));
assertThat(currentView, instanceOf(Button.class));
TouchUtils.clickView(this, currentView);
// Wait for the welcome page to start...
currentActivity = getInstrumentation().waitForMonitorWithTimeout(monitor, 5);
assertThat(currentActivity, is(notNullValue()));
// Make sure we are logged in...
currentView = currentActivity.findViewById(welcome_message);
assertThat(currentView, is(notNullValue()));
assertThat(currentView, instanceOf(TextView.class));
assertThat(((TextView)currentView).getText().toString(), is("Welcome, MyUsername!"));
}
}
This code is obviously not very readable. I have actually extracted it into a simple library with an English-like API so I can just say things like this:
type("myUsername").intoThe(username_field);
click(login_button);
I've tested to a depth of about 4 activities and am satisfied that the approach works though as I said, there appears to be an occasional timing issue I have not completely figured out. I am still interested in hearing of any other ways of testing across activities.
Take a look at Robotium
'a open-source test framework created to make automatic black-box testing of Android applications significantly faster and easier than what is possible with Android instrumentation tests out-of-the-box.'
Homepage:
http://www.robotium.org/
Source:
http://github.com/jayway/robotium
Please note that the Robotium project is maintained by the company I work for
You could always use Robotium. It supports blackbox testing just like Selenium but for Android. You will find it at Robotium.org
I'm surprised no one has mentioned some of the leading automated functional testing tools. Compared with Robotium, these don't require writing Java code.
MonkeyTalk: an open-source tool backed by the company Gorilla Logic. Pros: provides recording as well as a higher-level scripting language easier for non-technical users, and is cross-platform (includes iOS). Given those benefits as requirements, we've found this to be the best solution. It also allows customization beyond what can be done in their scripting language using Javascript.
Calabash-Android: an open-source tool for Cucumber-style features. Pros: write features in the Gherkin language which is Business Readable, Domain Specific Language that lets you describe software’s behavior without detailing how that behavior is implemented. Similar but not exact support is available for iOS in cucumber-ios. Recording capabilities are not as good, as they produce a binary output.
A couple of other references:
Here are some additional comparisons between Robotium,
Monkeytalk and Calabash. It mentions TestDroid as another
possibility.
This blog mentions the above plus NativeDriver and Bot-bot.
I created a record-and-playback tool for Android and made it available on GitHub. It's easy to configure and use, requires no programming, runs against real devices (which do not have to be rooted) and automatically saves screenshots as it plays tests.
First of all, use 'ActivityInstrumentationTestCase2', not 'InstrumentationTestCase', as your base class. I use Robotium and routinely test across multiple Activities. I found that I have to specify the login activity as the generic type (and class argument to the constructor).
The 'ActivityInstrumentationTestCase2' constructor ignores the package argument and does not require it. The constructor that takes the package is deprecated.
From the Javadocs:
"ActivityInstrumentationTestCase2(String pkg, Class activityClass)
This constructor is deprecated. use ActivityInstrumentationTestCase2(Class) instead"
Using the recommended base class allows the framework to handle certain boilerplate, like starting your activity. That's done by the call to 'getActivity()', if necessary.
Found this useful with a couple of modifications.
Firstly getInstrumentation().waitForIdleSync() will cure the flakiness SingleShot speaks of
and also InstrumentationTestCase has a lauchActivity function that can replace the start activity lines.
you can do it like this to avoid the flake waiting times out of sync :
final Button btnLogin = (Button) getActivity().findViewById(R.id.button);
Instrumentation instrumentation = getInstrumentation();
// Register we are interested in the authentication activity...
Instrumentation.ActivityMonitor aMonitor =
instrumentation.addMonitor(mynextActivity.class.getName(), null, false);
getInstrumentation().runOnMainSync(new Runnable() {
public void run() {
btnLogin.performClick();
}
});
getInstrumentation().waitForIdleSync();
//check if we got at least one hit on the new activity
assertTrue(getInstrumentation().checkMonitorHit(aMonitor, 1));
I'm working on pretty much the same thing, and I'll probably go with a variation on the accepted answer to this question, but I did come across Calculuon (gitHub) during my searches for a solution.
I haven't personally used it, but ApplicationTestCase looks like it might be what you're looking for.
Will accepted approach work with different Activities from different Applications, signed by different certificates? If not, Robotium the best way to test activities within same application.
There is another way to do the multiple activity using ActivityInstrumentation Class..
Its a normal automation scenario...
First get the focus of what ever object you want and then send a key
Simple as that
sample code
button.requestFocus();
sendKeys(KeyEvent.KEYCODE_ENTER);
Only thing is understanding the every API calls will help us.
This answer is based on the accepted answer but modified to solve the timing issue which for me became consistent after adding about a half dozen tests. #pajato1 gets the credit for solving the timing issue, as cited in the accepted answer comments.
/**
* Creates a test Activity for a given fully qualified test class name.
*
* #param fullyQualifiedClassName The fully qualified name of test activity class.
*
* #return The test activity object or null if it could not be located.
*/
protected AbstractTestActivity getTestActivity(final String fullyQualifiedClassName) {
AbstractTestActivity result = null;
// Register our interest in the given activity and start it.
Log.d(TAG, String.format("Running test (%s) with main class: %s.", getName(), fullyQualifiedClassName));
instrumentation = getInstrumentation();
Intent intent = new Intent(Intent.ACTION_MAIN);
intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
intent.setClassName(instrumentation.getTargetContext(), fullyQualifiedClassName);
// Wait for the activity to finish starting
Activity activity = instrumentation.startActivitySync(intent);
// Perform basic sanity checks.
assertTrue("The activity is null! Aborting.", activity != null);
String format = "The test activity is of the wrong type (%s).";
assertTrue(String.format(format, activity.getClass().getName()), activity.getClass().getName().equals(fullyQualifiedClassName));
result = (AbstractTestActivity) activity;
return result;
}
Try the Monkey tool testing
Step 1:
open the android studio terminal(Tools-> open terminal)
Step 2:
In order to use monkey , open up a command prompt and just naviagte to the following directory.
export PATH=$PATH:/home/adt-bundle-linux-x86-20140702/sdk/platform-tools
Step 3:
add this monkey command into terminal and press enter..
see the magic in your emulator.
adb shell monkey -p com.example.yourpackage -v 500
500- it is the frequency count or the number of events to be sent for testing.
you can change this count..
More reference,
http://www.tutorialspoint.com/android/android_testing.htm
http://androidtesting.blogspot.in/2012/04/android-testing-with-monkey-tool.html
Related
I'm using UIAutomator test framework for long tests (concerning to my acceptance test). And I need to wait until some activity is started.
I decided to use By.clazz (android.support.test.uiautomator package) methods to find activity object. I expected that something like
uiDevice.wait(Until.findObject(By.clazz(SomeActivity.class)), 30000);
will work. But it doesn't. I suppose that object of my activity cannot be found. I tried to use other By.clazz methods with different params but without success.
So, my code is pretty simple:
UiDevice uiDevice = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation());
/*.... do something...
like click on buttons which will open some activities...
*/
//does not work, time value just for sample
uiDevice.wait(Until.findObject(By.clazz(SomeActivity.class)), 30000);
I found workaround solution with using By.res, like
uiDevice.wait(Until.findObject(By.res(BASIC_PACKAGE, "someMainIdInSomeFragment")), 30000);
But I have very complicated structure of the app with base activities and so on. I often have the same layout for different activities with load different fragments. So I need to know that we started exactly SomeActivity ,regardless of loaded fragments.
So, the questions are:
Is it possible to use By.clazz for Activity to find its object?
Is there some another way to find activity object with UIAutomator?
Did I do everything right? Or maybe there are some mistakes? Is it possіble to do with UiAutomator?
Thanks!
Using class with UiObject2
Find the EditText, make sure to click on it (legacySetText would do it implicitly), and set your text.
val input = By.clazz(EditText::class.java.canonicalName)
device.wait(Until.findObject(input), TIMEOUT).apply {
click()
text = "your_text"
}
Yes, could be done through the id.
// android:id="#+id/widget_id"
By.res(applicationId, "widget_id")
Your syntax seems good to me. Just make sure no spinner (or any other widget) is blocking your view during the click attempt.
I am using Mokitio in android to run unit test cases.
.
What i am trying to do: There is a block of code in onCreate event
of the activity
I am trying not to run this block of code during Running Unit test
cases and run it during app regularly.
Is it possible to do something like that using mokito because mokito synchronizes for activity life cycle
The proper solution here is to change your design a bit. You should not think in terms of code blocks, but in terms of functionality.
The way of preventing that some x lines of code are run in a certain environment, but are not in some other context ... is by using proper OO means.
Meaning: first create an interface that describes the functionality of those lines of code we are talking about:
public interface DoTheFoo {
public void foo(Bar bar);
}
Then you create a "production" implementation DoTheFooImpl of that interface (which as a side effect: you might be able to write proper unit tests for as well).
Finally: within your class that needs that functionality, use dependency injection to acquire an object providing the DoTheFoo interface. In your production environment, that would be a DoTheFooImpl object; but for your unit testing, you would simply create an mock for it - configured to do nothing upen calls to foo().
Of course that sounds like a bit of work; but the point is: currently, your design is somehow deficient. And instead of trying to go for dirty hacks/workarounds, consider looking at your design to identify a more elegant way to resolve your problem.
I'm programming an app using android studio. I want to know in which way I can do a tutorial that users will see only the first time that use the app. Tutorial like image or screenshoots
Can someone help me? Thanks
I encountered this thread while looking for a solution for running a tutorial only at the first time (as rbaleksandar suggested), so in case it will be helpful for someone someday, here's a template of a solution that works for me (using the SharedPreferences API):
#Override
protected void onResume() {
super.onResume();
String tutorialKey = "SOME_KEY";
Boolean firstTime = getPreferences(MODE_PRIVATE).getBoolean(tutorialKey, true);
if (firstTime) {
runTutorial(); // here you do what you want to do - an activity tutorial in my case
getPreferences(MODE_PRIVATE).edit().putBoolean(tutorialKey, false).apply();
}
}
EDIT - BONUS - If you're into app tutorial - I'm messing now with the ShowcaseView library (which is amazing - try it out). Apparently they have some shortcut for that issue using a method called singleShot(long) - its input is a key for the SharedPreferences, and it does the exact same thing - runs only in the first activation. Example of usage (taken from here):
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_single_shot);
Target viewTarget = new ViewTarget(R.id.button, this);
new ShowcaseView.Builder(this)
.setTarget(viewTarget)
.setContentTitle(R.string.title_single_shot)
.setContentText(R.string.R_string_desc_single_shot)
.singleShot(42)
.build();
}
You could always code your own solution, but, let us not reinvent the wheel.
Check this Android Library:
Tour Guide Repository
It allows you to add pointers in your screen, so the user knows where is he supposed to touch next.
It's pretty easy to use, you only need to point to the element you want the user to touch.
From the doc:
Let's say you have a button like this where you want user to click on:
Button button = (Button)findViewById(R.id.button);
You can add the tutorial pointer on top of it by:
TourGuide mTourGuideHandler = TourGuide.init(this).with(TourGuide.Technique.Click)
.setPointer(new Pointer())
.setToolTip(new ToolTip().setTitle("Welcome!").setDescription("Click on Get Started to begin..."))
.setOverlay(new Overlay())
.playOn(button);
Hope this helps!
Some links to libraries for creating introduction and/or tutorial screens.
Horizontal cards like Google Now:
https://github.com/PaoloRotolo/AppIntro
Tutorial screen:
https://github.com/amlcurran/ShowcaseView
As far as I understand the question is not How do I create a tutorial? (as the people who have already posted an answer have concluded) but instead How to show a tutorial upon first launch only?. So here are my two cents on this topic:
I'm not familiar with how your Android app stores its configuration data but I will assume that it's either in a database (SQLite) or a text file (plaintext, YAML, XML - whatever). Add a configuration entry to wherever the app's settings are being stored - something like tutorial_on : false, tutorial_on : 1 etc. depending on the format the configuration is represented in.
The way configurations work is that whenever an app (or software in general) is launched it has to be loaded in the app itself. So add the following to your app (where and how is up to you and your app design):
Check tutorial_on entry
If tutorial_on is set to true/1 whatever
2.1 Display tutorial
2.2 Change tutorial_on to false/0 whatever
2.3 Store the result in your configuration
Continue using the app
By doing so the first time your app launches the flag responsible for displaying the tutorial will be toggled and afterwards every time you start the app the toggle flag will be read leading to omitting the tutorial.
Personally I would suggest that you an option similar to Don't show tutorial anymore along with a description how to re-enable it (by triggering some action in that app's menu etc.). This has two major benefits:
Improved user experience - users like to have control (especially over trivial matters such as showing or hiding a tutorial). Whenever you take the control away from them, they get pissed off.
Enable your user to re-learn forgotten things - a general rule of thumb is to create apps that should not burden the user with a lot of stuff to remember. That is why things should be self-explanatory. However sometimes you may want to do that nonetheless. By adding the possibility that the user re-launches (by simply resetting the tutorial_on flag and repeating the steps from above) the tutorial allows just that - refreshing a user's memory.
Everyone using Robotium for GUI testing.
Can you tell me what Android native test framework cannot do that Robotium can do?
As I know Robotium can be used as black box testing, so I do not need to know about application resources.
What else?
Robotium
pros:
support hybrid applications with webViews.
you can bound your test to unique IDs of tested application whenever with UIatomator to write complicated test cases will need great amount of work from you
in UIatomator is used UiAutomatorTestCase Instrumentation which doesn't give you possibilities to call current activity and check that loaded appropriate one, you can't call connectivity- or audio- managers for wi-fi or sound tests. In Robotium you can easily call such gems and that is greatly improve efficiency of your tests
Open source project, so you can modify tool to your needs
cons:
No possibility to tests multi-process application where android:process tag used for different activities.
example code:
UIautomator
pros:
Test case is independent from the process at which tested application works. So it can be used in places where are used additional libraries or activity is run in other process also useful if for test needed switching between several applications.
cons:
Works only on Android version 4.1 or more!
You can't use source IDs when you get ui-object instances. That mean if an application structure changed on one layout you need to refactor your test (this problem can be solved by using tags for each ui-element)
You can't get current activity or Instrumentation. That mean you are limited in development of your tests and not used many of android's api methods for your tests.
Hard to debug, you need to have script for buiding and starting your test fast and see output
hierarchy viewer:
I think both these tools are good ones, definitely possibility to switch between applications during test is awesome. You should select tool depending from your needs. I suggest Robotium for tests where you don't need to switch between applications because it have simple methods and opportunity to use Android API for writing flexible and smart tests in short code which can cover even testing of a web page inside webview and milti-process applications are very unusual.
The difference between Robotium and the native tools is the fact that with Robotium is really simple to write a test. It's basically point and click, from the Solo instance object.
Here you can download the JAR files and an example project to test it by yourself.
UPDATE
As an example, I'm testing an Activity with some Edit Text, a spinner and a pop-up dialog that shows up when I click a spinner option. Note here that with other methods, filling the fields of the pop up dialog is a real pain.
Here is how to declare the test class and Robotium's initialization
import com.jayway.android.robotium.solo.Solo; //Using Robotium
//Robotium uses ActivityInstrumentationTestCase2.
//Note here the use of the template
public class AddNewQuestionTests extends
ActivityInstrumentationTestCase2<AddNewQuestion> {
public AddNewQuestionTests(Class<AddNewQuestion> name) {
super(name);
}
public AddNewQuestionTests() {
super(AddNewQuestion.class);
}
private Solo solo;
protected void setUp() throws Exception {
super.setUp();
//Initialize Solo with the instrumentation and the activity under test
solo = new Solo(getInstrumentation(), getActivity());
}
And here is the test method:
public void testHappyPathAddScaleQuestion() {
// Type question title
solo.clickOnEditText(0); //find the EditText, and click it
solo.enterText((EditText) getActivity().findViewById(
//find the EditText, and put some string
R.id.activity_add_new_question_editText_question_title),
"Question title scale ");
// Type question description
solo.clickOnEditText(1);
solo.enterText((EditText) getActivity().findViewById(
R.id.activity_add_new_question_editText_question_description),
"Question description scale");
// Type the question
solo.clickOnEditText(2);
solo.enterText((EditText) getActivity().findViewById(
R.id.activity_add_new_question_editText_question),
"Question scale");
// Click the spinner and then the "Scale" question type
//Press an spinner option
solo.pressSpinnerItem(0, 4);
//Wait for the popup dialog title to show up. When robotium reads it, continue working solo.waitForText(getActivity().getResources().getString(R.string.activity_add_new_question_scale_selection_dialog_message));
// Type minimum and maximum ranges
solo.clickOnEditText(0);
solo.searchText(getActivity().getResources().getString(R.string.activity_add_new_question_maximum_value_hint));
solo.clickOnView(solo.getCurrentEditTexts().get(0));
solo.enterText(0, "34");
solo.clickOnView(solo.getCurrentEditTexts().get(0));
solo.enterText(1, "55");
// Click ok to close the dialog
solo.clickOnButton(getActivity().getResources().getString(R.string.OK));
// Click ok to get an ok message
solo.clickOnButton(getActivity().getResources().getString(R.string.OK));
//Wait for the ok toast message
boolean flagOKDatabase=solo.waitForText(getActivity().getResources().getString(R.string.database_success_storing_data),1,120);
assertEquals("Something wrong happened with the database", true, flagOKDatabase);
}
There is no downside to using robotium over the instrumentation framework native to android. That is because when using robotium you can still do everything you could of done without it but you also have access to lots of helpful functions already.
There are other android automation frameworks out there though that are definitely worth looking at. If you have any code in a web view these are especially helpful as this is where robotium really lets itself down.
https://github.com/calabash/calabash-android
https://github.com/calabash-driver/calabash-driver
http://testdroid.com/
I am trying to create unit tests for a REST client that does some API calls. The client works fine in the live application, but I can't get it to run in the test case.
Apparantly, LoaderTestCase.getLoaderResultSynchronously() could be used here (at least according to Android reference, but it will not accept my loader. The code:
public void testGetEventInfo() {
// Init some vars
...
// Create & execute loader
RESTLoader loader = new RESTLoader(getContext(),
RESTLoader.HTTPVerb.GET, action, params, LOADER_GET_NEWS);
getLoaderResultSynchronously(loader);
}
This yields the error getLoaderResultSynchronously(Loader) in the type LoaderTestCase is not applicable for the arguments (RESTLoader).
RESTLoader extends AsyncLoader. Note that I'm using the supportlibrary, maybe the Loader in there is incompatible? The documentation gives no information on this.
I've tried to solve this in several ways, though none seem to work:
Registered a listener to loader. However, the callback never triggers
Using CountdownLatch (also with a listener). Again, no trigger/countdown timeout.
Playing around with the type template (), without success.
Similar solutions on SO, though again failing to reach the listener.
Does anybody know why getLoaderResultSynchronously will not accept the loader? Or another clean way of testing the Loader, including a way to test return data? I can test handling the return data in a separate case, but I would also like to test the actual data.
Sincerely,
Have you taken a look at the source code? You'll find the following import statements:
import android.content.Loader;
import android.content.Loader.OnLoadCompleteListener;
It doesn't look like Android offers a support version for LoaderTestCase. The easiest solution would be to temporarily change to the non-support LoaderManager (that is, have your class make use of the android.content.Loader instead), test your app, and then switch back to the support implementation. You might also consider copying the testing source code into your project, import the support LoaderManager, and execute it directly. I'm not familiar with the test libraries for Loaders but it doesn't seem outwardly obvious that this would cause any major issues.
You can get sources from LoaderTestCase here, create SupportLoaderTestCase class from that sources in your test project and modify all namespaces to support library namespaces (e.g. change android.content.Loader with android.support.v4.content.Loader). Than you can extend your test case from SupportLoaderTestCase (not from LoaderTestCase) and use it without problems
The method you are trying to call (getLoaderResultSynchronously) accepts an object of type android.content.Loader. If your RESTLoader class is not of that EXACT type then you will get this error. I suspect your class directly or indirectly extends android.support.v4.content.Loader, which would explain the error.
I am not aware of a back-port of LoaderTestCase that would support testing of this type of class.