unit test android - android

Unit testing android application is often more difficult than expected because there are classes that do not have a public constructor (like NotificationManager) and classes that have methods that is not possible to override like Context.getText.
For NotificationManager I used a delegator. I would do the same for the context but that means that in all the classes that use the context (many) I need to use my own context that can't even be derived from Context. Then whenever I need to pass the context to an Android API I need to get the real context from within my wrapper.
Is this correct? Is there another way to do it? What is the rationale of having those methods declared as final?
Is there someone that has really written unit tests for big applications for Android?
EDIT
I found the reason why the getText method is defined as final: it's a template method. So it is enough to override the called non final methods
I also found this that works but it is a bit weird

Question is a little bit vague so I am going to write a long answer.
there are classes that do not have a public constructor (like NotificationManager)
Because there are some system-wide resources/components (for instance NotificationManager) which Android doesn't like application to construct/instantiate on demand, instead, these resources are centralized and managed by system and application should always use system provided API to acquire them.
and classes that have methods that is not possible to override like Context.getText
Generally in Java, method marked as final means they are protected by API and API developer doesn't want the default behaviour to be overridden by application developer. In particular for this method Context.getText(), it actually use Template method pattern, check out herschel's comment below and the source code for more details.
Then the question is how do we properly test our code written by using these android context based resources/components?
The answer given by Android SDK is Test Project (via InstrumentationTestRunner). Generally, when using InstrumentationTestRunner, behind the sense, system will install both test.apk (build from Test Project) and app.apk (build from Application Project), and use test.apk manipulate app.apk for testing. This is happened Regardless of whether you use AndroidTestCase API (referred as JUnit in diagram below) or InstrumentationTestCase API (referred as Instrumentation in diagram below) The corresponding testable android context based resoures/components (i.e. Activity, NotificationManager or TextView) are all acquired from the real running app.apk instance. In case you create/use your own MockContext in Test Project, you can see from the diagram that mock object is injected into the running application by InstrumentationTestRunner.
The drawback of using instrumentation test is about efficiency, it goes into the complete running life cycle (start emulator -> install and start both test.apk and app.apk and fire InstrumentationTestRunner) even though you just need test a single method from a single class (as you mentioned in comment). I agree with Ollie, it is a hostile design.
This is where Robolectric come in and play, quouting from their website: Robolectric is a unit test framework that de-fangs the Android SDK jar so you can test-drive the development of your Android app. Tests run inside the JVM on your workstation in seconds.
From my own experience, I have an application consists of around 10 activities and several Service with some other custom views/widgets doing mainly http communication with remote server, and use both instrumentation test (via Test Project) and Robolectic testing Android Context based code. I can pretty much test almost every single class/public method of my application as long as time is allowed.

Android is pretty hostile to testing in many respects.
You might want to take a look at Roboelectric
http://pivotal.github.com/robolectric/
I assume you've read these:
http://developer.android.com/guide/topics/testing/index.html
http://developer.android.com/resources/tutorials/testing/helloandroid_test.html

The *TestCase classes that Android provides for you should be able to solve the testing issues you are encountering. In most of these instances it will spawn a context for you that you can override if you wish. I would suggest starting there, or asking more specific questions about how you would test one thing or another.

Related

Find methods for xposed hooks

Im currently working with the xposed framework on Android 8.0. How is it possible to find all the method names of an app to hook? For example, i want to hook the method that is called if i add a new contact. Would it be possible to reverse engineer the apk to insert an logcat output into all methods, that shows which method was called?
There a various methods for identifying the relevant methods.
The first and most important one is knowledge of the official Android API. All Java/Kotlin based apps at some point use the classes and methods defined i the API. And the most important fact is that because the Android API belongs to Android and not to the app it can not be obfuscated.
Second you can reverse engineer the app itself using tools like Jadx, apktool, Ghidra, JEB... Just make sure that the tool you use does not rename the class names (e.g. to avoid name collisions or make obfuscated class and method names better readable) or at least allows you always show you the original class and method name. Because trying to hook a method by it's name will fail if you use a name generated by the APK reversing tool.
Also a very helpful tool that allows to identify a lot o the internal on a running program is frida-trace. As long the the executed app has no anti-debugging or anti-frida measures in place you can attach frida-trace at any time to an app on a rooted device and create execution traces you can later use to hook the methods using xposed or directly using frida.

How to initialize emulated external dependencies in a Kotlin Android UI test

I'll be developing an Android app in Kotlin and I'm trying to figure out how to initialize emulated dependencies. For example, the app will make API calls to a server, obtain the user's location from a location provider, pull down images from a content management system, store data locally in a database and in Android's Shared Preferences, and do math based on the current date/time. Thus there are a lot of external dependencies I want to emulate, including the current date/time so I can verify age calculation, etc.
My goal for testing is just to validate my app's screens using an Android instrumented test. I don't want any dependency on real external systems because testing those systems is the responsibility of the developers of those systems.
While reading Android'd documentation Consider whether to use test doubles, I noticed it offered a good tip: "Tip: Check with the library authors to see if they provide any officially-supported testing infrastructures, such as fakes, that you can reliably depend on." However, the documentation doesn't really explain how to initialize a 3rd party test infrastructure.
Below are what I understand so far about my options, but they all come back to a fundamental thing I don't understand: how does an Android app under test know it should operate in a test mode vs. a production mode?
Mocking such as Mockito or MockK: Mocking seems like a special case of Dependency Injection tailored for testing. The examples I've seen involve testing a class or a method, not a full scale system. The examples typically show how you mock a class and pass it to the class/method under test. But with a full scale system, the test code operates on widgets referenced via Espresso. There is no access to the classes where the logic is. My impression is mocking is for unit testing, not UI testing. But maybe someone can explain how to use mocking for UI testing:
a) Suppose an external dependency is initialized deep in the call stack. If I define a mock in my test code's setup function (e.g. a method annotated with #Before), how do I pass that down to the place in the code that depends on it?
b) I keep reading that mocks don't work in Kotlin because Kotlin defines all classes as final. There seem to be a few work arounds. But does does Google/Android officially recommend one of these (I haven't read it in their documentation).
Dependency Injection such as Dagger 2: If mocking isn't viable for UI testing, then should I use Dependency Injection? From what I understand, it seems Dagger 2 deals with issue 1.a above by defining a top-level Component and a tree of Modules which can provide dependencies at any layer of the stack. For testing, it seems like I would just provide a different Component that emulates the real dependencies.
a) In an Android instrumented test, where do instantiate a Dagger 2 Component designed for testing. How do I make sure that Component is used rather than the Component intended for production?
Prepare before launching test: I can see how I could customize build.gradle to prepare my test environment before my application is launched. For example, I could pass a flag to my app so that when the Application's onCreate() gets called, I can configure my system to prepare emulated dependencies via Dependency Injection, Mocking, or even just a custom implementation. For example, some external dependencies have a test mode where I would need to pass a flag to them so they work in test mode. I'm not clear how that sort of thing reconciles with Dependency Injection or Mock but I guess I could see how I could use those mechanisms as a wrapper to pass the test flag or not. In the following post someone wanted to mock a location provider and to do that they modified their build.gradle file to set things up before the Android test infrastructure started.
How to set Allow Mock Location on Android Device before executing AndroidTest with uiautomator and espresso?
In conclusion, I want to test a Kotlin Android app's UI using Android instrumented test with Espresso but I don't know how to setup the test so that external dependencies use emulation code rather than production code. Should I use mocking, Dependency Injection, or customize via build.gradle? Can someone help me get my thinking on track?
After much searching, I've discovered that the Android ActivityTestRule allows you to defer launching the Activity. This gives the test code time to initialize emulated dependencies as demonstrated in Fast Mocked UI Tests on Android Kotlin.

Android Integration tests without emulator

I am wondering if there is an option to test client library which uses Android (mock) and calls backend running on some endpoint?
The use case could be:
Have a library called 'A' that requires android's context (extracts data from it) and Google Services (could be mock). Then call A.sendData(context) that collects data such as advertising id, phone info, ...
So the test lifecycle could be:
A.sendData(context) → waits for server response → assert result.
I've read everywhere that Robolectric can't be used as an integration framework. I've created an Instrumented test with some UI which did the trick BUT running it takes ages and doesn't work properly on remote CI (timeouts, sometimes it passes sometimes it doesn't).
I need to mock 'just' part of Android device (no need for UI) and test if backend returns valid data so probably no need for emulator. OR if the emulator is needed I would like to create a test that uses emulator but there is no need to write UI test.
Is out there anything like this?
It seems to me all you're trying to do is validate a network component sends the appropriate data to a server and handles the appropriate response. For this kind of test, I think Robolectric would suit the needs just fine. It should be able to fill the Context of your app which can then be injected in to your component. All while being run through the JUnit framework which is significantly faster than instrumentation tests.
Though, if all you're trying to do is make sure your network component works. Then you may not actually need the Android framework. You could instead rewrite your class using pure Java. Just strip out the items you need from Context, and inject those in to your classes. Then you can use the JUnit testing framework which runs on the computer instead of the device. It's much faster and doesn't require a phone. This works as long as they're not Android specific components such as Drawable. This will also not work if you require Google Services.
A very tedious method is to use Mockito and PowerMock to mock the methods of your Context and Play Services. You just mock the methods that your Context class is going to use. This can get really complicated though. You can use this alongside Robolectric so you only have to mock what you need to mock.
As far as integrations tests are concerned, there's absolutely no need to write a UI test when writing integration tests. Basically what the integration tests do is create a test Application that runs on the phone. If you don't provide an UI, then you won't use an UI. It'll just show a blank screen while the test runs. You can use InstrumentationRegistery#getContext() to retrieve the context that's used in your application. Then inject the context in to your components that need it.
Though, this can sometimes even be a pain and instrumentation tests take longer even without any UI. But, if you want to make sure it works with an actual Android Context with actual Google Services, this is the best way.
So the basic strategy is to write a whole bunch of small, fast unit tests using Robolectric on JUnit framework which can be run a bunch of times. Then write your comprehensive validation tests using instrumentation framework to make sure everything works together nicely.

Mocking android applications like the contacts app using Mockito

I am trying to write functional tests for an android application. The problem is that most functional testing frameworks that I have explored (calabash-android, robotium) have a limitation on the number of activities from different applications that can be tested in the same test. So if in one workflow I need to select some contacts from the android contact picker I cant test that entire flow because the contact picker activity is part of the android contacts application and the framework can't test an activity from my application and the contacts application at the same time.
One possible solution my team thought of was to mock out the call to the contacts activity to return a dummy intent with contact information so that our application workflow can be tested. We are trying to use mockito to achieve this. However I am stuck pretty early on. Here is what I am trying to do
MyActivity mockActivity = mock(MyActivity.class);
when(mockActivity.startActivityForResult(<?>,anyInt())).thenReturn(fakeIntent);
I am not sure what to put in the first parameter in the second line. I have tried Intent.class and android.content.Intent.class however it throws a compile error. If anyone has worked with mocking activities using mockito, some help will be greatly appreciated.
P.S. - If I understand correctly mocking is used more in unit testing than functional testing. So these tests would be more of a hybrid. If anyone has a better suggestion on how to go about these functional tests on android I am all ears.
It's hard to answer this without knowing the signature of your startActivityForResult method, but the general idea is to use any(Xxx.class), where Xxx is the type of the parameter. So either
when(mockActivity.startActivityForResult(any(Xxx.class),anyInt())).thenReturn(fakeIntent);
or the (kind of equivalent)
doReturn(fakeIntent).when(mockActivity).startActivityForResult(any(Xxx.class),anyInt());
The issue is that you cannot really "mock" (actually "spy") on the activity you're testing since it is created out of your control by Android's instrumentation code. In a unit-test environment where you would have control, you could follow the mock(MyActivity.class) or spy(myActivityInstance) path (spy would actually be better because you could re-use most of the activity's original implementation), but here not.
The only solution I found for this dilemma was to move certain functionality out of the activity into utility classes, ideally utilizing roboguice for that (#ContextSingletons can be used to process activity results). Then, in your test project, you would create your own test guice injector, set that as base application injector before you call getActivity() for the first time and then let the activity work on your mocked utility class.
I outlined the complete process here.

Using Android Test Framework

Android provides various packages for testing like
AndroidTestCase
ApplicationTestCase
InstrumentationTestCase
ActivityInstrumentationTestCase2 ActivityTestCase
I need to know how to decide which package is best suitable for testing my app. Some info is provided in this link
http://developer.android.com/reference/android/test/package-summary.html
But I need more clarity on this...
TestCase – Plain old JUnit test case. It can be extended to test utility classes that are not tied to the Android framework.
AndroidTestCase – It extends JUnit’s TestCase. It’s a lighter
testing class compared to
ActivityTestCase. It doesn’t need to
launch an activity to run it. Its
getContext() method allows you to get
an injected context if you need one.
Since you can get a context from this
class, you can inflate your UI objects
to test their behaviors.
ActivityInstrumentationTestCase2 – It’s the newer version of ActivityInstrumentationTestCase. ActivityInstrumentationTestCase is deprecated in Android SDK 1.5. It’s a heavier testing class compared to AndroidTestCase. It provides UI and functional testing for a single activity. You can get an injected activity that you are testing on by calling its getActivity() method. The activity being tested is launched and finished before and after each test.
ActivityUnitTestCase – It gives
the tested activity an isolated
environment. When using it to test an
activity, the activity is not attached
to the system. This gives you more
control over what kind of environment
that you want your activity to be
tested in.
ApplicationTestCase – It provides testing for Application classes. It can be used to test the life cycle of an application.
InstrumentationTestRunner – The runner that runs the Android test
cases.
I just found this..Hope this helps for others...If u want more details like when and how to use, see the APIDemos test application in the samples directory within android SDK.
Please see the class hierarchy graph drawn by myself using PowerPoint.
The accepted answer gives enough info in words. I just to make it clear using graph :)
For the InstrumentationTestCase #Zorb asked, it's parent class for ActivityTestCase among others. It enables you to call the getInstrumentation method to get an instance of instrumentation so that you can operate application, activity, and so on.
INTRO
To clarify your question and collocate the classes you are asking for, it is essential BEFORE to divide testing in two categories. JUnit tests (in theory plain Java) and Instrumentation tests (that also are part of the JUnit test package but allow you to test more SDK Android related functionalities).
Traditional JUnit tests isolate a section of code and run a test. Instrumentation tests access instead more inclusively the android components. BUT ALSO THE INSTRUMENTATION TESTS DERIVE FROM THE JUNIT PACKAGE although they are instantiated in the system even before any application code is run and this explain why are slower, furthermore they need an emulator or phone that run the app you are testing to be executed.
(Follow in Bold all the class you mention, but there are others even more used that I will write without without the bold character).
****FIRST PART**(Junit tests)**
A) Junit tests that extend TextCase (are usually faster than Instrumentation, and combine well with Mock framweworks)
AndroidTestCase: It allows to have access to the Context of the Activity you are testing and their resources, it is a base class that extends more specialized subclasses, it is ideal to access databases, filesystem data. You can easily call getContext() and access the Resources without establishing a direct contact with the Activities as will be with Instrumentation tests.
ApplicationTestCase that controls the environment where you text the application, mainly the context and the life cycle. Other really useful extensions of AndroidTestCase allow you to control Loaders,Services,and Content Providers, but for some reason still not any Broadcast receiver that you can call it [indirectly][1] with the method InstrumentationRegistry.getTargetContext() and then instantiating BroadCastReceiver. Also it is really common to use different Mock framework with Junit that is usually faster than InstrumentationTests
-.-.-.-.-.-.-.-.--
****SECOND PART**(Instrumentation tests)**
B) Instrumentation tests, that extend TestCase
Are considered functional tests, focused more to insure that they work well with the user side, the View part of MVC.They are usually slower than the other mentioned category
InstrumentationTestCase is a base class useful to send key events to UI to simulate QWERTY keyboard keys or DPAD buttons,also it launches the Activity that have to be tested, and even Intents
ActivityTestCase Usually is not used alone,it just has common code for the subclasses, but in the case you are not satisfied by the 3 classes that inherit from this one(see below) like a new future component you can use it by the time will not exist a TestCase class dedicated.
ActivityInstrumentationTestCase2 is the most used Instrumentation class to write functional tests, from the constructor is invoked the instance of the Activity to test. Basically you call the activity with getActivity() and you can pratically run any method of that Activity.
ActivityInstrumentationTestCase, is deprecated, and ActivityUnitTestCase that although is under the Instrumentation is more similar to an unit test
I found this tutorial from droidcon 09 to be really informative. Complete with downloadable working source code examples.
Edit: The link seems to be dead, as pointed out in comments.
It walks through creating a Celcius to Fahrenheit temperature converter test-first using ActivityInstrumentationTestCase2, TestCase and AndroidTestCase.

Categories

Resources