When running android connected device tests, state that persists across application instances, such as permissions and files stored by the application, cause tests to interfere with each other.
For example, if I want to write a test for application behaviour when I deny runtime permissions and another test for application behaviour when I allow runtime permissions, then I must be very careful that the tests run in the correct order. If the allow test ran before the deny test, then the deny test would fail, because the permissions settings would already have been granted.
Another example, in a shopping app, the application may store the contents of the basket in the apps internal file storage to allow the basket to survive application termination and reboots. Testing the behaviour of the shopping basket then becomes very difficult, as the tests interfere with each other.
What is the solution to this problem?
Be sure to clean up the state after each test case. Tests which depend on the order run are considered a bad practice. In a lost of cases, you can implement a teardown() method (annotated with #AfterTest if you are using JUnit4) to do the clean up.
Create mock state objects during tests that can be injected into your app. I'm still new to this particular approach, so I don't have a lot of advice here. Some googling should help you get started.
Mocking the state / Injecting special state objects for the tests is a solution for most problems but not the Runtime permission case
Related
Firstly, apologies for the ridiculous title but I really could not come up with something more accurate.
I'm writing an app that interfaces with a particular type of network device. I can write instrumented tests that interact with said device just fine but those tests obviously only work if the device is accessible if the system running the tests are on the same network. Hence, using Firebase to run the tests is impossible because Firebase does not have access to the type of device the app interacts with.
However, it leads me back to a more generic question. How do you handle instrumented tests for functionality that is not publicly accessible? Be it special devices or networks, login credentials (hardcoding login credentials seem wrong), etc.
Is there any way to mock them? So say when you press a button you can mock a positive result?
Hope this made a sliver of sense.
Thanks
Sure, you can.
For example, mock framework - https://github.com/mockito/mockito
From Firebase docs they say:
Multiple processes
Crash Reporting creates a separate background process to upload crash
info. If your app extends the Android Application class, you must
ensure it is multi-process safe. Otherwise, it may cause concurrency
issues. When an app extends an Application object, this object gets
instantiated for each process in a multi-process app. Two important
things to watch for are:
If the implementation of this object accesses any out-of-process state
(a database, the file system, shared preferences, etc), or performs
other actions not safe for a multi-process environment, concurrency
issues might arise. This is because multiple instances of the
Application object may run simultaneously. Many third-party libraries
keep out-of-process state (e.g. in a local database) and are subject
to the same concurrency issues if they are initialized from the
Application object. If your app fits the description above and you
plan to use Crash Reporting in your app, we strongly encourage you to
consider moving the Application logic to Content Providers, or to
Android Activities. Any Application logic that is not safe for a
multi-process environment can have unintended effects on your app.
How can I check from my Application class if theres another instance inside the Application onCreate to avoid crashes with Fabric or others?
Generally speaking, you don't "check to see" if there is another Application object from another process. You simply assume that there is always exactly one Application object created for every process in your app, and ensure for yourself that it will be safe to run in conjunction with other Applications objects in other processes. Just don't access any shared read/write resources from Application and you will be fine.
If you must initialize something from only the main process, a more reliable way of doing this is to create a ContentProvider (declare in your manifest and create an object for it like any other component), and use its onCreate(). ContentProviders are only created and initialized from the main process - never from other processes. This way you can be sure that your init will not be duplicated in any other process.
Or if you don't want to deal with this at all, just wait until Crash Reporting comes out of beta to full release, as it will not use an extra process at that time in the future. We (Google) can't say exactly when that will be, but we're not wasting any time in getting the full release published.
I need to test a use case where the application starts from a clean state - i.e. the process has not been running before the test starts. From what I see from logcat, all instrumentation tests run under one single process instance/session, so the outcome of the test in my case depends on whether or not it runs as #1 or not. It should not be this way - as we all know, unit tests (or instrumentation tests) should be autonomous.
Is there any way with the standard Android instrumentation test tools and functions I can force the TestRunner to restart the process before a given test? If not, are there hacks or third-party libraries that can help me achieve that? Or is there any way I can specifically say that test X must be run first (worst option but still)?
In specific, my test relates to the launching of activities through intents, and the intent flags (e.g. FLAG_ACTIVITY_CLEAR_TOP) in addition to the Activity launch mode (e.g. singleTop) and the state of the process, very much dictates the outcome of the test.
Assuming you are running with Espresso, there is not a clean way to pull this off. This is because Espresso runs in the same process as the application and thus killing the app will kill Espresso.
The question is, do you need all the logic you want to execute in your Application or could it be ported to your Activity.onCreate()? With Espresso restarting an Activity is doable. If there is a need to restart the application because of global/singletons, removing these may be necessary. If this cannot be done you can look at other test automation frameworks like Appium which has some support for this.
I was reading "Multitasking the Android Way" by Dianne Hackborn and found one sentence I don't quite understand (I bolded the puzzling part):
A common misunderstanding about Android multitasking is the difference
between a process and an application. In Android these are not tightly
coupled entities: applications may seem present to the user without an
actual process currently running the app; multiple applications may
share processes, or one application may make use of multiple processes
depending on its needs; the process(es) of an application may be kept
around by Android even when that application is not actively doing
something.
How is it even possible? To "present" an application to the user, we need a least one of its activities to be in resumed state, and for this we need the underlying Linux process up and running, right? Is she talking about the application launcher or something?
To "present" an application to the user, we need a least one of its activities to be in resumed state, and for this we need the underlying Linux process up and running, right?
There are a few ways in which the user can see an app's UI without the app having a currently running process, such as:
The app can have an app widget on the home screen
The app can have a Notification in the status bar, or on a Wear device
The app can have an entry in the recent-tasks list
My interpretation is that "applications may seem present" refers to something like these.
as far as i understand each android process runs in a "sandbox" environment, what happen if an android application is a multi process application? do all processes use the one single sandbox since all the processes use the same user id?
As Android uses the Linux kernel for sanboxing, the real sandbox is per (unix) user id, rather than per process. And the Dalvik VM itself makes no attempt at sandboxing.
Of course proccesses owned by the same user id are somewhat isolated from each other, but tools like kill(), ptrace() and the /proc filesystem which pierce process isolation are available - and governed primarily by user id isolation.
There are a couple of oddities about how code maps to processes however:
Either an Activity or a Service can be designated in the manifest to run in a distinct process, but this will still be a process owned by the package user id
Distinct application packages with the same signing certificate can use the shared user id feature to share a sandbox; in some cases this can result in their code running in the same process.
The multiprocess attribute of the <activity> tag says that the activity could run in the process of whatever starts it - it's not entirely clear if this could mean it would end up running under a foreign userid (an idea that seems more risky for the caller than the callee).
The isolatedprocess attribute of the <service> says that it runs the service in an isolated process with "no permissions of its own" - while it doesn't come out and explicitly say so, I suspect this may mean that this would be a process owned by a distinct, minimally privileged user id.
A more definitive explanation on these last two points would be good. Perhaps I'll look at the implementing code when I have more time.
Each process is associated with a separate runtime (JVM) and sandbox in Android.
For example, if you tune couple of your activities to run on a separate processes (to take advantage of having a distinct memory space) then you end up with having 2 sandboxes when you launch the app and have to manage IPC if needed.
Another example is - if you set two different apps to share the same user id then you won't need IPC since they can see each other's data, but that doesn't mean they run on the same process or sandbox. You need to set process attribute of application element in the manifest.
Out of the context, you can also have multiple components belongs to separate apps to easy sharing and saving memory.