Expected
Creating a nested test within a parameterized test in JUnit5.
There are many conditions for the Android ViewModel using param. tests. I want to organize the tests within the param. test to improve output readability.
#ExtendWith(InstantExecutorExtension::class)
class ContentViewModelTest {
private fun `FeedLoad`() = Stream.of(
FeedLoadTest(isRealtime = false, feedType = MAIN, timeframe = DAY, lceState = LOADING),
FeedLoadTest(isRealtime = false, feedType = MAIN, timeframe = DAY, lceState = CONTENT))
#ParameterizedTest
#MethodSource("FeedLoad")
fun `Feed Load`(test: FeedLoadTest) {
#Nested
class FeedLoadNestedTest {
#Test
fun `all fields are included`() {
assertThat(4).isEqualTo(2 + 2)
}
#Test
fun `limit parameter`() {
assertThat(4).isEqualTo(3 + 2)
}
}
...
}
data class FeedLoadTest(val isRealtime: Boolean, val feedType: FeedType,
val timeframe: Timeframe, val lceState: LCE_STATE)
}
Observed
The normal parameterized assertions [not depicted] work as expected. The nested FeedLoadNestedTest does not run within the Stream of parameterized FeedLoad tests.
#Sam Brannen, thanks for the feedback!
Sam has indicated on GitHub, #Nested annotation on local classes will not be a viable option.
We have no plans to support #Nested on local classes defined within the scope of a method (function in Kotlin).
Solution
Implement multiple parameterized tests that pass in the same stream.
This will allow for assertions and logic to be organized into separate parameterized functions while testing the same data passed in via the stream.
#ParameterizedTest
#MethodSource("FeedLoadStream")
fun `Feed Load Part One`(test: FeedLoadTest) {
...
}
#ParameterizedTest
#MethodSource("FeedLoadStream")
fun `Feed Load Part Two`(test: FeedLoadTest) {
...
}
#ParameterizedTest
#MethodSource("FeedLoadStream")
fun `Feed Load Part Three`(test: FeedLoadTest) {
...
}
Related
I just added DataStore to our codebase. After that, I found that all sequential UI tests are failing - the first one in a test case pass but next fails with There are multiple DataStores active for the same file.
I provide a data store instance using Hilt
#InstallIn(SingletonComponent::class)
#Module
internal object DataStoreModule {
#Singleton
#Provides
internal fun provideConfigurationDataStore(
#ApplicationContext context: Context,
configurationLocalSerializer: ClientConfigurationLocalSerializer
): DataStore<ClientConfigurationLocal> = DataStoreFactory.create(
serializer = configurationLocalSerializer,
produceFile = { context.dataStoreFile("configuration.pb") }
)
}
I guess this is happening because In a Hilt test, the singleton component’s lifetime is scoped to the lifetime of a test case rather than the lifetime of the Application.
Any ideas on how to workaround this?
I had the same issue.
One solution I tried but which didn't work (correctly) is to make sure the tests, once done, remove the dataStore files (the whole folder) and close the scope (the overridden scope that you manage in a "manager" class), like so:
https://github.com/wwt/testing-android-datastore/blob/main/app/src/androidTest/java/com/wwt/sharedprefs/DataStoreTest.kt
I had this in a finished() block of a TestWatcher used for these UI tests. For some reason, this was not enough so I ended up not looking deeper into why.
Instead I just used a simpler solution: the UI tests would use their own Dagger component, which has its own StorageModule module, which provides its own IStorage implementation, which for UI tests is backed just by an in-memory map, whereas on a production Dagger module would back it up via a DataStore:
interface IStorage {
suspend fun retrieve(key: String): String?
suspend fun store(key: String, data: String)
suspend fun remove(key: String)
suspend fun clear()
I prefer this approach in my case as I don't need to test the actual disk-persistance of this storage in UI tests, but if I had needed it, I'd investigate further into how to reliably ensure the datastore folder and scope are cleaned up before/after each UI test.
I was having the same issues and I came out with a workaround. I append a random number to the file name of the preferences for each test case and I just delete the whole datastore file afterward.
HiltTestModule
#Module
#TestInstallIn(
components = [SingletonComponent::class],
replaces = [LocalModule::class, RemoteModule::class]
)
object TestAppModule {
#Singleton
#Provides
fun provideFakePreferences(
#ApplicationContext context: Context,
scope: CoroutineScope
): DataStore<Preferences> {
val random = Random.nextInt() // generating here
return PreferenceDataStoreFactory
.create(
scope = scope,
produceFile = {
// creating a new file for every test case and finally
// deleting them all
context.preferencesDataStoreFile("test_pref_file-$random")
}
)
}
}
#After function
#After
fun teardown() {
File(context.filesDir, "datastore").deleteRecursively()
}
I'd suggest for more control + better unit-test properties (ie. no IO, fast, isolated) oblakr24's answer is a good clean way to do this; abstract away the thing that you don't own that has behavior undesirable in tests.
However, there's also the possibility these tests are more like end-to-end / feature tests, so you want them to be as "real" as possible, fewer test doubles, maybe just faking a back-end but otherwise testing your whole app integrated. If so, you ought to use the provided property delegate that helps to ensure a singleton, and declare it top-level, outside a class, as per the docs. That way the property delegate will only get created once within the class-loader, and if you reference it from somewhere else (eg. in your DI graph) that will get torn down and recreated for each test, it won't matter; the property delegate will ensure the same instance is used.
A more general solution, not limited to Hilt, would be to mock Context.dataStoreFile() function with mockk to return a random file name.
I like this approach as it doesn't require any changes on the production code.
Example of TestWatcher:
class CleanDataStoreTestRule : TestWatcher() {
override fun starting(description: Description) {
replaceDataStoreNamesWithRandomUuids()
super.starting(description)
}
override fun finished(description: Description) {
super.finished(description)
removeDataStoreFiles()
}
private fun replaceDataStoreNamesWithRandomUuids() {
mockkStatic("androidx.datastore.DataStoreFile")
val contextSlot = slot<Context>()
every {
capture(contextSlot).dataStoreFile(any())
} answers {
File(
contextSlot.captured.filesDir,
"datastore/${UUID.randomUUID()}",
)
}
}
private fun removeDataStoreFiles() {
InstrumentationRegistry.getInstrumentation().targetContext.run {
File(filesDir, "datastore").deleteRecursively()
}
}
}
and then use it in tests:
class SomeTest {
#get:Rule
val cleanDataStoreTestRule = CleanDataStoreTestRule()
...
}
The solution assumes that you use Context.dataStoreFile() and that the file name does not matter. IMO the assumptions are reasonable in most cases.
I have a question related to Unit testing in android.
The app is written in Kotlin, MVVM architecture, Dagger-Hilt, Room, etc…
I have written all the tests for Room DAOs, according to official docs. I have created fake repositories so I can test some Managers/Helpers (these classes encapsulate logic I have to reuse in many ViewModels) that handle business logic. Now I need to test ViewModels which have these Managers/Helpers as dependencies.
I don’t want to fall into trap of re-testing the same code all over again, the question is how to test ViewModels?
Should I only test the parameters that are passed to functions in these Managers/Helpers, and write assertions for that, or what to do?
Thanks in advance!
That is what mocks are for. You should test only logic of ViewModel ignoring logic inside Managers/Helpers. Everything else should be mocked/faked:
#Test
fun `invalid login`() {
runBlocking {
//prepare
val validator = mock<Validator> {
on { errorFor(any()) }
.thenReturn("Something wrong")
}
val authNavigator = spy(loginNavigator)
val tracker = spy(ActionsTracker(mock()))
val trackers = Trackers(tracker, ViewsTracker(mock()), ClicksTracker(mock()), ImpressionsTracker(mock()))
assertViewModel(
validation = AuthViewModel.LoginValidation(validator, validator),
authNavigator = authNavigator,
trackers = trackers
) {
email.set("test")
password.set("test")
//assert
assertNull(emailError.get())
assertNull(passwordError.get())
//act
login()
//assert
assertNotNull(emailError.get())
assertNotNull(passwordError.get())
verify(authNavigator, never()).navigateToHome()
verify(tracker, never()).userPropertiesChanged(any(), any())
}
}
}
The error I have:
The code with the error:
#RunWith(PowerMockRunner::class)
#PrepareForTest(PotatoProvider::class, PotatoConsumer::class)
class WantedButNotInvoked {
#Mock
lateinit var potatoConsumer: PotatoConsumer
#Test
fun potato() {
Observable.just(Potato()).subscribe(potatoConsumer)
verify(potatoConsumer).accept(Potato())
//verify(potatoConsumer).accept(any()) //-> This fails too with the same reason
}
}
data class Potato(val value: Int = 1)
class PotatoConsumer : Consumer<Potato> {
override fun accept(t: Potato?) {
println(t)
}
}
So I making subscribe with this mock(potatoConsumer), and the rxJava have called 'accept', and mockito mark it as interaction, but mockito thinks this interaction is not what I'm expecting, why?
Versions of libraries used her:
mockitoVersion = '2.8.9'
mockitoAndroidVersion = '2.7.22'
powerMockVersion="2.0.2"
kotlinMockito="2.1.0"
rxKotlin = "2.3.0"
rxJavaVersion = "2.2.10"
Kinda workaround
Some fields mocked by powermock, fails on 'verify', but fields mocked with mockito is not;
Mockito can't mock not opened fields, without mock-maker-inline, but mockito conflicts with Powermock mock-maker-inline;
Powermock can delegate calls of mock-maker-inline to other mock-maker-inline(https://github.com/powermock/powermock/wiki/PowerMock-Configuration)
Use Mockito.mock on the failed fields instead of #Mock/Powermock mock injection
Example of the "green" potato test method using PowerMockRunner
#Test
fun potato() {
potatoConsumer = mock() // <-
Observable.just(Potato()).subscribe(potatoConsumer)
verify(potatoConsumer).accept(potato)
}
I am not familiar with PowerMock but I tried this test and it passes:
#Test
fun potato() {
fakePotatoProvider = Mockito.mock(PotatoProvider::class.java)
potatoConsumer = Mockito.mock(PotatoConsumer::class.java)
`when`(fakePotatoProvider.getObservable()).thenReturn(Observable.just(Potato()))
fakePotatoProvider.getObservable().subscribe(potatoConsumer)
verify(potatoConsumer).accept(Potato())
}
Maybe because you aren't passing the same instance of Potato(). Try to refactor your code to this
#Test
fun potato() {
val testPotato = Potato()
`when`(fakePotatoProvider.getObservable()).thenReturn(Observable.just(testPotato))
fakePotatoProvider.getObservable().subscribe(potatoConsumer)
verify(potatoConsumer).accept(testPotato)
}
As I mentioned above, the reason why it might be failing is the constant creation of new instances when passing your Potato object, hance that comparison fails.
Setup:
In our project (at work - I cannot post real code), we have implemented clean MVVM. Views communicate with ViewModels via LiveData. ViewModel hosts two kinds of use cases: 'action use cases' to do something, and 'state updater use cases'. Backward communication is asynchronous (in terms of action reaction). It's not like an API call where you get the result from the call. It's BLE, so after writing the characteristic there will be a notification characteristic we listen to. So we use a lot of Rx to update the state. It's in Kotlin.
ViewModel:
#PerFragment
class SomeViewModel #Inject constructor(private val someActionUseCase: SomeActionUseCase,
someUpdateStateUseCase: SomeUpdateStateUseCase) : ViewModel() {
private val someState = MutableLiveData<SomeState>()
private val stateSubscription: Disposable
// region Lifecycle
init {
stateSubscription = someUpdateStateUseCase.state()
.subscribeIoObserveMain() // extension function
.subscribe { newState ->
someState.value = newState
})
}
override fun onCleared() {
stateSubscription.dispose()
super.onCleared()
}
// endregion
// region Public Functions
fun someState() = someState
fun someAction(someValue: Boolean) {
val someNewValue = if (someValue) "This" else "That"
someActionUseCase.someAction(someNewValue)
}
// endregion
}
Update state use case:
#Singleton
class UpdateSomeStateUseCase #Inject constructor(
private var state: SomeState = initialState) {
private val statePublisher: PublishProcessor<SomeState> =
PublishProcessor.create()
fun update(state: SomeState) {
this.state = state
statePublisher.onNext(state)
}
fun state(): Observable<SomeState> = statePublisher.toObservable()
.startWith(state)
}
We are using Spek for unit tests.
#RunWith(JUnitPlatform::class)
class SomeViewModelTest : SubjectSpek<SomeViewModel>({
setRxSchedulersTrampolineOnMain()
var mockSomeActionUseCase = mock<SomeActionUseCase>()
var mockSomeUpdateStateUseCase = mock<SomeUpdateStateUseCase>()
var liveState = MutableLiveData<SomeState>()
val initialState = SomeState(initialValue)
val newState = SomeState(newValue)
val behaviorSubject = BehaviorSubject.createDefault(initialState)
subject {
mockSomeActionUseCase = mock()
mockSomeUpdateStateUseCase = mock()
whenever(mockSomeUpdateStateUseCase.state()).thenReturn(behaviorSubject)
SomeViewModel(mockSomeActionUseCase, mockSomeUpdateStateUseCase).apply {
liveState = state() as MutableLiveData<SomeState>
}
}
beforeGroup { setTestRxAndLiveData() }
afterGroup { resetTestRxAndLiveData() }
context("some screen") {
given("the action to open the screen") {
on("screen opened") {
subject
behaviorSubject.startWith(initialState)
it("displays the initial state") {
assertEquals(liveState.value, initialState)
}
}
}
given("some setup") {
on("some action") {
it("does something") {
subject.doSomething(someValue)
verify(mockSomeUpdateStateUseCase).someAction(someOtherValue)
}
}
on("action updating the state") {
it("displays new state") {
behaviorSubject.onNext(newState)
assertEquals(liveState.value, newState)
}
}
}
}
}
At first we were using an Observable instead of the BehaviorSubject:
var observable = Observable.just(initialState)
...
whenever(mockSomeUpdateStateUseCase.state()).thenReturn(observable)
...
observable = Observable.just(newState)
assertEquals(liveState.value, newState)
instead of the:
val behaviorSubject = BehaviorSubject.createDefault(initialState)
...
whenever(mockSomeUpdateStateUseCase.state()).thenReturn(behaviorSubject)
...
behaviorSubject.onNext(newState)
assertEquals(liveState.value, newState)
but the unit test were being flaky. Mostly they would pass (always when ran in isolation), but sometime they would fail when running the whole suit. Thinking it is to do with asynchronous nature of the Rx we moved to BehaviourSubject to be able to control when the onNext() happens. Test are now passing when we run them from AndroidStudio on the local machine, but they are still flaky on the build machine. Restarting the build often makes them pass.
The tests which fail are always the ones where we assert the value of LiveData. So the suspects are LiveData, Rx, Spek or their combination.
Question: Did anyone have similar experiences writing unit tests with LiveData, using Spek or maybe Rx, and did you find ways to write them which solve these flakiness issues?
....................
Helper and extension functions used:
fun instantTaskExecutorRuleStart() =
ArchTaskExecutor.getInstance().setDelegate(object : TaskExecutor() {
override fun executeOnDiskIO(runnable: Runnable) {
runnable.run()
}
override fun isMainThread(): Boolean {
return true
}
override fun postToMainThread(runnable: Runnable) {
runnable.run()
}
})
fun instantTaskExecutorRuleFinish() = ArchTaskExecutor.getInstance().setDelegate(null)
fun setRxSchedulersTrampolineOnMain() = RxAndroidPlugins.setInitMainThreadSchedulerHandler { Schedulers.trampoline() }
fun setTestRxAndLiveData() {
setRxSchedulersTrampolineOnMain()
instantTaskExecutorRuleStart()
}
fun resetTestRxAndLiveData() {
RxAndroidPlugins.reset()
instantTaskExecutorRuleFinish()
}
fun <T> Observable<T>.subscribeIoObserveMain(): Observable<T> =
subscribeOnIoThread().observeOnMainThread()
fun <T> Observable<T>.subscribeOnIoThread(): Observable<T> = subscribeOn(Schedulers.io())
fun <T> Observable<T>.observeOnMainThread(): Observable<T> =
observeOn(AndroidSchedulers.mainThread())
I didn't used Speck for unit-testing. I've used java unit-test platform and it works perfect with Rx & LiveData, but you have to keep in mind one thing. Rx & LiveData are async and you can't do something like someObserver.subscribe{}, someObserver.doSmth{}, assert{} this will work sometimes but it's not the correct way to do it.
For Rx there's TestObservers for observing Rx events. Something like:
#Test
public void testMethod() {
TestObserver<SomeObject> observer = new TestObserver()
someClass.doSomethingThatReturnsObserver().subscribe(observer)
observer.assertError(...)
// or
observer.awaitTerminalEvent(1, TimeUnit.SECONDS)
observer.assertValue(somethingReturnedForOnNext)
}
For LiveData also, you'll have to use CountDownLatch to wait for LiveData execution. Something like this:
#Test
public void someLiveDataTest() {
CountDownLatch latch = new CountDownLatch(1); // if you want to check one time exec
somethingTahtReturnsLiveData.observeForever(params -> {
/// you can take the params value here
latch.countDown();
}
//trigger live data here
....
latch.await(1, TimeUnit.SECONDS)
assert(...)
}
Using this approach your test should run ok in any order on any machine. Also the wait time for latch & terminal event should be as low as possible, the tests should run fast.
Note1: The code is in JAVA but you can change it easily in kotlin.
Note2: Singleton are the biggest enemy of unit-testing ;). (With static methods by their side).
The issue is not with LiveData; it is the more common problem - singletons. Here the Update...StateUseCases had to be singletons; otherwise if observers got a different instance they would have a different PublishProcessor and would not get what was published.
There is a test for each Update...StateUseCases and there is a test for each ViewModel into which Update...StateUseCases is injected (well indirectly via the ...StateObserver).
The state exists within the Update...StateUseCases, and since it is a singleton, it gets changed in both tests and they use the same instance becoming dependent on each other.
Firstly try to avoid using singletons if possible.
If not, reset the state after each test group.
I'm using coroutines to do an asynchronous call on pull to refresh like so:
class DataFragment : Fragment(), SwipeRefreshLayout.OnRefreshListener {
// other functions here
override fun onRefresh() {
loadDataAsync()
}
private fun loadDataAsync() = async(UI) {
swipeRefreshLayout?.isRefreshing = true
progressLayout?.showContent()
val data = async(CommonPool) {
service?.getData() // suspending function
}.await()
when {
data == null -> showError()
data.isEmpty() -> progressLayout?.showEmpty(null, parentActivity?.getString(R.string.no_data), null)
else -> {
dataAdapter?.updateData(data)
dataAdapter?.notifyDataSetChanged()
progressLayout?.showContent()
}
}
swipeRefreshLayout?.isRefreshing = false
}
}
Everything here works fine when I actually put it on a device. My error, empty, and data states are all handled well and the performance is good. However, I'm also trying to unit test it with Spek. My Spek test looks like this:
#RunWith(JUnitPlatform::class)
class DataFragmentTest : Spek({
describe("The DataFragment") {
var uut: DataFragment? = null
beforeEachTest {
uut = DataFragment()
}
// test other functions
describe("when onRefresh") {
beforeEachTest {
uut?.swipeRefreshLayout = mock()
uut?.onRefresh()
}
it("sets swipeRefreshLayout.isRefreshing to true") {
verify(uut?.swipeRefreshLayout)?.isRefreshing = true // says no interaction with mock
}
}
}
}
The test is failing because it says that there was no interaction with the uut?.swipeRefreshLayout mock. After some experimenting, it seems this is because I'm using the UI context via async(UI). If I make it just be a regular async, I can get the test to pass but then the app crashes because I'm modifying views outside of the UI thread.
Any ideas why this might be occurring? Also, if anyone has any better suggestions for doing this which will make it more testable, I'm all ears.
Thanks.
EDIT: Forgot to mention that I also tried wrapping the verify and the uut?.onRefresh() in a runBlocking, but I still had no success.
If you want to make things clean and consider using MVP architecture in the future you should understand that CourutineContext is external dependency, that should be injected via DI, or passed to your presenter. More details on topic.
The answer for your question is simple, you should use only Unconfined CourutineContext for your tests. (more)
To make things simple create an object e.g. Injection with:
package com.example
object Injection {
val uiContext : CourutineContext = UI
val bgContext : CourutineContext = CommonPool
}
and in test package create absolutely the same object but change to:
package com.example
object Injection {
val uiContext : CourutineContext = Unconfined
val bgContext : CourutineContext = Unconfined
}
and inside your class it will be something like:
val data = async(Injection.bgContext) {service?.getData()}.await()