Develop Reference

Use Spek with suspend method

I am trying to create a unit test using Spek framework and nhaarman mockito kotlin in my Android Kotlin project. The problem is that when there is nested suspend method I don't know how to mock response.This is how I'm trying
I defined:
val testCoroutineDispatcher = TestCoroutineDispatcher()
val testCoroutineScope = TestCoroutineScope(testCoroutineDispatcher)
and before any describe
beforeGroup {
Dispatchers.setMain(testCoroutineDispatcher) //not sure if this is working properly
}
afterGroup {
Dispatchers.resetMain() // reset main dispatcher to the original Main dispatcher
testCoroutineScope.cleanupTestCoroutines()
}
and this is my group
describe("Test view model") {
val contentRepository by memoized(CachingMode.SCOPE) { mock<ContentRepository>() }
val contentViewModel by memoized(CachingMode.SCOPE) {
ContentViewModel(contentRepository)
}
describe("When something happens") {
beforeGroup {
testCoroutineScope.runBlockingTest {
whenever(contentRepository.fetchAllContents(0, 10))
.thenReturn(Result.success(content))//This is suspend
contentViewModel.setContentPage(0)
}
}
it("should fetch all content from repository with page 0") {
verifyBlocking(contentRepository) {
fetchAllClassContents(0, 10)
}
}
}
}
})
But Im getting the following error
Argument(s) are different! Wanted:
classContentRepository.fetchAllClassContents(
0,
10,
Continuation at viewmodel.ContentViewModelSpek$1$3$1$4$1.invokeSuspend(ContentViewModelSpek.kt:92)
);
-> at repository.ContentRepository.fetchAllClassContents(ContentRepository.kt:23)
Actual invocation has different arguments:
contentRepository.fetchAllContents(
0,
10,
Continuation at viewmodel.ContentViewModel$setContentPage$1.invokeSuspend(ContentViewModel.kt:26)
);
It seem like mock, method execution and assertion are running in different scopes
I can't find any guide that helps me create test with coroutine
Thanks in advance

Testing Android Room with Kotlin Flow

I'm trying to test a Room DAO exposing functions that return Flows. The following test won't pass and I'm struggling to see why :
#Test
fun `observeHomeCoursesFeatured() does not return courses that are no longer featured`() = runBlocking {
val outputList: MutableList<List<HomeCourse>> = mutableListOf()
launch { subject.observeHomeCoursesFeatured().collect { outputList.add(it) } }
subject.saveHomeCoursesFeatured(listOf(getHomeCourseFeatured1(), getHomeCourseFeatured2()))
subject.saveHomeCoursesFeatured(listOf(getHomeCourseFeatured1()))
assertEquals(2, outputList.size)
assertEquals(listOf(getHomeCourseFeatured1(), getHomeCourseFeatured2()), outputList[0])
assertEquals(listOf(getHomeCourseFeatured1()), outputList[1])
}
It fails at assertEquals(2, outputList.size) saying that outputList is empty.
This test passes :
#Test
fun `observeHomeCoursesFeatured() does not return courses that are no longer featured`() = runBlocking {
subject.saveHomeCoursesFeatured(listOf(getHomeCourseFeatured1(), getHomeCourseFeatured2()))
assertEquals(listOf(getHomeCourseFeatured1(), getHomeCourseFeatured2()), subject.observeHomeCoursesFeatured().first())
subject.saveHomeCoursesFeatured(listOf(getHomeCourseFeatured1()))
assertEquals(listOf(getHomeCourseFeatured1()), subject.observeHomeCoursesFeatured().first())
}
The second test passing, shows that my DAO is working fine and it is more a question of threading and concurrency between the test thread and the thread that Room uses to trigger Flow changes.
I already added #get:Rule val archRule = InstantTaskExecutorRule() in my test. I also build my test DB with this :
db = Room.inMemoryDatabaseBuilder(ctx, CoreDatabase::class.java)
.setTransactionExecutor(Executors.newSingleThreadExecutor())
.allowMainThreadQueries()
.build()
What am I missing ?

launch is asynchronous, so you have a race condition.
#Test
fun `observeHomeCoursesFeatured() does not return courses that are no longer featured`() = runBlocking {
val job = async { subject.observeHomeCoursesFeatured().take(2).toList() }
subject.saveHomeCoursesFeatured(listOf(getHomeCourseFeatured1(), getHomeCourseFeatured2()))
subject.saveHomeCoursesFeatured(listOf(getHomeCourseFeatured1()))
val outputList = job.await()
assertEquals(2, outputList.size)
assertEquals(listOf(getHomeCourseFeatured1(), getHomeCourseFeatured2()), outputList[0])
assertEquals(listOf(getHomeCourseFeatured1()), outputList[1])
}

Unit testing coroutines runBlockingTest: This job has not completed yet

Please find below a function using a coroutine to replace callback :
override suspend fun signUp(authentication: Authentication): AuthenticationError {
return suspendCancellableCoroutine {
auth.createUserWithEmailAndPassword(authentication.email, authentication.password)
.addOnCompleteListener(activityLifeCycleService.getActivity()) { task ->
if (task.isSuccessful) {
it.resume(AuthenticationError.SignUpSuccess)
} else {
Log.w(this.javaClass.name, "createUserWithEmail:failure", task.exception)
it.resume(AuthenticationError.SignUpFail)
}
}
}
}
Now I would like to unit testing this function. I am using Mockk :
#Test
fun `signup() must be delegated to createUserWithEmailAndPassword()`() = runBlockingTest {
val listener = slot<OnCompleteListener<AuthResult>>()
val authentication = mockk<Authentication> {
every { email } returns "email"
every { password } returns "pswd"
}
val task = mockk<Task<AuthResult>> {
every { isSuccessful } returns true
}
every { auth.createUserWithEmailAndPassword("email", "pswd") } returns
mockk {
every { addOnCompleteListener(activity, capture(listener)) } returns mockk()
}
service.signUp(authentication)
listener.captured.onComplete(task)
}
Unfortunately this test failed due to the following exception : java.lang.IllegalStateException: This job has not completed yet
I tried to replace runBlockingTest with runBlocking but the test seems to wait in an infinite loop.
Can someone help me with this UT please?
Thanks in advance

As can be seen in this post:
This exception usually means that some coroutines from your tests were scheduled outside the test scope (more specifically the test dispatcher).
Instead of performing this:
private val networkContext: CoroutineContext = TestCoroutineDispatcher()
private val sut = Foo(
networkContext,
someInteractor
)
fun `some test`() = runBlockingTest() {
// given
...
// when
sut.foo()
// then
...
}
Create a test scope passing test dispatcher:
private val testDispatcher = TestCoroutineDispatcher()
private val testScope = TestCoroutineScope(testDispatcher)
private val networkContext: CoroutineContext = testDispatcher
private val sut = Foo(
networkContext,
someInteractor
)
Then in test perform testScope.runBlockingTest
fun `some test`() = testScope.runBlockingTest {
...
}
See also Craig Russell's "Unit Testing Coroutine Suspend Functions using TestCoroutineDispatcher"

In case of Flow testing:
Don't use flow.collect directly inside runBlockingTest. It should be wrapped in launch
Don't forget to cancel TestCoroutineScope in the end of a test. It will stop a Flow collecting.
Example:
class CoroutinesPlayground {
private val job = Job()
private val testDispatcher = StandardTestDispatcher()
private val testScope = TestScope(job + testDispatcher)
#Test
fun `play with coroutines here`() = testScope.runBlockingTest {
val flow = MutableSharedFlow<Int>()
launch {
flow.collect { value ->
println("Value: $value")
}
}
launch {
repeat(10) { value ->
flow.emit(value)
delay(1000)
}
job.cancel()
}
}
}

This is not an official solution, so use it at your own risk.
This is similar to what #azizbekian posted, but instead of calling runBlocking, you call launch.
As this is using TestCoroutineDispatcher, any tasks scheduled to be run without delay are immediately executed. This might not be suitable if you have several tasks running asynchronously.
It might not be suitable for every case but I hope that it helps for simple cases.
You can also follow up on this issue here:
https://github.com/Kotlin/kotlinx.coroutines/issues/1204
If you know how to solve this using the already existing runBlockingTest and runBlocking, please be so kind and share with the community.
class MyTest {
private val dispatcher = TestCoroutineDispatcher()
private val testScope = TestCoroutineScope(dispatcher)
#Test
fun myTest {
val apiService = mockk<ApiService>()
val repository = MyRepository(apiService)
testScope.launch {
repository.someSuspendedFunction()
}
verify { apiService.expectedFunctionToBeCalled() }
}
}

According to my understanding, this exception occurs when you are using a different dispatcher in your code inside the runBlockingTest { } block with the one that started runBlockingTest { }.
So in order to avoid this, you first have to make sure you inject Dispatchers in your code, instead of hardcoding it throughout your app. If you haven't done it, there's nowhere to begin because you cannot assign a test dispatcher to your test codes.
Then, in your BaseUnitTest, you should have something like this:
#get:Rule
val coroutineRule = CoroutineTestRule()
#ExperimentalCoroutinesApi
class CoroutineTestRule(
val testDispatcher: TestCoroutineDispatcher = TestCoroutineDispatcher()
) : TestWatcher() {
override fun finished(description: Description?) {
super.finished(description)
Dispatchers.setMain(testDispatcher)
}
override fun starting(description: Description?) {
super.starting(description)
Dispatchers.resetMain()
testDispatcher.cleanupTestCoroutines()
}
}
Next step really depends on how you do Depedency Injection. The main point is to make sure your test codes are using coroutineRule.testDispatcher after the injection.
Finally, call runBlockingTest { } from this testDispatcher:
#Test
fun `This should pass`() = coroutineRule.testDispatcher.runBlockingTest {
//Your test code where dispatcher is injected
}

There is an open issue for this problem: https://github.com/Kotlin/kotlinx.coroutines/issues/1204
The solution is to use the CoroutineScope intead of the TestCoroutinScope until the issue is resolved, you can do by replacing
#Test
fun `signup() must be delegated to createUserWithEmailAndPassword()`() =
runBlockingTest {
with
#Test
fun `signup() must be delegated to createUserWithEmailAndPassword()`() =
runBlocking {

None of these answers quite worked for my setup due to frequent changes in the coroutines API.
This specifically works using version 1.6.0 of kotlin-coroutines-test, added as a testImplementation dependency.
#Test
fun `test my function causes flow emission`() = runTest {
// calling this function will result in my flow emitting a value
viewModel.myPublicFunction("1234")
val job = launch {
// Force my flow to update via collect invocation
viewModel.myMemberFlow.collect()
}
// immediately cancel job
job.cancel()
assertEquals("1234", viewModel.myMemberFlow.value)
}

If you have any
Channel
inside the launch, you must call to
Channel.close()
Example code:
val channel = Channel<Success<Any>>()
val flow = channel.consumeAsFlow()
launch {
channel.send(Success(Any()))
channel.close()
}

runBlockingTest deprecated since 1.6.0 and replaced with runTest.

You need to swap arch background executor with one that execute tasks synchronously. eg. For room suspend functions, live data etc.
You need the following dependency for core testing
androidTestImplementation 'androidx.arch.core:core-testing:2.1.0'
Then add the following at the top of test class
#get:Rule
val instantExecutor = InstantTaskExecutorRule()
Explanations
InstantTaskExecutorRule A JUnit Test Rule that swaps the background executor used by the
Architecture Components with a different one which executes each task
synchronously.
You can use this rule for your host side tests that use Architecture
Components

As I mentioned here about fixing runBlockingTest, maybe it could help you too.
Add this dependency if you don't have it
testImplementation "androidx.arch.core:core-testing:$versions.testCoreTesting" (2.1.0)
Then in your test class declare InstantTaskExecutorRule rule:
#get:Rule
val instantTaskExecutorRule = InstantTaskExecutorRule()

Mockito.verify didn't see method exectution, even if it was

The error I have:
The code with the error:
#RunWith(PowerMockRunner::class)
#PrepareForTest(PotatoProvider::class, PotatoConsumer::class)
class WantedButNotInvoked {
#Mock
lateinit var potatoConsumer: PotatoConsumer
#Test
fun potato() {
Observable.just(Potato()).subscribe(potatoConsumer)
verify(potatoConsumer).accept(Potato())
//verify(potatoConsumer).accept(any()) //-> This fails too with the same reason
}
}
data class Potato(val value: Int = 1)
class PotatoConsumer : Consumer<Potato> {
override fun accept(t: Potato?) {
println(t)
}
}
So I making subscribe with this mock(potatoConsumer), and the rxJava have called 'accept', and mockito mark it as interaction, but mockito thinks this interaction is not what I'm expecting, why?
Versions of libraries used her:
mockitoVersion = '2.8.9'
mockitoAndroidVersion = '2.7.22'
powerMockVersion="2.0.2"
kotlinMockito="2.1.0"
rxKotlin = "2.3.0"
rxJavaVersion = "2.2.10"
Kinda workaround
Some fields mocked by powermock, fails on 'verify', but fields mocked with mockito is not;
Mockito can't mock not opened fields, without mock-maker-inline, but mockito conflicts with Powermock mock-maker-inline;
Powermock can delegate calls of mock-maker-inline to other mock-maker-inline(https://github.com/powermock/powermock/wiki/PowerMock-Configuration)
Use Mockito.mock on the failed fields instead of #Mock/Powermock mock injection
Example of the "green" potato test method using PowerMockRunner
#Test
fun potato() {
potatoConsumer = mock() // <-
Observable.just(Potato()).subscribe(potatoConsumer)
verify(potatoConsumer).accept(potato)
}

I am not familiar with PowerMock but I tried this test and it passes:
#Test
fun potato() {
fakePotatoProvider = Mockito.mock(PotatoProvider::class.java)
potatoConsumer = Mockito.mock(PotatoConsumer::class.java)
`when`(fakePotatoProvider.getObservable()).thenReturn(Observable.just(Potato()))
fakePotatoProvider.getObservable().subscribe(potatoConsumer)
verify(potatoConsumer).accept(Potato())
}

Maybe because you aren't passing the same instance of Potato(). Try to refactor your code to this
#Test
fun potato() {
val testPotato = Potato()
`when`(fakePotatoProvider.getObservable()).thenReturn(Observable.just(testPotato))
fakePotatoProvider.getObservable().subscribe(potatoConsumer)
verify(potatoConsumer).accept(testPotato)
}
As I mentioned above, the reason why it might be failing is the constant creation of new instances when passing your Potato object, hance that comparison fails.

Flakiness in tests on Android using LiveData, RxJava/RxKotlin and Spek

Setup:
In our project (at work - I cannot post real code), we have implemented clean MVVM. Views communicate with ViewModels via LiveData. ViewModel hosts two kinds of use cases: 'action use cases' to do something, and 'state updater use cases'. Backward communication is asynchronous (in terms of action reaction). It's not like an API call where you get the result from the call. It's BLE, so after writing the characteristic there will be a notification characteristic we listen to. So we use a lot of Rx to update the state. It's in Kotlin.
ViewModel:
#PerFragment
class SomeViewModel #Inject constructor(private val someActionUseCase: SomeActionUseCase,
someUpdateStateUseCase: SomeUpdateStateUseCase) : ViewModel() {
private val someState = MutableLiveData<SomeState>()
private val stateSubscription: Disposable
// region Lifecycle
init {
stateSubscription = someUpdateStateUseCase.state()
.subscribeIoObserveMain() // extension function
.subscribe { newState ->
someState.value = newState
})
}
override fun onCleared() {
stateSubscription.dispose()
super.onCleared()
}
// endregion
// region Public Functions
fun someState() = someState
fun someAction(someValue: Boolean) {
val someNewValue = if (someValue) "This" else "That"
someActionUseCase.someAction(someNewValue)
}
// endregion
}
Update state use case:
#Singleton
class UpdateSomeStateUseCase #Inject constructor(
private var state: SomeState = initialState) {
private val statePublisher: PublishProcessor<SomeState> =
PublishProcessor.create()
fun update(state: SomeState) {
this.state = state
statePublisher.onNext(state)
}
fun state(): Observable<SomeState> = statePublisher.toObservable()
.startWith(state)
}
We are using Spek for unit tests.
#RunWith(JUnitPlatform::class)
class SomeViewModelTest : SubjectSpek<SomeViewModel>({
setRxSchedulersTrampolineOnMain()
var mockSomeActionUseCase = mock<SomeActionUseCase>()
var mockSomeUpdateStateUseCase = mock<SomeUpdateStateUseCase>()
var liveState = MutableLiveData<SomeState>()
val initialState = SomeState(initialValue)
val newState = SomeState(newValue)
val behaviorSubject = BehaviorSubject.createDefault(initialState)
subject {
mockSomeActionUseCase = mock()
mockSomeUpdateStateUseCase = mock()
whenever(mockSomeUpdateStateUseCase.state()).thenReturn(behaviorSubject)
SomeViewModel(mockSomeActionUseCase, mockSomeUpdateStateUseCase).apply {
liveState = state() as MutableLiveData<SomeState>
}
}
beforeGroup { setTestRxAndLiveData() }
afterGroup { resetTestRxAndLiveData() }
context("some screen") {
given("the action to open the screen") {
on("screen opened") {
subject
behaviorSubject.startWith(initialState)
it("displays the initial state") {
assertEquals(liveState.value, initialState)
}
}
}
given("some setup") {
on("some action") {
it("does something") {
subject.doSomething(someValue)
verify(mockSomeUpdateStateUseCase).someAction(someOtherValue)
}
}
on("action updating the state") {
it("displays new state") {
behaviorSubject.onNext(newState)
assertEquals(liveState.value, newState)
}
}
}
}
}
At first we were using an Observable instead of the BehaviorSubject:
var observable = Observable.just(initialState)
...
whenever(mockSomeUpdateStateUseCase.state()).thenReturn(observable)
...
observable = Observable.just(newState)
assertEquals(liveState.value, newState)
instead of the:
val behaviorSubject = BehaviorSubject.createDefault(initialState)
...
whenever(mockSomeUpdateStateUseCase.state()).thenReturn(behaviorSubject)
...
behaviorSubject.onNext(newState)
assertEquals(liveState.value, newState)
but the unit test were being flaky. Mostly they would pass (always when ran in isolation), but sometime they would fail when running the whole suit. Thinking it is to do with asynchronous nature of the Rx we moved to BehaviourSubject to be able to control when the onNext() happens. Test are now passing when we run them from AndroidStudio on the local machine, but they are still flaky on the build machine. Restarting the build often makes them pass.
The tests which fail are always the ones where we assert the value of LiveData. So the suspects are LiveData, Rx, Spek or their combination.
Question: Did anyone have similar experiences writing unit tests with LiveData, using Spek or maybe Rx, and did you find ways to write them which solve these flakiness issues?
....................
Helper and extension functions used:
fun instantTaskExecutorRuleStart() =
ArchTaskExecutor.getInstance().setDelegate(object : TaskExecutor() {
override fun executeOnDiskIO(runnable: Runnable) {
runnable.run()
}
override fun isMainThread(): Boolean {
return true
}
override fun postToMainThread(runnable: Runnable) {
runnable.run()
}
})
fun instantTaskExecutorRuleFinish() = ArchTaskExecutor.getInstance().setDelegate(null)
fun setRxSchedulersTrampolineOnMain() = RxAndroidPlugins.setInitMainThreadSchedulerHandler { Schedulers.trampoline() }
fun setTestRxAndLiveData() {
setRxSchedulersTrampolineOnMain()
instantTaskExecutorRuleStart()
}
fun resetTestRxAndLiveData() {
RxAndroidPlugins.reset()
instantTaskExecutorRuleFinish()
}
fun <T> Observable<T>.subscribeIoObserveMain(): Observable<T> =
subscribeOnIoThread().observeOnMainThread()
fun <T> Observable<T>.subscribeOnIoThread(): Observable<T> = subscribeOn(Schedulers.io())
fun <T> Observable<T>.observeOnMainThread(): Observable<T> =
observeOn(AndroidSchedulers.mainThread())

I didn't used Speck for unit-testing. I've used java unit-test platform and it works perfect with Rx & LiveData, but you have to keep in mind one thing. Rx & LiveData are async and you can't do something like someObserver.subscribe{}, someObserver.doSmth{}, assert{} this will work sometimes but it's not the correct way to do it.
For Rx there's TestObservers for observing Rx events. Something like:
#Test
public void testMethod() {
TestObserver<SomeObject> observer = new TestObserver()
someClass.doSomethingThatReturnsObserver().subscribe(observer)
observer.assertError(...)
// or
observer.awaitTerminalEvent(1, TimeUnit.SECONDS)
observer.assertValue(somethingReturnedForOnNext)
}
For LiveData also, you'll have to use CountDownLatch to wait for LiveData execution. Something like this:
#Test
public void someLiveDataTest() {
CountDownLatch latch = new CountDownLatch(1); // if you want to check one time exec
somethingTahtReturnsLiveData.observeForever(params -> {
/// you can take the params value here
latch.countDown();
}
//trigger live data here
....
latch.await(1, TimeUnit.SECONDS)
assert(...)
}
Using this approach your test should run ok in any order on any machine. Also the wait time for latch & terminal event should be as low as possible, the tests should run fast.
Note1: The code is in JAVA but you can change it easily in kotlin.
Note2: Singleton are the biggest enemy of unit-testing ;). (With static methods by their side).

The issue is not with LiveData; it is the more common problem - singletons. Here the Update...StateUseCases had to be singletons; otherwise if observers got a different instance they would have a different PublishProcessor and would not get what was published.
There is a test for each Update...StateUseCases and there is a test for each ViewModel into which Update...StateUseCases is injected (well indirectly via the ...StateObserver).
The state exists within the Update...StateUseCases, and since it is a singleton, it gets changed in both tests and they use the same instance becoming dependent on each other.
Firstly try to avoid using singletons if possible.
If not, reset the state after each test group.