When I run android CTS full test using below command
run cts --plan CTS
Every time it shows different result for some of the packages, I mean some packages some tests passes/fails randomly every time I re-run full test. But when I run package individually (The package in which some tests failed), all the tests passes in it.
Why I am seeing this behavior?
Environment:
OS: Android L
CTS version: 5.1_r7
It happen some time some test failed randomly because some time that test condition is satisfied some time not and some time because of timeout test may fail.
Some cts tests involve specific timeouts set for some event to occur. For example if you are running cts test related to data calls like turning mobile data off/on and timeout to get mobile data connected is set to 10 seconds, then some time this test will pass and sometime it will fail. In this case, increasing that timeout will resolve this issue.
Regarding issue of test case failing when running multiple packages, there could be possibility that test case before failed one has not set device in a neutral/original state for next test. It is a good practice to revert all changes made during a test while exiting a test case.
Related
I had an issue to run my ui test
#Test
fun firstUi() {
onView(withId(R.id.home_profile_tab)).perform(click())
onView(withId(R.id.enter)).perform(click())
onView(withId(R.id.tvCountryCode)).check(matches(withText("+964")))
}
This test run and passed
But the issue is, after running test and reaching to first line, the firs perform(click)) executed after around 90 seconds, and it is almost constant and every time it takes 90 seconds
But after that (90sec) other lines executed and test completed around 4 seconds and passed successfully
Base on android documentation:
Each time your test invokes onView(), Espresso waits to perform the
corresponding UI action or assertion until the following
synchronization conditions are met:
The message queue is empty.
There are no instances of AsyncTask currently executing a task.
All developer-defined idling resources are
idle.
So how and where can I investigate more to detect to root cause of issue???
Or what I'm doing wrong???
With the help of this post I found what was my issue
I live in iran and most of services because on sanctions are banned here
So I looked in Frames, so I found that some AsyncTask are exist for some sdk, that are trying to send request to their server, but because of ban they couldn't
So they keep retrying, maybe after 90sec without a successful request call they give up, and Espresso requirements meet to continue running tests
My Solution was using VPN, so sdk can successfully make request to their servers
My test suite runs fine on iOS, the app has many background processes running that prevent large portions of the tests from working in synchronized mode. The desynced commands run properly on the iOS simulator, but when I run them on the android emulator it is as if I never called await device.disableSynchronization().
The tests still hang and the console logs:
The app is busy, due to:
- Enqueued timers
- Animations running on screen
Any ideas about how to fix this?
Further review shows the error message:
The app has not responded to the network requests below:
The odd thing is the listed network request has been completed. The request upon which the app purportedly has not responded is a button push that navigates to a new screen which incurs a somewhat lengthy network request. After the network request completes other processes continually run necessitating the disabled synchronization. Since Detox is still waiting on some response from the app about the button tap, it does not move on to the next, desynchronized, actions. Is there any way to ensure that Detox receives the response of this .tap()?
A deeper investigation reveals that this is likely caused by a lingering animation that, for whatever reason device.disablesynchronization() ignores on Android builds. I am now working on mocking this file based on the out-of-date documentation provided by wix.
I am running android-cts . The commands I run are mentioned below,
cts-tf
run-cts --plan --cts-camera
It is being running for two days . How do I stop the current task and save the existing logs .
Also it is mentioned in documentation ,that the cts logs will be stored in
CTS_ROOT/android-cts/results/start_time.zip
But I dont see a start_time.zip in the location specified .
There are few ways by which you can stop the CTS invocation and result will be generated of your runs,
Unplug the USB Cables from the devices, this will make tradefed to not detect any device and once timeout occurs, it will generate result on testcases and modules it ran.
Kill command, just write the kill in the tradefed, and it will kill and stop the invocation threads for the runs. once that is done, it will generate the result but make sure not to give kill command more than 1 or else it will exit from tradefed without generating the result.
I have written code to run tasks on job scheduler. Testing it on real device works fine. But i wanted a way to unit test it so that i can be sure it is covering all conditions like job success/failure/ runs when conditions are met, periodic etc. Please suggest me a way to write unit tests for these scenarios
You can run jobs on-demand via adb, as well as force an immediate timeout of an executing job as though it had reached the end of its timeslice. See the output of
adb shell cmd jobscheduler
for a summary of useful commands.
I have a problem with not working tests on Jenkins - they are taking too much time.
Jenkins run Project with more than 200 tests, some of them fails or hang out. How can I set timeout for single test and abort it and go to next test?
I assume that:
[31mFAILED [0m
junit.framework.AssertionFailedError...]]
means 31m need to pass to set test to failure
You might be able to define a timeout for your test class: https://github.com/junit-team/junit/wiki/Timeout-for-tests
Beside that you should investigate the reason why your test takes ages to fail/succeed and fix/improve it.