So I want to build my tests and app, and then run the instrumentation tests whenever I want, even when I do not have access to the code (and hence, cannot use cAT task)
I have tried using assembleAndroidTest to generate test apk. But when I install this apk (as well as the app apk) and run am instrumentation with correct arguments (-e emma true, and with correct runner class verified using pm list instrumentation), I get no .ec file under /data/app/<package-name>. There is no error however.
I am using JaCoCo and when I run the tests using cAT, the coverage report is generated so I guess my JaCoCo configs are Ok.
Related
If I:
Create a brand-new Android Studio 3.5.1 project (Kotlin, API 21, "Empty Activity" template)
Run the app from inside the IDE
Confirm the app is installed and has a launcher icon
Run the connectedAndroidDebugTest Gradle task (from inside Android Studio or via gradlew)
The app winds up being uninstalled by the test run. I get that behavior even if I add a testApplicationId value to defaultConfig to have the test code use a different application ID.
How do I stop that behavior? How can I run instrumented tests from the command line, without disturbing an existing app installation?
The connectedCheck task has the type DeviceProviderInstrumentTestTask. For a simple test run on one device it uses a SimpleTestRunner, which in
turn uses a SimpleTestRunnable to actually execute the test. Here you find a structure of
try {
// connect to device
// install all APKs
// run tests
} catch(Exception e) {
// handle error
} finally {
// get test report
// uninstall all APKs
// disconnect from device
}
I'm not perfectly sure if I've found the most recent implementations, but this exact behavior dates back several years. So I guess you can't achieve what you're asking for.
Maybe try to run it via adb like this:
adb shell am instrument -w com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner
It will not uninstall your app.
here it is described in more details.
The instrumentation installs 2 APKs: the APK under test and the APK with the test code.
It also uninstalls both APKs before it tries to install the new ones and I don't know if it's possible to prevent the uninstall itself.
testApplicationId changes only application id for the APK with the test code (which is normally the same as for the main APK with ".test" appended) the application id of the APK under test remains still the same. But it is possible to create separate buildType for the APK under test (with exactly the same configuration as the debug build type) and use that.
Then connectedAndroidXYZTest could be used to run the tests (or createXYZCoverageReport).
I tried to test an app with Firebase Test lab but get an error saying "testOnly APKs are not allowed."
C:\temp>gcloud firebase test android run --type instrumentation --app myapp.apk --test myapp-androidTest.apk --device model=Nexus10,version=22,locale=en,orientation=landscape --timeout 300s
Have questions, feedback, or issues? Get support by visiting:
https://firebase.google.com/support/
Uploading [myapp.apk] to Firebase Test Lab...
Uploading [myapp-androidTest.apk] to Firebase Test Lab...
Raw results will be stored in your GCS bucket at [https://console.developers.google.com/storage/browser/test-lab-j9zwyqscmy0rw-k53tazzivjxvu/2017-10-16_11:56:43.691000_AcnJ/]
Test [matrix-1vxb29yr0b2z7] has been created in the Google Cloud.
Firebase Test Lab will execute your instrumentation test on 1 device(s).
Creating individual test executions...failed.
ERROR: (gcloud.firebase.test.android.run)
Matrix [matrix-1vxb29yr0b2z7] failed during validation: "testOnly" found in the Manifest. testOnly APKs are not allowed.
C:\temp>
But the firebase test works when I run it in Android Studio 3.0 RC1 by setting the deployment target options to "Firebase Test Lab Device Matrix".
I searched "testOnly" in my project but not found it. It looks like it was gradle injected the attribute into AndroidManifest.xml.
Does anyone know how to resolve it?
I have the same probllem. I finally resolved by generating the apk with 'Build > Build APK(s)'
The APk is generated at : app\build\outputs\apk\debug
The apk is generated witouth the 'testOnly' label and it runs on firebase.
I found this info on https://developer.android.com/studio/run/index.html
It was Android Studio added the field. Refer to the document developer.android.com/guide/topics/manifest/… "Android Studio automatically adds this attribute when you click Run". Not using Android Studio, I built an APK from the command line then I can upload it to Firebase Test Lab for testing.
When you use the instant run option and the APK generated in this fashion cannot be used for any kind of firebase test.
Instead select the build option above and then select the build APK option and the the APK generated in this fashion can be used for the firebase test.
I have made a library project consisting of an Android module and no application. This module relies on Android-specific functionality.
How do I run these tests with Firebase?
It seems like Firebase requires an app apk and a test apk, while I only have a test apk (*-androidTest.apk). I normally run tests locally without issues using ./gradlew connectedCheck.
I have a web app that is using Firebase FCM to communicate with my android device.
On receiving a certain data message through Firebase, I need android to run Instrumentation tests (such tests are run in a separate test application or "test APK" which is developed in the androidTest folder).
How can I make an Intent that will launch the test APK?
You can't run instrumentation tests in response to an external stimulus like this. Instrumentation tests are always driven through a JUnit test harness, which requires yet some other controlling process to kick them off using the am instrument command line program on the device. This is typically Android Studio, which will install both your app APK and your test APK, then kick off the tests. Firebase Test Lab will also take both APKs kick off your tests on physical and virtual devices.
I'm investigating a cloud-based solution for our UIAutomator 2.0 testing needs, and I'm having quite a few issues with uploading APKs for our test project.
We have a stand-alone test project that exists separately from the main Android application project. The package for the project is in the format com.company.project and the package for the test project is com.company.project.test.
1) If I attempt to upload a test APK that has all the tests in androidTest (and the gradle uses androidTestCompile for dependencies), Firebase console complains that:
"Unable to find instrumentation package for com.company.project.test"
This is likely because our test project is standalone, and the tests actually live under com.company.project.test.test.
2) On the other hand, if I change the location of tests to java main folder in the project and change androidTestCompile to compile for the gradle dependencies, when I attempt to upload the test APK Firebase console complains again:
"We experienced an error while validating your APK. Please verify the APK is correct and upload again."
I can't get a single test running since I'm blocked both ways.
Anyone else attempt to use Firebase Test Lab with a similar project hierarchy? Help!
Finally figured it out. Don't even bother with the web UI, use the gcloud terminal to run your tests instead. It's WAY more customizable.
https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run
You can specify the --test-package to whatever your test package is, whether it's a standalone project or not.