A basic Flutter app failed to launch on Firebase Test Lab for Android. I can see in the video that the app never came up so it never even tried running my tests, it just timed out. In the logs, I do not see any errors from my application; there are errors and warnings but I can't understand any of them nor do I know if they're ok to ignore. I used Firebase Test Lab on other projects and I'm accustomed to seeing lots of errors which I assume come from Firebase Test Lab's infrastructure. I typically look through the errors to try to find something that might pertain to my app. In this case I don't see anything that seems to be related to my app.
The app is from the code lab https://codelabs.developers.google.com/codelabs/flutter-codelab-first#0 .
The instructions for Firebase setup are at https://github.com/flutter/flutter/tree/main/packages/integration_test#firebase-test-lab .
The test passes when run locally by running the following command on my Mac:
./gradlew app:connectedAndroidTest -Ptarget=`pwd`/../integration_test/app_test.dart
The command I run from my CI (gitlab) is as follows:
gcloud firebase test android run --type instrumentation --app build/app/outputs/apk/debug/app-debug.apk --test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk
I got the same timeout by initiating the test through the Firebase Test Lab UI.
I also posted this question at https://firebase-community.slack.com/archives/C1MTSQ5QT/p1674438388007639 .
Any ideas for what I could investigate?
I was able to get Firebase Test Lab to test a Flutter App by using this Espresso "bridge" https://pub.dev/packages/espresso. So instead of writing Flutter tests with the integrationDriver from package:integration_test/integration_test_driver.dart as described in https://github.com/flutter/flutter/tree/main/packages/integration_test#driver-entrypoint, I can use the enableFlutterDriverExtension() from package:flutter_driver/driver_extension.dart .
Not ideal.
Related
I followed this tutorial to create tests in android studio: https://developer.android.com/training/testing/unit-testing/instrumented-unit-tests#run-ctl. I successfully ran those tests on my physical device. Then I created the test run configuration for test lab, also followed the same tutorial. When I want to run this configuration I am receiving the following error:
What am I doing wrong? Why am I not authorised for my own project? I was able to add my firebase project normally. I am using only free spark billing in Firebase.
When you prepare your test run configuration, make sure you select your Firebase project (the one you created and to which you have access) from the list of all cloud projects you see. Also, if you created your project after your login in Android Studio, it might help to log out and re-login (using the icon in the top right corner of Android Studio).
I have setup a project in Firebase to run Instrumented Test for my app. I created the project from Android Studio, then I went to the Firebase Console and moved the project to Blaze Plan. I logged in and setup the project in Android Studio, so far so good, the thing is that when I try to run the test in Firebase using the Sample Spark Configuration but Android Studio never finishes to upload the APK, I don't have any error neither, what I get in Android Studio RUN console are the following lines:
Testing started at 11:29 ...
Using Cloud Storage Bucket location test-bucket-id
Uploading app APK ...
So, it seems there are some issue witht the bucket or with the project, but I'm not able to read any error anywhere. So, where can I access any log to know what is going on? Does a firebase project expose logs though its web UI or does Android Studio allows me to get some kind of logs besides what I get on the build window?
I had the same issue. Turned out there wasn't anything wrong with the project or the bucket. All I had to do was specify the package of my instrumented tests instead of using the "all in module" option in the run configuration.
I am using Firebase test labs along with Jenkins to run a Robo test after each build of my Android app. Currently I use a Yaml file to setup all the configurations I need. I supply a username and password using the Robo Directives. However, I now want to supply a Robo Script that controls the sign in process a little bit more than just supplying the login details. I have made the script in Android studio, but I don't see anywhere in the documentation that allows me to supply the location of the Robo Script from the Yaml file or command line, does anyone know if this is possible?
I have tried the documentation and various links from here - https://firebase.google.com/docs/test-lab/command-line
This feature is available today in beta, but not as a full release. You can use the --robo-script flag to specify a script to upload when using
gcloud beta firebase test android run
It is not available in the release version:
gcloud firebase test android run
After some time in beta, new features will be promoted to full release. See the CLI docs for the beta version here.
I know this is an old question but I was looking for the same thing and I find it :
https://cloud.google.com/sdk/gcloud/reference/beta/firebase/test/android/run#--robo-script
Using like this
gcloud firebase test android run --app app-release.apk --robo-script script.json
I have googled everything so far, but I couldn’t find anything how to record a Robo Test of you Android Ionic application. With the recorded test I can submit it to Firebase Test Lab to run it over multiple devices.
I came this far:
I opened my project in Android Studio
I opened Test Lab settings via Tools > Firebase > Test Lab
Then when I try to record a Robo script, it shows only one event: ‘Delay 60000 milliseconds’
Does anyone have a clue on how to do this?
Test Lab currently only supports native Android apps. Other frameworks, such as Ionic, are not supported.
I tried to test an app with Firebase Test lab but get an error saying "testOnly APKs are not allowed."
C:\temp>gcloud firebase test android run --type instrumentation --app myapp.apk --test myapp-androidTest.apk --device model=Nexus10,version=22,locale=en,orientation=landscape --timeout 300s
Have questions, feedback, or issues? Get support by visiting:
https://firebase.google.com/support/
Uploading [myapp.apk] to Firebase Test Lab...
Uploading [myapp-androidTest.apk] to Firebase Test Lab...
Raw results will be stored in your GCS bucket at [https://console.developers.google.com/storage/browser/test-lab-j9zwyqscmy0rw-k53tazzivjxvu/2017-10-16_11:56:43.691000_AcnJ/]
Test [matrix-1vxb29yr0b2z7] has been created in the Google Cloud.
Firebase Test Lab will execute your instrumentation test on 1 device(s).
Creating individual test executions...failed.
ERROR: (gcloud.firebase.test.android.run)
Matrix [matrix-1vxb29yr0b2z7] failed during validation: "testOnly" found in the Manifest. testOnly APKs are not allowed.
C:\temp>
But the firebase test works when I run it in Android Studio 3.0 RC1 by setting the deployment target options to "Firebase Test Lab Device Matrix".
I searched "testOnly" in my project but not found it. It looks like it was gradle injected the attribute into AndroidManifest.xml.
Does anyone know how to resolve it?
I have the same probllem. I finally resolved by generating the apk with 'Build > Build APK(s)'
The APk is generated at : app\build\outputs\apk\debug
The apk is generated witouth the 'testOnly' label and it runs on firebase.
I found this info on https://developer.android.com/studio/run/index.html
It was Android Studio added the field. Refer to the document developer.android.com/guide/topics/manifest/… "Android Studio automatically adds this attribute when you click Run". Not using Android Studio, I built an APK from the command line then I can upload it to Firebase Test Lab for testing.
When you use the instant run option and the APK generated in this fashion cannot be used for any kind of firebase test.
Instead select the build option above and then select the build APK option and the the APK generated in this fashion can be used for the firebase test.