My company will be releasing a communication app for Android in the coming weeks/months. It is an embedded app on a Nexus 7 2013 device, running AOSP with some minor modifications. This makes typical remote updates complicated as there's an extra step we must run to install the APK/libs into the OS system folders.
The plan is to do Alpha and Beta trials, and we have started gathering volunteers. We'll be loading the APK onto devices and giving them out initially. I've found plenty of information on generally preparing for release (unit testing, signing your APK, etc) but I'm looking for information on Alpha and Beta releases.
Problems:
We need to update the app remotely without bringing the devices in
We need to gather as much (useful) data as possible on usage, device statistics, bugs and crashes
Research & Possible Solutions:
Developer Console Alpha/Beta testing - https://developer.android.com/distribute/googleplay/developer-console.html
App Blade or other third party support tools - https://appblade.com
Analytics libraries
Flurry - https://developer.yahoo.com/analytics
Mixpanel - https://mixpanel.com/android-analytics
Google Analytics - https://developers.google.com/analytics/devguides/collection/android/v4/
Amplitude - https://amplitude.com/behavioral-analytics-platform
Localytics ($) - https://www.localytics.com
Fetch logcat programmatically on the device and upload to our server
Save specific in-app logs/statistics to internal SharedPreferences or SQLite database and upload to our server periodically
Gather written feedback from Alpha/Beta testers via surveys, email communication, etc.
If the device is in hand, we can debug crashes by:
Dump logcat to file - "adb logcat -d > logcat.txt"
Check kernel logs - /proc/last_kmsg
Check dump - /data/dont_panic
Questions:
Have you run an Alpha/Beta trial using Google Play Developer Console? Was it useful?
Have you used any third party tools such as App Blade to push releases and gather statistics?
What are the best practices for gathering detailed statistics/feedback from your apps once they've been released?
Thanks in advance
Alpha/Beta tests are extremely useful. You can create google+ communities for Alpha and beta testers. Then add those community for sharing the app with specified users. A new version can be easily deployed and notified to alpha and beta users.
Nups
Multiple ways actually: Write your own beacon kind of system to gather information from device and send those using Alarm Manager periodically. For storing logs, sql should be used.
Yes. I've tried running a private Alpha and a public beta using the dev console and G+ groups. This is really useful if the testers are used to g+ and how it works as we gathered feedback there. Also this is the only way that I know of to seamlessly "push" the updates to the users. The only downside is that the dev console lacks an api so you can't upload builds automatically from a ci-server.
Tried Test Flight before it was bought by Apple but didn't use it for feedback, only distribution. In the most simple cases I've used Dropbox.
I would recommend the the following feedback "stack":
Crashlytics for exception reports (they also do beta distribution now but haven't tried it yet) - http://crashlytics.com
Google analytics for metrics like time spent on certain parts, load times etc
G+ groups for direct feedback
Related
In the past we were able to deploy private versions of our app to EMM's (e.g. VMWare, MobileIron) and test out managed configurations. But today, we are unable to test new app updates within a managed environment.
Android https://developer.android.com/work/managed-configurations
With new Google updates EMMs are no longer able to upload private versions of our app if the app package id conflicts with a publicly available app on Google Play. For regulatory reasons we are unable to just change the package id and test because it is technically not testing the same binaries. Best we can do now is simulate a managed environment using Test DCP : https://play.google.com/store/apps/details?id=com.afwsamples.testdpc
Is this the best we can test without publicly releasing the app update to Google Play? We have contacted VMWare and basically got the same answer but would like a confirmation. Uploading the app to a closed testing track on Google Play and then trying to importing to EMM did not work either.
iOS https://www.appconfig.org/ios/
Basically the same issue for iOS. Apple has kind of removed the Enterprise Developer Account which we previously used to sign and upload our own versions to EMM. The new eligibility requirements are too much. Alternatively, none of the EMMs work with TestFlight. And for iOS we do not know of any app like Android Test DCP to simulate a managed environment. I read a few github chains and Apple forums where companies are just releasing the app publicly then testing to make sure everything works. That can't be right, right?
In the Huawei Store, we could publish an app upgrade via "Version/Upgrade" menu. But once we publish it will directly appear in the store.
Is there any way to publish an app on the Huawei developer console for testing like Beta/Alpha texting in the Google Play console?
Yes. AppGallery Console provides Open testing feature for developers to publish their APK to selected users before releasing it for all. The Opening Testing service is still in beta state. To launch an open test, you should send an email for the application. You can follow the this official link to apply for an Open Test.
You can try HMS Cloud Testing which provides a complete set of automatic test processes based on real mobile phone use. It tests automatically the compatibility, stability, performance, and power consumption of Android apps, without manual intervention.
Please refer the following articles for details.
A Novice Journey towards Cloud Testing - Part I
A Novice Journey towards Cloud Testing - Part II
Cloud Testing: - Variety of tests on your Android App
My app is now in the alpha status and I would like to distribute it to a few people (QA, alpha testers...) without making the app public. I know, that google play offers a private channel to distribute apps in alpha/beta state to some people, but according to the website it takes a few hours until the application is available. However, I would like to have it available for users as soon as a new build is ready and uploaded. I thought about setting up a private android repository (e.q f-droid), but unfortunately I could not find anything in the documentation about securing the android repository - I want it to be protected somehow (key, password...).
Another possibility would be to set up a web-server and host the different build versions on that server. Then the testers have to navigate to that site and download the appropriate build version. However, from the usability point of view it isn't that nice - user have to download the version manually and install it afterwards. Also for the QA guys it would be nicer, if it is easier to switch between different android builds.
How would you solve this problem? Is there already any software out there, that is suitable for that task?
Any help is really appreciated.
I know Google says it can take a few hours, in my experience, it happens very quickly, a few minutes at most. I don't know if there are times when it takes longer.
I thought about setting up a private android repository (e.q f-droid), but unfortunately I could not find anything in the documentation about securing the android repository - I want it to be protected somehow (key, password...).
While perhaps not in the docs, F-Droid does support HTTP Authentication on its repositories. For example, if using Apache you can add a relevant .htpasswd and .htaccess file in the webroot. This will result in the webserver sending back a 401 and asking the client to authenticate. F-Droid responds by prompting with a username/password dialog.
I would suggest using Crashlytics. Crashlytics is now integrated in Fabric.
Crashlytics allows you to distrubute beta versions of your app. Testers only have to install the install the Crashlytics app, and enable installing apps from different sources. You specify which users can test the app by emailadress.
The app allows testers to switch between any version of your app, and install any possible other apps they are invited for.
You can also integrate Crashlytics in your app. If you do you get notified of errors that occured complete with stacktraces, and additional information about the device. This makes bugfixing easier.
We are adding Android Auto and iOS CarPlay support to the existing Android/iOS versions of an app. We are able to successfully test the Auto application using the Android Media Browser simulator as directed by the Android developer documentation.
We also have a stereo head unit that supports both Auto and CarPlay. We are able to use the CarPlay app successfully on the head unit, and we are able to use published Auto apps on the head unit. However, we can't see our development app on the actual device.
The Auto documentation is still a little bit thin, but I'm gathering based on some wording I've seen that Auto apps get some special flag (or similar) added by Google Play when they pass review:
Before making the app available to Android Auto users, Google Play
submits your app for review against the Auto App Quality criteria and
notifies you of the result. If your app is approved, Google Play makes
that app available to Android Auto users.
Based on this, is at all possible to run Auto apps on hardware before they've already been published and approved through Google Play?
This seems like a frustrating chicken-and-egg problem. We'd like to have the confidence that things look good on actual hardware and on target devices before publishing.
It is now possible to test your Android Auto apps on Auto-enabled head units. The procedure is to upload your app to the Play Store in an alpha channel, which you can then install to your device and test in a car. You will even receive feedback from the Auto review team for your app. [Wayne Piekarski]
Follow this
You can also install the Desktop Head Unit (DHU) to test it in software before submitting it to the Play Store. I used this method to make sure most of the quality issues were resolved with my app before submitting it. The DHU does not require the apk to be signed by Google.
Also, submitting it through the store usually takes several hours before you can test. The DHU is, obviously, immediate feedback.
Here's the link: https://developer.android.com/training/auto/testing/index.html
The short answer is no, you can not. It is due to the driver safety review. It will not be able to run on the real device until the app is approved.
But I think, you can contact Google and they can do something about it, if you really need to test it in your car.
I am going to have a alpha release of my app, which is not yet on the market. I want to have the app APK link sent out to friends via email so they can download it from my site CDN.
One question here: if I want to give them updates, what will be a good way? Can I download the new APK within the app, and somehow install the APK to replace the old one without anything to do with the market? So my friends can have the app upgraded while it is still in alpha release?
When I did it, I used Zubhium -- they were a web service with a small API that you could install into your app, giving you a mini "app store"-style backend and handling distribution for you. It would host and distribute your APK, connect up to their server when the app launched, check for updates, invalidate old versions, gather crash logs for you, etc. It was very good.
Zubhium are now https://www.vessel.io -- I presume they still have the above features as part of their now-much-bigger service, but when I checked you had to give them a credit card number even to sign up for the free plan, so I've not played with it.
A friend of mine uses http://testflightapp.com for iOS, and it looks like they have an Android version now, so that's certainly worth checking out. A quick search also shows up http://applover.me. #Janusz recommends http://hockeyapp.net/features in his comment.
As #Nanne points out in his answer, the Play Store itself now lets you distribute to limited alpha- and beta-test groups. That looks like it has fairly minimal features compared to the third-party services (no A/B testing, etc.), but will be familiar and free. And it doesn't need an extra SDK rolled into your app.
So, my general answer is that there's more than one professional beta-testing API/service that you could use, that they're generally very useful, quite easy to roll into your app, solve all the problems you're anticipating and more, and often have a free plan to get started. I'd recommend picking one of them rather than trying to roll your own solution.
If you want this only to be able to release your app in Alpha, and maybe later in beta, take a look at the android market again.
Check out this link: https://support.google.com/googleplay/android-developer/answer/3131213?hl=en
It boils down to the fact that you can have an Alpha-test, and a beta test, each with selected users. You can upload your app as normal, so you'll have updates via the market, but not everyone can download your app.
For the beta at least, you can select a community that is the source of your users, so all that are in that community could be testers.
This is the best method for testing I believe.