I am working on an Android app that I want to run on all Android devices without any problems, but I'm having a hard time trying to determine how many device configurations and how many hardware devices I should have for testing.
There are several features that I am using which are not supported in the emulator, as mentioned at the bottom of Using the Android Emulator in the Dev Guide:
Emulator Limitations
In this release, the limitations of the emulator include:
No support for placing or receiving actual phone calls. You can
simulate phone calls (placed and received) through the emulator
console, however.
No support for USB connections No support for camera/video capture
(input).
No support for device-attached headphones No support for
determining connected state
No support for determining battery charge level and AC charging
state
No support for determining SD card insert/eject
No support for Bluetooth
The Multiple APK Support page states:
...we encourage you to develop and publish a single APK that supports as many device configurations as possible...
Android Market allows you to publish multiple APKs under the same
application listing. Android Market then supplies each APK to the
appropriate devices based on configuration support you've declared in
the manifest file of each APK.
I would assume that it's not necessary to cover every single permutation of all the available feature sets, but it's not clear to me how many device configurations would be considered sufficient to cover all the devices available.
If I wanted to confirm on hardware that my app is almost guaranteed to work on every device, how many phones and how many tablets would I need? Also, are there any special considerations I need to make for manufacturer- or carrier-specific modifications?
Answer Updated on 11/15/2011
Use android website stats to glean info on the most popular os version(s).
Use android website stats to get the most popular DPI/screensize combination(s).
Use a combination of 1&2 to determine the specs of devices you want to cover.
Prepare a rough listing of devices which would meet (3) above.
If you have A lot of money, -or- if you work for a big company then order the phones. If not, then as others have already mentioned choose the one device which can provide you with max coverage. Sadly, the difference between the emulator and devices can be ridiculous at times. The other option that i found out only recently is that you can outsource this process to companies which exclusively focus on this market need. Unfortunately, I don't have any specific company to recommend in this area. Good luck !
Updates:
I came across a couple of companies that are offering Android Testing as a Service. These would be good options to test your APKs without having to purchase the devices.
You may want to give APKUDO a try: http://www.apkudo.com/
"
See your app run on every Android device. Free.
Apkudo helps developers release with confidence by installing, executing, and exercising their apps on every Android™ device, with detailed test reports & streaming video. Currently in private beta.
"
[Update - 3/25/2013]
TestDroid also has a cloud testing service
http://testdroid.com/product/testdroid-cloud
"
Test online on Real devices Testdroid Cloud is the simplest way to test your application against various real Android devices – from different manufacturers, with different HW platforms, OS versions and screen resolutions. You can also pick and choose on what devices to run your tests, just on the devices you are interested in
"
AppThwack is another solution
Ranorex is yet another solution (http://www.ranorex.com/mobile-automation-testing/android-test-automation.html)
One of the big problems with android is that each manufacturer may change pretty much whatever he wants in the firmware. It may have many advantages, but it might be a big problem when trying to test compatibility.
You should have at least one device, but there is no limitation, or even a number that is "enough", the more devices you test on - the better.
Regarding carrier modifications, usually those are smaller and will change less than manufacturer modifications, but this might change as well.
I have encountered many issues with many different phones for Android. Each phone has its little quirks and so the more phones you test on the better. I had an app that had well over 200K downloads and was really surprised about some of the issues. Now there are things I do and stay away from due to these quirks I discovered. Also each version of Android has its issues to. So I guess my answer is, get as many phones from at least each of the versions of OS's from say 1.6 to 3.x.
ListViews with custom Adapters is one, I created custom adapter to add icons to each item in a ListView. I had repeated crashes from 2.2 phones like DroidX to name one but on others it worked fine. It ran fine on a G1 too. Also with ListViews and SimpleExpandableListAdapter are fraught with issues on other phones where the icons don't line up or on large lists it just crashes the app. I stay away from ListView now since I want to try and keep my apps deployable across many different OS's
Pertaining to the other answer here are the list of phones I encountered for mostly US deployment. Not sure why the Samsung Galaxy S is listed 3 times or Motorola Droid II is twice, may have to do with OS versions.
1 Motorola Droid X 9.3%
2 HTC Evo 4G 6.2%
3 Samsung Galaxy S 4.9%
4 HTC Desire HD 4.4%
5 Motorola Droid 4.4%
6 HTC Droid Incredible 4.1%
7 Motorola Droid II 3.4%
8 Samsung Galaxy S 3.3%
9 Samsung Galaxy S 2.7%
10 Motorola Droid II 2.4%
This is why when I program I keep in mind to be agnostic to the OS API and just try to make sure it works for the majority of the users.
One possible way of deciding how many mobiles you'd need for testing is to look at the market using your application.
One of my applications has the following mobile phone breakdown (be aware this is in the UK). So in my region these 10 mobiles will give you approxpimately 70% coverage
1 HTC Wildfire 15.2%
2 Samsung Galaxy S2 10.5%
3 HTC Desire 10.2%
4 Samsung Galaxy S 9.7%
5 HTC Desire S 8.6%
6 HTC Desire HD 7.5%
7 Samsung Galaxy Ace 3.1%
8 Samsung Europa 2.7%
9 ZTE Blade 2.1%
10 Samsung Galaxy Tab 2.1%
Your aim should be to cover as more android users as possible for your project. Important parameters that you need to cover are by api level, ppi, screen size and resolution.
How it could be achieved:
1. Find most popular devices for your project (for example using google analytics)
2. Group those devices by parameters
3. Find most popular groups
4. Find most popular devices in each group
Such approach is realized on http://choosedevice.com.
So based on your budget you should buy devices from step 4, starting from devices from most popular group.
Related
How come Android Studio only provides Nexus phones in "Device in Editor" instead of Samsung?
I just downloaded Android Studio this week. Here's the phones available in "Device In Editor":
When I go to "Create New Virtual Device", again only Nexus Phones are available, no Samsung. What's going on?
Also, why is the PLAY STORE check only for the Nexus 5X and Nexus 5 devices. Why not any newer devices?
The purpose of the devices in 'device in editor' is not to cover every existing phone. Rather, it is to offer a spectrum of various screen sizes, densities, ratios, etc. So it really does not matter what type of device it is, as long as there is a diverse set of devices and screens offered (which there is). As for the play store thing, I am not sure.
The short answer is because Google isn't Samsung. Why aren't there LG configs in the Device Editor? Or HTC? Because there are thousands of devices and it would be impractical to manage, even if those OEMs provided the info. Fortunately, they don't need to be pre-defined in there because you can configure your own AVD with any resolution, memory config, etc, to create any device you need to emulate. More or less.
Of course, there's more to it than just defining the right resolution, etc. Many OEMs, including Samsung, roll their own OS updates, so even if you configure an AVD to match the S7, it won't really be accurate because the emulator will be running stock 7.1.2 (for example) and not Samsung's own flavor of 7.1.2. But it'll be close enough for design work and general testing.
Going back to Samsung specifically, they do provide device details and skins to help you configure the emulator to more or less match their devices. You can find info about it here:
http://developer.samsung.com/galaxy/emulator-skin
As for the Play Store indicator, Google has a brief and fairly unsatisfying explanation:
Notice that only some hardware profiles are indicated to include Play Store. This indicates that these profiles are fully CTS compliant and may use system images that include the Play Store app.
https://developer.android.com/studio/run/managing-avds.html
I am about to port my application on android. Since I am new to this system I would like to ask for tips on android phones for testing my app. The number of phones is pretty huge so it is pretty hard to choose the right one. Also I wasn't able to find some kind of official chart of system distribution.
I know that it really depends on hardware and how much my app demands from hardware. It is pretty basic app and I am able to run this app at 30fps on iPhone 4S.
I did some research and people are talking about the latest phones but I think that it is not necessary to buy the newest one. Also, is there option to buy for example tablet and emulate android devices on this tablet?
So my goal is to focus on devices people have the most.
Thank you for any advice.
You can run many of your tests on the Android emulator. Using it together
with the DDMS will allow you to simulate several real life situations
(arriving call, arriving sms, change position...)
You can use BlueStacks which is an emulator put on steroids.
TestFairy and UTest are awsome communities for crowd testing were you can test ~any device you can think of.
Still, it is a good idea, especially if you plan your run at Android to be long,
to maintain a small stash of real devices.
Which devices should you buy? Which OS version should they run?
The Android dashboard is the official source for answering some of these questions. It gives an
up to date statistics of worldwide distribution by OS version, svreen size/density and more:
OS Distribution
Screen type Distribution
So, true to July 2014 you would want devices running Jelly Bean, Gingerbread & ICS and you screens of types normal/hdpi, normal/xxhdpi and normal/xhdpi. This list can grow but even this will provide you with a good sample of the Android device ecosystem.
As for manufacturers, the picture is clear: Samsung is the winner. Exact number &
distributions are harder to find here, but a Google search might help.
If my memory serves me write some 65% of all Android devices are Samsungs made with
Galaxy S3 taking 15% (of ENTIRE Android market) and Galaxy S4 = 10%.
Device screen features:
Galaxy S2 is normal/hdpi
Galaxy S3 is normal/xhdpi
Galaxy S4/S5 is normal/xxhdpi
These will probably be your best bet. You will do smartly to install on them 2-3 different OS versions, at
minimum Jelly Bean and ICS.
The Android Development Tools in Eclipse and Android Studio both have emulators that can be used to emulate different devices, including tablets and phones. You don't need to buy anything to be able to test your app on an emulator. The emulators run on your computer.
For more information
As for what devices most people have, while I don't have official statistics, looking at my own applications' downloads, most people are using either a Nexus 4 or Nexus 5.
I am beginning Android app development and I would like to know which device would give me the most comprehensive testing environment. I would like my app to be compatible with the most number of devices without breaking the bank by buying a dozen devices. Just testing on the emulator is not an option because it runs at 1/10th of normal speed.
OS Version
Should I purchase a device with the most common OS? As of now, 2.2 is the most popular. Obviously, a 2.2 device wouldn't be able to test run an app made for 3.0. But is the reverse also true? If I buy a 3.0 device, would that allow me to test all lower versions?
Tablet or Phone
Given that my app won't be dealing with phone calls, should I be buying a phone or tablet? Are the two sufficiently the same except for resolution? Has your app ever worked on a tablet but failed on a phone? The Apple iPad has a neat emulation mode for iPhone apps. Is this feature also available on Android tablets?
Price is also a big consideration when choosing a tablet over a phone. I'm not going to sign up for another cellular plan, so most phones would have me fork over $500+. That would come out to be the same price as the Motorola Xoom. But arguably Android phones are much more popular than tablets, so a Xoom wouldn't let me see what most of my users would be seeing.
In general it's up to you, I suggest you Google Nexus S, because of NFC module in that phone (only that phone nowadays as I good know have NFC), besides that it has everything you need. Big plus for screen.
Android 3.0 is made generally for tablets but of course you can write application in 3.0 and run it on 2.2 or 2.3 just look for backward compatibility, and use reflection.
Also good to know that it does not depend of your phone which version you have it's up to SDK ver which you choose during creating project, in general you can set 7 as min SDK and test it on phone with 9.
If you are creating applications which requires a lot of custom drawing/animating such a game, then a slew of devices would be ideal to test against resolutions, DPI, RAM, CPU power, etc.
If you are using more of the native components then you do not have to worry as much about the different screen types and sizes as these tend to scale very well.
This question was brought up at GDC's Android Developer Day, you may be able to find the video online. Basically, it boiled down to trying to get one of each kind of device, low end, mid, high. It was also recommended that you built for the newest SDK, then start building against older ones, fixing, testing, rebuilding.
Therefore, I would personally recommend the Nexus S.
The company I work for does Android development on the original G1, HTC Hero, Samsung Epic 4G, and HTC Incredible.
My boss recently gave me an HTC Wildfire phone for doing an Android version of one of our mobile apps, which is all fine and good, except that the app requires API level 8 (ie Android 2.2), and the phone itself only has Android 2.1 installed. After much googling, I discovered that this phone does support Android 2.2, and carriers will be rolling out updates over the air for this model phone "in the near future".
Since I'm not entirely certain what our lazy carrier's definition of "near future" is, it would be nice if I could download images for particular releases and install them on my phone. Coming from the world of the iPhone, it seems that this should be something rather necessary in order to successfully develop an Android app.
Yet, it doesn't seem that Google offers any Android OS updates themselves. Have I no choice but to wait for the carrier to release an update?
Edit: Thanks for the suggestions regarding the emulator, but I'm asking about testing on hardware only. I already have a working 2.2 emulator image, but I need to test on hardware now, since the emulator does a very poor job of things like video playback and audio processing.
There's a few ways you can test your application on Android 2.2:
1) You can use the Emulator provided by Google in the SDK. It is possible to create a virtual machine of any version of Android with various hardware options. However, the emulator does have some limitations such as not supporting certain hardware options (such as bluetooth) and it's also dreadfully slow to boot.
2) If you have to use an actual phone with 2.2, you can root your phone and flash a custom ROM. I'm not particularly familiar with how to root the Wildfire, or what custom ROMs are available for that specific phone, but Cyanogen Mod has pretty good ROM support for HTC phones, as well as an active forum with tutorials.
No, it's not possible to download images before OTA. You could check CyanogemMod from time to time - they're working on Wildfire support. Also you could use an emulator.
If you are wanting to bounce around with newer versions, I recommend the Nexus One as a dev phone if you can get your hands on one. Or, if you can deal with the constraints, start with a built-in AVD emulator with API 8 until your phone gets the OTA update.
The Motorola Droid is also getting/has 2.2 on it. At least in the States, in seems that most of the phones on Verizon already have it (non-scientific, just from talking to people I know). So if that phone is available, it may be a decent choice.
I am an iphone application developer, all iphones have very similar Operating systems, and the size and everything is the same, so I don't need to create applications specific for each iphone.
But with android there are different phones, different sizes, So How can I possibly know that my app works, and looks fine on all these devices
I don't already own an android but I completed my first application using the simulator. Which android phone do u suggest for testing? I am an AT&T user and Iuse an iphone. Can I simply insert my sim-card in the new android to be able to test my app on the device?
The android virtual device lets you change the virtual hardware to match all of the current shipping phones. There are ways to receive device crash reports from a shipped application to help diagnose force closes or other grievous errors.
You need an unlocked phone to use another SIM card. The Nexus One is available unlocked out of the gate, and comes with an AT&T version. The only AT&T (as sold by AT&T) Android phone (Backflip) is stupidly crippled and should be avoided. You can also buy an Android Dev Phone, which is currently "middle of the pack" in terms of hardware. You can also easily find numerous other Android capable handsets - pick depending on your target market. Some phones are now old: for instance, the G1 is on its way out.
you won't need to worry about different screen size . take a look at Supporting Multiple Screen article . I'm not sure for the late ones because I'm also working with Android Emulator .
1 - Take a look at this Google IO session for load of tips on making your app work across devices. It's not that hard at all really
2 - I'd recommend the Nexus One if you need a phone just to develop on. It's representative of what you'll be seeing in devices in the coming years. If you want a current representative phone take a look at the HTC Legend (Mid-end) or the HTC Wildfire (Low-end). There's also the Droid and things like that but you can only get them on Verizon I think. I'm not too knowledgeable about availability in the US, I'm from the UK.