Is there a way to ask an Android device what audio and video Codecs it supports for encoding?
I found devices that do not support some of the codecs listed as mandatory in
http://developer.android.com/guide/appendix/media-formats.html
and there seem to be devices supporting additional codec not listed there.
That could be interesting for you:
private static MediaCodecInfo selectCodec(String mimeType) {
int numCodecs = MediaCodecList.getCodecCount();
for (int i = 0; i < numCodecs; i++) {
MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i);
if (!codecInfo.isEncoder()) {
continue;
}
String[] types = codecInfo.getSupportedTypes();
for (int j = 0; j < types.length; j++) {
if (types[j].equalsIgnoreCase(mimeType)) {
return codecInfo;
}
}
}
return null;
}
Found it here. AS you can see you get the number of installed codecs with MediaCodecList.getCodecCount();. With MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i); you get information about a specific codec out of the list. codecInfo.getName() for example tells you title/name of the codec.
Is there a way to ask an Android device what audio and video Codecs it supports for encoding?
I really wish there were, but there is not, at least through ICS.
Jelly Bean offers a MediaCodec class. While it does not have a "give me a list of supported codecs", it does have createEncoderByType(), where you pass in a MIME type. Presumably, that will throw a RuntimeException or return null if your desired MIME type is not supported. And I cannot promise that just because MediaCodec reports that an encoder is available that it is guaranteed to work from, say, MediaRecorder.
The simplest way is using
MediaCodecList(MediaCodecList.ALL_CODECS).codecInfos
It returns a array of all encoders and decoders available on your devices like this image.
And then, you can use filter to query the specific encoders and decoders you are looking for. For example:
MediaCodecList(MediaCodecList.ALL_CODECS).codecInfos.filter {
it.isEncoder && it.supportedTypes[0].startsWith("video")
}
This returns all available video encoders.
Here is an updated code based on Jonson's answer, written in Kotlin and not using deprecated methods:
fun getCodecForMimeType(mimeType: String): MediaCodecInfo? {
val mediaCodecList = MediaCodecList(MediaCodecList.REGULAR_CODECS)
val codecInfos = mediaCodecList.codecInfos
for (i in codecInfos.indices) {
val codecInfo = codecInfos[i]
if (!codecInfo.isEncoder) {
continue
}
val types = codecInfo.supportedTypes
for (j in types.indices) {
if (types[j].equals(mimeType, ignoreCase = true)) {
return codecInfo
}
}
}
return null
}
Related
public void getCodecInfo() {
int numCodecs = MediaCodecList.getCodecCount();
for (int i = 0; i < numCodecs; i++) {
MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i);
if (!codecInfo.isEncoder()) {
continue;
}
String[] types = codecInfo.getSupportedTypes();
for (int j = 0; j < types.length; j++) {
MediaCodecInfo.CodecCapabilities capabilities = codecInfo.getCapabilitiesForType(types[j]);
Log.d("CodecCapabilities", new Gson().toJson(capabilities));
//MediaCodecInfo.VideoCapabilities videoCapabilities = capabilities.getVideoCapabilities();
//Log.d("videoCapabilities", new Gson().toJson(videoCapabilities));
}
}
}
The above gave me this, what does the following number for profile and level tell me anything related to the video capabilities?
{"colorFormats":[21,2130708361],"profileLevels":[{"level":2048,"profile":1},{"level":2048,"profile":2},{"level":2048,"profile":8}],"flags":0}
If I uncomment these two lines in the above code, it crashes with this error message:
java.lang.NoSuchMethodError: android.media.MediaCodecInfo$CodecCapabilities.getVideoCapabilities
How can I query the android device, to find out the video capabilities? I'd like to know the max video bitrate and video resolution the device is capable to handle.
I guess that you are testing your code on a device running API < 21? If is the case, the getVideoCapabilies method is available only on devices running Android >= 21
Anyway, to get bitrate and supported width & height Ranges (API >=21 too...Humm may be related to getVideoCapabilies availability... I don't know :) ), you can use :
Range<Integer> bitrateRange = videoCapabilities.getBitrateRange();
Range<Integer> heightRange = videoCapabilities.getSupportedHeights();
Range<Integer> widthRange = videoCapabilities.getSupportedWidths();
You can take a look at this gist that I published a few days ago to get all capabilities for a given codec (Colors, profiles & levels are printed by names instead of numbers, which could be very helpful): TestMediaCodecInfoAsyncTask.java
I am using MediaCodec API to decode a H264 video stream using a SurfaceView as the output surface. The decoder is configured successfully without any errors. When I try to finally render the decoded video frame onto the SurfaceView using releaseOutputBuffer(bufferIndex, true), it throws MediaCodec.CodecException, however the video is rendered correctly.
Calling getDiagnosticInfo() and getErrorCode() on the exception object return an error code of -34, but I can't find in the docs what this error code means. The documentation is also very unclear about when this exception is thrown.
Has anyone faced this exception/error code before? How can I fix this?
PS: Although the video works fine but this exeception is thrown at everyreleaseOutputBuffer(bufferIndex, true), call.
Android media-codec is very dependant on the device vendor. Samsung is incredibly problematic other devices running the same code will run fine. This has been my life for the last 6 months.
The best approach to do although it can feel wrong is to try + catch + retry. There are 4 distinct places where the MediaCodec will throw exceptions:
Configuration - NativeDecoder.Configure(...);
Start - NativeDecoder.Start();
Render output - NativeDecoder.ReleaseOutputBuffer(...);
Input - codec.QueueInputBuffer(...);
NOTE: my code is in Xamarin but the calls map very closely to raw java.
The way you configure your format description also matters. The media-codec can crash on NEXUS devices if you don't specify:
formatDescription.SetInteger(MediaFormat.KeyMaxInputSize, currentPalette.Width * currentPalette.Height);
When you catch any exception you will need to ensure the mediacodec is reset. Unfortunatly reset isnt available to older api-levels but you can simulate the same effect with:
#region Close + Release Native Decoder
void StopAndReleaseNativeDecoder() {
FlushNativeDecoder();
StopNativeDecoder();
ReleaseNativeDecoder();
}
void FlushNativeDecoder() {
if (NativeDecoder != null) {
try {
NativeDecoder.Flush();
} catch {
// ignore
}
}
}
void StopNativeDecoder() {
if (NativeDecoder != null) {
try {
NativeDecoder.Stop();
} catch {
// ignore
}
}
}
void ReleaseNativeDecoder() {
while (NativeDecoder != null) {
try {
NativeDecoder.Release();
} catch {
// ignore
} finally {
NativeDecoder = null;
}
}
}
#endregion
Once you catch the error when you pass new input you can check:
if (!DroidDecoder.IsRunning && streamView != null && streamView.VideoLayer.IsAvailable) {
DroidDecoder.StartDecoder(streamView.VideoLayer.SurfaceTexture);
}
DroidDecoder.DecodeH264FrameBuffer(payload, payloadSize, frameDuration, presentationTime, isKeyFrame);
Rendering to a texture-view seems to be the most stable option currently. But the device fragmentation has really hurt android in this area. We have found cheaper devices such as a the Tesco Hudl to be of the most stable for video. Even had up to 21 concurrent videos on screen at 1 time. Samsung S4 can get around 4-6 depending on the resolution/fps but something like the HTC can work as well as the Hudl. Its been a wake up call and made me realise samsung devices are literally copying apple design and twiddling with the android-sdk and actually breaking a lot of functionality along the way.
It is most probably an issue with the codec you are using. Try using something like this to
private static MediaCodecInfo selectCodec(String mime){
int numCodecs = MediaCodecList.getCodecCount();
for(int i = 0; i < numCodecs; i++){
MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i);
if(!codecInfo.isEncoder()){
continue;
}
String[] types = codecInfo.getSupportedTypes();
for(int j = 0; j < types.length; j++){
if(types[j].equalsIgnoreCase(mime)){
return codecInfo;
}
}
}
return null;
}
And then setting your encoder with:
MediaCodecInfo codecInfo = selectCodec(MIME_TYPE);
mEncoder = MediaCodec.createCodecByName(codecInfo.getName());
That may resolve your error by ensuring that the Codec you've chosen is fully supported.
I'm using the selectColorFormat method from bigflake to get supported input color format of encoder of MediaCodec.It works well on some tested devices except Huawei Mate7. The returned color format is COLOR_FormatYUV420Planar.So I configure it as the input format.But when encoding I found the input buffer's capacity is not correct,which should be 3*width*height/2.But it's smaller than that.So when queueInputBuffer called error occured. Then I observed the capabilities.colorFormats list, it has more than ten color formats.But on the normal devices the list has about 2 or 3 color formats.Then I chose the color format to COLOR_FormatYUV420SemiPlanar and it works well on Mate7 too.
So I doubt whether this is a bug of Mate7 version or the bug of selectColorFormat method?
And I find the anwser of FAQ Q5 has :
#20 COLOR_FormatYUV420PackedPlanar (also I420)
#39 COLOR_FormatYUV420PackedSemiPlanar (also NV12)
#0x7f000100 COLOR_TI_FormatYUV420PackedSemiPlanar (also also NV12)
What's the meaning of also I420 and also NV12?Does it mean the layout is all the same with I420 or NV12?Because I have tested I420 and NV12, but haven't come across these three colors device.
The selectColorFormat method is as belows:
private MediaCodecInfo selectCodec(String mimeType) {
int numCodecs = MediaCodecList.getCodecCount();
for (int i = 0; i < numCodecs; i++) {
MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i);
if (!codecInfo.isEncoder()) {
continue;
}
String[] types = codecInfo.getSupportedTypes();
for (int j = 0; j < types.length; j++) {
if (types[j].equalsIgnoreCase(mimeType)) {
return codecInfo;
}
}
}
return null;
}
private int selectColorFormat(String mimeType) {
MediaCodecInfo codecInfo = selectCodec(mimeType);
int colorFormat = 0;
MediaCodecInfo.CodecCapabilities capabilities = codecInfo.getCapabilitiesForType(mimeType);
for (int i = 0; i < capabilities.colorFormats.length; i++) {
if (isRecognizedFormat(capabilities.colorFormats[i])) {
colorFormat = capabilities.colorFormats[i];
break;
}
}
return colorFormat;
}
private boolean isRecognizedFormat(int colorFormat) {
switch (colorFormat) {
// these are the formats we know how to handle for this test
case MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar:
case MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420PackedPlanar:
case MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar:
case MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420PackedSemiPlanar:
case MediaCodecInfo.CodecCapabilities.COLOR_TI_FormatYUV420PackedSemiPlanar:
return true;
default:
return false;
}
}
All devices running Android 4.3 (API 18) or later support YUV input in I420 or NV12 formats. I can say this with confidence because they are part of the CTS tests that all "official" Android devices must pass.
The color constants returned by the codecs have multiple values that map to either I420 or NV12. You can see in the bigflake sample code how this is handled (e.g. generateFrame() in EncodeDecodeTest).
The input buffer capacity should be equal to or slightly larger than width * height * 3 / 2. If you are encountering buffer overrun exceptions, make sure you are clearing the buffer before copying data into it.
I tried to get the SLDeviceVolumeItf interface of the RecorderObject on Android but I got the error: SL_RESULT_FEATURE_UNSUPPORTED.
I read that the Android implementation of OpenSL ES does not support volume setting for the AudioRecorder. Is that true?
If yes is there a workaround? I have a VOIP application that does not worl well on Galaxy Nexus because of the very high mic gain.
I also tried to get the SL_IID_ANDROIDCONFIGURATION to set the streamType to the new VOICE_COMMUNINCATION audio-source but again I get error 12 (not supported).
// create audio recorder
const SLInterfaceID id[2] = { SL_IID_ANDROIDSIMPLEBUFFERQUEUE, SL_IID_ANDROIDCONFIGURATION };
const SLboolean req[2] = { SL_BOOLEAN_TRUE, SL_BOOLEAN_TRUE };
result = (*engine)->CreateAudioRecorder(engine, &recorderObject, &audioSrc, &audioSnk, 2, id, req);
if (SL_RESULT_SUCCESS != result) {
return false;
}
SLAndroidConfigurationItf recorderConfig;
result = (*recorderObject)->GetInterface(recorderObject, SL_IID_ANDROIDCONFIGURATION, &recorderConfig);
if(result != SL_RESULT_SUCCESS) {
error("failed to get SL_IID_ANDROIDCONFIGURATION interface. e == %d", result);
}
The recorderObject is created but I can't get the SL_IID_ANDROIDCONFIGURATION interface.
I tried it on Galaxy Nexus (ICS), HTC sense (ICS) and Motorola Blur (Gingerbread).
I'm using NDK version 6.
Now I can get the interface. I had to use NDK 8 and target-14.
When I tried to use 10 as a target, I had an error compiling the native code (dirent.h was not found).
I had to use target-platform-14.
I ran into a similar problem. My results were returning the error code for not implemented. However, my problem was that I wasn't creating the recorder with the SL_IID_ANDROIDCONFIGURATION interface flag.
apiLvl = (*env)->GetStaticIntField(env, versionClass, sdkIntFieldID);
SLint32 streamType = SL_ANDROID_RECORDING_PRESET_GENERIC;
if(apiLvl > 10){
streamType = SL_ANDROID_RECORDING_PRESET_VOICE_COMMUNICATION;
I("set SL_ANDROID_RECORDING_PRESET_VOICE_COMMUNICATION");
}
result = (*recorderConfig)->SetConfiguration(recorderConfig, SL_ANDROID_KEY_RECORDING_PRESET, &streamType, sizeof(SLint32));
if (SL_RESULT_SUCCESS != result) {
return 0;
}
Even i tried to find a way to change the gain in OpenSL, looks like there is no api/interface for that. i implemented a work around by implementing a simple shift gain multiplier
void multiply_gain(void *buffer, int bytes, int gain_val)
{
int i = 0, j = 0;
short *buffer_samples = (short*)buffer;
for(i = 0, j = 0; i < bytes; i+=2,j++)
{
buffer_samples[j] = (buffer_samples[j] >> gain_val);
}
}
But here the gain is multiplied/divided (based on << or >>) by a factor or 2. if you need a smoother gain curve, you need to write a more complex digital gain function.
which codecs are supported for audio in android? G711 is supported. But is there any other like G729,...etc?
Take a look here:
http://developer.android.com/guide/appendix/media-formats.html
Im not sure if G.729 is explicitly available. But it might be used as a module in one of the other codecs that are listed on the page above.
HTH,
Sriram.
Take a look at: http://developer.android.com/reference/android/media/MediaCodecInfo.html
Have a piece of code which tell's all the avaible codecs (audio and video) in the device.
private List<String> mSupportedCodecs;
private void getSupportedCodecs() {
int numCodecs = MediaCodecList.getCodecCount();
mSupportedCodecs = new ArrayList<String>(numCodecs);
for (int i = 0; i < numCodecs; i++) {
MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i);
mSupportedCodecs.add(codecInfo.getName());
}
}