Develop Reference

Android MediaCodec releaseOutputBuffer throws MediaCodec.CodecException when decoding H264 video

I am using MediaCodec API to decode a H264 video stream using a SurfaceView as the output surface. The decoder is configured successfully without any errors. When I try to finally render the decoded video frame onto the SurfaceView using releaseOutputBuffer(bufferIndex, true), it throws MediaCodec.CodecException, however the video is rendered correctly.
Calling getDiagnosticInfo() and getErrorCode() on the exception object return an error code of -34, but I can't find in the docs what this error code means. The documentation is also very unclear about when this exception is thrown.
Has anyone faced this exception/error code before? How can I fix this?
PS: Although the video works fine but this exeception is thrown at everyreleaseOutputBuffer(bufferIndex, true), call.

Android media-codec is very dependant on the device vendor. Samsung is incredibly problematic other devices running the same code will run fine. This has been my life for the last 6 months.
The best approach to do although it can feel wrong is to try + catch + retry. There are 4 distinct places where the MediaCodec will throw exceptions:
Configuration - NativeDecoder.Configure(...);
Start - NativeDecoder.Start();
Render output - NativeDecoder.ReleaseOutputBuffer(...);
Input - codec.QueueInputBuffer(...);
NOTE: my code is in Xamarin but the calls map very closely to raw java.
The way you configure your format description also matters. The media-codec can crash on NEXUS devices if you don't specify:
formatDescription.SetInteger(MediaFormat.KeyMaxInputSize, currentPalette.Width * currentPalette.Height);
When you catch any exception you will need to ensure the mediacodec is reset. Unfortunatly reset isnt available to older api-levels but you can simulate the same effect with:
#region Close + Release Native Decoder
void StopAndReleaseNativeDecoder() {
FlushNativeDecoder();
StopNativeDecoder();
ReleaseNativeDecoder();
}
void FlushNativeDecoder() {
if (NativeDecoder != null) {
try {
NativeDecoder.Flush();
} catch {
// ignore
}
}
}
void StopNativeDecoder() {
if (NativeDecoder != null) {
try {
NativeDecoder.Stop();
} catch {
// ignore
}
}
}
void ReleaseNativeDecoder() {
while (NativeDecoder != null) {
try {
NativeDecoder.Release();
} catch {
// ignore
} finally {
NativeDecoder = null;
}
}
}
#endregion
Once you catch the error when you pass new input you can check:
if (!DroidDecoder.IsRunning && streamView != null && streamView.VideoLayer.IsAvailable) {
DroidDecoder.StartDecoder(streamView.VideoLayer.SurfaceTexture);
}
DroidDecoder.DecodeH264FrameBuffer(payload, payloadSize, frameDuration, presentationTime, isKeyFrame);
Rendering to a texture-view seems to be the most stable option currently. But the device fragmentation has really hurt android in this area. We have found cheaper devices such as a the Tesco Hudl to be of the most stable for video. Even had up to 21 concurrent videos on screen at 1 time. Samsung S4 can get around 4-6 depending on the resolution/fps but something like the HTC can work as well as the Hudl. Its been a wake up call and made me realise samsung devices are literally copying apple design and twiddling with the android-sdk and actually breaking a lot of functionality along the way.

It is most probably an issue with the codec you are using. Try using something like this to
private static MediaCodecInfo selectCodec(String mime){
int numCodecs = MediaCodecList.getCodecCount();
for(int i = 0; i < numCodecs; i++){
MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i);
if(!codecInfo.isEncoder()){
continue;
}
String[] types = codecInfo.getSupportedTypes();
for(int j = 0; j < types.length; j++){
if(types[j].equalsIgnoreCase(mime)){
return codecInfo;
}
}
}
return null;
}
And then setting your encoder with:
MediaCodecInfo codecInfo = selectCodec(MIME_TYPE);
mEncoder = MediaCodec.createCodecByName(codecInfo.getName());
That may resolve your error by ensuring that the Codec you've chosen is fully supported.

Arduino Mega ADK strange acc.write() behavior (huge delay) NAK issue seemd to be confirmed. Android generates NAK's as a result of acc.write

I'm trying to communicate between Arduino Mega Adk (ADK 2011) and android device.
Something goes ok, but something goes completely wrong.
Transfer data from Android to Arduino via acc.read from Arduino side works fine.
But when i try to send some bytes from Arduino to Android - something strange happens.
First of all here is Arduino sketch:
#include <Max3421e_constants.h>
#include <Max3421e.h>
#include <Usb.h>
#include <AndroidAccessory.h>
#define COMMAND_LED 0x2
#define TARGET_PIN_18 0x12
#define TARGET_PIN_19 0x13
#define V_ON 0x1
#define V_OFF 0x0
#define PIN_18 18
#define PIN_19 19
#define INPUT_PIN 30
AndroidAccessory acc("Google, Inc.",
"DemoKit",
"Ololo device board",
"1.0",
"http://www.android.com",
"0000000012345678");
byte rcvmsg[3];
byte sndmsg[3];
int buttonState = 0;
void setup();
void loop();
void led_setup(){
pinMode(PIN_18, OUTPUT);
pinMode(PIN_19, OUTPUT);
pinMode(INPUT_PIN, INPUT);
}
void setup()
{
Serial.begin(115200);
Serial.print("\r\nStart");
led_setup();
acc.powerOn();
}
void loop()
{
if (acc.isConnected()) {
buttonState = digitalRead(INPUT_PIN);
if (buttonState == 1){
sndmsg[0] = 0x2;
sndmsg[1] = 0x1;
sndmsg[2] = 0x1;
int len = acc.write(sndmsg, 3);
digitalWrite(PIN_19, HIGH);
}
else {
//Nothing here for test
}
}
//usefull test for button
buttonState = digitalRead(INPUT_PIN);
if (buttonState == 1){
digitalWrite(PIN_19, HIGH);
}
else {
digitalWrite(PIN_19, LOW);
}
}
Ok. When acc.write() is executed it takes up to ~1 second to transfer data to android. And this time doesn't depend on number of bytes in sndmsg. Only if i execute acc.write(sndmsg,0) (sending 0 bytes) - everything goes fast.
That is a little bit disturbing. I've tried to change board to another one but have got the same result.
Any advices? may be that is a common bug, but there is no such much information in web.
UPD:
Wrote some very simple code, that only sends 3 bytes via acc.write.
here it is:
#include <Max3421e_constants.h>
#include <Max3421e.h>
#include <Usb.h>
#include <AndroidAccessory.h>
AndroidAccessory acc("Google, Inc.",
"DemoKit",
"Demokit board",
"1.0",
"http://www.android.com",
"0000000012345678");
byte msg[3];
unsigned long time;
void setup();
void loop();
void led_setup(){
}
void setup()
{
Serial.begin(115200);
Serial.print("\r\nStart");
acc.powerOn();
}
void loop()
{
if (acc.isConnected()) {
Serial.print("\r\nis Connected");
msg[0] = 0x1;
msg[1] = 0x1;
msg[2] = 0x1;
//Serial.print("\r\nSending");
time = millis();
Serial.print ("\r\nBefore write\r\n");
Serial.print (time);
acc.write(msg, 3);
time = millis();
Serial.print("\r\nAfter write: \r\n");
Serial.print (time);
//delay(500);
}
}
And it's debug output is:
is Connected
Before write
6983
After write:
10958
is Connected
Before write
10958
After write:
14491
is Connected
and so on. So on some reasons acc.write takes a lot of time and there is no data in the android app.
New UPD (19.01.2015):
Today i've performed some experiments. Here are results.
First, i've looked into AndroidAccessory.cpp and found write function:
int AndroidAccessory::write(void *buff, int len)
{
usb.outTransfer(1, out, len, (char *)buff);
return len;
}
Ok, then i looked into usb host shield library and found there usb.cpp with outTransfer fucntion, that returns error code if ouccured and 0x00 if everything is ok.
So i modified write function to return an error code instead of lenght, like this:
int AndroidAccessory::write(void *buff, int len)
{
byte rcode;
rcode = usb.outTransfer(1, out, len, (char *)buff);
return int(rcode);
}
and recived "4" as result.
According to MAX3421Econstants.h it is hrNAK (0x04) error code.
Any ideas? Looks like accessory does not recive NAKs from Android and write fails as a result.
Situation update:
Did some research. There is a hell of NAK's when accessory is connected. Here is dump from usb connector:

i found the solution. And it is very simple - i didn't setup communication with accessory correctly.
This is not an Arduino problem. Arduino works fine.
It is just how android interacts with android accessory.
So, results:
When AndroidAccessory is plugged to Android and Android haven't setup
communication with the accessory yet, Android OS will send a lot of
USB NAKs to the accessory and this is normal.
You must be careful
during setuping communication with the accessory. If you make some
mistakes, you can receive same situation: Probably possible to write
to the accessory, but accessory isn't possible to write to the
android.
If UsbManager opens accessory correctly, it stops sending
NAKs and starts recieve data from arduino.
It is a little bit strange for me, because it was really hard to found a problem: i have an application, written according to this manual: http://developer.android.com/guide/topics/connectivity/usb/accessory.html But, because i'm not very familiar with android, it seems that i've done some mistakes and receive strange behavior:
i was able to write to arduino
i was able to retrive information about arduino as android accessory
it was possible to ask for permissions
but when arduino tries to write to android, it recievs a lot of NAKs
So i decided to rewrite program in more simple way, just with one function and tried to do it in right way. And after some debugging it finally started to work as i want.
So thank everyone, who spend time for reading this.
Bonus: dump of normal packet, ended with EOP not NAK
UPD 26.01.2015:
I found problem and it was in my android code.
here is explanation:
Android developer's manual said that function, which set up communication with accessory must start it's own thread in which all communications with input and output streams are held. Smth like this:
private void openAccessory() {
Log.d(TAG, "openAccessory: " + accessory);
mFileDescriptor = mUsbManager.openAccessory(mAccessory);
if (mFileDescriptor != null) {
FileDescriptor fd = mFileDescriptor.getFileDescriptor();
mInputStream = new FileInputStream(fd);
mOutputStream = new FileOutputStream(fd);
Thread thread = new Thread(null, this, "AccessoryThread");
thread.start();
}
}
I really messed this thing up. Forgot about creating new thread in openAccessory function and tried to do it in different place. And recieve a hell of NAK's. Then i've changed my code and add some debug-like run() function like this:
public void run() {
final byte[] buffer = new byte[16384];
int ret = 0;
int i = 0;
while (i<50) {
try {
ret = mInputStream.read(buffer);
i++;
Thread.sleep(500);
} catch (IOException e) {
break;
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
And until this run method exists (while i < 50) android reads arduino correctly. And when thread ends up (i > 50) - Arduino starts to readout Error Code 4 (NAK) from Android.
That's all folks.

android get usb_interface_descriptor, usb_endpoint_descriptor

In the method usb_device_new() they are reading the usb_device_descriptor with this function call:
read(fd, device->desc, sizeof(device->desc));
First of all how does that even work, to read from the file descriptor after you have opened the device. How do you know that the usb_device_descriptor is what will be read? I find that somewhat confusing.
And the second question that I have is how can I read the usb_interface_descriptor and the usb_endpoint_descriptor from the usb_device?
Thanks.

I have found how they manage to retrive the interface and endpoints details:
while ((desc = usb_descriptor_iter_next(&iter)) != NULL)
{
if (desc->bDescriptorType == USB_DT_INTERFACE)
{
struct usb_interface_descriptor *interface = (struct usb_interface_descriptor *) desc;
interfaceValues.add(interface->bInterfaceNumber);
interfaceValues.add(interface->bInterfaceClass);
interfaceValues.add(interface->bInterfaceSubClass);
interfaceValues.add(interface->bInterfaceProtocol);
interfaceValues.add(interface->bNumEndpoints);
}
else if (desc->bDescriptorType == USB_DT_ENDPOINT)
{
struct usb_endpoint_descriptor *endpoint = (struct usb_endpoint_descriptor *) desc;
endpointValues.add(endpoint->bEndpointAddress);
endpointValues.add(endpoint->bmAttributes);
endpointValues.add(__le16_to_cpu(endpoint->wMaxPacketSize));
endpointValues.add(endpoint->bInterval);
}
}
I just had to dig a little bit deeper in the framework, in this file.

fopen/fread APK Assets from NativeActivity on Android

I have only been able to find solutions dated 2010 and earlier. So I wanted to see if there was a more up-to-date stance on this.
I'd like to avoid using Java and purely use C++, to access files (some less-or-more than 1MB) stored away in the APK. Using AssetManager means I can't access files like every other file on every other operating system (including iOS).
If not, is there a method in C++ where I could somehow map fopen/fread to the AssetManager APIs?

I actually found pretty elegant answer to the problem and blogged about it here.
The summary is:
The AAssetManager API has NDK bindings. This lets you load assets from the APK.
It is possible to combine a set of functions that know how to read/write/seek against anything and disguise them as a file pointer (FILE*).
If we create a function that takes an asset name, uses AssetManager to open it, and then disguises the result as a FILE* then we have something that's very similar to fopen.
If we define a macro named fopen we can replace all uses of that function with ours instead.
My blog has a full write up and all the code you need to implement in pure C. I use this to build lua and libogg for Android.

Short answer
No. AFAIK mapping fread/fopen in C++ to AAssetManager is not possible. And if were it would probably limit you to files in the assets folder. There is however a workaround, but it's not straightforward.
Long Answer
It IS possible to access any file anywhere in the APK using zlib and libzip in C++.
Requirements : some java, zlib and/or libzip (for ease of use, so that's what I settled for). You can get libzip here: http://www.nih.at/libzip/
libzip may need some tinkering to get it to work on android, but nothing serious.
Step 1 : retrieve APK location in Java and pass to JNI/C++
String PathToAPK;
ApplicationInfo appInfo = null;
PackageManager packMgmr = parent.getPackageManager();
try {
appInfo = packMgmr.getApplicationInfo("com.your.application", 0);
} catch (NameNotFoundException e) {
e.printStackTrace();
throw new RuntimeException("Unable to locate APK...");
}
PathToAPK = appInfo.sourceDir;
Passing PathToAPK to C++/JNI
JNIEXPORT jlong JNICALL Java_com_your_app(JNIEnv *env, jobject obj, jstring PathToAPK)
{
// convert strings
const char *apk_location = env->GetStringUTFChars(PathToAPK, 0);
// Do some assigning, data init, whatever...
// insert code here
//release strings
env->ReleaseStringUTFChars(PathToAPK, apk_location);
return 0;
}
Assuming that you now have a std::string with your APK location and you have zlib on libzip working you can do something like this:
if(apk_open == false)
{
apk_file = zip_open(apk_location.c_str(), 0, NULL);
if(apk_file == NULL)
{
LOGE("Error opening APK!");
result = ASSET_APK_NOT_FOUND_ERROR;
}else
{
apk_open = true;
result = ASSET_NO_ERROR;
}
}
And to read a file from the APK:
if(apk_file != NULL){
// file you wish to read; **any** file from the APK, you're not limited to regular assets
const char *file_name = "path/to/file.png";
int file_index;
zip_file *file;
struct zip_stat file_stat;
file_index = zip_name_locate(apk_file, file_name, 0);
if(file_index == -1)
{
zip_close(apk_file);
apk_open = false;
return;
}
file = zip_fopen_index(apk_file, file_index, 0);
if(file == NULL)
{
zip_close(apk_file);
apk_open = false;
return;
}
// get the file stats
zip_stat_init(&file_stat);
zip_stat(apk_file, file_name, 0, &file_stat);
char *buffer = new char[file_stat.size];
// read the file
int result = zip_fread(file, buffer, file_stat.size);
if(result == -1)
{
delete[] buffer;
zip_fclose(file);
zip_close(apk_file);
apk_open = false;
return;
}
// do something with the file
// code goes here
// delete the buffer, close the file and apk
delete[] buffer;
zip_fclose(file);
zip_close(apk_file);
apk_open = false;
Not exactly fopen/fread but it gets the job done. It should be pretty easy to wrap this to your own file reading function to abstract the zip layer.

Android app restarts automatically after a crash

My app is partly written in native app using C/C++. The problem is that whenever C/C++ part crashes for some reason the app dies and then restarts automatically. This causes all kinds of messy problems
Now of course, it should not crash in the native part and I'm trying to weed out all reasons why it would happen. However, if it does happen I'd like to:
Quit gracefully
If it does die, at least not try to restart automatically.
I'm curious as to why this behaviour happens. After some search I tried putting the following line in the main activity element of the AndroidManifest.xml:
android:finishOnTaskLaunch="true"
but the automatic restore still happens.
Anyone knows why this is happening and how to change it?
UPDATE:
I think a more fundamental question is,
Is there something similar to a callback if there is a native crash?
One of the answers suggested 'handling crash signals'. I'd be grateful for any links on how it can be done at an application or module level.
As it stands currently, if there is a crash the app just disappears, there's nothing in logcat, so no debugging is possible.

Try to handle crash signals (SIGSEGV etc.) and send kill to yourself in signal handler. This trick helps me.
Example:
#include <signal.h>
#include <unistd.h>
static void signal_handler(int signal, siginfo_t *info, void *reserved)
{
kill(getpid(),SIGKILL);
}
extern "C" jint JNI_OnLoad(JavaVM* vm, void* /*reserved*/)
{
struct sigaction handler;
memset(&handler, 0, sizeof(handler));
handler.sa_sigaction = signal_handler;
handler.sa_flags = SA_SIGINFO;
sigaction(SIGILL, &handler, NULL);
sigaction(SIGABRT, &handler, NULL);
sigaction(SIGBUS, &handler, NULL);
sigaction(SIGFPE, &handler, NULL);
sigaction(SIGSEGV, &handler, NULL);
sigaction(SIGSTKFLT, &handler, NULL);
return(JNI_VERSION_1_6);
}
UPDATE2
if you want to see crashlog in android logcat you should use this signal handler
static void signal_handler(int signal, siginfo_t *info, void *reserved)
{
struct sockaddr_un addr;
size_t namelen;
socklen_t alen;
int s, err;
char name[] = "android:debuggerd";
namelen = strlen(name);
// Test with length +1 for the *initial* '\0'.
if ((namelen + 1) > sizeof(addr.sun_path)) {
errno = EINVAL;
return;
}
/* This is used for abstract socket namespace, we need
* an initial '\0' at the start of the Unix socket path.
*
* Note: The path in this case is *not* supposed to be
* '\0'-terminated. ("man 7 unix" for the gory details.)
*/
memset (&addr, 0, sizeof addr);
addr.sun_family = AF_LOCAL;
addr.sun_path[0] = 0;
memcpy(addr.sun_path + 1, name, namelen);
alen = namelen + offsetof(struct sockaddr_un, sun_path) + 1;
s = socket(AF_LOCAL, SOCK_STREAM, 0);
if(s < 0) return;
RETRY_ON_EINTR(err,connect(s, (struct sockaddr *) &addr, alen));
if (err < 0) {
close(s);
s = -1;
}
pid_t tid = gettid();
if(s>=0)
{
/* debugger knows our pid from the credentials on the
* local socket but we need to tell it our tid. It
* is paranoid and will verify that we are giving a tid
* that's actually in our process
*/
int ret;
RETRY_ON_EINTR(ret, write(s, &tid, sizeof(unsigned)));
if (ret == sizeof(unsigned)) {
/* if the write failed, there is no point to read on
* the file descriptor. */
RETRY_ON_EINTR(ret, read(s, &tid, 1));
//notify_gdb_of_libraries();
}
close(s);
}
wait(NULL);
kill(getpid(),SIGKILL);
}
I took it from android source (can't insert link because android.git.kernel.org is down), but I am not sure that it will work in future Android releases

By default your application should not be automatically restarting. Generally one would have to register for this kind of thing, e.g. via the AlarmManager/keep alives.
Do you have a service as part of your application?