Develop Reference

MediaCodec Error 0xfffffff3 with video decoder

I'm compressing a video with audio on android using MediaCodec. When I run the code in a try/catch the output video is fine apart from the last part of the audio is missing.
I've had to rewrite the code and also convert it into Kotlin but nothing has worked so far. I've seen a few similar issues on here but the error code is slightly different as it is more of an audio issue whereas this looks like an issue with the video extraction.
Here's the code where the error is happening. The line with videoDecoder.queueInputBuffer is exactly the crash.
videoExtractorDone = !videoExtractor.advance()
if (videoExtractorDone) {
if (false) Log.d(TAG, "video extractor: EOS")
videoDecoder.queueInputBuffer(decoderInputBufferIndex, 0, 0, 0, videoDecoderOutputBufferInfo.flags)
}
How can I solve this issue so that the clip has full audio? I can't lose any of the audio from this process.

An AAC audio stream is playable in VLC for Android, but not in Exoplayer

I have an RTMP stream I want to play in my app using the Exoplayer library. My setup for that is as follows:
TrackSelector trackSelector = new DefaultTrackSelector();
RtmpDataSourceFactory rtmpDataSourceFactory = new RtmpDataSourceFactory(bandwidthMeter);
ExtractorsFactory extractorsFactory = new DefaultExtractorsFactory();
factory = new ExtractorMediaSource.Factory(rtmpDataSourceFactory);
factory.setExtractorsFactory(extractorsFactory);
createSource();
mPlayer = ExoPlayerFactory.newSimpleInstance(mActivity, trackSelector, new DefaultLoadControl(
new DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE),
1000, // min buffer
3000, // max buffer
1000, // playback
2000, //playback after rebuffer
DefaultLoadControl.DEFAULT_TARGET_BUFFER_BYTES,
true
));
vwExoPlayer.setPlayer(mPlayer);
mPlayer.addListener(mVideoStreamHandler);
mPlayer.addVideoListener(new VideoListener() {
#Override
public void onVideoSizeChanged(int width, int height, int unappliedRotationDegrees, float pixelWidthHeightRatio) {
Log.d("hasil", "onVideoSizeChanged: w:" + width + ", h:" + height);
String res = width + "x" + height;
resolution.setText(res);
}
#Override
public void onRenderedFirstFrame() {
}
});
Where createSource() is as follows:
private void createSource() {
mMediaSource180 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_180));
mMediaSource360 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_360));
mMediaSource720 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_720));
mMediaSourceAudio = factory.createMediaSource(Uri.parse(API.GAME_AUDIO_STREAM_URL));
}
My current problem is that only the first three ExtractorMediaSources work fine in Exoplayer. The mMediaSourceAudio refuses to play in Exoplayer, but works just fine in the VLC Media Player for Android.
Right now I have a suspicion that the format is AAC-LTP, or whatever AAC variant that requires a codec available in VLC but not in default Android. However, I do not have access to the encoding process so I don't know for sure.
If this isn't the case, what is it?
EDIT:
I've been debugging the BandwidthMeter and added a MediaSourceEventListener. When I use the normal Video sources, onDownstreamFormatChanged() gets called, but not when I use that Audio Stream source.
In addition, the BandwidthMeter works fine, with bytes always downloaded in all parts of the stream and more bytes when the video stream comes in, but only in the Audio only stream that, when I call mPlayer.getBufferedPosition(), the returned value is always 0. Also, when I use the Audio Stream source, no OMX code was called - no decoders were set up.
Am I seeing a malformed audio stream, or do I need to change my Exoplayer's settings?
EDIT 2:
Further debugging reveals that, in all the Video streams and Audio stream, the same FlvExtractor is used. Even though the Video streams have the avc video track encoding and mp4a-latm audio track encoding. Is this normal?

Turns out it's because the stream was recognized to have two tracks/sampleQueues. One Audio track, and one track with null format. That null track was supposed to be the video track, which was supposed to exist according to the stream's flvHeader flag.
For now, I get around this by creating a custom MediaSource using a custom MediaPeriod. Said custom MediaPeriod having code to separate the video and audio tracks of the SampleQueues, then using the audio-only SampleQueue[] instead of the source SampleQueue[] when I want to play the audio-only stream.
Though this gives me another point of concern: There's something one can do to alter the 'has audio track (flag & 0x04) and video track (flag & 0x01)' flag in the rtmp stream, right?

Thanks for the comments, I'm new to ExoPlayer. But your comments helped me in debugging and getting multiple workarounds to the issue.
I tried to use custom MediaSource and custom MediaPeriod to address this audio issue. I have observed video format data coming after audio data incase of video+audio wowza stream, so the function maybeFinishPrepare() will wait for getting both video and audio format tag data before invoking onPrepared, incase if video tagData is received first. Incase of audio data received first, it wont wait and will call onPrepare().
With the above changes, I was able to play audio alone and video_audio wowza streams, where rtmp tagHeader with tagTypes were coming in the order of video tagData and then followed by audio data.
I wasn't able to use the same patch with srs server to play both audio_only and video_audio streams with the same changes. srs server is giving tagData in the order of audio and then video tagData,
So, I debugged further in FlvExtractor. In readFlvHeader, I have overriden the hasAudio and hasVideo variables. These variables will be set based on the first few tagHeaders(5 or 6). I used peekFully on input for 6 times in a loop. In each loop after fetching tagType and tagDataSize, tagDataSize is used to input.advancePeekPosition(), and tagType is used to identify whether we have audio/video format data in tagData. After peeking for first 6 consecutive tagHeaders, I was able to get actual values of hasAudio and hasVideo, and ignored the flvHeaders.flags, which were used to set these variables.
Custom FlvExtractor workaround, looked cleaner than custom MediaSource/MediaPeriod, as we will create those many tracks as necessary, as we are setting proper hasVideo/hasAudio values.

Two-thirds of screen is green when encode with MediaCodec

I am working on a project that needs to process a video using OpenGL on Android. I decided to use MediaCodec and I managed to get it works with the help from ExtractDecodeEditEncodeMuxTest. The result is quite good, I have it receives a video, extracts the tracks, decodes the videotrack, edits with OpenGL, and encodes to a video file.
The problem is that the result video can be play well on Android, but when it comes to iOS, two-thirds of the screen is green.
I tried to solve with the suggestions from here, here, and here, experiment different formats for the encoder, but the problem is still the same.
Could someone suggest me the reasons that can cause this problem and how to fix it?
This is the video when it's played on iOS
This is the configuration for the encoder
MediaCodec mediaCodec = MediaCodec.createEncoderByType("video/avc");
MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 540, 960);
mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 2000000);
mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 30);
mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, CodecCapabilities.COLOR_FormatSurface);
mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
Update
I wonder if i had any mistake with the video orientation, because the working partial of the output video has the same ratio as the desired output resolution, but in horizontal orientation. The input is vertical recorded, so does the desired output.
Here is the code of the decoder configuration:
inputFormat.setInteger(MediaFormat.KEY_WIDTH, 540);
inputFormat.setInteger(MediaFormat.KEY_HEIGHT, 960);
inputFormat.setInteger("rotation-degrees", 90);
String mime = inputFormat.getString(MediaFormat.KEY_MIME);
MediaCodec decoder = MediaCodec.createDecoderByType(mime);
decoder.configure(inputFormat, surface, null, 0);
Update Dec 25: I've tried different resolutions and orientations when configuring both encoder and decoder to check if the video's orientation is the problem or not, but the output video just got rotated, the green problem is still there.
I also tried "video/mp4v-es" for the encoder, the result video is viewable on Mac, but the iPhone cannot even play it.

I've just solved it. The reason turns out to be the MediaMuxer, it wraps the h264 raw stream in some sort of container that iOS cant understand. So instead of using MediaMuxer, I write the raw h264 stream from the encoder to a file, and use mp4parser to mux it into a mp4 file.

I know the answer now: It has to do with the fps range. I changed the fps rate on my camera params and on the media codec and suddenly it worked!

Android MediaCodec video wrong color and playing too fast

I am encoding a video using MediaCodec using Camera's setPreviewCallback.
(I follow this example Encoding H.264 from camera with Android MediaCodec).
I use the follow setting for the MediaCodec:
mediaCodec = MediaCodec.createEncoderByType("video/avc");
MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 1280, 720);
mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 8000000);
mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 30);
mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
mediaCodec.start();
My Camera setting are:
Camera.Parameters parameters = mCamera.getParameters();
parameters.setPreviewFormat(ImageFormat.NV21);
parameters.setPictureSize(previewWidth, 1280);
parameters.setPreviewSize(previewWidth, 720);
parameters.setPreviewFrameRate(30);
parameters.setPreviewFpsRange(5000,30000);
mCamera.setParameters(parameters);
I got a video but there are two problems:
its colors are wrong.
its playing too fast.
Here is an example video
http://www.youtube.com/watch?v=I1Eg2bvrHLM&feature=youtu.be
Does any of you guys know what are the causes of this two problem? And may be tell me some ways to solve this problem because I am totally lost/confused now.
thanks for reading and would appreciate any comments and opinions.

The YUV formats used by Camera output and MediaCodec input have their U/V planes swapped.
If you are able to move the data through a Surface you can avoid this issue; however, you lose the ability to examine the YUV data. An example of recording to a .mp4 file from a camera can be found on bigflake.
Some details about the colorspaces and how to swap them is in this answer.
There is no timestamp information in the raw H.264 elementary stream. You need to pass the timestamp through the decoder to MediaMuxer or whatever you're using to create your final output. If you don't, the player will just pick a rate, or possibly play the frames as fast as it can.

Android MediaCodec : Decoding is success but encoded file is not playable

I am trying to write a video compression application that will run on Jellybean version of Android. Till now I could decode the given input to video/raw format and it is playable with mplayer. My problem is that when I Encode this video/raw item into video/avc format with width = 320, height = 240, bitRate = (480*1024), frameRate = 20, iFrameInterval = 7 and colorFormat = YUV420Planar, the output file is not playable by just double clicking on it. Can anyone suggest me a way to play it using any player.? Or can you tell me if I had made any mistake in giving the above parameters like bitrate, framerate etc..
Thanks in advance.! :)