I am working on a project that needs to process a video using OpenGL on Android. I decided to use MediaCodec and I managed to get it works with the help from ExtractDecodeEditEncodeMuxTest. The result is quite good, I have it receives a video, extracts the tracks, decodes the videotrack, edits with OpenGL, and encodes to a video file.
The problem is that the result video can be play well on Android, but when it comes to iOS, two-thirds of the screen is green.
I tried to solve with the suggestions from here, here, and here, experiment different formats for the encoder, but the problem is still the same.
Could someone suggest me the reasons that can cause this problem and how to fix it?
This is the video when it's played on iOS
This is the configuration for the encoder
MediaCodec mediaCodec = MediaCodec.createEncoderByType("video/avc");
MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 540, 960);
mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 2000000);
mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 30);
mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, CodecCapabilities.COLOR_FormatSurface);
mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
Update
I wonder if i had any mistake with the video orientation, because the working partial of the output video has the same ratio as the desired output resolution, but in horizontal orientation. The input is vertical recorded, so does the desired output.
Here is the code of the decoder configuration:
inputFormat.setInteger(MediaFormat.KEY_WIDTH, 540);
inputFormat.setInteger(MediaFormat.KEY_HEIGHT, 960);
inputFormat.setInteger("rotation-degrees", 90);
String mime = inputFormat.getString(MediaFormat.KEY_MIME);
MediaCodec decoder = MediaCodec.createDecoderByType(mime);
decoder.configure(inputFormat, surface, null, 0);
Update Dec 25: I've tried different resolutions and orientations when configuring both encoder and decoder to check if the video's orientation is the problem or not, but the output video just got rotated, the green problem is still there.
I also tried "video/mp4v-es" for the encoder, the result video is viewable on Mac, but the iPhone cannot even play it.
I've just solved it. The reason turns out to be the MediaMuxer, it wraps the h264 raw stream in some sort of container that iOS cant understand. So instead of using MediaMuxer, I write the raw h264 stream from the encoder to a file, and use mp4parser to mux it into a mp4 file.
I know the answer now: It has to do with the fps range. I changed the fps rate on my camera params and on the media codec and suddenly it worked!
Related
I'm compressing a video with audio on android using MediaCodec. When I run the code in a try/catch the output video is fine apart from the last part of the audio is missing.
I've had to rewrite the code and also convert it into Kotlin but nothing has worked so far. I've seen a few similar issues on here but the error code is slightly different as it is more of an audio issue whereas this looks like an issue with the video extraction.
Here's the code where the error is happening. The line with videoDecoder.queueInputBuffer is exactly the crash.
videoExtractorDone = !videoExtractor.advance()
if (videoExtractorDone) {
if (false) Log.d(TAG, "video extractor: EOS")
videoDecoder.queueInputBuffer(decoderInputBufferIndex, 0, 0, 0, videoDecoderOutputBufferInfo.flags)
}
How can I solve this issue so that the clip has full audio? I can't lose any of the audio from this process.
I have to decode an H.264 stream with the Android MediaCodec API. Problem is I don't know at this point the size of the output stream (I mean resolution of frames : 640x360? 1280x720? 1920x1080? ...). I use my own access/demuxer code to get the input stream as a ByteBuffer object. First step to create my MediaCodec is:
MediaCodec mediaCodec = MediaCodec.createDecoderByType("video/avc");
Then, to configure it, I would like to have the proper MediaFormat information to configure my MediaCodec object fine. Currently, I do:
MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 1280, 720);
But the size is not always 1280x720 and my MediaFormat needs more information (on some test devices, decoder produces weird results but works fine on others). I read the doc and I saw that we can get the proper MediaFormat using a MediaExtractor and getTrackFormat(/* idx */);. But how can I use MediaExtractor with my raw ByteBuffer (setDataSource(...) can only handle file descriptors or uris)? Is there an other way to process?
Thanks!
EDIT: I extracted SPS and PPS from my stream and provided it to the codec at the configuration step (passing by the MediaFormat) doing:
mediaFormat.setByteBuffer("csd-0", bufferSPS);
mediaFormat.setByteBuffer("csd-1", bufferPPS);
/*...*/
mediaCodec.configure(mediaFormat, null, null, 0);
This is my first question so please let me know if I missed anything!
I want to copy the video using android Media Codec class. (form.mp4 -> to.mp4)
I did decoding video and audio . but i have no idea how to incorde video and audio. I already saw http://bigflake.com/mediacodec/ this page. But i can't find simple example.
mFormat = MediaFormat.createVideoFormat("video/avc", mMetadata.V_WIDTH,mMetadata.V_HEIGHT);
mFormat.setInteger(MediaFormat.KEY_WIDTH, mMetadata.V_WIDTH);
mFormat.setInteger(MediaFormat.KEY_HEIGHT, mMetadata.V_HEIGHT);
mFormat.setInteger(MediaFormat.KEY_BIT_RATE, (int) mMetadata.BIT_RATE);
mFormat.setInteger(MediaFormat.KEY_FRAME_RATE, mMetadata.FRAME_RATE);
mFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
mFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, mMetadata.I_FRAME_INTERVAL);
videoEncoder = MediaCodec.createEncoderByType("video/avc");
videoEncoder.configure(mFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
videoEncoder.start();
I did Media-codec start. but I don't know how connect row data of video. please save me from this hell.
I am encoding a video using MediaCodec using Camera's setPreviewCallback.
(I follow this example Encoding H.264 from camera with Android MediaCodec).
I use the follow setting for the MediaCodec:
mediaCodec = MediaCodec.createEncoderByType("video/avc");
MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 1280, 720);
mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 8000000);
mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 30);
mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
mediaCodec.start();
My Camera setting are:
Camera.Parameters parameters = mCamera.getParameters();
parameters.setPreviewFormat(ImageFormat.NV21);
parameters.setPictureSize(previewWidth, 1280);
parameters.setPreviewSize(previewWidth, 720);
parameters.setPreviewFrameRate(30);
parameters.setPreviewFpsRange(5000,30000);
mCamera.setParameters(parameters);
I got a video but there are two problems:
its colors are wrong.
its playing too fast.
Here is an example video
http://www.youtube.com/watch?v=I1Eg2bvrHLM&feature=youtu.be
Does any of you guys know what are the causes of this two problem? And may be tell me some ways to solve this problem because I am totally lost/confused now.
thanks for reading and would appreciate any comments and opinions.
The YUV formats used by Camera output and MediaCodec input have their U/V planes swapped.
If you are able to move the data through a Surface you can avoid this issue; however, you lose the ability to examine the YUV data. An example of recording to a .mp4 file from a camera can be found on bigflake.
Some details about the colorspaces and how to swap them is in this answer.
There is no timestamp information in the raw H.264 elementary stream. You need to pass the timestamp through the decoder to MediaMuxer or whatever you're using to create your final output. If you don't, the player will just pick a rate, or possibly play the frames as fast as it can.
I am trying to write a video compression application that will run on Jellybean version of Android. Till now I could decode the given input to video/raw format and it is playable with mplayer. My problem is that when I Encode this video/raw item into video/avc format with width = 320, height = 240, bitRate = (480*1024), frameRate = 20, iFrameInterval = 7 and colorFormat = YUV420Planar, the output file is not playable by just double clicking on it. Can anyone suggest me a way to play it using any player.? Or can you tell me if I had made any mistake in giving the above parameters like bitrate, framerate etc..
Thanks in advance.! :)