So, I have encoded a h264 elementary stream with MediaCodec by collectings the frames using Camera's onPreviewFrame method.(using Encoding H.264 from camera with Android MediaCodec). Then, I generated an mp4 video using the h264 stream. Unfortunately, it doesn't have any audio in it.
I notice that MediaCodec should also allow encoding audio because it has settings for audio codecs.
Now is there any ways to add audio to the h264 stream?
thanks for reading and would appreciate any comments or suggestions.
A given instance of MediaCodec will encode either video or audio. You would need to create a second instance of MediaCodec to do the audio encoding, and then combine the streams with the MediaMuxer class (introduced in Android 4.3, API 18).
There are examples of using MediaMuxer on bigflake, but at the time I'm writing this there isn't one that demonstrates combining audio and video (they're just "muxing" the video into a .mp4 file). It should be enough to demonstrate how to use the class though.
Related
I'm trying to mux a mp4 file using mediaMuxer. I've already set SPS, PPS and replaced the 4-bit header. When the h.264 frame formed with single slice, everything is ok, but when I change it into multi-slice, the result is wrong.
sounds multi-slice is not fully supported in android mediaMuxer, it will cause compatible issue when plays on iPhone/QuickTime player.
There is some good documentation on this site called big flake about how to use media muxer and mediacodec to encode then decode video as mp4, or extract video then encode it again and more stuff.
But it doesnt seem that there is a way to encode audio with video at the same time, no documentation or code about this. It doesn't seem impossible.
Question
Do you know any stable way of doing it that will work on all devices greater than android 18?
Why no one implemented it, is it hard to implement?
You have to create 2 Mediacodec instances, one for video and one for audio and then use MediaMuxer to mux the video with audio after encoding, you can take a look at ExtractDecodeEditEncodeMuxTest.java and at this project to capture camera/mic and save to mp4 file using Mediamuxer and Mediacodec
I’m trying to show live preview from one android device to another.
Here is what I did,
Sender : 1.Camera frame(YUV) -> 2.Mediacodec (encode h264 byte[])-> 3.MediaMuxer-> 4.mp4
I'm sending output of media encoder via socket connection.
Receiver : 5.byte[] via socket ->6. Mediacodec (decoder) -> 7.Play.
Till step 5 everything works fine.
However I'm not able to decode the byte[]. What is missing here? I guess, I'm not able to send(don't know how to send) sps and pps properly! also how to test that what I'm sending is correct data?
Thanks.
You are muxing to mp4 (container format) the encoded h264 video. However, on the decoding side, you are not using demuxer (or parser/splitter). Media codec decoder can take elementary video not any container format.
Use mediaextractor to demux in android. (https://developer.android.com/reference/android/media/MediaExtractor.html)
I am developing an App that need audio-video encoding simultaneously. I checked the Google Grafika https://github.com/google/grafika CameraCaptureActivity, but it only encodes the video data. How can I encode audio data with other MediaCodec object and create a mp4 with MediaMuxer that contains both audio and video.
I have been asked to display a video stream (the stream is not from HTTP)in android, the stream is raw H.264 which is Recorded and encoded in a PC ,and I get it through WIFI.
When I get the stream, can use the MediaCodec decoder to decode the stream and display it?
Yes. Configure the MediaCodec as a "video/avc" decoder, and pass an output Surface to the configure() call.
The MediaCodec API is pretty low-level, and there's not a lot of sample code available. It might be easier to use MediaPlayer instead.
Update:
There's now a bunch of sample code here. Most of it makes use of Android 4.3 (API 18) features, but if you don't need MediaMuxer or Surface input to MediaCodec it'll work on API 16.
See the Video Encoding Recommendations here