I take camera preview frames from android camera in 640x480 (sufficient to me) and do some modifications over them. But now I need to encode those to new MP4 file (with audio).
Is this some how possible? I can't use ffmpeg due to its not so good license, but I've found Stagefright framework which should be probably capable of doing that. But I did not find any sort of official documentation or tutorials to do such a thing I need to do.
Is there a way to do it with this framework please? I don't need codes, I would be very glad just for pointing me the right direction.
There is one scenario where the use-case described is realized. Consider a scenario where the Camera output is fed to an OpenGL library where some effects are applied on the preview frames which need to be recorded.
Well in this case, you can use the traditional MediaRecorder with GrallocSource instead of CameraSource. The setup would look like thus:
MediaRecorder is set up with the GrallocSource. The input surfaces for recording are provided by the Camera + OpenGL combined operation which implement a SurfaceTextureClient. A good example for this can be found in SurfaceMediaSource_test modules.
stagefright is quite good if you must support API 9 and higher. But this framework is not official, as you saw. You can use the sample code in platform/frameworks/av at your risk.
The google source includes CameraSource, which provides the camera frames directly to the encoder. While this approach may be much more efficient (the pixels are not copied to the user space at all), it does not allow manipulation. It is possible to modify the C++ source, but I strongly recommend to access the Camera in Java, and pass the preview frames via JNI to stagefrght (OpenMAX) encoder. On some devices, this may force you to use software encoder. You must convert the video frames to YUV planar format for the encoder. See libyuv for optimized converters.
If you can restrict your support to API 16 and higher, it is safer to use the official Java MediaCdec API.
Related
I want to know, if we can manage Googles new CameraX api's vide capturing speed, for example record video in 2x speed or in slow mode. Any sugestion?
Currently you can't record a video in 2x speed or any other speed different than 1x using CameraX.
Alternatively you can use FFmpeg to create a slow/fast motion video from an input video.
There are few ways to create a slow/fast motion video using ffmpeg, see guide - this process can be done in a few ways, the first one mentioned in the linked page is lossless and requires no re encoding so it is very fast.
If you don't know much about ffmpeg there are plenty of articles online and guides.
for using it in android I personally prefer this library, but there are plenty more option out there.
I am trying to create a video from series of images in android.
I have come across these three options MediaCodec, ffmpeg using ndk and jcodec. Can someone let me know which one of them is best and easiest. I didn't find any proper documentation so can somebody please post their working example?
If you are talking about API 4.3+ in general you need to get input surface from encoder, copy image to the texture that comes along with the surface, put correct timestamp and send it back to encoder. and do it
fps (frames per second) * resulted video duration in seconds
times. encoder bitstream after encoder should go to the muxer, so finally you will get mp4 file.
It requires rather much coding:)
I suggest you to try free Intel Media Pack: https://software.intel.com/en-us/articles/intel-inde-media-pack-for-android-tutorials
It has a sample - JpegSubstituteEffect, it allows to create videos from images. The idea is to take a dummy video (black video and quiet audio) and to substitute all black frame by coping images. It could be easily enhanced to creating a video from series of images. I know a couple of applications in Google Play making the same using Media Pack
I tried JCodec 1.7 for Android. This is very simple compared to the other two options and works. There is class SequenceEncoder in the android package that accepts Bitmap instances and encodes them in to video. I ended up cloning this class into my app to override some of the settings e.g. fps. Problem with JCodec is that performance is dismal - encoding single 720x480 pixels frame takes just about 45 seconds. I wanted to do timelapse videos possibly at fullHD and was initially thinking any encoder will do as I was not expecting encoding a frame to take more than a second (minimal interval between frames in my app is 3 seconds). As you can guess with 45 seconds per frame JCodec is not a fit.
I am monitoring your question for other answers that may be helpful.
The MediaCodec/MediaMuxer way seems ok but it is insanely complex. I need to learn quite a bit about OpenGL ES, video formats and some Android voodoo to get this going. Ohh and this only works on the latest crop of phones 4.3+. This is real shame for Google with all of their claims to fame. I found some Stackoverflow discussions on the topic. Two sub-paths exist - the OpenGL way is device independent. There is another way which involves transcoding your RGB Bitmap data to YUV. the catch with YUV is that there are 3 flavours of it depending on the device HW - planar, semi planar and semi planar compressed (I am not sure if a 4th way is not coming in the future so...).
Here are couple useful links on the OpenGL way
CTS test - https://android.googlesource.com/platform/cts/+/jb-mr2-release/tests/tests/media/src/android/media/cts/ExtractDecodeEditEncodeMuxTest.java
Many CTS like tests - http://bigflake.com/mediacodec/#EncodeDecodeTest
Two of them seem to be relevant and useful
EncodeAndMuxTest - http://bigflake.com/mediacodec/EncodeAndMuxTest.java.txt
CameraToMpegTest - http://bigflake.com/mediacodec/CameraToMpegTest.java.txt (I believe this to be closest to what I need, just need to understand all the OpenGL voodoo and get my Bitmap in as texture filling up the entire frame i.e. projections, cameras and what not comes into play)
ffmpeg route does not seem direct enough too. Something in C++ accepting stuff from Java...I guess some weird conversions of the Bitmap to byte[]/ByteBuffer will be first needed - cpu intensive and slow. I actually have JPEG byte[] but am not sure this will come handy to the ffmpeg library. I am not sure if ffmpeg is taking leverage of the GPU or other HW acceleration so it may well end up at 10 seconds per frame and literally baking the phone.
FFmpeg can implement this task. You first need compile ffmpeg library in Android (refer to this article "How to use Vitamio with your own FFmpeg build")
You could refer the samples in FFmpeg to figure out how to implement your task.
In Android implement your task in C++; then use JNI to integrate the C++ code into your Android app.
I want to create live painting video as export feature for a painting application.
I can create a video with a series of images, with the use of a library ( FFMPEG or MediaCodec). But, this would require too much processing power to compare the images and encode the video.
While drawing, I know exactly which pixels are changed. So, I can save lot of processing if I can pass this info to FFMPEG, instead of having the FFMPEG figure this out from the images.
Is there away to efficiently encode the video for this purpose ?
It should not require "too much processing power" for MediaCodec. Because, for example, device is capable to write video in real time, some of them write full HD video.There's another thing : each MediaCodec's encoder requires pixel data in specific format, you should query API for supported capabilities before using the API. Also it will be tricky to make your app work on all devices with MediaCodec if your app produces only one pixel format, because probably not all of devices will support it(another words: different vendors have different MediaCodec implementation).
I want to decode and display h264 frame over my screen. Now all I wanted to do is to use omx codec to decode my frames as it decodes in hardware which helps achieve fps for high resolutions. Till now all I come to know is there are different way to access hardware decoder. I come across some names like stagefright, omxcodec, openmax. All are these are different things. I want to decode frame by frame. and upto my knowledge android libraries accept only file path or url of media. But here my requirement is different. I need to decode frame by frame. So can someone tell me where can I start digging? I come to know that in latest version of android, they have published api to decode. But I want to support all version of android. So any suggestion or guidance will be great.
Thanks.
You are right, indeed at the top level libstagefright library has a "stream" interface.
That is, you create audio- or video-source, configure codecs, and then start the process of encoding or decoding.
However, using an interface OMXCodec you can obtain access to the individual frames.
Unfortunately, I do not know any source of official OMXCodec documentations.
The only source - these are examples of the code in the Android repository.
If you want to know more about how to use OMXCodec, you can view the AwesomePlayer.cpp file
(http://gitorious.org/rowboat/frameworks-base/blobs/rowboat-ics/media/libstagefright/AwesomePlayer.cpp)
Using the library libstagefright is clear from the StagefrightRecorder.cpp (in the same repository)
Also useful information about the OMXCodec architecture is
here
P.S. Remember that the OMXCodec API not stable and may vary from version to version...
I am testing imaging algorithms using a android phone's camera as input, and need a way to consistently test the algorithms. Ideally I want to take a pre-recorded video feed and have the phone 'pretend' that the video feed is a live video from the camera.
My ideal solution would be where the app running the algorithms has no knowledge that the video is pre-recorded. I do not want to load the video file directly into the app, but rather read it in as sensor data if at all possible.
Is this approach possible? If so, any pointers in the right direction would be extremely helpful, as Google searches have failed me so far
Thanks!
Edit: To clarify, my understanding is that the Camera class uses a camera service to read video from the hardware. Rather than do something application-side, I would like to create a custom camera service that reads from a video file instead of the hardware. Is that doable?
When you are doing processing on a live android video feed you will need to build your own custom camera application that feeds you individual frames via the PreviewCallback interface that Android provides.
Now, simulating this would be a little bit tricky seen as the format for the preview frames will generally be in the NV21 format. If you are using a pre-recorded video, I don't think there is any clear way of reading frames one by one unless you try the getFrameAtTime method which will give you bitmaps in an entirely different format.
This leads me to suggest that you could probably test with these Bitmaps (though I'm really not sure what you are trying to do here) from the getFrameAtTime method. In order for this code to then work on a live camera preview, you would need to have to convert your NV21 frames from the PreviewCallback interface into the same format as the Bitmaps from getFrameAtTime, or you could then adapt your algorithm to process NV21 format frames. NV21 is a pretty neat format, presenting color and luminance data separately, but it can be tricky to use.