So, In my application, I am able to show effects(like blur filter, gaussian) to video that comes from Camera using GPUImage library.
Basically, I (library) will take the input from the Camera, get's the raw byte data, converts it into RGBA format from YUV format, then applies effects to this image and displays on the Surface of GLSurfaceView using OpenGL. finally, to the user, it looks like a video with effects applied.
Now I want to record the frames of Surface as a video using MediaCodec API.
but this discussion says that we can not pass a predefined Surface to the MediaCodec.
I have seen some samples at bigflake where he is creating Surface using MediaCodec.createInputSurface() but for me, Surface comes from the GLSurfaceView.
So, how can I record a frames of a Surface as a video?
I will record the audio parallelly, merge that video and audio using FFMPEG and present to the user as a Video with effects applied.
You can see a complete example of this in Grafika.
In particular, the "Show + capture camera" activity records camera output to .mp4. It also demonstrates applying some simple image processing techniques in the GL shader. It uses a GLSurfaceView and a convoluted dance to keep the recording going across orientation changes.
Also possibly of interest, the "Record GL app with FBO" activity records OpenGL ES rendering a couple different ways. It uses plain SurfaceView and is much more straightforward.
Related
I need to record video with timestamp on each video frame. I see a example in cts which use InputSurace.java and OutputSurface.java to connect Decoder and Encoder to transcode video files. Is it possible to reuse these two android java class to implement a timestamp video recorder?
I try to use OutputSurface as Camera preview output and use InputSurface as MediaCodec Encoder input but sounds like only record 2 or 3 frames then it hang there forever!
Take your time and explore this link to get an idea how to feed the Camera preview into a video file. Once you are confident about the mechanism, you should be able to feed the MediaCodec input surface with some kind of OpenGL magic to put extra graphics on the top of the Camera's preview. I would recommend to tweak the example code's drawExtra() as a start.
I want to use GPUImage Android to process video in real time. I see example that creating pictures with different filters but I didnt find any example of recording video with filters. Is this possible with GPUImage Android?
android-gpuimage library does not support video recording but you can try using the android-gpuimage-videorecording library. It is a fork of the gpu-image for android that provides also the video recording functionality
android-gpuimage-videorecording
see the GPUImageMovieWriter class
It should point you in the right direction for developing your own video writer on top of GPUImage.
The idea is to:
draw on current screen surface
switch to encoder input surface and draw previous frame buffer again on it
switch back to screen surface
other useful links: EGL surface helper, Media encoder
GPUVideo-android
I know GPUVideo-android. This library apply video filter on generate an Mp4 and on ExoPlayer video and Video Recording with Camera2.
Android MediaCodec API is used this library.
Let's try camera record , video preview and video generate.
I am doing a project on image processing stuff. I receive a raw h264 video stream in real time and decode it using MediaCodec. I have successfully displayed the decoded video on a TextureView or SurfaceView. Now I want to process each frame, do something to it using OpenCV4Android and show the updated video frame on the screen. I know OpenCV has a sample project that demonstrates how to process video frames from the phone camera, but I wonder how to do it if I have another video source.
Also I have some questions on TextureView:
What does the onSurfaceTextureUpdated() from SurfaceTextureListener do? If I call getBitmap() in this function, then does that mean I get each frame of the video? And what about SurfaceTexture.onFrameAvailableListener?
Is it possible to use a hidden TextureView as an intermediate, extract its frames for processing and render it back to another surface, say, OpenGL ES texture for displaying?
The various examples in Grafika that use Camera as input can also work with input from a video stream. Either way you send the video frame to a Surface.
If you want to work with a frame of video in software, rather than on the GPU, things get more difficult. You either have to receive the frame on a Surface and copy it to a memory buffer, probably performing an RGB-to-YUV color conversion in the process, or you have to get the YUV buffer output from MediaCodec. The latter is tricky because a few different formats are possible, including Qualcomm's proprietary tiled format.
With regard to TextureView:
onSurfaceTextureUpdated() is called whenever TextureView receives a new frame. You can use getBitmap() to get every frame of the video, but you need to pace the video playback to match your filter speed -- TextureView will drop frames if you fall behind.
You could create a "hidden TextureView" by putting other View elements on top of it, but that would be silly. TextureView uses a SurfaceTexture to convert the video frames to OpenGL ES textures, then renders them as part of drawing the View UI. The bitmap data is retrieved with glReadPixels(). You can just use these elements directly. The bigflake ExtractMpegFramesTest demonstrates this.
I want to use GPUImage Android to process video in real time. I see example that creating pictures with different filters but I didnt find any example of recording video with filters. Is this possible with GPUImage Android?
android-gpuimage library does not support video recording but you can try using the android-gpuimage-videorecording library. It is a fork of the gpu-image for android that provides also the video recording functionality
android-gpuimage-videorecording
see the GPUImageMovieWriter class
It should point you in the right direction for developing your own video writer on top of GPUImage.
The idea is to:
draw on current screen surface
switch to encoder input surface and draw previous frame buffer again on it
switch back to screen surface
other useful links:
EGL surface helper,
Media encoder
GPUVideo-android
This library apply video filter on generate an Mp4 and on ExoPlayer video and Video Recording with Camera2.
Android MediaCodec API is used this library.
This library has many types of filters.
Filters link
currently, I am playing a video on a GLSurfaceView using OpenGL ES 2.0. Now, I am searching for a way to encode this video played on the surface view into a MP4 video using MediaCodec.
Therefore, I found the bigflake example, which seems to solve my issue perfectly (http://bigflake.com/mediacodec/EncodeAndMuxTest.java.txt).
However, it seems that I am too stupid to set the input source right. This example uses mEncoder.createInputSurface() to create the input source, however I have a GLSurfaceView where the video is actually played. So how do I set my own surface as input source for the encoder?
Since you are using GLSurfaceView, you need to insert intercepting code in onDrawFrame(), while allocating surface in onSurfaceCreated().
Input surface can be created as usual after setting up encoder parameters.
Interceptor can be done in a form of copying egl scene into frame buffer via copying shader. And then do swapbuffer to encode frame.
Try look at tutorial for arbitrary elg scene capturing at
https://software.intel.com/en-us/articles/intel-inde-media-pack-for-android-tutorials-video-capturing-for-opengl-applications