I am working on a project where we need to record the rendered OpenGL surface. (for example if we use GLsurfaceView, we need to record the surface along with the audio from the MIC)
Presently I am using MediaRecorder API by setting the video source as the VIDEO_SOURCE_GRALLOC_BUFFER.
I am using the following sample as the base code
I wanted to know ....
Is this the right way? . Is there any better alternate ?
The sample test given in the link is recording the audio and video of the EGLSURFACE but it is not displayed properly.
What might be the reason? Any help/pointers is really appreciated.
thanks,
Satish
The code you reference isn't a sample, but rather internal test code that exercises a non-public interface. SurfaceMediaSource could change or disappear in a future release and break your code.
Update:
Android 4.3 (API 18) allows Surface input to MediaCodec. The EncodeAndMuxTest sample demonstrates recording OpenGL ES frames to a .mp4 file.
The MediaRecorder class doesn't take Surface input, so in your case you'd need to record the audio separately and then combine it with the new MediaMuxer class.
Update #2:
Android 5.0 (API 21) allows Surface input to MediaRecorder, which is often much more convenient than MediaCodec. If you neeed to use MediaCodec, there is an example showing three different ways of recording OpenGL ES output with it in Grafika's "record GL activity".
The MediaProjection class can also be useful for screen recording.
Related
I want to use GPUImage Android to process video in real time. I see example that creating pictures with different filters but I didnt find any example of recording video with filters. Is this possible with GPUImage Android?
android-gpuimage library does not support video recording but you can try using the android-gpuimage-videorecording library. It is a fork of the gpu-image for android that provides also the video recording functionality
android-gpuimage-videorecording
see the GPUImageMovieWriter class
It should point you in the right direction for developing your own video writer on top of GPUImage.
The idea is to:
draw on current screen surface
switch to encoder input surface and draw previous frame buffer again on it
switch back to screen surface
other useful links: EGL surface helper, Media encoder
GPUVideo-android
I know GPUVideo-android. This library apply video filter on generate an Mp4 and on ExoPlayer video and Video Recording with Camera2.
Android MediaCodec API is used this library.
Let's try camera record , video preview and video generate.
I want to use GPUImage Android to process video in real time. I see example that creating pictures with different filters but I didnt find any example of recording video with filters. Is this possible with GPUImage Android?
android-gpuimage library does not support video recording but you can try using the android-gpuimage-videorecording library. It is a fork of the gpu-image for android that provides also the video recording functionality
android-gpuimage-videorecording
see the GPUImageMovieWriter class
It should point you in the right direction for developing your own video writer on top of GPUImage.
The idea is to:
draw on current screen surface
switch to encoder input surface and draw previous frame buffer again on it
switch back to screen surface
other useful links:
EGL surface helper,
Media encoder
GPUVideo-android
This library apply video filter on generate an Mp4 and on ExoPlayer video and Video Recording with Camera2.
Android MediaCodec API is used this library.
This library has many types of filters.
Filters link
So, In my application, I am able to show effects(like blur filter, gaussian) to video that comes from Camera using GPUImage library.
Basically, I (library) will take the input from the Camera, get's the raw byte data, converts it into RGBA format from YUV format, then applies effects to this image and displays on the Surface of GLSurfaceView using OpenGL. finally, to the user, it looks like a video with effects applied.
Now I want to record the frames of Surface as a video using MediaCodec API.
but this discussion says that we can not pass a predefined Surface to the MediaCodec.
I have seen some samples at bigflake where he is creating Surface using MediaCodec.createInputSurface() but for me, Surface comes from the GLSurfaceView.
So, how can I record a frames of a Surface as a video?
I will record the audio parallelly, merge that video and audio using FFMPEG and present to the user as a Video with effects applied.
You can see a complete example of this in Grafika.
In particular, the "Show + capture camera" activity records camera output to .mp4. It also demonstrates applying some simple image processing techniques in the GL shader. It uses a GLSurfaceView and a convoluted dance to keep the recording going across orientation changes.
Also possibly of interest, the "Record GL app with FBO" activity records OpenGL ES rendering a couple different ways. It uses plain SurfaceView and is much more straightforward.
I'm looking at the class MediaRecorder of the Android SDK, and I was wondering if it can be used to record a video made from a Surface.
Example: I want to record what I display on my surface (a video game?) into a file.
As I said in the title: I'm not looking to record anything from the camera.
I think it is possible by overriding most of the class, but I'd very much like some ideas...
Beside, I'm not sure how the Camera class is used in MediaRecorder, and what I should get from my Surface to replace it.
Thank you for your interest!
PS: I'm looking at the native code used my MediaRecorder to have some clue, maybe it will inspire someone else:
http://www.netmite.com/android/mydroid/frameworks/base/media/jni/
The ability to record from a Surface was added in Android Lollipop. Here is the documentation:
http://developer.android.com/about/versions/android-5.0.html#ScreenCapture
Android 4.3 (API 18) adds some new features that do exactly what you want. Specifically, the ability to provide data to MediaCodec from a Surface, and the ability to store the encoded video as a .mp4 file (through MediaMuxer).
Some sample code is available here, including a patch for a Breakout game that records the game as you play it.
This is unfortunately not possible at the Java layer. All the communication between the Camera and Media Recorder happens in the native code layers, and there's no way to inject non-Camera data into that pipeline.
While Android 4.1 added the Media Codec APIs, which allow access to the device's video encoders, there's no easy-to-use way to take the resulting encoded streams and save them as a standard video file. You'd have find a library to do that or write one yourself.
You MAY wish to trace the rabbit hole from a different folder in AOSP
frameworks/av/media
as long as comfortable with NDK (C/C++, JNI, ...) and Android (permissions, ...)
Goes quite deep and am not sure about how far you can go on a non-rooted device.
Here's an article on how to draw on a EGLSurface and generate a video using MediaCodec:
https://www.sisik.eu/blog/android/media/images-to-video
This uses OpenGL ES but you can have MediaCodec provide a surface that you can then obtain a canvas from to draw on. No need for OpenGL ES.
how to play video using opengl es in android?
Your question is a bit vague. If all you want to do is to play a video in a GL surface, it's possible. See the Media Player sample code that Dave was referring to in his sample. All you have to do there is to replace the SurfaceView with a GLSurfaceView in both the MediaPlayerDemo_Video.java file as well as in the corresponding layout file (mediaplayer_2.xml).
Also you need to create a custom Renderer class (one that implements the GLSurfaceView.Renderer interface) and set it to your GLSurfaceView.
After you do all this, you will have your video playing on a GL surface, but that would be all.
If, on the other hand, you want to manipulate the video, i.e. to draw the video frames in a GL texture and add effects to it or transform it (for example scale, rotate, etc.), then I'm afraid this can't be done. The reason is that you don't have access to the raw video frames in your application.
I'm not sure why you're metioning OpenGL here, but probably the easiest way to play video is using the VideoView class. You'll want to have a look at the MediaPlayer class too.
You might find the Audio and Video page in the Android Developer Documentation helpful.
There's VideoView sample code and MediaPlayer sample code in the API demos provided with the Android SDK.