Is there a way to record square (640x640) videos and concat them in Android? I looked up in the Internet and found some solutions. The solution seems to be "ffmpeg". However, to use ffmpeg I need to dive into NDK and build ffmpeg from its sources. Is there a solution by only using the Android SDK?
My basic needs are:
Record multiple videos (square format)
Resize captured videos (i.e. 480x480 to 640x640)
Concat captured videos
Rotate final video (clockwise 90)
Final output will be in mp4 or mpg format
Is there a solution by only using the Android SDK?
Not really.
Your primary video recording option is MediaRecorder, and it supports exactly nothing of what you list. For example, there is no requirement for any Android device to support taking square videos.
You are also welcome to use the camera preview stuff to assemble your own videos from individual frames. Vine does this, AFAIK. There, you could perhaps use existing Bitmap facilities to handle the cropping, resizing, and rotating. However, this will be slow, and doing this work in a way that can keep up with a reasonable frame rate will be difficult. Also, I do not know if there is a library that can stitch those frames together into a video, or blend in any sort of audio (camera previews are pure images).
Related
My android app plays videos in Exoplayer 2, and now I'd like to play a video backwards.
I searched around a lot and found only the idea to convert it to a gif and this from WeiChungChang.
Is there any more straight-forward solution? Another player or a library that implements this for me is probably too much to ask, but converting it to a reverse gif gave me a lot of memory problems and I don't know what to do with the WeiChungChang idea. Playing only mp4 in reverse would be enough tho.
Videos are frequently encoded such that the encoding for a given frame is dependent on one or more frames before it, and also sometimes dependent on one or more frames after it also.
In other words to create the frame correctly you may need to refer to one or more previous and one or more subsequent frames.
This allows a video encoder reduce file or transmission size by encoding fully the information for every reference frame, sometimes called I frames, but for the frames before and/or after the reference frames only storing the delta to the reference frames.
Playing a video backwards is not a common player function and the player would typically have to decode the video as usual (i.e. forwards) to get the frames and then play them in the reverse order.
You could extend ExoPlayer to do this yourself but it may be easier to manipulate the video on the server side if possible first - there exist tools which will reverse a video and then your players will be able to play it as normal, for example https://www.videoreverser.com, https://www.kapwing.com/tools/reverse-video etc
If you need to reverse it on the device for your use case, then you could use ffmpeg on the device to achieve this - see an example ffmpeg command to do this here:
https://video.stackexchange.com/a/17739
If you are using ffmpeg it is generally easiest to use via a wrapper on Android such as this one, which will also allow you test the command before you add it to your app:
https://github.com/WritingMinds/ffmpeg-android-java
Note that video manipulation is time and processor hungry so this may be slow and consume more battery than you want on your mobile device if the video is long.
Here is my scenario:
Download an avi movie from the web
Open a bitmap resource
Overlay this bitmap at the bottom of the movie on all frames in the background
Save the video on extarnal storage
The video length is 15 seconds usually
Is this possible to achieve using MediaMuxer ? Any info on the matter is gladly received
I've been looking to http://bigflake.com/mediacodec/#DecodeEditEncodeTest (Thanks #fadden) and it says there:
"Decoding the frame and copying it into a ByteBuffer with
glReadPixels() takes about 8ms on the Nexus 5, easily fast enough to
keep pace with 30fps input, but the additional steps required to save
it to disk as a PNG are expensive (about half a second)"
So having almost 1 sec/frame is not acceptable. From what I am thinking one way would be to save each frame as PNG, open it, add the bitmap overlay on it and then save it. However this would take an enormous time to accomplish.
I wonder if there is a way to do things like this:
Open video file from external storage
Start decoding it
Each decoded frame will be altered with the bitmap overlay in memory
The frame is sent to an encoder.
On iOS I saw that there a way to take the original audio + original video + an image and add them in a container and then just encode the whole thing...
Should I switch to ffmpeg ? How stable and compatible is ffmpeg ? Am I risking compatibility issues with android 4.0+ devices ? Is there a way to use ffmpeg to acomplish this ? I am new to this domain and still doing research.
Years later edit:
Years have passed since the question and ffmpeg isn't really easy to add to a commercial software in terms of license. How did this evolved? Newer versions of android are more capable on this with the default sdk?
Some more time later edit
I got some negative votes for posting info as an answer so I'll edit the original question. Here is a great library which, from my testing does apply watermark to video and does it with progress callback making it a lot easier to show progress to the user and also uses the default android sdks. https://github.com/MasayukiSuda/Mp4Composer-android
This library generate an Mp4 movie using Android MediaCodec API and apply filter, scale, and rotate Mp4.
Sample code, could look like:
new mp4Composer(sourcePath, destinationPath)
.filter(new GlWatermarkFilter(watermarkBitmap)
.listener(){
#Override
private void onProgress(double value){}
#Override
private void onCompleted(double value){
runOnUiThread( () ->{
showSneakbar
}
}
#Override
private void onCancelled(double value){}
#Override
private void onFailed(Exception e){}
}).start();
Testing on emulator, seems to work fine on android 8+ while on older generates a black video file.However, when testing on real device seems to work.
I don't know much about the MediaMuxer but ffmpeg does support overlaying functionality. FFMPEG has various filters one of them is overlay filter. What I understand is you want to overlay an image (i.e. png) on the video, ffmpeg surely is a useful framework to do this job. You can set the output format you can set the co-ordinates of the image which is to be overplayed.
E.g.
ffmpeg -i input.avi -i logo.png -filter_complex 'overlay=10:main_h-overlay_h-10' output.avi
Above command adds overlays logo.png on the input.avi video file in bottom left corner.
More information about the filters is available at following website,
https://www.ffmpeg.org/ffmpeg-filters.html#overlay-1
If this is a solution to your problem you need the C code equivalent to the above command. You also need to see the performance of the ffmpeg because it a pure software framework.
Hope I have understood your question correctly and this helps.
If you need do this without ffmpeg on Android device:
Start from : https://github.com/google/grafika
The answer on your question between Play video (PlayMovieActivity.java) and Record Gl App (RecordFBOActivity.java) examples.
Steps:
Setup mInputWindowSurface as Video Encoder Input Surface.
Decode frame from video stream using MoviePlayer as video (external) texture.
Draw this video texture on Surface.
Draw watermark on the same Surface over video texture.
Notify MediaCodec that surface ready for encoding:
mVideoEncoder.frameAvailableSoon();
mInputWindowSurface.setPresentationTime(timeStampNanos);
and then goto Step 2.
Don't forget to adjust speed of decoding. Just remove SpeedControlCallback which in example set to decode 60 FPS video.
Advantages of this way:
Media Codec use hardware decoder/encoder for video processing.
You can change bit rate of result video.
You can try INDE Media Pack - https://software.intel.com/en-us/articles/intel-inde-media-pack-for-android-tutorials
It has transcoding\remuxing functionality as MediaComposer class and several sample effects like JpegSubstituteEffect - it shows how substitute video frame by a picture from jpg file and TextOverlayEffect to overlay text on video frame etc. It could be easily enhanced to watermark effect
This is what worked for me:
ffmpeg -i input.avi -i logo.png -filter_complex 'overlay=10:main_h-overlay_h-10' -strict -2 output.avi
ffmpeg recommended the usage -strict -2 inorder to allow the usage of experimental codecs. without the inclusion, the accepted answer above fails to work.
I am working on application that does some real time image processing on camera frames. For that, I use preview callback's method onPreviewFrame. This works fine for cameras that support preview frames that have resolution at least 640x480 or larger. But when camera does not support such large camera preview resolution, application is programmed to refuse processing such frames. Now, the problem I have is with phones like Sony Xperia Go. It is a very nice device that can record video up to resolution 1280x720, but unfortunately maximum camera preview size is 480x320, which is too small for my needs.
What I would like to know is how to obtain these larger camera frames (up to 1280x720 or more)? Obviously it has to be possible because camera application has the ability to record videos in that resolution - therefore this application somehow must be able to access those larger frames. How to do the same from my application?
Application has to support Android 2.1 and later, but I would be very happy even if I find the solution for my problem only for Android 4.0 or newer.
This question is similar to http://stackoverflow.com/questions/8839109/processing-android-video-frame-by-frame-while-recording, but I don't need to save the video - I only need those high resolution video frames...
It seems the only thing you can do is decoding frames from MediaRecoder data.
You may use ffmpeg to decode recoreder data from LocalSocket.
Hope the following open source projects may help:
ipcamera-for-android: https://code.google.com/p/ipcamera-for-android/
spydroid-ipcamera: https://code.google.com/p/spydroid-ipcamera/
You should probably take a look at the OpenCV library.
It has methods that allow you to receive full frames.
I have an impression: video preview size is small, and is slow, slower than the set video recording frame rate.
I was once trying to look for solutions on this. It seems a better way is to get the video stream from the video recorder, then directly process the data from the video stream.
You could find some examples on Android ip-camera.
You can use this library:
https://github.com/natario1/CameraView
This library has addFrameProcessor listener that in process function has Frame parameter.
If you need to record video while frame processing, you need to use from takeVideoSnapshot function of CameraView. takeVideo stop frame processing until complete video recording in latest version I tested 2.6.4.
I'm making an app that takes a video and does some computation on the video. I need to carry out this computation on individual frames of the video. So, I have two questions:
Are videos in Android stored as a sequence of pictures? (I've seen a lot of Android devices that advertise having 25-30 fps cameras) If yes, can I, as a developer, get access to these frames that make up a video and how so?
If not, is there any way for me to generate at least 15-20 distinct frames per second from a video taken on an android device? (and of course, do the computation on those frames generated)
Videos are stored as videos. To manipulate frames one can use FFMPEG library. There are FFMPEG ports to Android such as in Dolphin opensource player. This would require C/C++ programming with NDK though.
Is it possible to record video with overlay view? While recording the video I have displayed one small image on the overlay view. What I want to do is I want those overlay image along with the video recorded. So when I will open that recorded video, I will be able to see that overlapped image that recorded with video also.
Friends, I need this solution ASAP. Please suggest proper solution :)
Unfortunately, there is no way in the current Android API to get between the camera input and the encoder. Any solution would either involve capturing frames from the video source, overlaying the additional image, and then including an encoder for the captured frames. Even in native code with NEON optimizations on a fast system, this is going to be a slow process. Alternatively, the whole stream could be post-processed in a similar fashion, but this would also require a decoder.
For future reference: This is possible using the CameraView library, at least in "snapshot video" mode.