Using OpenCV 4.5.2 + FFMPEG on an android app
I'm trying to convert an .avi video file into a .mp4 file using x264, by running
ffmpeg -i input.avi -c:v libx264 output.mp4
The transcoding is processed correctly but, when I play the video, the colors are a bit... saturated?
This transcoding is part of the following flow:
Grab a .mov video file
Use OpenCV VideoCapture and VideoWriter to write text on the video frames (output is .avi)
Then I need to convert .avi file into .mp4 so it's reproducible on exoplayer.
In step 2. I'm looping all video frames and writing them to a new file, writing a text on them.
val videoWriter = VideoWriter(
outputFilePath,
VideoWriter.fourcc('M', 'J', 'P', 'G'),
15.0,
Size(1920.0, 1088.0),
true
)
val frame = Mat()
videoCapture.read(frame)
Imgproc.putText(
frame,
"This is a text",
Point(200.0, 200.0),
3,
5.0,
Scalar(
255.0,
124.0,
124.0,
255.0
),
1
)
videoWriter.write(frame)
I know that step 2. is probably not corrupting the frames because in my sample app, I'm displaying all frames in an ImageView, and they all match the original .mov video. So, my guess is that the issue is occurring on 3.
I'm using 'com.arthenica:mobile-ffmpeg-min-gpl:4.4' for android to execute the FFMPEG command as follows:
FFmpeg.executeAsync("-i $outputFilePath -c:v libx264 -y ${mp4File.path}")
where outputFilePath is the path for the .avi file and mp4File is an existing empty .mp4 file.
So I guess what I'm looking for is a way to have a lossless video color transcoding between .avi and .mp4 files.
Here's a screenshot of my sample app. The image on top is the last frame of the .avi video. The image on the bottom is the last frame played on a video player for the .mp4 transcoded video. This frame color difference is noticeable throughout the whole video.
EDIT: After some digging, I found out that the issue is that the VideoWritter is messing with the RGB colors. I still don't know the reason why this is happeninng.
Figured it out myself with some debug assitance from #llogan.
So, it looks like VideoCapture exports frames with BGR format, thus the Red and Blue colors being switched out. In order to fix my issue all I had to do was to convert the frame from BGR to RGB using the OpenCV utility method:
val frame = Mat()
val frame1 = Mat()
videoCapture.read(frame)
Imgproc.cvtColor(frame, frame1, Imgproc.COLOR_BGR2RGB)
videoWriter.write(frame1)
Related
I wanted to make a Video Editor application for Android. But I am not sure whether only FFMPEG is enough or not. In this application I want to add features like Trim, Split, Filter, Speed, Rotation, Transition, Text, Music etc. I think only FFMPEG is not enough, some other Library or Domain I have to know. Can any one help on this ?
ffmpeg can do all of these:
Trim: -ss and/or -t/-to options. trim and atrim filters.
Split: same as trim
Filter: See FFmpeg Filters.
Speed: setpts, atempo, rubberband filters
Rotation: rotate, transpose, hflip, vflip filters. -metadata rotate option. Also see How to rotate a video with FFmpeg?
Transition: xfade filter
Text: drawtext and subtitles filters. Also see Text on video ffmpeg.
Music: add as input file and mux. amerge and amix filters. Also see How to overlay/downmix two audio files using ffmpeg?, FFMPEG mux video and audio (from another file)
How to add text with animation on video in ffmpeg?
Now I am trying to add video management function on android app.
If you have a experiences on it, please help me.
I added the visible, marquee,several text lines animation on video in ffmpeg.
It works well.
Here is my code.
ffmpeg -i input.mp4 -vf "[in]drawtext=fontfile=/path/to/font.ttf: text='First Line': fontcolor=red: fontsize=40: x=(w-text_w)/2: y=if(lt(t\,3)\,(-h+((3*h-200)*t/6))\,(h-200)/2):enable='between(t,2.9,50)',drawtext=fontfile=/path/to/font.ttf: text='Second Line': fontcolor=yellow: fontsize=30: x=if(lt(t\,4)\,(-w+((3*w-tw)*t/8))\,(w-tw)/2): y=(h-100)/2:enable='between(t,3.5,50)',drawtext=fontfile=/path/to/font.ttf: text='Third Line': fontcolor=blue: fontsize=50: x=if(lt(t\,5)\,(2*w-((3*w+tw)*t/10))\,(w-tw)/2): y=h/2:enable='between(t,4.5,50)',drawtext=fontfile=/path/to/font.ttf: text='Fourth Line': fontcolor=black: fontsize=20: x=(w-text_w)/2: y=if(lt(t\,6)\,(2*h-((3*h-100)*t/12))\,(h+100)/2):enable='between(t,5.5,50)'[out]" out.mp4
Here "input.mp4" is your input video file and it will output with the name "out.mp4"
I have recorded a video from camera in my app and saved in device storage.Now I want to reverse the video such that it plays from backwards.i.e. if video is of 10 seconds then the last frame at 10th second will become first frame and it starts playing from there to 1st second first frame.I want to save the reversed video in a file.How should i proceed in that?
If you are prepared to use ffmpeg you can use this approach - it essentially breaks the video into frames and then builds it again in reverse order:
https://stackoverflow.com/a/8137637/334402
There are several ways to use ffmpeg in Android but the 'wrapper' approach is one which I have found a reasonable blend of performance and ease of use. Some example Android ffmpeg wrapper:
http://hiteshsondhi88.github.io/ffmpeg-android-java/
https://github.com/guardianproject/android-ffmpeg
It's worth being aware that this will be time-consuming on a Mobile - if you have the luxury of being able to upload to a server and doing the reversal there it might be quicker.
Thanks to Mick for giving me an idea to use ffmpeg for reversing video.
I have posted code at github for reversing video along with performing other video editing operation using ffmpeg and complete tutorial in my blog post here.
As written in my blog post,
For reversing video,first we need to divide video into segments with
duration of 10 seconds or less because reverse video command for
ffmpeg will not work for long duration videos unless your device has
32 GB of RAM.
Hence,to reverse a video-
1.Divide video into segments with duration of 10 seconds or less.
2.Reverse the segmented videos
3.Concatenate reversed segmented videos in reverse order.
For dividing video into segments with duration of 6 seconds we can use
the below command-
String[] complexCommand = {"-i", inputFileAbsolutePath, "-c:v",
"libx264", "-crf", "22", "-map", "0", "-segment_time", "6", "-g", "9",
"-sc_threshold", "0", "-force_key_frames", "expr:gte(t,n_forced*6)",
"-f", "segment", outputFileAbsolutePath};
Here,
-c:v libx264
encodes all video streams with libx264
-crf
Set the quality for constant quality mode.
-segment_time
time for each segment of video
-g
GOP size
-sc_threshold
set scene change threshold.
-force_key_frames expr:gte(t,n_forced*n)
Forcing a keyframe every n seconds
After segmenting video,we need to reverse the segmented videos.For
that we need to run a loop where each segmented video file will be
reversed.
To reverse a video with audio(without removing its audio) we can use
the below command-
String command[] = {"-i", inputFileAbsolutePath, "-vf", "reverse",
"-af", "areverse", outputFileAbsolutePath};
To reverse a video with audio removing its audio we can use the below
command-
String command[] = {"-i", inputFileAbsolutePath, "-an", "-vf",
"reverse", outputFileAbsolutePath};
To reverse a video without audio we can use the below command-
String command[] = {"-i",inputFileAbsolutePath, "-vf", "reverse",
outputFileAbsolutePath};
After reversing segmented videos,we need to concatenate reversed
segmented videos in reverse order.For that we sort videos on the basis
of last modified file using Arrays.sort(files,
LastModifiedFileComparator.LASTMODIFIED_REVERSE).
Then, to concatenate reversed segmented videos(with audio) we can use the below
command-
String command[] =
{"-i",inputFile1AbsolutePath,"-i",inputFile2AbsolutePath
.....,"-i",inputFileNAbsolutePath,"-filter_complex","[0:v0] [0:a0]
[1:v1] [1:a1]...[N:vN] concat=n=N:v=1:a=1 [v]
[a],"-map","[v]","-map","[a]", outputFileAbsolutePath};
To concatenate reversed segmented videos(without audio) we can use the below
command-
String command[] =
{"-i",inputFile1AbsolutePath,"-i",inputFile2AbsolutePath
.....,"-i",inputFileNAbsolutePath,"-filter_complex","[0:0] [1:0]
[2:0]...[N:0] concat=n=N:v=1:a=0",outputFileAbsolutePath};
Here,
-filter_complex [0:v0] [0:a0] [1:v1] [1:a1]…[N:vN] tells ffmpeg what streams to send to the concat filter.In the above case, video stream 0
[0:v0] and audio stream 0 [0:a0] from input 0,video stream 1 [1:v1]
and audio stream 1 [1:v1] from input 1 and so on.
concat filter is used to concatenate audio and video streams, joining
them together one after the other.The filter accepts the following
options:
n
Set the number of segments. Default is 2.
v
Set the number of output video streams, that is also the number of
video streams in each segment. Default is 1.
a
Set the number of output audio streams, that is also the number of
audio streams in each segment. Default is 0.
I recorded a Video for limited time. Now i want to fetch all frames of video. I am using the below code and by using it i am able to get frames but i am not getting all video frames. 3 to 4 frames are repeated then i got a different frame. But as we all know we can get 25- 30 frames in a second to display smooth video. How to get all frames.
for (int i = 0; i < 30; i++) {
Bitmap bArray = mediaMetadataRetriever.getFrameAtTime(
1000000 * i,
MediaMetadataRetriever.OPTION_CLOSEST);
savebitmap(bArray, 33333 * i);
}
I don't want to use NDK. I got this link don't know what should be the value for "argb8888". I am getting error here. Can anyone explain how to do it.
Getting frames from Video Image in Android
I faced the same problem before and the Android's MediaMetadataRetriever seems not appropriated for this task since it doesn't have a good precision.
I used a library called "FFmpegMediaMetadataRetriever" in android studio:
Add this line in your build.graddle under module app:
compile 'com.github.wseemann:FFmpegMediaMetadataRetriever:1.0.14'
Rebuild your project.
Use the FFmpegMediaMetadataRetriever class to grab frames with higher
precision:
FFmpegMediaMetadataRetriever med = new FFmpegMediaMetadataRetriever();
med.setDataSource("your data source");
and in your loop you can grab frame using:
Bitmap bmp = med.getFrameAtTime(i*1000000, FFmpegMediaMetadataRetriever.OPTION_CLOSEST);
To get image frames from video we can use ffmpeg.For integrating FFmpeg in android we can use precompiled libraries like ffmpeg-android.
To extract image frames from a video we can use below command
String[] complexCommand = {"-y", "-i", inputFileAbsolutePath, "-an",
"-r", "1/2", "-ss", "" + startMs / 1000, "-t", "" + (endMs - startMs)
/ 1000, outputFileAbsolutePath};
Here,
-y
Overwrite output files
-i
ffmpeg reads from an arbitrary number of input “files” specified by the -i option
-an
Disable audio recording.
-r
Set frame rate
-ss
seeks to position
-t
limit the duration of data read from the input file
Here in place of inputFileAbsolutePath you have to specify the absolute path of video file from which you want to extract images.
For complete code check out this on my repository .Inside extractImagesVideo() method I am running command for extracting images from video.
For complete tutorial regarding integration of ffmpeg library and using ffmpeg commands to edit videos, check out this post which I have written on my blog.
You need to do :
Decode the video.
Present the decoded images at least as fast as 24 images / second. I
suppose you can skip this step.
Save the decoded images.
It appears that decoding the video would be the most challenging step. People and companies have spent years developing codecs (encoder / decoder) for various video formats.
Use this library JMF for FFMPEG.
I am trying to write a video compression application that will run on Jellybean version of Android. Till now I could decode the given input to video/raw format and it is playable with mplayer. My problem is that when I Encode this video/raw item into video/avc format with width = 320, height = 240, bitRate = (480*1024), frameRate = 20, iFrameInterval = 7 and colorFormat = YUV420Planar, the output file is not playable by just double clicking on it. Can anyone suggest me a way to play it using any player.? Or can you tell me if I had made any mistake in giving the above parameters like bitrate, framerate etc..
Thanks in advance.! :)