Develop Reference

use FFMpeg in xamarin android

I'm using xamarin.android to create an android application. for working on the output of recorded audios in android for changing its pitch and the other thing with NWaves package, I need to change its format to .wav. I tried to use many audio convertors but all of them threw exceptions, but it should work with FFmpeg. However, I used this code but it doesn't generate any file.
List<string> cmd = new List<string>();
cmd.Add("ffmpeg");
cmd.Add("-i");
cmd.Add("/storage/emulated/0/Android/data/com.companyname.pushersvc/demo.3GP");
cmd.Add("/storage/emulated/0/Android/data/com.companyname.pushersvc/test.wav");
string cmdParams = string.Join(" ", cmd);
await FFMpeg.Xamarin.FFMpegLibrary.Run(
Application.Context,
cmdParams
);
return new FileInfo(path);
I read this and FFMpeg docs, but all of them used .mp4 convertor.
I also read ffmpeg doc for audio converting which you can see the code, which is not working

How to write metadata to mp4 file using mp4parser?

I'm using mp4parser to mux h264 and aac file which are re-encoded from orginal video file,how can I write the metadata of the original video to the new mp4 file? Or is there a common method to write metadata to mp4 file?

metadata and MP4 is a really problem. There is no generally supported specification. But this is only one part of the problem.
Prob (1): When to write metadata
Prob (2): What to write
Prob (1) is relatively easy to solve: Just extend the DefaultMp4Builder or the FragmentedMp4Builder on your own and override the
protected ParsableBox createUdta(Movie movie) {
return null;
}
with something meaningful. E.g.:
protected ParsableBox createUdta(Movie movie) {
UserDataBox udta = new UserDataBox();
CopyrightBox copyrightBox = new CopyrightBox();
copyrightBox.setCopyright("All Rights Reserved, me, myself and I, 2015");
copyrightBox.setLanguage("eng");
udta.addBox(copyrightBox);
return udta;
}
some people used that to write apple compatible metadata but even though there are some classes in my code I never really figured out what works and what not. You might want to have a look into Apple's specification here
And yes: I'm posting this a year to late.

It seems that the 'mp4parser' library (https://code.google.com/p/mp4parser/), supports writing Metadata to mp4 files in Android. However, I've found there's little-to-no documentation on how to do this, beyond a few examples in their codebase. I've had some luck with the following example, which writes XML metadata into the 'moov/udta/meta' box:
https://github.com/copiousfreetime/mp4parser/blob/master/examples/src/main/java/com/googlecode/mp4parser/stuff/ChangeMetaData.java

If you consider the alternatives you might want to look at JCodec for this purpose. It now has the org.jcodec.movtool.MetadataEditor API (and a matching CLI org.jcodec.movtool.MetadataEditorMain).
Their documentation contains many samples: http://jcodec.org/docs/working_with_mp4_metadata.html
So basically when you want to add some metadata you need to know what key(s) it corresponds to. One way to find out is to inspect a sample file that already has the metadata you need. For this you can run the JCodec's CLI tool that will just print out all the existing metadata fields (keys with values):
./metaedit <file.mp4>
Then when you know the key you want to work with you can either use the same CLI tool:
# Changes the author of the movie
./metaedit -f -si ©ART=New\ value file.mov
or the same thing via the Java API:
MetadataEditor mediaMeta = MetadataEditor.createFrom(new
File("file.mp4"));
Map<Integer, MetaValue> meta = mediaMeta.getItunesMeta();
meta.put(0xa9415254, MetaValue.createString("New value")); // fourcc for '©ART'
mediaMeta.save(false); // fast mode is off
To delete a metadata field from a file:
MetadataEditor mediaMeta = MetadataEditor.createFrom(new
File("file.mp4"));
Map<Integer, MetaValue> meta = mediaMeta.getItunesMeta();
meta.remove(0xa9415254); // removes the '©ART'
mediaMeta.save(false); // fast mode is off
To convert string to integer fourcc you can use something like:
byte[] bytes = "©ART".getBytes("iso8859-1");
int fourcc =
ByteBuffer.wrap(bytes).order(ByteOrder.BIG_ENDIAN).getInt();
If you want to edit/delete the android metadata you'll need to use a different set of fucntion (because it's stored differently than iTunes metadata):
./metaedit -sk com.android.capture.fps,float=25.0 file.mp4
OR alternatively the same through the API:
MetadataEditor mediaMeta = MetadataEditor.createFrom(new
File("file.mp4"));
Map<String, MetaValue> meta = mediaMeta.getKeyedMeta();
meta.put("com.android.capture.fps", MetaValue.createFloat(25.));
mediaMeta.save(false); // fast mode is off

extract all video frames in Android

I recorded a Video for limited time. Now i want to fetch all frames of video. I am using the below code and by using it i am able to get frames but i am not getting all video frames. 3 to 4 frames are repeated then i got a different frame. But as we all know we can get 25- 30 frames in a second to display smooth video. How to get all frames.
for (int i = 0; i < 30; i++) {
Bitmap bArray = mediaMetadataRetriever.getFrameAtTime(
1000000 * i,
MediaMetadataRetriever.OPTION_CLOSEST);
savebitmap(bArray, 33333 * i);
}
I don't want to use NDK. I got this link don't know what should be the value for "argb8888". I am getting error here. Can anyone explain how to do it.
Getting frames from Video Image in Android

I faced the same problem before and the Android's MediaMetadataRetriever seems not appropriated for this task since it doesn't have a good precision.
I used a library called "FFmpegMediaMetadataRetriever" in android studio:
Add this line in your build.graddle under module app:
compile 'com.github.wseemann:FFmpegMediaMetadataRetriever:1.0.14'
Rebuild your project.
Use the FFmpegMediaMetadataRetriever class to grab frames with higher
precision:
FFmpegMediaMetadataRetriever med = new FFmpegMediaMetadataRetriever();
med.setDataSource("your data source");
and in your loop you can grab frame using:
Bitmap bmp = med.getFrameAtTime(i*1000000, FFmpegMediaMetadataRetriever.OPTION_CLOSEST);

To get image frames from video we can use ffmpeg.For integrating FFmpeg in android we can use precompiled libraries like ffmpeg-android.
To extract image frames from a video we can use below command
String[] complexCommand = {"-y", "-i", inputFileAbsolutePath, "-an",
"-r", "1/2", "-ss", "" + startMs / 1000, "-t", "" + (endMs - startMs)
/ 1000, outputFileAbsolutePath};
Here,
-y
Overwrite output files
-i
ffmpeg reads from an arbitrary number of input “files” specified by the -i option
-an
Disable audio recording.
-r
Set frame rate
-ss
seeks to position
-t
limit the duration of data read from the input file
Here in place of inputFileAbsolutePath you have to specify the absolute path of video file from which you want to extract images.
For complete code check out this on my repository .Inside extractImagesVideo() method I am running command for extracting images from video.
For complete tutorial regarding integration of ffmpeg library and using ffmpeg commands to edit videos, check out this post which I have written on my blog.

You need to do :
Decode the video.
Present the decoded images at least as fast as 24 images / second. I
suppose you can skip this step.
Save the decoded images.
It appears that decoding the video would be the most challenging step. People and companies have spent years developing codecs (encoder / decoder) for various video formats.
Use this library JMF for FFMPEG.

OpenSL ES decode 24bit FLAC

I am trying to decode a FLAC file with 24bit sample format using OpenSL ES on Android. Originally, I had my SLDataFormat_PCM for the SLDataSink setup like this.
_pcm.formatType = SL_DATAFORMAT_PCM;
_pcm.numChannels = 2;
_pcm.samplesPerSec = SL_SAMPLINGRATE_44_1;
_pcm.bitsPerSample = SL_PCMSAMPLEFORMAT_FIXED_16;
_pcm.containerSize = SL_PCMSAMPLEFORMAT_FIXED_16;
_pcm.channelMask = SL_SPEAKER_FRONT_LEFT | SL_SPEAKER_FRONT_RIGHT;
_pcm.endianness = SL_BYTEORDER_LITTLEENDIAN;
This is working well for basically any data format. Luckily the samplesPerSec is not respected (I don't want resampling).
Now I want to support the full bit-depth of a FLAC file with 24bit samples. When using this format, it apparently performs a bit-depth conversion, because once I load the file, and then check the ANDROID_KEY_PCMFORMAT_BITSPERSAMPLE info, it is 16.
When I put bitsPerSample = SL_PCMSAMPLEFORMAT_FIXED_24; or SL_PCMSAMPLEFORMAT_FIXED_32, then OpenSL ES rejects it
E/libOpenSLES(22706): pAudioSnk: bitsPerSample=32
W/libOpenSLES(22706): Leaving Engine::CreateAudioPlayer (SL_RESULT_CONTENT_UNSUPPORTED)
Any idea how this is meant to work? Is Android currently restricted to 16 bit int only?
I would also accept 32bit float, but I don't suppose that will work either.

Currently it only supports 8 and 16 bits
Sources:
Android source code (line 60)
Article (PCM data format section)

Retrieve album art using FFmpeg

I'm developing an Android application that relies on FFmpeg to retrieve audio metadata. I know it's possible to retrieve album art programmatically using FFMpeg. However, once you have decoded the art (a video frame within an MP3) how do generate an image file (a PNG) for use within an application? I've search all over but can't seem to find a working example.
Edit, here is the solution:
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
void retrieve_album_art(const char *path, const char *album_art_file) {
int i, ret = 0;
if (!path) {
printf("Path is NULL\n");
return;
}
AVFormatContext *pFormatCtx = avformat_alloc_context();
printf("Opening %s\n", path);
// open the specified path
if (avformat_open_input(&pFormatCtx, path, NULL, NULL) != 0) {
printf("avformat_open_input() failed");
goto fail;
}
// read the format headers
if (pFormatCtx->iformat->read_header(pFormatCtx) < 0) {
printf("could not read the format header\n");
goto fail;
}
// find the first attached picture, if available
for (i = 0; i < pFormatCtx->nb_streams; i++)
if (pFormatCtx->streams[i]->disposition & AV_DISPOSITION_ATTACHED_PIC) {
AVPacket pkt = pFormatCtx->streams[i]->attached_pic;
FILE* album_art = fopen(album_art_file, "wb");
ret = fwrite(pkt.data, pkt.size, 1, album_art);
fclose(album_art);
av_free_packet(&pkt);
break;
}
if (ret) {
printf("Wrote album art to %s\n", album_art_file);
}
fail:
av_free(pFormatCtx);
// this line crashes for some reason...
//avformat_free_context(pFormatCtx);
}
int main() {
avformat_network_init();
av_register_all();
const char *path = "some url";
const char *album_art_file = "some path";
retrieve_album_art(path, album_art_file);
return 0;
}

To use ffmpeg programmatically, I think you would have to call read_apic() in libavformat (which is part of ffmpeg).
From the commandline, you can apparently do this:
ffmpeg -i input.mp3 -an -vcodec copy cover.jpg
The commandline behaviour implies that the cover art image is seen as just another video stream (containing just one frame), so using libavformat in the usual way you would to demux the video part of a stream should produce that image.
Sample code for demuxing: ffmpeg/docs/examples/demuxing.c The first (and only) AVPacket that would be obtained from demuxing the video stream in an mp3 would contain the JPEG file (still encoded as JPEG, not decoded).
AVFormatContext* fmt_ctx;
// set up fmt_ctx to read first video stream
AVPacket pkt;
av_read_frame(fmt_ctx, &pkt);
FILE* image_file = fopen("image.jpg", "wb");
int result = fwrite(pkt.data, pkt.size, 1, image_file);
fclose(image_file);
If there are multiple images, I think they would be seen as separate video streams, rather than as separate packets in the same stream. The first stream would be the one with the largest resolution.
All this is probably implemented internally in terms of read_apic().
The ID3v2 spec allows for any image format, but recommends JPEG or PNG. In practice all images in ID3 are JPEG.
EDIT: Moved some of the less useful bits to postscript:
P.S. ffmpeg -i input.mp3 -f ffmetadata metadata.txt will produce an ini-like file containing the metadata, but the image is not even referred to in there, so that is not a useful approach.
P.S. There may be multiple images in an ID3v2 tag. You may have to handle the case when there is more than one image or more than one type of image present.
P.S. ffmpeg is probably not the best software for this. Use id3lib, TagLib, or one of the other implementations of ID3. These can be used either as libraries (callable from the language of your choice) or as commandline utilities. There is sample C++ code for TagLib here: How do I use TagLib to read/write coverart in different audio formats? and for id3lib here: How to get album art from audio files using id3lib.

As an addition the answer above, I needed a way to resize the output image as well so I found the below command while experimenting with the command in the current answer:
ffmpeg -i input.mp3 -filter:v scale=-2:250 -an output.jpeg
So this basically scales the output image to whatever ratio or value you want.