Develop Reference

How to write file to external storage using superpowered SDK in android?

While implementing a project using superpowered for Android audio effects like flunge, echo, reverb etc I am able to make recording through native and write file in external storage, then using superpowered crossfader example I open that file and apply effects on it that works fine.
Now I need to write the output file in external storage with applied effects but don't know how to do this.
There are iOS examples for this like SuperpoweredOfflineProcessingExample,
but I didn't find its solution for Android file. Any help will be highly appreciated to make audio output wav file with effects.

I had a requirement to apply effect on just recorded audio (so I have original wav and apply effect to it).
Here is a shapshot of method which apply effect on original file and save it to separate file:
applyEffect(const char *input, const char *output, int effectId) {
SuperpoweredDecoder *decoder = new SuperpoweredDecoder();
const char *openError = decoder->open(input, false);
if (openError) {
delete decoder;
return false;
};
FILE *fd = createWAV(output, decoder->samplerate, 2);
if (!fd) {
delete decoder;
return false;
};
float effectMix = 0.5f;
SuperpoweredFX *effect = NULL;
if (effectId == 0) {
effect = new SuperpoweredEcho(decoder->samplerate);
((SuperpoweredEcho *) effect)->setMix(effectMix);
} else if (effectId == 1) {
effect = new SuperpoweredReverb(decoder->samplerate);
((SuperpoweredReverb *) effect)->setMix(effectMix);
}
if (effect == NULL) {
delete decoder;
return false;
}
effect->enable(true);
// Create a buffer for the 16-bit integer samples coming from the decoder.
short int *intBuffer = (short int *)malloc(decoder->samplesPerFrame * 2 * sizeof(short int) + 16384);
// Create a buffer for the 32-bit floating point samples required by the effect.
float *floatBuffer = (float *)malloc(decoder->samplesPerFrame * 2 * sizeof(float) + 1024);
// Processing.
while (true) {
// Decode one frame. samplesDecoded will be overwritten with the actual decoded number of samples.
unsigned int samplesDecoded = decoder->samplesPerFrame;
if (decoder->decode(intBuffer, &samplesDecoded) == SUPERPOWEREDDECODER_ERROR) {
break;
}
if (samplesDecoded < 1) {
break;
}
// Apply the effect.
// Convert the decoded PCM samples from 16-bit integer to 32-bit floating point.
SuperpoweredShortIntToFloat(intBuffer, floatBuffer, samplesDecoded);
effect->process(floatBuffer, floatBuffer, samplesDecoded);
// Convert the PCM samples from 32-bit floating point to 16-bit integer.
SuperpoweredFloatToShortInt(floatBuffer, intBuffer, samplesDecoded);
}
// Write the audio to disk.
fwrite(intBuffer, 1, samplesDecoded * 4, fd);
};
// Cleanup.
closeWAV(fd);
delete decoder;
delete effect;
free(intBuffer);
free(floatBuffer);
return true;
}
New file will be created with applied effect.
Hope it will help you somehow!

Android encoded stream (H264/AAC) audio does not play in Flash Player

I'm working on adding a live broadcasting feature to an Android app. I do so through RTMP and make use of the DailyMotion Android SDK, which in turn makes use of Kickflip.
Everything works perfect, except for the playback of the audio on the website (which makes use of Flash). The audio does work in VLC, so it seems to be an issue with Flash being unable to decode the AAC audio.
For the audio I instantiate an encoder with the "audio/mp4a-latm" mime type. The Android developer docs state the following about this mime type: "audio/mp4a-latm" - AAC audio (note, this is raw AAC packets, not packaged in LATM!). I expect that my problem lies here, but yet I have not been able to find a solution for it.
Pretty much all my research, including this SO question about the matter pointed me in the direction of adding an ADTS header to the audio byte array. That results in the following code in the writeSampleData method:
boolean isHeader = false;
if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
isHeader = true;
} else {
pts = bufferInfo.presentationTimeUs - mFirstPts;
}
if (mFirstPts != -1 && pts >= 0) {
pts /= 1000;
byte data[] = new byte[bufferInfo.size + 7];
addADTStoPacket(data, bufferInfo.size + 7);
encodedData.position(bufferInfo.offset);
encodedData.get(data, 7, bufferInfo.size);
addDataPacket(new AudioPacket(data, isHeader, pts, mAudioFirstByte));
}
The addADTStoPacket method is identical to the one in the above mentioned SO post, but I will show it here regardless:
private void addADTStoPacket(byte[] packet, int packetLen) {
int profile = 2; //AAC LC
//39=MediaCodecInfo.CodecProfileLevel.AACObjectELD;
int freqIdx = 4; //44.1KHz
int chanCfg = 1; //CPE
// fill in ADTS data
packet[0] = (byte)0xFF;
packet[1] = (byte)0xF9;
packet[2] = (byte)(((profile-1)<<6) + (freqIdx<<2) +(chanCfg>>2));
packet[3] = (byte)(((chanCfg&3)<<6) + (packetLen>>11));
packet[4] = (byte)((packetLen&0x7FF) >> 3);
packet[5] = (byte)(((packetLen&7)<<5) + 0x1F)
packet[6] = (byte)0xFC;
}
The variables in the above method match the settings I have configured in the application, so I'm pretty sure that's fine.
The data is written to the output stream in the following method of the AudioPacket class:
#Override
public void writePayload(OutputStream outputStream) throws IOException {
outputStream.write(mFirstByte);
outputStream.write(mIsAudioSpecificConfic ? 0 : 1);
outputStream.write(mData);
}
Am I missing something here? I could present more code if necessary, but I think this covers the most related parts. Thanks in advance and I really hope someone is able to help, I've been stuck for a couple of days now...

Why AVCodecContext extradata is NULL?

I am trying to decode h264 video using ffmpeg and stagefright library. I'm using this example.
The example shows how to decode mp4 files, but i want to decode only h264 video.
Here is piece of my code..
AVFormatSource::AVFormatSource(const char *videoPath)
{
av_register_all();
mDataSource = avformat_alloc_context();
avformat_open_input(&mDataSource, videoPath, NULL, NULL);
for (int i = 0; i < mDataSource->nb_streams; i++)
{
if (mDataSource->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
{
mVideoIndex = i;
break;
}
}
mVideoTrack = mDataSource->streams[mVideoIndex]->codec;
size_t bufferSize = (mVideoTrack->width * mVideoTrack->height * 3) / 2;
mGroup.add_buffer(new MediaBuffer(bufferSize));
mFormat = new MetaData;
switch (mVideoTrack->codec_id == CODEC_ID_H264)
{
mConverter = av_bitstream_filter_init("h264_mp4toannexb");
mFormat->setCString(kKeyMIMEType, MEDIA_MIMETYPE_VIDEO_AVC);
if (mVideoTrack->extradata[0] == 1) //SIGSEGV Here
{
mFormat->setCString(kKeyMIMEType, MEDIA_MIMETYPE_VIDEO_AVC);
mFormat->setData(kKeyAVCC, kTypeAVCC, mVideoTrack->extradata,
mVideoTrack->extradata_size);
}
}
mFormat->setInt32(kKeyWidth, mVideoTrack->width);
mFormat->setInt32(kKeyHeight, mVideoTrack->height);
}
mVideoTrack->extradata is NULL. What i'm doing wrong?? My question is, what should be in mVideoTrack->extradata for kKeyAVCC ??
Please help me, I need Your help.
Thanks in advance.

If your input is a raw h.264 file, It is already in annex B format. So you do not need to do the "h264_mp4toannexb" conversion. In addition, in annex B, the SPS/PPS are sent inline with the first (or every) IDR frame. So no extra data is needed. Read more here: Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream

How to mix / overlay two mp3 audio file into one mp3 file (not concatenate)

I want to merge two mp3 files into one mp3 file.for example if 1st file is 1min and 2nd file is 30 sec then the output should be one min. In that one min it should play both the files.

First of all, in order to mix two audio files you need to manipulate their raw representation; since an MP3 file is compressed, you don't have a direct access to the signal's raw representation. You need to decode the compressed MP3 stream in order to "understand" the wave form of your audio signals and then you will be able to mix them.
Thus, in order to mix two compressed audio file into a single compressed audio file, the following steps are required:
decode the compressed file using a decoder to obtain the raw data (NO PUBLIC SYSTEM API available for this, you need to do it manually!).
mix the two raw uncompressed data streams (applying audio clipping if necessary). For this, you need to consider the raw data format obtained with your decoder (PCM)
encode the raw mixed data into a compressed MP3 file (as per the decoder, you need to do it manually using an encoder)
More info aboud MP3 decoders can be found here.

I am not sure if you want to do it on an Android phone (looks like that because of your tags), but if I'm right maybe try LoopStack, it's a mobile DAW (did not try it myself).
If you are just "mixing" two files without adjusting the output volume your output might clip. However I am not sure if it's possible to "mix" two mp3 files without decoding them.
If it is okay for you to merge them on your PC try Audacity, it's a free desktop DAW.

I have not done it in Android but I had done it using Adobe flex. I guess the logic remains the same. I followed the following steps:
I extracted both the mp3s into two byte arrays. (song1ByteArray, song2ByteArray)
Find out the bigger byte array. (Let's say song1ByteArray is the larger one).
Create a function which returns the mixed byte array.
private ByteArray mix2Songs(ByteArray song1ByteArray, ByteArray song2ByteArray){
int arrLength=song1ByteArray.length;
for(int i=0;i<arrLength;i+=8){ // here if you see we are incrementing the length by 8 because a sterio sound has both left and right channels 4 bytes for left +4 bytes for right.
// read left and right channel values for the first song
float source1_L=song1ByteArray.readFloat();// I'm not sure if readFloat() function exists in android but there will be an equivalant one.
float source1_R=song1ByteArray.readFloat();
float source2_L=0;
float source2_R=0;
if(song2ByteArray.bytesAvailable>0){
source2_L=song1ByteArray.readFloat();//left channel of audio song2ByteArray
source2_R=song1ByteArray.readFloat(); //right channel of audio song2ByteArray
}
returnResultArr.writeFloat((source_1_L+source_2_L)/2); // average value of the source 1 and 2 left channel
returnResultArr.writeFloat((source_1_R+source_2_R)/2); // average value of the source 1 and 2 right channel
}
return returnResultArr;
}

1. Post on Audio mixing in Android
2. Another post on mixing audio in Android
3. You could leverage Java Sound to mix two audio files
Example:
// First convert audiofile to audioinputstream
audioInputStream = AudioSystem.getAudioInputStream(soundFile);
audioInputStream2 = AudioSystem.getAudioInputStream(soundFile2);
// Create one collection list object using arraylist then add all AudioInputStreams
Collection list=new ArrayList();
list.add(audioInputStream2);
list.add(audioInputStream);
// Then pass the audioformat and collection list to MixingAudioInputStream constructor
MixingAudioInputStream mixer=new MixingAudioInputStream(audioFormat, list);
// Finally read data from mixed AudionInputStream and give it to SourceDataLine
nBytesRead =mixer.read(abData, 0,abData.length);
int nBytesWritten = line.write(abData, 0, nBytesRead);
4. Try AudioConcat that has a -m option for mixing
java AudioConcat [ -D ] [ -c ] | [ -m ] | [ -f ] -o outputfile inputfile ...
Parameters.
-c
selects concatenation mode
-m
selects mixing mode
-f
selects float mixing mode
-o outputfile
The filename of the output file
inputfile
the name(s) of input file(s)
5. You could use ffmpeg android wrapper using a syntax and approach as explained here

This guy used the JLayer library in a project quite similar to yours. He also gives you a guide on how to integrate that library in your android application directly recompiling the jar.
Paraphrasing his code it is very easy to accomplish your task:
public static byte[] decode(String path, int startMs, int maxMs)
throws IOException, com.mindtherobot.libs.mpg.DecoderException {
ByteArrayOutputStream outStream = new ByteArrayOutputStream(1024);
float totalMs = 0;
boolean seeking = true;
File file = new File(path);
InputStream inputStream = new BufferedInputStream(new FileInputStream(file), 8 * 1024);
try {
Bitstream bitstream = new Bitstream(inputStream);
Decoder decoder = new Decoder();
boolean done = false;
while (! done) {
Header frameHeader = bitstream.readFrame();
if (frameHeader == null) {
done = true;
} else {
totalMs += frameHeader.ms_per_frame();
if (totalMs >= startMs) {
seeking = false;
}
if (! seeking) {
SampleBuffer output = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream);
if (output.getSampleFrequency() != 44100
|| output.getChannelCount() != 2) {
throw new com.mindtherobot.libs.mpg.DecoderException("mono or non-44100 MP3 not supported");
}
short[] pcm = output.getBuffer();
for (short s : pcm) {
outStream.write(s & 0xff);
outStream.write((s >> 8 ) & 0xff);
}
}
if (totalMs >= (startMs + maxMs)) {
done = true;
}
}
bitstream.closeFrame();
}
return outStream.toByteArray();
} catch (BitstreamException e) {
throw new IOException("Bitstream error: " + e);
} catch (DecoderException e) {
Log.w(TAG, "Decoder error", e);
throw new com.mindtherobot.libs.mpg.DecoderException(e);
} finally {
IOUtils.safeClose(inputStream);
}
}
public static byte[] mix(String path1, String path2) {
byte[] pcm1 = decode(path1, 0, 60000);
byte[] pcm2 = decode(path2, 0, 60000);
int len1=pcm1.length;
int len2=pcm2.length;
byte[] pcmL;
byte[] pcmS;
int lenL; // length of the longest
int lenS; // length of the shortest
if (len2>len1) {
lenL = len1;
pcmL = pcm1;
lenS = len2;
pcmS = pcm2;
} else {
lenL = len2;
pcmL = pcm2;
lenS = len1;
pcmS = pcm1;
}
for (int idx = 0; idx < lenL; idx++) {
int sample;
if (idx >= lenS) {
sample = pcmL[idx];
} else {
sample = pcmL[idx] + pcmS[idx];
}
sample=(int)(sample*.71);
if (sample>127) sample=127;
if (sample<-128) sample=-128;
pcmL[idx] = (byte) sample;
}
return pcmL;
}
Note that I added attenuation and clipping in the last rows: you always have to do both when mixing two waveforms.
If you don't have memory/time requirements you can make an int[] of the sum of the samples and evaluate what is the best attenuation to avoid clipping.

To merge (overlap) two sound files, you can use This FFMPEG library.
Here is the Documentation
In their sample you can just enter the command you want. So lets talk about the command that we need.
-i [FISRST_FILE_PATH] -i [SECOND_FILE_PATH] -filter_complex amerge -ac 2 -c:a libmp3lame -q:a 4 [OUTPUT_FILE_PATH]
For first and second file paths, you will get the absolute path of the sound file.
1- If it is on storage then it is a sub folder for Environment.getExternalStorageDirectory().getAbsolutePath()
2- If it is assets so it should be a sub folder for file:///android_asset/
For the output path, Make sure to add the extension
ex.
String path = Environment.getExternalStorageDirectory().getAbsolutePath() + "/File Name.mp3"

I didn't get any fine solution.but we can do some trick here.. :)
You can assign both mp3 files to two different MediaPlayer object.then play both files at a time with a button.compare both mp3 files to find the longest duration.after that Use a AudioReorder to record to that duration. it will solve your problem..I know its not a right way but hope it will help you.. :)

How to encode non-camera video in Android

I am working on an android application in which a video is dynamically generated by compositing a sequence of animation frames. I tried to use the Android Media Recorder API for this but have not found a way to get it to accept a non-camera source as input. I have been attempting to use a FFMPEG port (based on the Rockplayer build) but am running into difficulties with missing functions since I am using it as an encoder, not a decoder.
The iPhone version of this app uses AVAssetWriter from the AVFoundation framework.
Is there an easier way to do this or am I stuck slugging it out with FFMPEG?

This may help (see the note on resolution though):-
How to encode using the FFMpeg in Android (using H263)
I'm not sure if they did a custom build of ffmpeg, or not, if so they may be able to offer advice on porting a more feature complete version.
-Anthony

Opencv has ViewBase class which takes the input from the camera as a frame and represent the frame as a bitmap , you can extand the class View base and make it for your own use , even though installing opencv on the android isn't very easy.
When you extend SampleCvViewBase you will have the following function which you can use pretty much hard work but the best I can think of.
#Override
protected Bitmap processFrame(VideoCapture capture) {
capture.retrieve(picture, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);
if (Utils.matToBitmap(picture, bmp))
return bmp;
bmp.recycle();
return null;
}

You can use a pure Java open source library called JCodec ( http://jcodec.org ).
It contains a simple yet working H.264 encoder and MP4 muxer. The class below uses JCodec low level API and should be what you need ( CORRECTED ):
public class SequenceEncoder {
private SeekableByteChannel ch;
private Picture toEncode;
private RgbToYuv420 transform;
private H264Encoder encoder;
private ArrayList<ByteBuffer> spsList;
private ArrayList<ByteBuffer> ppsList;
private CompressedTrack outTrack;
private ByteBuffer _out;
private int frameNo;
private MP4Muxer muxer;
public SequenceEncoder(File out) throws IOException {
this.ch = NIOUtils.writableFileChannel(out);
// Transform to convert between RGB and YUV
transform = new RgbToYuv420(0, 0);
// Muxer that will store the encoded frames
muxer = new MP4Muxer(ch, Brand.MP4);
// Add video track to muxer
outTrack = muxer.addTrackForCompressed(TrackType.VIDEO, 25);
// Allocate a buffer big enough to hold output frames
_out = ByteBuffer.allocate(1920 * 1080 * 6);
// Create an instance of encoder
encoder = new H264Encoder();
// Encoder extra data ( SPS, PPS ) to be stored in a special place of
// MP4
spsList = new ArrayList<ByteBuffer>();
ppsList = new ArrayList<ByteBuffer>();
}
public void encodeImage(BufferedImage bi) throws IOException {
if (toEncode == null) {
toEncode = Picture.create(bi.getWidth(), bi.getHeight(), ColorSpace.YUV420);
}
// Perform conversion
for (int i = 0; i < 3; i++)
Arrays.fill(toEncode.getData()[i], 0);
transform.transform(AWTUtil.fromBufferedImage(bi), toEncode);
// Encode image into H.264 frame, the result is stored in '_out' buffer
_out.clear();
ByteBuffer result = encoder.encodeFrame(_out, toEncode);
// Based on the frame above form correct MP4 packet
spsList.clear();
ppsList.clear();
H264Utils.encodeMOVPacket(result, spsList, ppsList);
// Add packet to video track
outTrack.addFrame(new MP4Packet(result, frameNo, 25, 1, frameNo, true, null, frameNo, 0));
frameNo++;
}
public void finish() throws IOException {
// Push saved SPS/PPS to a special storage in MP4
outTrack.addSampleEntry(H264Utils.createMOVSampleEntry(spsList, ppsList));
// Write MP4 header and finalize recording
muxer.writeHeader();
NIOUtils.closeQuietly(ch);
}
public static void main(String[] args) throws IOException {
SequenceEncoder encoder = new SequenceEncoder(new File("video.mp4"));
for (int i = 1; i < 100; i++) {
BufferedImage bi = ImageIO.read(new File(String.format("folder/img%08d.png", i)));
encoder.encodeImage(bi);
}
encoder.finish();
}
}
You can get JCodec jar from a project web-site.