Develop Reference

How to use AudioTrack in static mode

I want to play float arrays on Android. I wanted to go the (what I thought was the) easy way first so thought I'd use AudioTrack. I chose to use static mode because the audio stream is not continuous. In the documentation it says:
In static buffer mode, copies the data to the buffer starting at offset 0, and the write mode is ignored. Note that the actual playback of this data might occur after this function returns.
So initially my implementation looked like:
// constructor ..
audioTrack = new AudioTrack(new AudioAttributes.Builder()
.setUsage(AudioAttributes.USAGE_ASSISTANCE_NAVIGATION_GUIDANCE)
.setContentType(AudioAttributes.CONTENT_TYPE_SONIFICATION).build(),
new AudioFormat.Builder()
.setEncoding(AudioFormat.ENCODING_PCM_FLOAT)
.setSampleRate(sample_rate)
.setChannelMask(AudioFormat.CHANNEL_OUT_STEREO).build(),
sample_rate * 2 * 10, AudioTrack.MODE_STATIC, AudioManager.AUDIO_SESSION_ID_GENERATE);
[...]
public ... () { // function
audioTrack.write(sound, 0, sound.length, AudioTrack.WRITE_NON_BLOCKING);
audioTrack.play()
Which lead to the first problem that a sound could only play once. I found I need to be calling stop() and reloadStaticData() after play(). But this results in the first sound not being played at all. And any following sounds to be played wrongly (meaning the sound is wrong).
I tried out putting stop() and reloadStaticData() after write() and before play(), then the first sound is played but the next ones are wrong (they're a cut off version of the first sound instead of a different sound).
From these experiences it seems to me like write() is not actually writing to the start of the buffer. In general how to use AudioTrack is completely unclear to me. stop() seems to stop playback. But I have to call it after play() to reset the playback head position.

Turns out my phone speakers couldn't play the sound... this is my solution
public void play_sound(float[] sound) {
if (audioTrack.getPlayState() == AudioTrack.PLAYSTATE_PLAYING) {
audioTrack.stop();
audioTrack.reloadStaticData();
}
float[] buffer = new float[buffer_size_in_bytes / 4];
for (int i = 0; i < sound.length; i++) {
buffer[i] = sound[i];
}
// Last arg is ignored.
audioTrack.write(buffer, 0, buffer.length, AudioTrack.WRITE_BLOCKING);
audioTrack.play();
}

Android MediaCodec decoder input/output frame count

I'm working on video transcoding in Android, and using the standard method as these samples to extract/decode a video. I test the same process on different devices with different video devices, and I found a problem on the frame count of decoder input/output.
For some timecode issues as in this question, I use a queue to record the extracted video samples, and check the queue when I got a decoder frame output, like the following codes:
(I omit the encoding-related codes to make it clearer)
Queue<Long> sample_time_queue = new LinkedList<Long>();
....
// in transcoding loop
if (is_decode_input_done == false)
{
int decode_input_index = decoder.dequeueInputBuffer(TIMEOUT_USEC);
if (decode_input_index >= 0)
{
ByteBuffer decoder_input_buffer = decode_input_buffers[decode_input_index];
int sample_size = extractor.readSampleData(decoder_input_buffer, 0);
if (sample_size < 0)
{
decoder.queueInputBuffer(decode_input_index, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
is_decode_input_done = true;
}
else
{
long sample_time = extractor.getSampleTime();
decoder.queueInputBuffer(decode_input_index, 0, sample_size, sample_time, 0);
sample_time_queue.offer(sample_time);
extractor.advance();
}
}
else
{
DumpLog(TAG, "Decoder dequeueInputBuffer timed out! Try again later");
}
}
....
if (is_decode_output_done == false)
{
int decode_output_index = decoder.dequeueOutputBuffer(decode_buffer_info, TIMEOUT_USEC);
switch (decode_output_index)
{
case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
{
....
break;
}
case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
{
....
break;
}
case MediaCodec.INFO_TRY_AGAIN_LATER:
{
DumpLog(TAG, "Decoder dequeueOutputBuffer timed out! Try again later");
break;
}
default:
{
ByteBuffer decode_output_buffer = decode_output_buffers[decode_output_index];
long ptime_us = decode_buffer_info.presentationTimeUs;
boolean is_decode_EOS = ((decode_buffer_info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0);
if (is_decode_EOS)
{
// Decoder gives an EOS output.
is_decode_output_done = true;
....
}
else
{
// The frame time may not be consistent for some videos.
// As a workaround, we use a frame time queue to guard this.
long sample_time = sample_time_queue.poll();
if (sample_time == ptime_us)
{
// Very good, the decoder input/output time is consistent.
}
else
{
// If the decoder input/output frame count is consistent, we can trust the sample time.
ptime_us = sample_time;
}
// process this frame
....
}
decoder.releaseOutputBuffer(decode_output_index, false);
}
}
}
In some cases, the queue can "correct" the PTS if the decoder gives error value (e.g. a lot of 0s). However, there are still some issues about the frame count of decoder input/output.
On an HTC One 801e device, I use the codec OMX.qcom.video.decoder.avc to decode the video (with MIME types video/avc). The sample time and PTS is matched well for the frames, except the last one.
For example, if the extractor feeds 100 frames and then EOS to the decoder, the first 99 decoded frames has the exactly same time values, but the last frame is missing and I get output EOS from the decoder. I test different videos encoded by the built-in camera, by ffmpeg muxer, or by a video processing AP on Windows. All of them have the last one frame disappeared.
On some pads with OMX.MTK.VIDEO.DECODER.AVC codec, things becomes more confused. Some videos has good PTS from the decoder and the input/output frame count is correct (i.e. the queue is empty when the decoding is done.). Some videos has consistent input/output frame count with bad PTS in decoder output (and I can still correct them by the queue). For some videos, a lot of frames are missing during the decoding. For example, the extractor get 210 frames in a 7 second video, but the decoder only output the last 180 frames. It is impossible to recover the PTS using the same workaround.
Is there any way to expect the input/output frame count for a MediaCodec decoder? Or more accurately, to know which frame(s) are dropped by the decoder while the extractor gives it video samples with correct sample time?

Same basic story as in the other question. Pre-4.3, there were no tests confirming that every frame fed to an encoder or decoder came out the other side. I recall that some devices would reliably drop the last frame in certain tests until the codecs were fixed in 4.3.
I didn't search for a workaround at the time, so I don't know if one exists. Delaying before sending EOS might help if it's causing something to shut down early.
I don't believe I ever saw a device drop large numbers of frames. This seems like an unusual case, as it would have been noticeable in any apps that exercised MediaCodec in similar ways even without careful testing.

Record audio loops through sample rates?

I am trying to read and understand audio in Android. In my search I came along this article. Here he has wrote a code to record audio in wav format. But there is one thing I don't fully understand, and that is the first loop of his code:
public class ExtAudioRecorder
{
private final static int[] sampleRates = {44100, 22050, 11025, 8000};
public static ExtAudioRecorder getInstanse(Boolean recordingCompressed)
{
ExtAudioRecorder result = null;
if(recordingCompressed)
{
result = new ExtAudioRecorder( false,
AudioSource.MIC,
sampleRates[3],
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT);
}
else
{
int i=0;
do
{
result = new ExtAudioRecorder( true,
AudioSource.MIC,
sampleRates[i],
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT);
} while((++i<sampleRates.length) & !(result.getState() == ExtAudioRecorder.State.INITIALIZING));
}
return result;
He gives like a basic information about it, but I don't get this completely. Does this has anything to do with the performance of different types of Android devices? Anyway, hope somebody can brighten this up for me :)

He is trying to initialize the audio recorder with different sample rates, from these {44100, 22050, 11025, 8000}.
Depending on the underlying hardware, not all sample rates may be supported by the device.
Although the documentation says:
"44100Hz is currently the only rate that is guaranteed to work on all devices, but other rates such as 22050, 16000, and 11025 may work on some devices."
I think the author has written code to make sure that if initialization at a sample rate fails, an attempt is made to initialize at some other sample rate, unless the initialization is successful, which is given by the check he is making in the loop condition.

android java audio dsp sites or android sound library?

anyone know of any usefull links for learning audio dsp for android?
or a sound library?
im trying to make a basic mixer for playing wav files but realised i dont know enough about dsp, and i cant find anything at all for android.
i have a wav file loaded into a byte array and an AudioTrack on a short loop.
how can i feed the data in?
i expect this post will be ignored but its worth a try.
FileInputStream is = new FileInputStream(filePath);
BufferedInputStream bis = new BufferedInputStream(is);
DataInputStream dis = new DataInputStream(bis);
int i = 0;
while (dis.available() > 0) {
byteData[i] = dis.readByte(); //byteData
i++;
}
final int minSize = AudioTrack.getMinBufferSize( 44100, AudioFormat.CHANNEL_CONFIGURATION_STEREO, AudioFormat.ENCODING_PCM_16BIT );
track = new AudioTrack( AudioManager.STREAM_MUSIC, 44100, AudioFormat.CHANNEL_CONFIGURATION_STEREO, AudioFormat.ENCODING_PCM_16BIT,
minSize, AudioTrack.MODE_STREAM);
track.play();
bRun=true;
new Thread(new Runnable() {
public void run() {
track.write(byteData, 0, minSize);
}
}).start();

I'll give this a shot just because I was in your position a few months ago...
If you already have the wav file audio samples in a byte array, you simple need to pass the samples to the audio track object (lookup the write() methods).
To mix audio together you simply add the sames from each track. For example, add the first sample from track 1 to track 2, add the second sample from track 1 to track 2 and so on. The end result would ideally be a third array containing the added samplws which you pass to the 'write' method of your audio track instance.
You must be mindful of clipping here. If your data type 'short' then the maximum value allowed is 32768. A simple way to ensure that your added samples do not exceed this limit is to peform the addition and store the result in a variable whose data type is larger than a short (eg. int) and evaluate the result. If it's greater than 32768 then make it equal to 32768 and cast it back to a short.
int result = track1[i] + track2[i];
if(result > 32768) {
result = 32768;
}
else if(result < -32768) {
result = -32768;
}
mixedAudio[i] = (short)result;
Notice how the snippet above also tests for the minimum range of a short.
Appologies for the lack of formatting here, I'm on my mobile phone on a train :-)
Good luck.

Android AudioRecord Supported Sampling Rates

I'm trying to figure out what sampling rates are supported for phones running Android 2.2 and greater. We'd like to sample at a rate lower than 44.1kHz and not have to resample.
I know that all phones support 44100Hz but was wondering if there's a table out there that shows what sampling rates are valid for specific phones. I've seen Android's documentation (
http://developer.android.com/reference/android/media/AudioRecord.html) but it doesn't help much.
Has anyone found a list of these sampling rates??

The original poster has probably long since moved on, but I'll post this in case anyone else finds this question.
Unfortunately, in my experience, each device can support different sample rates. The only sure way of knowing what sample rates a device supports is to test them individually by checking the result of AudioRecord.getMinBufferSize() is non negative (which means there was an error), and returns a valid minimum buffer size.
public void getValidSampleRates() {
for (int rate : new int[] {8000, 11025, 16000, 22050, 44100}) { // add the rates you wish to check against
int bufferSize = AudioRecord.getMinBufferSize(rate, AudioFormat.CHANNEL_CONFIGURATION_DEFAULT, AudioFormat.ENCODING_PCM_16BIT);
if (bufferSize > 0) {
// buffer size is valid, Sample rate supported
}
}
}

Android has AudioManager.getProperty() function to acquire minimum buffer size and get the preferred sample rate for audio record and playback. But yes of course, AudioManager.getProperty() is not available on API level < 17. Here's an example code sample on how to use this API.
// To get preferred buffer size and sampling rate.
AudioManager audioManager = (AudioManager) this.getSystemService(Context.AUDIO_SERVICE);
String rate = audioManager.getProperty(AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE);
String size = audioManager.getProperty(AudioManager.PROPERTY_OUTPUT_FRAMES_PER_BUFFER);
Log.d("Buffer Size and sample rate", "Size :" + size + " & Rate: " + rate);
Though its a late answer, I thought this might be useful.

Unfortunately not even all phones support the supposedly guaranteed 44.1kHz rate :(
I' ve been testing a Samsung GalaxyY (GT-S5360L) and if you record from the Camcorder source (ambience microphone), the only supported rates are 8kHz and 16kHz. Recording # 44.1kHz produces utter garbage and # 11.025kHz produces a pitch-altered recording with slightly less duration than the original sound.
Moreover, both strategies suggested by #Yahma and #Tom fail on this particular phone, as it is possible to receive a positive, minimum-buffer size from an unsupported configuration, and worse, I've been forced to reset the phone to get the audio stack working again, after attempting to use an AudioRecord class initialized from parameters that produce a supposedly valid, (non-exception raising) AudioTrack or AudioRecord instance.
I'm frankly a little bit worried at the problems I envision when releasing a sound-app to the wild. In our case, we are being forced to introduce a costly sample-rate-conversion layer if we expect to reuse our algorithms (expecting a 44.1kHz recording rate)on this particular phone model.
:(

I have a phone (Acer Z3) where I get a positive buffer size returned from AudioRecord.getMinBufferSize(...) when testing 11025 Hz. However, if I subsequently run
audioRecord = new AudioRecord(...);
int state = audioRecord.getState();
if (state != AudioRecord.STATE_INITIALIZED) ...
I can see that this sampling rate in fact does not represent a valid configuration (as pointed out by user1222021 on Jun 5 '12). So my solution is to run both tests to find a valid sampling rate.

This method gives the minimum audio sample rate supported by your device.
NOTE : You may reverse the for loop to get the maximum sample rate supported by your device (Don't forget to change the method name).
NOTE 2 : Though android doc says upto 48000(48khz) sample rate is supported ,I have added all the possible sampling rates (as in wikipedia) since who know new devices may record UHD audio in higher (sampling) framerates.
private int getMinSupportedSampleRate() {
/*
* Valid Audio Sample rates
*
* #see <a
* href="http://en.wikipedia.org/wiki/Sampling_%28signal_processing%29"
* >Wikipedia</a>
*/
final int validSampleRates[] = new int[] { 8000, 11025, 16000, 22050,
32000, 37800, 44056, 44100, 47250, 48000, 50000, 50400, 88200,
96000, 176400, 192000, 352800, 2822400, 5644800 };
/*
* Selecting default audio input source for recording since
* AudioFormat.CHANNEL_CONFIGURATION_DEFAULT is deprecated and selecting
* default encoding format.
*/
for (int i = 0; i < validSampleRates.length; i++) {
int result = AudioRecord.getMinBufferSize(validSampleRates[i],
AudioFormat.CHANNEL_IN_DEFAULT,
AudioFormat.ENCODING_DEFAULT);
if (result != AudioRecord.ERROR
&& result != AudioRecord.ERROR_BAD_VALUE && result > 0) {
// return the mininum supported audio sample rate
return validSampleRates[i];
}
}
// If none of the sample rates are supported return -1 handle it in
// calling method
return -1;
}

I'd like to provide an alternative to Yahma's answer.
I agree with his/her proposition that it must be tested (though presumably it varies according to the model, not the device), but using getMinBufferSize seems a bit indirect to me.
In order to test whether a desired sample rate is supported I suggest attempting to construct an AudioTrack instance with the desired sample rate - if the specified sample rate is not supported you will get an exception of the form:
"java.lang.IllegalArgumentException: 2756Hz is not a supported sample rate"

public class Bigestnumber extends AsyncTask<String, String, String>{
ProgressDialog pdLoading = new ProgressDialog(MainActivity.this);
#Override
protected String doInBackground(String... params) {
final int validSampleRates[] = new int[]{
5644800, 2822400, 352800, 192000, 176400, 96000,
88200, 50400, 50000, 48000,47250, 44100, 44056, 37800, 32000, 22050, 16000, 11025, 4800, 8000};
TrueMan = new ArrayList <Integer> ();
for (int smaple : validSampleRates){
if(validSampleRate(smaple) == true) {
TrueMan.add(smaple);
}}
return null;
}
#Override
protected void onPostExecute(String result) {
Integer largest = Collections.max(TrueMan);
System.out.println("Largest " + String.valueOf(largest));
}
}
public boolean validSampleRate(int sample_rate) {
AudioRecord recorder = null;
try {
int bufferSize = AudioRecord.getMinBufferSize(sample_rate, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
recorder = new AudioRecord(MediaRecorder.AudioSource.MIC, sample_rate, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
} catch(IllegalArgumentException e) {
return false;
} finally {
if(recorder != null)
recorder.release();
}
return true;
}
This Code will give you Max Supported Sample Rate on your Android OS. Just Declare ArrayList <Integer> TrueMan; in your beggining of the class. Then you can use high sample rate in AudioTrack and AudioRecord to get better sound quality. Reference.

Just some updated information here. I spent some time trying to get access to recording from the microphone to work with Android 6 (4.4 KitKat was fine). The error shown was the same as I got for 4.4 when using the wrong settings for sample rate/pcm etc. But my problem was in fact that the Permissions in AndroidManifest.xml are no longer sufficient to request access to the Microphone and in fact this now needs to be done run time:
https://developer.android.com/training/permissions/requesting.html