Develop Reference

How to get smaller buffer size in multi-channel audio application with Oboe

I'm using Oboe 1.2 in an audio android application. When I call getFramesPerBurst(), which gives the endpoint buffer size, I get expected results (240 frames) if the number of output channels is set to 2. However when I set 4 output channels, the value returned by getFramesPerBurst() is around 960 (!). Is that normal ? Is that a limitation of the hardware (I tested on 4 different devices though, with different os version) ? A limitation of Oboe ? I notice also that this value is different than the value given by the property PROPERTY_OUTPUT_FRAMES_PER_BUFFER of AudioManager from the AudioService.
oboe::AudioStreamBuilder builder;
if (!oboe::AudioStreamBuilder::isAAudioRecommended()){
builder.setAudioApi(oboe::AudioApi::OpenSLES);
}
builder.setSharingMode(oboe::SharingMode::Exclusive);
builder.setFormat(oboe::AudioFormat::Float);
builder.setChannelCount(4);
builder.setCallback(&_oboeCallback);
builder.setPerformanceMode(oboe::PerformanceMode::LowLatency);
oboe::Result result = builder.openStream(&_stream);
if (result == oboe::Result::OK) {
int framePerBurst = _stream->getFramesPerBurst(); // gives value around 960 for 4 channels, 240 for 2 channels
_stream->setBufferSizeInFrames(2*framePerBurst);
}

Unless you are connecting to an audio device which actually has 4 independent channels (e.g. a USB audio interface or DJ controller like this one) then your 4 channel stream will need to be mixed into an N channel stream where N is the number of channels in your audio device. This could be 2 (stereo) for headphones or 1 (mono) for a built-in speaker.
The mixer introduces latency and larger buffer sizes. This is the difference in buffer sizes you see when you request a channel count of 2 vs 4.
For the lowest latency always leave the channel count unspecified when creating the stream, then do any channel count conversion inside your own app. There's an example of this here.

Call android's audiotrack to output custom square wave data, why is my square wave output wrong?

I am using the android audiotrack to output a square wave, check that the data generated by the square wave is not abnormal, but why is it outputting a horn waveform?
I checked all the possibilities, but I still don't know what went wrong. I confirm that the data can output a square wave, but the waveform of the android phone output will occasionally become this flared waveform. Is it related to the function or parameter of audioTrackF.write?
int afw = audioTrackF.write(mbitDateF, 0, mbitDateF.length, AudioTrack.WRITE_BLOCKING);

Android oboe c++ Some sounds distorted on playback

I'm using the Android oboe library for high performance audio in a music game.
In the assets folder I have 2 .raw files (both 48000Hz 16 bit PCM wavs and about 60kB)
std_kit_sn.raw
std_kit_ht.raw
These are loaded into memory as SoundRecordings and added to a Mixer. kSampleRateHz is 48000:
stdSN= SoundRecording::loadFromAssets(mAssetManager, "std_kit_sn.raw");
stdHT= SoundRecording::loadFromAssets(mAssetManager, "std_kit_ht.raw");
mMixer.addTrack(stdSN);
mMixer.addTrack(stdFT);
// Create a builder
AudioStreamBuilder builder;
builder.setFormat(AudioFormat::I16);
builder.setChannelCount(1);
builder.setSampleRate(kSampleRateHz);
builder.setCallback(this);
builder.setPerformanceMode(PerformanceMode::LowLatency);
builder.setSharingMode(SharingMode::Exclusive);
LOGD("After creating a builder");
// Open stream
Result result = builder.openStream(&mAudioStream);
if (result != Result::OK){
LOGE("Failed to open stream. Error: %s", convertToText(result));
}
LOGD("After openstream");
// Reduce stream latency by setting the buffer size to a multiple of the burst size
mAudioStream->setBufferSizeInFrames(mAudioStream->getFramesPerBurst() * 2);
// Start the stream
result = mAudioStream->requestStart();
if (result != Result::OK){
LOGE("Failed to start stream. Error: %s", convertToText(result));
}
LOGD("After starting stream");
They are called appropriately to play with standard code (as per Google tutorials) at required times:
stdSN->setPlaying(true);
stdHT->setPlaying(true); //Nasty Sound
The audio callback is standard (as per Google tutorials):
DataCallbackResult SoundFunctions::onAudioReady(AudioStream *mAudioStream, void *audioData, int32_t numFrames) {
// Play the stream
mMixer.renderAudio(static_cast<int16_t*>(audioData), numFrames);
return DataCallbackResult::Continue;
}
The std_kit_sn.raw plays fine. But std_kit_ht.raw has a nasty distortion. Both play with low latency. Why is one playing fine and the other has a nasty distortion?

I loaded your sample project and I believe the distortion you hear is caused by clipping/wraparound during mixing of sounds.
The Mixer object from the sample is a summing mixer. It just adds the values of each track together and outputs the sum.
You need to add some code to reduce the volume of each track to avoid exceeding the limits of an int16_t (although you're welcome to file a bug on the oboe project and I'll try to add this in an upcoming version). If you exceed this limit you'll get wraparound which is causing the distortion.
Additionally, your app is hardcoded to run at 22050 frames/sec. This will result in sub-optimal latency across most mobile devices because the stream is forced to upsample to the audio device's native frame rate. A better approach would be to leave the sample rate undefined when opening the stream - this will give you the optimal frame rate for the current audio device - then use a resampler on your source files to supply audio at this frame rate.

need to understad how AudioRecord and AudioTrack work for raw PCM capture and playback

I use the following code in a Thread to capture raw audio samples from the microphone and play it back through the speaker.
public void run(){
short[] lin = new short[SIZE_OF_RECORD_ARRAY];
int num = 0;
// am = (AudioManager) this.getSystemService(Context.AUDIO_SERVICE); // -> MOVED THESE TO init()
// am.setMode(AudioManager.MODE_IN_COMMUNICATION);
record.startRecording();
track.play();
while (passThroughMode) {
// while (!isInterrupted()) {
num = record.read(lin, 0, SIZE_OF_RECORD_ARRAY);
for(i=0;i<lin.length;i++)
lin[i] *= WAV_SAMPLE_MULTIPLICATION_FACTOR;
track.write(lin, 0, num);
}
// /*
record.stop();
track.stop();
record.release();
track.release();
// */
}
where record is an AudioRecord and track is an Audiotrack. I need to know in detail (and in a simplified way if possible) how the AudioRecord stores PCM data and AudioTrack plays PCM data. This is how I have understood it so far:
As the while() loop is continuously running, record obtains SIZE_OF_RECORD_ARRAY number of samples (which is 1024 for now) as shown in the figure. The samples get saved contiguously in the lin[] array of shorts (16 bit shorts, as I am using 16 bit PCM encoding). This is done by record.read(). Then track.write() places these samples in the speaker which is played by the hardware. Is this correct or am I missing something here?

As for how the samples are laid out in memory; they're just arrays of linear approximations to a sound wave, taken at discrete times (like your figure shows). In the case of stereo, the samples will be interleaved (LRLRLRLR...).
When it comes to the path the audio takes, you're essentially right, although there are a few more steps involved:
Writing data to your Java AudioTrack causes it to make a JNI (Java Native Interface) call to a native helper class, which in turn calls the native AudioTrack class.
The AudioTracks are owned by the AudioFlinger, which periodically takes data from all the AudioTracks on a given output thread (which have been mixed by the AudioMixer) and writes it to the audio HAL output stream class.
From there the data goes to the user-space ALSA library, and through a couple of intermediate steps to the kernel-space PCM driver. Then further on from there; typically going through some kind of DSP that applies various acoustic compensation filters, and eventually making it's way to the hardware codec, which controls the speaker DAC and amplifiers.
When recording from the internal microphone(s) you'd have more or less the same steps, except that they'd be done in the opposite order.
Note that some of these steps (essentially everything from the audio HAL and below) are platform-specific, and therefore might differ between platforms from different vendors (and even different platforms from the same vendor).

Play stereo tone in android

Similar topics on this question include only playing tone in mono where the left and right frequencies are the same.
My question is: how to generate a stereo tone such that the left channel has a different frequency than the right channel?
I thought of pre-recording .wav files but recording many .wav files and putting it on res folder is not a good idea.
i come across the SoundPool and AudioTrack class but I need a snippet showing the different frequencies of the left and right channel stored as buffer before playing the tone.
Or is there other ways? Please provide the desired snippet.

I have never tried this:
Initialize your left and right frequencies
//playback rate (1.0 = normal playback, range 0.5 to 2.0)
float lFrequency = 1.0;
float rFrequency = 1.0;
Initialize a SoundPool object
SoundPool sp = SoundPool(2, AudioManager.STREAM_MUSIC, 0);
Load your track twice (load function)
int sLeft = sp.load(mContext, R.raw.yourAudioFileId, 1);
int sRight = sp.load(mContext, R.raw.yourAudioFileId, 1);
Play the 2 sounds (one on Left and one on Right) using different rates (play function)
sp.play (sLeft, 1.0, 0.0, 0, 0, lFrequency);
sp.play (sRight, 0.0, 1.0, 0, 0, rFrequency);

From my understanding, if you use SoundPool it will play in stereo without any special configuration. From the documentation: "The SoundPool library uses the MediaPlayer service to decode the audio into a raw 16-bit PCM mono or stereo stream"
so as long as the file you play is in Stereo to begin with, it should play that way.

Instead of prerecording WAV files and embedding them in your application, you can instead generate the split-channel WAV audio in memory from code, and then either save it as a WAV file that you would play with SoundPool, or play the audio directly using AudioTrack (I'd recommend the latter).
The audio itself is just an array of (usually) 2-byte integers. With stereo, the left and right samples are interleaved throughout the array (so sample[0] is the first L sample, sample[1] is the first R sample, sample[2] is the second L sample etc.). So when your app starts up, you would create an array however long you need (with CD-quality audio, your array will need 88200 elements for each second of audio) then fill the samples with calculated values for your tone, then pass the array to AudioTrack for playing.
Or you would save the audio as a WAV file and play it with SoundPool (which might actually be better from a memory footprint standpoint). The WAV format is very simple to write (reading is more complicated): just a 44-byte header with various properties, and then the audio data itself.
I think I've answered a similar android question before, so I'm going to go see if I have some basic code for doing this.
Not me, but some good code for reading and writing WAV files in java:
http://computermusicblog.com/blog/2008/08/29/reading-and-writing-wav-files-in-java

I would use other software to generate an .ogg stereo file (don't use WAV files, they weight too much), wich is a very lightweight audio format wich works well in Android. I use the free Reaper, Audacity is easier, anyone would do. Just create two mono audio tracks, set their pan to full right and full left. Load your samples, and apply a pitch-change plugin to alter the frequency of one of the tracks. You can also generate them with a sintesizer. Then render all to a tone.ogg file.
Tone