I'm getting a very peculiar issue with my audio callbacks in my Android app (that's using NDK/OpenSL ES). I'm streaming audio output at 44.1 kHz and 512 frames (which gives me a callback time of 11.6 ms). In the callback, I'm synthesizing a couple of waveforms, filters, etc (like a synthesizer). Due to optimization I never reach over 5 ms of the callback time. However, when I turn on a specific effect (digital delay line), it starts to take a radically longer time in the callback. The digital delay line will jump from 7.5 ms (after all voices/filters have been processed) and jump up to 100 to 350 ms.
This is the most confusing part; after maybe 1 or 2 seconds, the digital delay execution time will jump from the extremely high time to 0.2 ms completion time per callback.
Why would the Android app take a long time to complete my digital delay processing code the first few callbacks and then die down to a very short and audio-happy time? I'm kind of at a loss right now and not sure how to fix this. To confirm, this only happens with the delay processing method. It's just a standard digital delay line (you can find some on github) and I feel like the algorithm isn't the problem here...
Kind of a pseudocode/rough sketch of what my audio callback code looks like:
static bool myAudioCallback(void *userData, short int *audIO, int numSamples, int srate) {
AudioData *data = (AudioData *)userData;
// Resets pointer array values to 0
for (int i = 0; i < numSamples; i++) data->buffer[i] = 0;
// Voice Generation Block
for (int voice = 0; voice < data->numVoices; voice++) {
// Reset voice buffers:
for (int i = 0; i < numSamples; i++) data->voiceBuffer[i] = 0;
// Generate Voice
data->voiceManager[voice]->generateVoiceBlock(data->voiceBuffer, numSamples);
// Sum voices
for (int i = 0; i < numSamples; i++) data->buffer[i] += data->voiceBuffer[i]];
}
// When app first starts, delayEnabled = false so user must click on a
// button on the UI to enable it.
// Trouble is that when we enable processDelay(double *buffer, in frames) the
// first time, we get a long execution time.
if (data->delayEnabled) {
data->delay->processDelay(data->buffer, numSamples);
}
// Conversion loop
for (int i = 0; i < numSamples; i++) {
double sample = clipOutput(data->buffer[i]);
audIO[2*i] = audIO[(2*i)+1] = CONV_FLT_TO_16BIT(sample * data->volume);
}
}
Thanks!
Not a great answer to the solution but this is what I did:
Before the user is able to do anything on the app, I turned on the delay and let it run its course for like 2 seconds before switching it off. This allows the callback to do its weird long 300 ms execution time while not destroying the audio.
Obviously this is not a great answer and if anyone can figure out a more logical explanation I would be more than happy to mark that as the answer.
Related
I'm started to implement Oboe c++ library for Android.
(According to Build a Musical Game using Oboe
I just scale the sample for increasing the volume and it works but with crackling popping.
can I increase the amplitude without getting the crackling popping?
I tried to save my sample sounds with a little bit gain but it sounds very bad.
Thanks.
Btw without increasing the volume, it sounds clear but very low volume compared to other music apps.
for (int i = 0; i < mNextFreeTrackIndex; ++i) {
mTracks[i]->renderAudio(mixingBuffer, numFrames);
for (int j = 0; j < numFrames * kChannelCount; ++j) {
audioData[j] += (mixingBuffer[j] * ((float)volume));
}
Edited:
int16_t Mixer::hardLimiter(int16_t sample) {
int16_t audioData = sample * volume;
if(audioData >= INT16_MAX){
return INT16_MAX;
}else if(audioData <= INT16_MIN){
return INT16_MIN;
}
return audioData;
};
The code which you've posted is from Mixer::renderAudio(int16_t *audioData, int32_t numFrames). Its job is to mix the sample values from the individual tracks together into a single array of 16-bit samples.
If you're mixing 2 or more tracks together without reducing the values first then you may exceed the maximum sample value of 32,767 (aka INT16_MAX). Doing so would cause wraparound (i.e. writing 32,768 to an int16_t will result in a value of -32,768 being stored) and therefore audible distortion/crackling.
With this in mind you could write a very basic (hard) limiter - do your volume scaling using an int32_t and only write it into the int16_t array if the value doesn't exceed the maximum, otherwise just write the maximum.
This isn't really a good approach though because you shouldn't be hitting the limits of 16-bit values. Better would be to scale down your input sample values first, then add a gain stage after the mixer (or on individual tracks inside the mixer) to bring the overall amplitude up to an acceptable level.
I'm trying to get a precise clock that is not influenced by other processes inside the app.
I currently use System.nanoTime() like below inside a thread.
I use to calculate the timing of each of the sixteen step.
Currently timed operations have sometime a perceptible delay that i try to fix.
I would like to know if there is a more precise way to obtaining timed operations, maybe by check the internal soundcard clock and use it to generate the timing i need.
I need it to send midi notes from android device to external audio sinthetizers and for audio i need precise timing
Is there anyone who can help me improve this aspect?
Thanks
cellLength = (long)(0.115*1000000000L);
for ( int x = 0; x < 16; x++ ) {
noteStartTimes[x] = x*cellLength ;
}
long startTime = System.nanoTime();
index = 0;
while (isPlaying) {
if (noteStartTimes[index] < System.nanoTime() - startTime) {
index++ ;
if (index == 16) { //reset things
startTime = System.nanoTime() + cellLength;
index = 0 ;
}
}
}
For any messages that you receive, the onSend callback gives you a timestamp.
For any messages that you send, you can provide a timestamp.
These timestamps are based on System.nanoTime(), so your own code should use this as well.
If your code is delayed (by its own processing, or by other apps, or by background services), System.nanoTime() will accurately report the delay. But no timer will function can make your code run earlier.
I'm curious about this.
I wanted to check which function was faster, so I create a little code and I executed a lot of times.
public static void main(String[] args) {
long ts;
String c = "sgfrt34tdfg34";
ts = System.currentTimeMillis();
for (int k = 0; k < 10000000; k++) {
c.getBytes();
}
System.out.println("t1->" + (System.currentTimeMillis() - ts));
ts = System.currentTimeMillis();
for (int i = 0; i < 10000000; i++) {
Bytes.toBytes(c);
}
System.out.println("t2->" + (System.currentTimeMillis() - ts));
}
The "second" loop is faster, so, I thought that Bytes class from hadoop was faster than the function from String class. Then, I changed the order of the loops and then c.getBytes() got faster. I executed many times, and my conclusion was, I don't know why, but something happen in my VM after the first code execute so that the results become faster for the second loop.
This is a classic java benchmarking issue. Hotspot/JIT/etc will compile your code as you use it, so it gets faster during the run.
Run around the loop at least 3000 times (10000 on a server or on 64 bit) first - then do your measurements.
You know there's something wrong, because Bytes.toBytes calls c.getBytes internally:
public static byte[] toBytes(String s) {
try {
return s.getBytes(HConstants.UTF8_ENCODING);
} catch (UnsupportedEncodingException e) {
LOG.error("UTF-8 not supported?", e);
return null;
}
}
The source is taken from here. This tells you that the call cannot possibly be faster than the direct call - at the very best (i.e. if it gets inlined) it would have the same timing. Generally, though, you'd expect it to be a little slower, because of the small overhead in calling a function.
This is the classic problem with micro-benchmarking in interpreted, garbage-collected environments with components that run at arbitrary time, such as garbage collectors. On top of that, there are hardware optimizations, such as caching, that skew the picture. As the result, the best way to see what is going on is often to look at the source.
The "second" loop is faster, so,
When you execute a method at least 10000 times, it triggers the whole method to be compiled. This means that your second loop can be
faster as it is already compiled the first time you run it.
slower because when optimised it doesn't have good information/counters on how the code is executed.
The best solution is to place each loop in a separate method so one loop doesn't optimise the other AND run this a few times, ignoring the first run.
e.g.
for(int i = 0; i < 3; i++) {
long time1 = doTest1(); // timed using System.nanoTime();
long time2 = doTest2();
System.out.printf("Test1 took %,d on average, Test2 took %,d on average%n",
time1/RUNS, time2/RUNS);
}
Most likely, the code was still compiling or not yet compiled at the time the first loop ran.
Wrap the entire method in an outer loop so you can run the benchmarks a few times, and you should see more stable results.
Read: Dynamic compilation and performance measurement.
It simply might be the case that you allocate so much space for objects with your calls to getBytes(), that the JVM Garbage Collector starts and cleans up the unused references (bringing out the trash).
Few more observations
As pointed by #dasblinkenlight above, Hadoop's Bytes.toBytes(c); internally calls the String.getBytes("UTF-8")
The variant method String.getBytes() which takes Character Set as input is faster than the one that does not take any character set. So for a given string, getBytes("UTF-8") would be faster than getBytes(). I have tested this on my machine (Windows8, JDK 7). Run the two loops one with getBytes("UTF-8") and other with getBytes() in sequence in equal iterations.
long ts;
String c = "sgfrt34tdfg34";
ts = System.currentTimeMillis();
for (int k = 0; k < 10000000; k++) {
c.getBytes("UTF-8");
}
System.out.println("t1->" + (System.currentTimeMillis() - ts));
ts = System.currentTimeMillis();
for (int i = 0; i < 10000000; i++) {
c.getBytes();
}
System.out.println("t2->" + (System.currentTimeMillis() - ts));
this gives:
t1->1970
t2->2541
and the results are same even if you change order of executions of loop. To discount any JIT optimizations, I would suggest run the tests in separate methods to confirm this (as suggested by #Peter Lawrey above)
So, Bytes.toBytes(c) should always be faster than String.getBytes()
I am currently writing some code for a sample sequencer in Android. I am using the AudioTrack class. I have been told the only proper way to have accurate timing is to use the timing of the AudioTrack. EG I know that if I write a buffer of X samples to AudioTrack playing at a rate of 44100 samples per second, that the time to write will be (1/44100)X secs.
Then you use that info to know what samples should be written when.
I am trying to implement my first attempt using this approach. I am using only one sample and am writing it as continuous 16th notes at a tempo of 120bpm. But for some reason it is playing at a rate of 240bpm.
First I checked my code to derive the time of a 16th (nanoseconds) note at tempo X. It checks outs.
private void setPeriod()
{
period=(int)((1/(((double)TEMPO)/60))*1000);
period=(period*1000000)/4;
Log.i("test",String.valueOf(period));
}
Then I verified that my code to get the time for my buffer to be played at 44100khz in nanoseconds and it is correct.
long bufferTime=(1000000000/SAMPLE_RATE)*buffSize;
So now I am left thinking that the audio track is playing at a rate that is different from 44100. Maybe 96000khz, which would explain the doubling of speed. But when I instantiate the audioTrack, it was indeed set to 44100khz.
final int SAMPLE_RATE is set to 44100
buffSize = AudioTrack.getMinBufferSize(SAMPLE_RATE, AudioFormat.CHANNEL_OUT_MONO,
AudioFormat.ENCODING_PCM_16BIT);
track = new AudioTrack(AudioManager.STREAM_MUSIC, SAMPLE_RATE,
AudioFormat.CHANNEL_OUT_MONO,
AudioFormat.ENCODING_PCM_16BIT,
buffSize,
AudioTrack.MODE_STREAM);
So I am confused as to why my tempo is being doubled. I ran a debug to compare time elapsed audioTrack to time elapsed system time, and the it seems that the audiotrack is indeed playing twice as fast as it should be. I am confused.
Just to make sure, this is my play loop.
public void run() {
// TODO Auto-generated method stub
int buffSize=192;
byte[] output = new byte[buffSize];
int pos1=0;//index for output array
int pos2=0;//index for sample array
long bufferTime=(1000000000/SAMPLE_RATE)*buffSize;
long elapsed=0;
int writes=0;
currTrigger=trigger[triggerPointer];
Log.i("test","period="+String.valueOf(period));
Log.i("test","bufferTime="+String.valueOf(bufferTime));
long time=System.nanoTime();
while(play)
{
//fill up the buffer
while(pos1<buffSize)
{
output[pos1]=0;
if(currTrigger&&pos2<sample.length)
{
output[pos1]=sample[pos2];
pos2++;
}
pos1++;
}
track.write(output, 0, buffSize);
elapsed=elapsed+bufferTime;
writes++;
//time passed is more than one 16th note
if(elapsed>=period)
{
Log.i("test",String.valueOf(writes));
Log.i("test","elapsed A.T.="+String.valueOf(elapsed)+" elapsed S.T.="+String.valueOf(System.nanoTime()-time));
time=System.nanoTime();
writes=0;
elapsed=0;
triggerPointer++;
if(triggerPointer==16)
triggerPointer=0;
currTrigger=trigger[triggerPointer];
pos2=0;
}
pos1=0;
}
}
}
edited : rephrased and updated due to initial erroneous assumption that system time was used to synchronize sequenced audio :)
As for audio playing back at twice the speed, this is a bit strange as the "write"-method of the AudioTrack is blocking until the native layer has enqueued the next buffer, are you sure the render loop isn't invoked from two different sources (although I assume from your example you invoke the loop from within a thread).
However, what is certain is that there is a time synchronization issue to address: the problem herein lies with the calculation of the buffer time you use in your example:
(1000000000/SAMPLE_RATE)*buffSize;
Which will always return 4353741 at a buffer size of 192 samples at a sample rate of 44100 Hz, thus disregarding any cues in tempo (for instance this will be the same at 300 BPM or 40 BPM), Now, in your example this doesn't have any consequences for the actual syncing per se, but I'd like to point this out as we'll return to it shortly further on in this text.
Also, nanoseconds are a nicely precise unit, but too much as milliseconds will suffice for audio operations. As such, I will continue the illustration in milliseconds.
Your calculation for the period of a 16th note at 120 BPM indeed checks out at the correct value of 125 ms. The previously mentioned calculation for the period corresponding to each buffer size is 4.3537 ms. This indicates you will iterate the buffer loop 28.7112 times before the time of a single sixteenth note passes. In your example however, you check whether the "offset" for this sixteenth note has passed at the END of the buffer iteration loop (where the period for a single buffer has already been added to the elapsed time!), by using:
elapsed>=period
Which will already lead to drift at the first occasion as at this moment "elapsed" would be at (192 * 29 iterations) 5568 samples (or 126.26 ms), rather than at (192 * 28.7112 iterations) 5512 samples (or 126 ms). This is a difference of 56 samples (or when speaking in time : 1.02 ms). This wouldn't of course lead to samples playing back FASTER than expected (as you stated), but already leads to a irregularity in playback. For the second 16th note (which would occur at the 57.4224th iteration, the drift would be 11136 - 11025 = 111 samples or 2.517 ms (more than half your buffer time!) As such, you must perform this check WITHIN the
while(pos1<buffSize)
loop, where you are incrementing the read pointer up until the size of the buffer has been reached. As such you will need to increase another variable by a fraction of the buffer period PER buffer sample.
I hope the above example illustrates why I'd initially proposed counting time by sample iterations rather than elapsed time (of course the samples DO indicate time, as they are merely translations of a unit of time to an amount of samples in a buffer, but you can use these numbers as the markers, rather than adding a fixed interval to a counter as in your render loop).
First of all, some convenience math to help you with getting these values :
// calculate the amount of samples are necessary for storing the given length of time
// ( in milliSeconds ) at the given sample rate ( in Hz )
int millisecondsToSamples( int milliSeconds, int sampleRate )
{
return ( int ) ( milliSeconds * ( sampleRate / 1000 ));
}
OR : These calculations which are more convenient when thinking in a musical context like you mentioned in your post. Calculate the amount of samples that are present in a single bar of music at the given sample rate ( in Hz ), tempo ( in BPM ) and time signature ( timeSigBeatUnit being the "4" and timeSigBeatAmount being the "3" in a time signature of 3/4 - although most sequencers limit themselves to 4/4 I've added the calculation for explaining the logic).
int samplesPerBeat = ( int ) (( sampleRate * 60 ) / tempo );
int samplesPerBar = samplesPerBeat * timeSigBeatAmount;
int samplesPerSixteenth = ( int ) ( samplesPerBeat / 4 ); // 1/4 of a beat being a 16th
etc.
The way you then write the timed samples into the output buffer is by keeping track of the "playback position' in your buffer callback, i.e. each time you write a buffer, you'll be incrementing the playback position with the length of the buffer. Returning to a musical context: if you were to be "looping a single bar of 120 bpm in 4/4 time", when the playback position would exceed (( sampleRate * 60 ) / 120 * 4 = 88200 samples, you reset it to 0 to "loop" from the beginning.
So let's assume you have two "events" of audio that occur in a sequence of a single bar of 4/4 time at 120 BPM. One event is to play on the 1st beat of a bar and lasts for a quaver (1/8 of a bar) and the other is to play on the 3rd beat of a bar and lasts for another quaver. These two "events" (which you could represent in a value object) would have the following properties, for the first event:
int start = 0; // buffer position 0 is at the 1st beat/start of the bar
int length = 11025; // 1/8 of the full bar size
int end = 11025; // start + length
and the second event:
int start = 44100; // 3rd beat (or half-way through the bar)
int length = 11025;
int end = 55125; // start + length
These value objects could have two additional properties such as "sample" which could be the buffer containing the actual audio and "readPointer" which would hold the last sample-buffer index the sequencer has read from last.
Then in the buffer write loop:
int playbackPosition = 0; // at start of bar
int maximumPlaybackPosition = 88200; // i.e. a single bar of 4/4 at 120 bpm
public void run()
{
// loop through list of "audio events" / samples
for ( CustomValueObject audioEvent : audioEventList )
{
// loop through the buffer length this cycle will write
for ( int i = 0; i < bufferSize; ++i )
{
// calculate "sequence position" from playback position and current iteration
int seqPosition = playbackPosition + i;
// sequence position within start and end range of audio event ?
if ( seqPosition >= audioEvent.start && seqPosition <= audioEvent.end )
{
// YES! write its sample content into the output buffer
output[ i ] += audioEvent.sample[ audioEvent.readPointer ];
// update the sample read pointer to the next slot (but keep in bounds)
if ( ++audioEvent.readPointer == audioEvent.length )
audioEvent.readPointer = 0;
}
}
// update playback position and keep within sequencer range for looping
if ( playbackPosition += bufferSize > maximumPosition )
playbackPosition -= maximumPosition;
}
}
This should give you a perfectly timed approach in writing audio. There's still some magic you have to work out when you're hitting the iteration where the sequence will loop (i.e. read the remaining unprocessed buffer length from the start of the sample for seamless looping) but I hope this gives you a general idea on a working approach.
I developed an auto-call application. The app reads a text file that includes a phone number list and calls for a few second, ends the call and then repeats.
My problem is that the app does not send calls after 10~16 hours. I don't know the reason exactly, but I guess that the problem is the CPU usage. My app's CPU usage is almost 50%! How do I reduce CPU usage?
Here is part of source code:
if(r_count.compareTo("0")!=0) {
while(index < repeat_count) {
count = 1;
time_count = 2;
while(count < map.length) {
performDial(); //start call
reject(); //end call
finishActivity(1);
TimeDelay("60"); // wait for 60sec
count = count + 2;
time_count = time_count + 2;
onBackPressed(); // press back button for calling next number
showCallLog();
finishActivity(0);
}
index++;
}
This is the TimeDelay() method source:
public void TimeDelay(String delayTime) {
saveTime = System.currentTimeMillis()/1000;
currentTime = 0;
dTime = Integer.parseInt(delayTime);
while(currentTime - saveTime < dTime) {
currentTime = System.currentTimeMillis()/1000;
}
}
TimeDelay() repeats in the while loop for a few times.
The reason it's using 50% of your CPU is that Android apparently won't let it use 100% of the CPU, which a loop like the one in your TimeDelay() ordinarily would. (Or else you have two CPUs and it is in fact using 100% of one CPU.) What you're doing is called a busy wait and it should be obvious why continually checking a condition will use lots of CPU. So don't do that. Use Thread.sleep() instead. Your app will then use no CPU at all during the wait.
Also, for God's sake, why are you passing a string and then parseInting it, rather than just passing an Integer in the first place? :-)
If your method take a long time to finish , especially the while loop. You should put Thread.sleep(50) inside your loop. This makes you processor be able to handle other processes.
Your CPU will be reduced. Not sure but you should try.
Hope you get good result.