I have an app that is streaming audio content and sometimes it just
stops all of the suddent.
the logcat windows shows --
AudioHardware pcm playback is going to standby
and that's it.
I saw on another thread (pun intended) that someone was saying it was
because he was using too many threads. Could that really be causing
this? Could i give the audio thread higher priority?
Anyway to prevent the audio hardware pcm from going to standby?
I too am having this problem, and have nearly exhausted all my resources trying to find a workaround. I found an article on how to prevent this with AudioRecord, and maybe the same principles he talks about there will apply to you as well:
http://hificorder.blogspot.com/
In my case, I am having lockups using SoundPool. Increasing the number of max streams allowed prevented this from happening in most cases, but it still happens in other situations. I really wish there was better information about how to handled this correctly, because as it is, this is a show stopping bug in my app which may force me to abandon it all together.
Be sure to post back if you have any findings to add.
Related
Whenever I play a sound effect in my LibGDX game on an Android device, the game stutters. I have tried the game on three Samsung devices:
On Galaxy S7 Edge (2016, Android 8) and Galaxy Tab S 10.5 (2014, Android 6.0.1) the game is still playable, but not running smoothly whenever there are multiple sound effects being played (looping a sound effects are not a problem).
However on Galaxy S20 Ultra (2020, Android 10) the game is unplayable: Every call to Sound.play() takes 2...4 ms and causes "AUDIO_OUTPUT_FLAG_FAST denied by server; frameCount 0 -> 54276" error. This error does not appear with the other devices, but Sound.play() still takes 1...2 ms which of course is a considerable portion of a 16 ms frame.
So what I think is pretty clear is that the problem is in the Sound.play() method, not for example the number of concurrent sounds playing (which I have limited to 8 but have tried 4 as well), or that the Android device would be too slow to process the sounds (in which case a 6 year old GT should not outperform this year's high-end S20), or that the sound effect files would be too large (the one I'm using for testing was originally a 3.8 kB WAV). And yes, I am using AssetManager to load the sounds in advance.
I have now spent two long days doing research, found about 15-20 topics on different forums about what I believe is the same or related issue, and tried out all the suggested fixes without any success:
Changing audio format from WAV to OGG
Different sample rates: 44.1k, 48k, 96k on both formats (with 96k, there is no stutter and no error, but no audible sound either)
Adding silence of 1 or 2 seconds to the end of the sound effect (which itself is 41 ms long), with all the combinations of the above formats and sample rates.
Some say that looping a silent sound clip "in background" has solved the problem, but I anyway have another sound (car engine) looping constantly in the game and that seems to have no effect.
I have also seen suggestions to use Music class instead of Sound, but it's not suitable for collision sound effects with Box2D because pitch cannot be adjusted.
The only workaround that I found but have not tried yet is playing the sounds on a different thread. I have not tried it because I'm not familiar with multithreading and have not been able to find a comprehensive enough guide on how to do it (properly) in LibGDX. I also assume that this approach would be problematic for any sounds which may have to be paused, stopped or adjusted during playback by some actor from the main thread. Furthermore, according to https://github.com/libgdx/libgdx/wiki/Threading, "You should never perform multi-threaded operations on anything that is graphics or audio related".
Therefore, before I even start familiarizing my self on that topic (multithreading), I just wanted to ask once more: Is there really no other solution? It just doesn't feel right that a high-end Android device from this year cannot start the playback of a small WAV sound any faster than in 4 ms. There are lot of games in Play store with working sound effects and smooth gameplay, are they all really using multithreading?
I don't have a complete answer, but I'll share some ideas here.
My own anecdotal experience is that sound operations such as starting sound playback tend to be too time-consuming for a typical render thread on Android. I've tried a few different approaches (AudioTrack, SoundPool, etc.), and as best I can remember have gotten similar results in each case.
Putting the audio on a different thread seems like the most practical solution. I understand the hesitance if you're not familiar with multithreading, and I think you're right to be cautious, especially when using a third-party library. However, for simple tasks, Android supplies some fairly straightforward tools, like HandlerThread and Handler, that could perhaps be leveraged.
As for the LibGDX documentation saying not to perform multi-threaded operations on anything audio related, it's not clear to me whether that means don't do anything audio-related on a thread other than the render thread, or if it just means to keep all audio on a single thread, but that that thread doesn't have to be the render thread. If it's the latter, then putting audio on a separate thread might be an option.
I took a quick look at the LibGDX source code. I'd have to spend more time to better understand what's going on, but I see use of both AudioTrack and SoundPool, and I'm pretty sure I've run into this issue with both.
But, I also see some signs of asynchronous sound functionality. There are some classes with 'asynchronous' in the name that use a dedicated handler thread. I don't know if this functionality is documented (I couldn't find the documentation immediately) or otherwise supported, but it does seem to be present in the source code. The comments say there are some limitations, but it's not immediately clear to me what they are.
As for communication between the render thread and an audio thread, it would add some complexity, but you should be able to do it fairly straightforwardly using handlers or other similar tools. In fact, that's what the LibGDX code I looked at does - it creates a HandlerThread and uses a Handler (naturally) to post to it. It can still be difficult, especially when using a third-party library where you don't control where all audio operations occur. For example, LibGDX may always set up the audio objects on a specific thread (e.g. the render thread), which means if you use another thread, you'll be using the objects on a thread other than that on which they were created. I doubt that would be an issue, but it depends on the technology. (For example, the documentation for ExoPlayer says that instances should only be used from a single thread.)
In my own code I'm doing all audio myself, so I control it and can put everything on the same thread. That might be difficult or impossible with LibGDX, but the presence of the 'asynchronous' audio classes may be a hint that playing audio on a different thread is safe to do. (And maybe you can make use of those classes, assuming they're a supported part of the API.)
In case someone else has this issue. In your AndroidLauncher, override this.
#Override
public AndroidAudio createAudio(Context context, AndroidApplicationConfiguration config) {
return new AsynchronousAndroidAudio(context, config);
}
You MUST make sure you don't have any SoundId actions (eg. some_sound.Stop(sound_id)). As those will not work with AsynchronousAndroidAudio and will crash the game. So check that before you publish your game.
I am developing an app that allows you to record many clips and stitches them back together before uploading. Whilst developing for Android 5 and up I think this is prevalent problem on all the platforms. Not sure if anyone came up with an usable solution. I seem not to be able to record 1s video as the MediaRecorder takes ~500ms to .start() and on different devices this time changes.
The only method that I found to be more or less accurate is to set up a file observer on media file at the moment when I start() and stop observing when I finish I tend to wait for the first 256bytes to be written before I measure the clip time.
Now then CAN ANYONE tell me whether this is the only thing I can do? Or is there a better method of measuring the delay? Or if you know of an approach I could take to write a recording app that will have immediate start and stop I will greatly appreciate that.
If not maybe point me in the direction of a different approach? Bear in mind I am an absolute beginner so answers like "use MediaCodec API" really not going to help me a lot. I searched for days an appropriate solution can't seem to find anything useful anywhere. Setting thread with a delayedPost is also not an answer.
My current task involves fixing an Android music player that sucks too much RAM. By taking a look at the "Apps" section of the system settings, I found a mysterious (for me) process called Media. Here is a screenshot:
I would like to know more about this process and about the reason why it appears every time a song is played and disappears every time the playback stops. The only reference I have been able to find in the official documentation is this one.
What does the MediaProvider class do?
Do you have any clue about what may be causing that process to be kept alive during the playback?
I solved it myself. Turned out that a Cursor was being left open (I don't know whether accidentally or willfully), thus causing the Media process to appear.
Still, it is weird that, neither in the official SDK documentation nor on the Net, there is no mention about any MediaProvider-ish thing.
I have searched for this online, but am still a bit confused (as I'm sure others will be if they think of something like this). I'd like to preface by saying that this is not for homework and/or profit.
I wanted to create an app that could listen to your microwave as you prepare popcorn. It would work by sounding an alarm when there's a certain time interval between pops (say 5-6 seconds). Again, this is simply a project to keep me occupied - not for a class.
Either way, I'm having trouble trying to figure out how to analyze the audio intake in real-time. That is, I need a way to log the time when a "pop" occurs. So that you guys don't think I didn't do any research into the matter, I've checked out this SO question and have extensively searched the AudioRecord function list.
I'm thinking that I will probably have to do something with one of the versions of read() and then compare the recorded audio every 2 seconds or so to the recorded audio of a "pop" (i.e. if 70% or more of the byte[] audioData array is the same as that of a popping sound, then log the time). Can anyone with Android audio input experience let me know if I'm at least on the right track? This is not a question of me wanting you to code anything for me, but a question as to whether I'm on the correct track, and, if not, which direction I should head instead.
I think I have an easier way.
You could use the MediaRecorder 's getMaxAmplitude method.
Anytime your recorder detects a big jump in amplitude, you have detected a corn pop!
Check out this code (ignore the playback part): Playing back sound coming from microphone in real-time
Basically the idea is that you will have to take the value of each 16-bit sample (which corresponds to the value of the wave at that time). Using the sampling rate, you can calculate the time between peaks in volume. I think that might accomplish what you want.
this may be a bit overkill, but there is a framework from MIT media labs called funf: http://code.google.com/p/funf-open-sensing-framework/
They already created classes for audio input and some analysis (FFT and the like), also saving to files or uploading is implemented as far as I've seen, and they handle most of the sensors available on the phone.
You can also get inspired from the code they wrote, which I think is pretty good.
I was looking at Android's SoundPool as a mechanism to implement sound effects in my generic game development library. It seemed ideal.
But a little bit of research indicates that there all kinds of bugs in SoundPool. Are the bugs in SoundPool still relevant?
Because I'm developing a library, any bugs in SoundPool become bugs in my library, and I want to insulate my users from that.
So my question is basically: what API should I use for audio?
Using AudioTrack and writing my own mixer is not out of the question. But obviously it would be preferable to avoid doing that. And is there any API to provide decoding for me?
I need to be able to play a reasonable number of simultaneous sound effects (at least 16, let's say), and have even more open. Sounds need to start playing with low latency. WAV files need to be supported (MP3/Ogg is unimportant). Sound effects need to support seamless looping and dynamic, individual volume adjustment. The Android app lifecycle needs to be properly supported.
I have heard there is a 1MB limit somewhere for SoundPool, this is probably acceptable for each individual sound effect but not for all buffers/sounds. Can someone tell me exactly what the limit is on?
Finally, I need to be able to play background music as well, in compressed formats, with low CPU load. I assume MediaPlayer is ideal for this. Can it be used in parallel with another API?
I know a few people have been using MediaPlayer to fill in for SoundPool. But does it support the features that I need?
Are there any other audio APIs I've missed?
Just to add some more recent feedback on this issue. I've been using SoundPool for some time in an app with a fairly large user base for key press sounds. Our use case:
Must be played immediately
Up to 3+ sounds in parallel
We make use of the setRate across it's full range [0.5f-2.0f]
I've now experienced two major device specific issue and have decided to cut my losses and switch away from SoundPool
A large number of 4.4 LG devices (mostly the LG G2/G3 line) were having a native crash with their implementation of SoundPool. This was fixed in an update (eventually) but we still have a lot of users with un-upgraded devices
Sony Xperia devices currently have all sorts of issue with SoundPool as reported by others. In my case, I've discovered that if you use setRate with rate > 1.0f the SoundPool with start throwing exceptions until your app quits (and burn through a bunch of battery in the process).
TL;DR; I no longer think it's worth the danger/hassle of debugging SoundPool
Stick with OGG files and SoundPool will do you just fine. It's the nature of the multi-platform beast that is Android that there WILL be hardware configurations that will not work with every significant program, no matter how diligently the programmers try.
If this is a large and well-funded project, add to the funding one of each major phone for testing. It's actually much cheaper than the programmer time spent researching and trying to guess what their performance is.
Sorry. Seems as if this isn't the answer that you were looking for. Good luck!
DISCLAIMER: I have a small amount of experience with MediaPlayer, and no successful experience with the other APIs I mention, and the following information is based on what I've read in the DOCs and what I've read from google searches.
You could use mediaplayer (for the background music) with other audio APIs, since MediaPlayer automatically runs on it's own thread, but I believe it has a high-ish cpu load, and I don't think it would take compressed bits very well, but I'm not too sure.
There's also JetPlayer http://developer.android.com/reference/android/media/JetPlayer.html which seems like a lot of work to use effectively, but it would work very well with playing background music, then playing other sounds as needed in the game. From what I read of the DOCs, it takes a MIDI file (I think?) and you mute and unmute tracks to make it work how you want it to.
I like AudioTrack because it gives you the ability to edit sounds at runtime by changing the frequencies of the sound, and SoundPool can do the same.
Though for your situation, AudioTrack doesn't seem like it would work well, since playing two sounds would require two threads because AudioTrack is blocking (I'm pretty sure).
And with SoundPool, I'm thinking that since you have 16 sounds, maybe take two threads with one SoundPool in each thread and apply 8 sounds to each SoundPool. I don't really know though, as I've never even tried using SoundPool.
And again, my information is not based on experience, just what it appears from what I've read, so I may be completely or maybe just slightly wrong, or heck, who knows.
And I don't really know anything about the SoundPool bugs, since I haven't researched it.