Whenever I play a sound effect in my LibGDX game on an Android device, the game stutters. I have tried the game on three Samsung devices:
On Galaxy S7 Edge (2016, Android 8) and Galaxy Tab S 10.5 (2014, Android 6.0.1) the game is still playable, but not running smoothly whenever there are multiple sound effects being played (looping a sound effects are not a problem).
However on Galaxy S20 Ultra (2020, Android 10) the game is unplayable: Every call to Sound.play() takes 2...4 ms and causes "AUDIO_OUTPUT_FLAG_FAST denied by server; frameCount 0 -> 54276" error. This error does not appear with the other devices, but Sound.play() still takes 1...2 ms which of course is a considerable portion of a 16 ms frame.
So what I think is pretty clear is that the problem is in the Sound.play() method, not for example the number of concurrent sounds playing (which I have limited to 8 but have tried 4 as well), or that the Android device would be too slow to process the sounds (in which case a 6 year old GT should not outperform this year's high-end S20), or that the sound effect files would be too large (the one I'm using for testing was originally a 3.8 kB WAV). And yes, I am using AssetManager to load the sounds in advance.
I have now spent two long days doing research, found about 15-20 topics on different forums about what I believe is the same or related issue, and tried out all the suggested fixes without any success:
Changing audio format from WAV to OGG
Different sample rates: 44.1k, 48k, 96k on both formats (with 96k, there is no stutter and no error, but no audible sound either)
Adding silence of 1 or 2 seconds to the end of the sound effect (which itself is 41 ms long), with all the combinations of the above formats and sample rates.
Some say that looping a silent sound clip "in background" has solved the problem, but I anyway have another sound (car engine) looping constantly in the game and that seems to have no effect.
I have also seen suggestions to use Music class instead of Sound, but it's not suitable for collision sound effects with Box2D because pitch cannot be adjusted.
The only workaround that I found but have not tried yet is playing the sounds on a different thread. I have not tried it because I'm not familiar with multithreading and have not been able to find a comprehensive enough guide on how to do it (properly) in LibGDX. I also assume that this approach would be problematic for any sounds which may have to be paused, stopped or adjusted during playback by some actor from the main thread. Furthermore, according to https://github.com/libgdx/libgdx/wiki/Threading, "You should never perform multi-threaded operations on anything that is graphics or audio related".
Therefore, before I even start familiarizing my self on that topic (multithreading), I just wanted to ask once more: Is there really no other solution? It just doesn't feel right that a high-end Android device from this year cannot start the playback of a small WAV sound any faster than in 4 ms. There are lot of games in Play store with working sound effects and smooth gameplay, are they all really using multithreading?
I don't have a complete answer, but I'll share some ideas here.
My own anecdotal experience is that sound operations such as starting sound playback tend to be too time-consuming for a typical render thread on Android. I've tried a few different approaches (AudioTrack, SoundPool, etc.), and as best I can remember have gotten similar results in each case.
Putting the audio on a different thread seems like the most practical solution. I understand the hesitance if you're not familiar with multithreading, and I think you're right to be cautious, especially when using a third-party library. However, for simple tasks, Android supplies some fairly straightforward tools, like HandlerThread and Handler, that could perhaps be leveraged.
As for the LibGDX documentation saying not to perform multi-threaded operations on anything audio related, it's not clear to me whether that means don't do anything audio-related on a thread other than the render thread, or if it just means to keep all audio on a single thread, but that that thread doesn't have to be the render thread. If it's the latter, then putting audio on a separate thread might be an option.
I took a quick look at the LibGDX source code. I'd have to spend more time to better understand what's going on, but I see use of both AudioTrack and SoundPool, and I'm pretty sure I've run into this issue with both.
But, I also see some signs of asynchronous sound functionality. There are some classes with 'asynchronous' in the name that use a dedicated handler thread. I don't know if this functionality is documented (I couldn't find the documentation immediately) or otherwise supported, but it does seem to be present in the source code. The comments say there are some limitations, but it's not immediately clear to me what they are.
As for communication between the render thread and an audio thread, it would add some complexity, but you should be able to do it fairly straightforwardly using handlers or other similar tools. In fact, that's what the LibGDX code I looked at does - it creates a HandlerThread and uses a Handler (naturally) to post to it. It can still be difficult, especially when using a third-party library where you don't control where all audio operations occur. For example, LibGDX may always set up the audio objects on a specific thread (e.g. the render thread), which means if you use another thread, you'll be using the objects on a thread other than that on which they were created. I doubt that would be an issue, but it depends on the technology. (For example, the documentation for ExoPlayer says that instances should only be used from a single thread.)
In my own code I'm doing all audio myself, so I control it and can put everything on the same thread. That might be difficult or impossible with LibGDX, but the presence of the 'asynchronous' audio classes may be a hint that playing audio on a different thread is safe to do. (And maybe you can make use of those classes, assuming they're a supported part of the API.)
In case someone else has this issue. In your AndroidLauncher, override this.
#Override
public AndroidAudio createAudio(Context context, AndroidApplicationConfiguration config) {
return new AsynchronousAndroidAudio(context, config);
}
You MUST make sure you don't have any SoundId actions (eg. some_sound.Stop(sound_id)). As those will not work with AsynchronousAndroidAudio and will crash the game. So check that before you publish your game.
I'm working on an app that uses a MediaPlayer object to play H.264 MP4 videos from a WallpaperService as it is a live wallpaper app. Battery drain occurs while the device (Nexus 5, Android 6.0.1) is idle and sleeping if I pause/stop the MediaPlayer with mediaPlayer.pause() or mediaPlayer.stop(). The drain is about 3-7%/hour as tested multiple times overnight. As soon as I release the media player with mediaPlayer.release(), the battery drain goes back to a more normal 1%/hour. I pause/stop the mediaPlayer when onVisibilityChanged calls false. The phone is reporting to be going to sleep in both the stock Android battery chart and Better Battery Stats.
How can this battery drain be explained if the CPU is going into a sleep state successfully?
EDIT: Something new I've discovered is that when calling mediaPlayer.setSurface(null) right before mediaPlayer.pause(), the idle battery use comes back to normal. I can then do mediaPlayer.setSurface(surface) to set it back before mediaPlayer.start(). The problem is there's some black artifacting for a couple of seconds after restarting.
I can't give you a precise answer but can give you what to look for. I suspect what is going on is that pause() is checking for events frequently enough to keep the processor from entering the deeper sleep/C-states. In contrast, stop() doesn't need to check for events and so allows the processor to enter a deep sleep state. I wrote an article on sleep states some years back.
I suspect that the writers of the function decided to check more frequently than is necessary. This is a very common mistake due to the developers thinking that shorter sleeps / more frequent checking result in better response (which it almost never does). You can check this by using a processor power monitor that actually checks the hardware sleep states. Unfortunately, most don't and only check for processor independent "equivalents".
So let's get back to your question: What can you do about it. I can give you some advice but none of it is very satisfying:
Check for an API or data structure that allows you to set the
checking interval for pause(). By the way, I don't know of any.
Write your own. Of course, this complicates writing platform independent apps
Use an alternative media player that has done this correctly
Hammer on google until it's fixed
Like I said, none of this is very satisfying. By the way, searching the net, I found evidence that this has happened more than once with new Android releases.
Good luck and let us know what happens.
I want to make an android app, that records a video stream and when the user does not push a button, everything before the last 120 seconds of the video stream gets deleted. This should run for hours so only ~50mb are in use all the time. Has anyone an idea how to record a video like a never-ending flow of data that allows me to access certain points and delete everything before those points?
I know this question is pretty general but I find it very hard to access android camera close to the hardware.
You'll probably run into file size limitations if nothing else.
A better approach would be to just keep recording 30-second videos, and delete any that are more than two minutes old until the user presses the "record" button, at which time you start keeping them.
Then splice them together into one long video afterwards.
By the way, this will kill your battery. I assume you're equipped to deal with that.
I'm looking to create a drum machine in ActionScript 3 (as an Adobe AIR Android app), which will keep to a user defined tempo (BPM).
I am struggling to find a way to keep the project in time, I have, at the moment, made it so that 5 different sounds are represented in rows of 8 squares, and the user can click each square to choose when to play that sound (hope this makes sense).
At the moment I am using Timer to keep the project in time, which is very laggy and inconsistent.
using timer is a bad idea for this, there I said it...
The issue is that the timer has a drift and fires several milliseconds later.
Try a simple test where you have a timer that executes every 500ms, and then compare the getTimer() count. What I have found in my experiments that the timer is continually off and it looks like it doesn't self correct. I've tried using a self-correcting timer, that changes the firing time based on the getTimer() difference since last run, but it's still not reliable. and anytime your processor's load picks up, the timer will be off anyway.
The correct way of dealing with this is to use byteArray data as a source for the sound. Based on the calculation of sampling resolution you can populate the stream with the data in advance, and the sound will play on time, pretty much guaranteed. I haven't gone as far as to create something that does this myself. But there are several libraries that you can utilize that can help you with this.
My top two decremented libraries are SiON and tonfall
you can see a sample of SiON here http://wonderfl.net/c/qf4b
and tonfall example at http://tonematrix.audiotool.com/
While I haven't tried them on android, I think either should work
I'm having a real hard time calculating the distance between two android phones using sound.
-the main idea is having 2 phones sync'ed on same time, making mobile A send a msg to mobile B to let him know he is playing sound soon. note that mobile A save this time.
-then mobile B sends "ok, u can go ahead" to mobile A while it starts recording the next 1 second or so.
-Then mobile A gets the "ok" and start playing a 1000Hz sound.
-Mobile B detect that freq and send its current time to mobileA
now we have all the info to calculate the distance. problem is that at theory this is all good, but when i implement this i have lots of random time added into the equation.
the main problem is that I cant point at the ABSOLUTE time when mobile B got the good freq.
I tried not recording the whole 1000 ms but lots of "mini" chunks of (12~24ms) but the time the mobile spend on the recorder_.startRecording()/recorder_.read()/recorder_.stop() commands is too much, and im missing the freq by lots of ms (each ms is equal to 30cm so i cant effort much errors...)
can any one tell me what im doing wrong or guiding me to better ways of doing that??
The main issue is the recording device cant point on the actual time he recorded the wanted freq.....
thanks in advanced,
Ofer.
Please have a look at new audio features introduced in API 19.