Prevent my audio app using NuPlayer on Android Lollipop 5.x? - android

I have an audio app that plays multiple tracks at the same time, each with their own mediaPlayer. Each track is reasonably long, upwards of two minutes.
So long as the tracks are encoded as ogg files, everything works great on Android 4.x. I've yet to encounter a device running stock 4.x that has any audio problems with this setup.
But on Lollipop 5.x there are a wide variety of audio problems - stuttering, tracks cutting out, and bluetooth audio almost never seems to work.
I've discovered that going into Developer options in 5.x and unchecking "use Nuplayer (experimental)" instantly solves these problems and returns to 4.x levels of performance.
Is there a way I can programmatically force my app to use the 4.x media stack (I believe it's called Awesomeplayer?) and not to use the new Nuplayer system? At least until I can discover the source of the Nuplayer problems?

Update:
Setting a partial wake lock on the MediaPlayer resolves this problem:
playerToPrepare.setWakeMode(context, PowerManager.PARTIAL_WAKE_LOCK);
A partial wake lock shouldn't have too big of an impact, and it seems like MediaPlayer itself cleans this up when playback completes.
-- Original Answer ---
So, I finally found a way to safely detect wether or not NuPlayer will be used or not on Lollipop. Seems like the best strategy for now is to inform the user to open Developer Settings and enable AwesomePlayer until Google fixes NuPlayer.
Sadly, there's no good way to change this setting for the user, we can just read its value unless you're signed as a system application.
This approach checks Android's system properties values to see if the user have enabled the use of AwesomePlayer or not under Developer Settings. Since Lollipop have NuPlayer on by default, if this value is disabled, we know NuPlayer will be used.
Drop SystemProperties.java into your project for access to read the system properties, do not change its package name from android.os (it calls through to its corresponding JNI methods, so needs to stay the same).
You can now check if the phone is Lollipop/5.0, if AwesomePlayer is enabled, and act accordingly if it's not (e.g. by opening the Developer Settings):
public void openDeveloperSettingsIfAwesomePlayerNotActivated(final Context context) {
final boolean useAwesome = SystemProperties.getBoolean("persist.sys.media.use-awesome", false);
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP && !useAwesome) {
final Intent intent = new Intent(Settings.ACTION_APPLICATION_DEVELOPMENT_SETTINGS);
context.startActivity(intent);
}
}

Enabling/Disabling NuPlayer didn't help. But I managed the wakelock part with a friendly UI. I'll look tonight as SysCtl from KitKat and compare it with the one on Lollipop, maybe I'll find something interesting.
So bluetooth stuttering is related to the dumb kernel on 5.02 that stutters the playback as soon as screen is off. I used a partial wakelock so the cpu stays active after screen off with this app. It works. No more stuttering. As for speakers that require high sample rate I just switched the cpu governor to performance. It's a workaround but the partial wakelock should work especially on bluetooth headphones. Here's the app's link https://play.google.com/store/apps/details?id=eu.thedarken.wl&hl=en

Related

How do some apps overcome phone recording restrictions?

Background
Phone recording is not really supported on Android, yet some devices support it to some extend.
This made various call recording apps gather as much possible information about devices and what should be done to them, and decide upon this what to do.
Some even offer root solutions.
One such example is boldbeast Call Recorder app, which offers a lot of various configurations to change:
"record mode" . Shows 14 modes for non-rooted devices, and up to 34 for rooted. Also shows "Alsa mode" as an option for it, for rooted devices.
Has "Tune Audio Effect ("auto tune a groupd of parameters") .
Has "Tune Audio Route", with the possible values of "Disabled", "Group1", "Group2", "Group3"
For rooted devices:
"change audio controls" ("auto change audio controls")
"change audio driver" (change audio drive settings to enable record mode 21,22,23,24,31,32,33,34")
For rooted devices: "start input stream"
The problem
If I'm in need to create a call recording app, there is no other way than to find the various workarounds for various devices, but as it seems other apps use terms that don't appear in the API.
I can't find any of those of the app I've mentioned, for example.
What I've found
Other than tons of questions of how to record calls on Android, showing that it doesn't work on all devices, I could find some interesting things. Here are my tries and insights so far:
There are some Audio recording sources we can use while preparing the recording (docs here) , but sadly in each device it might be different. For some, VOICE_CALL works, and for some, others. But at least we can try...
On OnePlus 2 with Android 6.0.1, incoming calls can be recorded using VOICE_CALL, but I can't make outgoing calls be recorded there, unless I use MIC as audio source together with speaker turned on. Somehow, the app I've mentioned succeeds recording it without any issues. I'm sure I will see other issues with other Android devices, as I've tried to address this whole topic in the past. Update: I've found this sample project (also here), which for some reason sleeps for 2 seconds on the UI thread between prepare and start calls of the mediaRecorder. It works fine, and when I did something similar (wait using Handler.postDelayed for 1 second), it worked fine too. The comment that was written there is "Sometimes prepare takes some time to complete".
On Galaxy S7 with Android 8, I've failed to get sound of the other side for outgoing calls AND incoming calls (even with MIC and speaker), no matter what I did, yet the app I've mentioned worked fine.
To let you try my POC of call recording, I've published an open source github repository here, having a sample that will record a single call, and let you listen to the most recent one, if all works well.
This "ViktorDegtyarev - CallRecLib" SDK , which doesn't seem to work at all, and crashes on various Android versions
These 2 old sample projects : rvoix , esnyder-callrecorder , both fail to actually record. The second doesn't even seem to work on Android 6.0.1 device, which it's supposed to support.
aykuttasil - CallRecorder sample and axet - android-call-recorder sample - both, just like on my POC, don't have any tweaking except for AudioSource, and because of this they fails to record on some cases, such as OnePlus 2 output-audio of outgoing calls.
Most third party apps only offer the AudioSource tweaking, but some (like "boldbeast") do offer more. One example is "Automatic Call Recorder" which has "configuration" (10 values to choose from, first is "default") and "method" (5 vales to choose from, first is "default"). Those apps probably do not want others to understand what those configurations mean, so they put general names. Or, it's just too complicated for everyone (especially for users), so they generalize the names.
There is an API of "setMode" here, but it doesn't seem to change upon calling it. I was thinking of maybe change the "channel" of where the call is being used, this way, but it doesn't work. It stays on the value of "2" during call, which is MODE_IN_CALL.
There are customized parameters that are available for various devices (each OEM and its own parameters), which can be set here and maybe even via JNI (here and here) , but I don't get where to get this information from (meaning which pairs of key-value are available). I've searched in a lot of places, but couldn't find any website that talks about which possible parameters are available, and for which devices.
I was thinking of using AudioRecord instead of MediaRecorder class for recording, thinking that it's a bit low level, so it could give me more power and access to customized capabilities, but it seems to be very similar to MediaRecorder, and even use the same audio sources (example here).
Another try I had with low level API, was even further, of using JNI (OpenSL ES for Android). For this, I couldn't find much information (except here and here), and only found the 2 samples of Google here (called "audio echo" and "native audio"), which are not about recording sound, or at least I don't see them occur.
Android P might have official way to record calls (read here and here). Testing on my Android P DP3 device (Pixel 2), I could record both sides fine in both incoming and outgoing calls, using "DEFAULT" as audio source, so maybe the API will finally be official and work on all Android versions. I wrote about it here and here.
I was thinking that maybe the Visualizer class could be a workaround of recording, but according to some StackOverflow post (here), the quality it extremely low, so I decided that maybe I shouldn't try it. Plus I couldn't find a sample of how to record from it.
I've found some parameters that might be available on some devices, here (found from here), all start with "AUDIO_PARAMETER_", but testing on Galaxy S7, all returned empty string. I've also found this website, that gave me the idea of using audioManager.setParameters("noise_suppression=off") together with MIC audio source, but this didn't seem to do anything in the case of Galaxy S7.
The questions
As opposed to other similar questions about this topic, I'm not asking how to record calls. I already know it's a very problematic and complex problem. I already know I will have to address various configurations, and that I will probably use a server to store all of them and find there the best match for each one.
What I want to ask is more about the tweaking and workarounds :
Is there a list of configurations for the various devices, Android versions, and what to choose for each?
Besides Audio source, which other configuration is possible to be used?
Which parameters are possible for the various devices and Android versions ? Are there any websites of the OEMs describing them?
What are the various terms in the app I've mentioned? Where can I find information of how to change them?
Which tools are available for rooted devices?
Is it possible to know which device supports call recording and which not, by using the API ?
About the workaround of OnePlus 2, to wait a moment till we start recording, why is it needed? Is it needed on all Android versions? Is it a known issue? Would 1 second be enough?
How come on the Galaxy S7 I've failed to record the other side even when using MIC&speaker?
EDIT: I've found this of accessibility service being able to help with call recording:
https://developer.android.com/guide/topics/media/sharing-audio-input#voice_call_ordinary_app
Not sure how to use it though. It seems "ACR Phone Dialer" uses it. If anyone knows how it can be done, please let me know.
I spent many weeks working on a Voicecall Recording App so I faced all your issues/questions/problems.
Moreover: my project had a low-priority so I didn't spent much time every day on it, so I worked on this App for many months while Android was changing under the hood (minor an major releases).
I was developing always on the same Galaxy Note 5 using its stock ROM (without Root) but I discovered that on the same device the behaviour was changing from one Android release to another without any explanation.
For example from Nougat 7.0 to 7.1.2 I was unable to record a voicecall using the same code as before.
Google has enforced_or_changed restrictions about voicecall recording many times.
At the beginning it was sufficient to use use VOICE_CALL AudioSource. Then manufactures has started to interprete this Value as they wanted, and the result was that one implementation was working well but another was not.
Then Reflection was needed to run undocumented/hidden methods to start voicecall recording.
Then Google has added a Runtime check, so calling them directly was not more possible even using Reflection.
However this method lack of stability because it was not guarantee that a method was using the same name on all devices.
Then I started to reverse-engineer currently working Apps that were working on newer Android version and I discovered that them were using a complete different and more secure approach. This takes me many weeks because all these Apps uses JNI Libraries trying to hide this method between Assembler code.
When I succesfully create a Test App which was recording well I tried the SAME code in many different devices and ROMs/Versions and surprisely it was working well.
This means that all those different methods you can see in these App Settings (I'm 98% sure about it) are just "fake" or just refers to OLD methods not more used.
A small different metion should be done for Rooted devices:
these devices could change AudioRoutes so a different approach can be used in this case.
[1] There isn't any list or website listing all supported devices or best method to do a successfully voicecall record
[6] It's not possibile to know which device supports Voicecall Recording
just using an API call. You have to try and catch Excepions...
[8] Recording by MIC+speaker suffers of many issues: (1) the caller will hear all your ambient sound so the privacy-bug is a big issue (2) the echo is a big problem (3) the recording volume is very low as the quality of recordered voice
According to my tests, one way to improve this is to have an AccessibilityService being active (no need to write there anything at all) while choosing voice-recognition as the audio source. Also it's recommended to have the speaker turned on because this will record the audio from the microphone.
This seems to exist in some call-recording apps.
Weird thing is that Google has written this as a rule on the Play Store:
The Accessibility API is not designed and cannot be requested for
remote call audio recording.
https://support.google.com/googleplay/android-developer/answer/11899428
No idea what the "remote" means here.
Anyway, I've updated the Github repository to include these additions.

Service lowers volume even when volumes are on max

In my app I have an additional library for SIP calling, which on some phones (Huawei 5.1) locks volume to around 50% (or even less) for some reason and it keeps it like that for the whole phone volume meanwhile service is running.
I tried setting:
am.SetStreamVolume(Android.Media.Stream.Music,am.GetStreamMaxVolume(Android.Media.Stream.Music), 0);
And playing beeping noise with MediaPlayer/Soundpool.
I also tried setting Microfone to
Mode.Ringtone / Mode.Normal,...
but the volume is still low when this library/service is running.
Is there any other setting option possible that could override some setting in the library, because if I look current volumes, it's all on MAX (Media/Notification/Ringtone/...)?
In this library, you have to initialize sampleRate / buff size, which I do with getting default values from Android
String rate = audioManager.getProperty(AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE);
String size = audioManager.getProperty(AudioManager.PROPERTY_OUTPUT_FRAMES_PER_BUFFER);
Some companies add a volume limit to protect user. I met similar issue before on samsung note 4. I am not sure whether APP has the permission to break this limit(mostly not allow).
In order to find the relative setting, please try this: Play music by the default music player, and press volume+. Keep volume up and then your phone may toast a dialog to tip you. And you can change this setting.
Hope it works :)

Not releasing MediaPlayer causes battery drain

I'm working on an app that uses a MediaPlayer object to play H.264 MP4 videos from a WallpaperService as it is a live wallpaper app. Battery drain occurs while the device (Nexus 5, Android 6.0.1) is idle and sleeping if I pause/stop the MediaPlayer with mediaPlayer.pause() or mediaPlayer.stop(). The drain is about 3-7%/hour as tested multiple times overnight. As soon as I release the media player with mediaPlayer.release(), the battery drain goes back to a more normal 1%/hour. I pause/stop the mediaPlayer when onVisibilityChanged calls false. The phone is reporting to be going to sleep in both the stock Android battery chart and Better Battery Stats.
How can this battery drain be explained if the CPU is going into a sleep state successfully?
EDIT: Something new I've discovered is that when calling mediaPlayer.setSurface(null) right before mediaPlayer.pause(), the idle battery use comes back to normal. I can then do mediaPlayer.setSurface(surface) to set it back before mediaPlayer.start(). The problem is there's some black artifacting for a couple of seconds after restarting.
I can't give you a precise answer but can give you what to look for. I suspect what is going on is that pause() is checking for events frequently enough to keep the processor from entering the deeper sleep/C-states. In contrast, stop() doesn't need to check for events and so allows the processor to enter a deep sleep state. I wrote an article on sleep states some years back.
I suspect that the writers of the function decided to check more frequently than is necessary. This is a very common mistake due to the developers thinking that shorter sleeps / more frequent checking result in better response (which it almost never does). You can check this by using a processor power monitor that actually checks the hardware sleep states. Unfortunately, most don't and only check for processor independent "equivalents".
So let's get back to your question: What can you do about it. I can give you some advice but none of it is very satisfying:
Check for an API or data structure that allows you to set the
checking interval for pause(). By the way, I don't know of any.
Write your own. Of course, this complicates writing platform independent apps
Use an alternative media player that has done this correctly
Hammer on google until it's fixed
Like I said, none of this is very satisfying. By the way, searching the net, I found evidence that this has happened more than once with new Android releases.
Good luck and let us know what happens.

Step Counter in Android: always on?

It is a well known issue that many Android phones switch off the accelerometer when the screen goes off. However something seems to have changed with Android Fit (the app). Fit keeps counting steps even when the screen goes off. If Fit is installed, then events are raised for step counting within the Fit environment and I am able to capture them using
Fitness.SensorsApi.findDataSources(mClient, new DataSourcesRequest.Builder()
.setDataTypes(DataType.TYPE_STEP_COUNT_CUMULATIVE)
I have tested this on a Samsung S4 and on a Oneplus One and in both cases the steps are counted.
How do they do that? What Android classes do they use?
My understanding is that the available method introduced since Kitkat is to implement a SensorEventListener. For example theelfismike provides code that implements this. However on many phones the step counting stops when the screen goes off. Interestingly the counting does not seem to stop if the Google Fit app is installed (hence I guess they keep the accelerometer on).
Am I missing something? Is the functionality of keeping counting steps after screen off available to the mortal programmers?
Thanks!
As Ilja said, your code runs even after the screen gets turned off. But in this case I guess we need a little different answer.
They definitely use a Service that keeps a wakelock and they query the sensors for data. Important part here is holding the wakelock - you have to prevent the device from going into sleep during lifetime of your service - if you don't want to miss some data.
But this approach will be drain the battery really fast, because in order to detect steps you need to process quite a lot of data from sensors.
Thats why there is sensor batching. That allows you to get continuous sensor data even without keeping the device awake. It basically stores the sensor events in a hw based queue right in the chip itself and only sends them to your app (service,..) at predefined intervals in batches. This allows you to do a 24/7 monitoring without draining the battery significantly. Please note that only supported chipsets can do that (you can find details in Android docs), in case of older phones you need to fallback to the hideous wakelock keeping method in order to get your data.
You can also just use Google Fit APIs, but this would only work when there're both Google Fit + Google Play Services installed on the device with monitoring turned on.
Every normal Thread is keep on working when the screen goes off or when the Activity lost its focus...but when the activity gets killed then all thread are killed...
However you can use services for longrunning tasks like asking the accelerometer for example

How does noise cancellation work in android?

I came across this relatively old post which describes how impressively Nexus One's noise cancellation works and I was wondering where can I find more information about its implementation in the OS software.
In particular:
How much of it is done using software and how much of it is done in
hardware?
Which modules in the Android source code are responsible for noise
cancellation?
Can I control its behavior via Android's API? (if so, which ones)
Does it also work with the microphone in the headset that comes with
Nexus One (4-pin 3.5mm jack) or does it work with the built-in
microphone only?
I only know the answer for the Nexus One, but:
It's done in hardware.
Not sure.
Nope.
Maybe?
For the N1, it works using a second microphone in the back, and comparing the two signals. I don't know exactly how this process is done (hardware or software), but I know there isn't an API for it. Also, it probably doesn't work for the external headset, since there's no second sound source to compare the first one to (unless the headset has two mics too, but I don't think it does).
About the Nexus One:
All hardware only configuration in software.
Sound drivers and sound system but only configuration.
No API possibly some prop configuration but I haven't been able to get that to work.
No, longer reply following.
I haven't found any indication that it uses the other microphone to do noise reduction for the headset. It wouldn’t make much sense either as it would most likely just try to cancel out with the noise from your pocket.
For most other android phones and for headset on the Nexus One I'm pretty sure that there is only some sort of filter to reduce input of sound that is not speech.
I have done some research on this that I tried to get some help with on the android porting and dev lists. There is a little further info:
http://groups.google.com/group/android-porting/browse_thread/thread/fe1b92065b75c6da?pli=1
With the reservation that I haven't looked at the latest and greatest versions of android.

Categories

Resources