Develop Reference

Android with Nexus 6 -- how to avoid decreased OpenSL audio thread priority relating to app focus?

I'm encountering a strange problem when trying to implement low-latency streaming audio playback on a Nexus 6 running Android 6.0.1 using OpenSL ES.
My initial attempt seemed to be suffering from starvation issues, so I added some basic timing benchmarks in the buffer completion callback function. What I've found is that audio plays back fine if I continually tap the screen while my app is open, but if I leave it alone for a few seconds, the callback starts to take much longer. I'm able to reproduce this behavior consistently. A couple of things to note:
"a few seconds" ~= 3-5 seconds, not long enough to trigger a screen change
My application's activity sets FLAG_KEEP_SCREEN_ON, so no screen changes should occur anyway
I have taken no action to try to increase the audio callback thread's priority, since I was under the impression that Android reserves high priority for these threads already
The behavior occurs on my Nexus 6 (Android 6.0.1), but not on a Galaxy S6 I also have available (Android 5.1.1).
The symptoms I'm seeing really seem like the OS kicks down the audio thread priority after a few seconds of non-interaction with the phone. Is this right? Is there any way I can avoid this behavior?

While watching the latest Google I/O 2016 audio presentation, I finally found the cause and the (ugly) solution for this problem.
Just watch the around one minute of this you tube clip (starting at 8m56s):
https://youtu.be/F2ZDp-eNrh4?t=8m56s
It explains why this is happening and how you can get rid of it.
In fact, Android slows the CPU down after a few seconds of touch inactivity to reduce the battery usage. The guy in the video promises a proper solution for this soon, but for now the only way to get rid of it is to send fake touches (that's the official recommendation).
Instrumentation instr = new Instrumentation();
instr.sendKeyDownUpSync(KeyEvent.KEYCODE_BACKSLASH); // or whatever event you prefer
Repeat this with a timer every 1.5 seconds and the problem will vanish.
I know, this is an ugly hack, and it might have ugly side effects which must be handled. But for now, it is simply the only solution.
Update:
Regarding your latest comment ... here's my solution.
I'm using a regular MotionEvent.ACTION_DOWN at a location outside of the screen bounds. Everything else interfered in an unwanted way with the UI. To avoid the SecurityException, initialize the timer in the onStart() handler of the main activity and terminate it in the onStop() handler. There are still situations when the app goes to the background (depending on the CPU load) in which you might run into a SecurityException, therefore you must surround the fake touch call with a try catch block.
Please note, that I'm using my own timer framework, so you have to transform the code to use whatever timer you want to use.
Also, I cannot ensure yet that the code is 100% bulletproof. My apps have that hack applied, but are currently in beta state, therefore I cannot give you any guarantee if this is working correctly on all devices and Android versions.
Timer fakeTouchTimer = null;
Instrumentation instr;
void initFakeTouchTimer()
{
if (this.fakeTouchTimer != null)
{
if (this.instr == null)
{
this.instr = new Instrumentation();
}
this.fakeTouchTimer.restart();
}
else
{
if (this.instr == null)
{
this.instr = new Instrumentation();
}
this.fakeTouchTimer = new Timer(1500, Thread.MIN_PRIORITY, new TimerTask()
{
#Override
public void execute()
{
if (instr != null && fakeTouchTimer != null && hasWindowFocus())
{
try
{
long downTime = SystemClock.uptimeMillis();
MotionEvent event = MotionEvent.obtain(downTime, downTime, MotionEvent.ACTION_DOWN, -100, -100, 0);
instr.sendPointerSync(event);
event.recycle();
}
catch (Exception e)
{
}
}
}
}, true/*isInfinite*/);
}
}
void killFakeTouchTimer()
{
if (this.fakeTouchTimer != null)
{
this.fakeTouchTimer.interupt();
this.fakeTouchTimer = null;
this.instr = null;
}
}
#Override
protected void onStop()
{
killFakeTouchTimer();
super.onStop();
.....
}
#Override
protected void onStart()
{
initFakeTouchTimer();
super.onStart();
.....
}

It is well known that the audio pipeline in Android 6 has been completely rewritten. While this improved latency-related issues in most cases, it is not impossible that it generated a number of undesirable side-effects, as is usually the case with such large-scale changes.
While your issue does not seem to be a common one, there are a few things you might be able to try:
Increase the audio thread priority. The default priority for audio threads in Android is -16, with the maximum being -20, usually only available to system services. While you can't assign this value to you audio thread, you can assign the next best thing: -19 by using the ANDROID_PRIORITY_URGENT_AUDIO flag when setting the thread's priority.
Increase the number of buffers to prevent any kind of jitter or latency (you can even go up to 16). However on some devices the callback to fill a new buffer isn’t always called when it should.
This SO post has several suggestions to improve audio latency on Anrdoid. Of particular interest are points 3, 4 and 5 in the accepted answer.
Check whether the current Android system is low-latency-enabled by querying whether hasSystemFeature(FEATURE_AUDIO_LOW_LATENCY) or hasSystemFeature(FEATURE_AUDIO_PRO).
Additionally, this academic paper discusses strategies for improving audio latency-related issues in Android/OpenSL, including buffer- and callback interval-related approaches.

Force resampling to native device sample rate on Android 6.
Use the device's native sample rate of 48000. For example:
SLDataFormat_PCM dataFormat;
dataFormat.samplesPerSec = 48000;

Computer AI opponent in turn based google play game

I am trying to make a four player turn based game.
Now in case their are less than 4 human player I want to start X human player and 4-X AI player. Is it possible with the google turn based multiplayer https://developers.google.com/games/services/android/turnbasedMultiplayer
Also in case a human player leaves I want to replace it with the AI player instead of cancelling the game. Does it support this ?

I am implementing this and have it working 90%. When it is a new turn in my game, I check if the next player is a BOT or HUMAN player. This is prearranged using a seating algorithm, so that I space out the HUMANs neatly. i.e. opposite when there are two human players. When the next player is a BOT, I take a turn, but do not advance the Particpant. That way it will persist and send an update without passing the turn to the other player. When the next player is a HUMAN, I do a take Turn and switch to the next player. The code looks something like this....
if (Players[playerTurn%4]==BOT) {
Games.TurnBasedMultiplayer.takeTurn(MainActivity.mHelper.getApiClient(), MainActivity.mMatch.getMatchId(),
MainActivity.mTurnData.persist(), myParticipantId).setResultCallback(
new ResultCallback() {
#Override
public void onResult(TurnBasedMultiplayer.UpdateMatchResult result) {
processResult(result);
}
});
state = GameState.Playing;
} else {
Games.TurnBasedMultiplayer.takeTurn(MainActivity.mHelper.getApiClient(), MainActivity.mMatch.getMatchId(),
MainActivity.mTurnData.persist(), nextParticipantId).setResultCallback(
new ResultCallback<TurnBasedMultiplayer.UpdateMatchResult>() {
public void onResult(TurnBasedMultiplayer.UpdateMatchResult result) {
processResult(result);
}
});
state = GameState.multiplayerWait;
}
I am not sure how to swap inactive players with another player. I don't think this is possible unless you run your own server or use something like Parse.

Android Step Counter -- Count does not match either fitbit / Samsung Step Counter. (Off by thousands)

The problem I'm having is that my count is totally off, from any pedometers, fitbit, or the Samsung Step Counter.
It appears to shut down, and not add any steps after awhile.
If I enter that I'm starting at 3000 for example, it calculates an offset and it stores as a shared preference. It tries to remain registered for the the Sensor. I also store the current steps, so that if the activity that is listening for Step Broadcasts is resumed, it will request for the steps to be output.
I have tried making the service provide notifications and be a foreground service, but the accuracy does not improve, and it uses a ton of power, I have tried a wakelock, with similiar results, not accurate, and uses too much power.
As it stands, my app does not show up in the power usage statistics, so it is hardly using power at all. The hardware sensor should be capturing steps from the accelerometers, and when it does wake up, it should output the newest steps. When it does update, it is closer to the value that I set as the step count.
I am using the Step Counter which was made available in kitkat or above, on some devices. The following code registers for the sensor.
Sensor sensor = sensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER);
if (null != sensor)
{
sensorManager.registerListener(this, sensor,
SensorManager.SENSOR_DELAY_NORMAL);
}
This is within the OnCreate Method of a Service, that I have constructed.
The Service is created as sticky. And uses a broadcast receiver to receive starting steps from an activity, to compute an offset. It also broadcasts the steps that have happened. Here is more of the code.
class MyBroadCastReceiver extends BroadcastReceiver {
#Override
public void onReceive(Context context, Intent intent) {
if (intent.getAction().equals(ACTION_REQUEST_STEPS))
{
SendStepBroadcast(currentSteps);
}
else if (intent.getAction().equals(ACTION_SET_STEPS))
{
setSteps = intent.getFloatExtra(
STEPS_OCCURRED, -1);
SendStepBroadcast(setSteps);
}
}
}
#Override
public void onSensorChanged(SensorEvent event) {
if (setSteps > -1) {
offset = setSteps - event.values[0] + 1;
SharedPreferences prefs = getSharedPreferences("com.halfwaythere",
MODE_PRIVATE);
prefs.edit().putFloat("offset", offset).apply();
setSteps = -1;
}
currentSteps = event.values[0] + offset;
SharedPreferences prefs = getSharedPreferences("com.halfwaythere",
MODE_PRIVATE);
prefs.edit().putFloat("currentSteps", currentSteps).apply();
SendStepBroadcast(currentSteps);
}
private void SendStepBroadcast(float steps) {
Intent broadcastSteps = new Intent();
broadcastSteps.setAction(ACTION_STEPS_OCCURRED);
broadcastSteps.putExtra(STEPS_OCCURRED, steps);
this.sendBroadcast(broadcastSteps);
}
In the Activity the following code is used to start the service:
#Override
protected void onStart() {
super.onStart();
Intent intent = new Intent(this, StepService.class);
startService(intent);
}
My most recent set of attempts to fix this, I tried the following:
Use android:process=":background" to start the service in it's own process. I will try this tomorrow in a field test, and see how it works.
I tried setting the count to zero, and found out my code would not allow, so in the above code I used > -1. Since -1 shouldn't be a valid step count.
All of the code above has test wrapped around it, and I've tried to find any edge cases, and have looked over stack overflow for pedometer problems with Step Counter on Samsung Galaxy S4. Or anything about best practices.
Thanks, and if you need any additional info, please let me know.

From my research:
When the screen is locked, the Hardware Sensor waits before outputting steps. It does count steps, but once you press the button on the side it wakes up, and it receives a SensorEvent.
Service was occasionally having Start Command being called. Was very important that I only register once, so I added a boolean that would be flipped once the initialization was called, and flip off after that. Service was paused, but not killed, and Start Command would run upon it being awoken.
Setting Service to Foreground, seems to be getting closer to the value from other pedometers. Uses more battery, but is keeping it going more, so that makes sense.
Did not require wakelock, it was only on for several milliseconds, but that did not cause the hardware sensor to send results.
Will have my Service run till goal is achieved, or offer the option to stop tracking at half way point. I really just wanted something to tell me that I would get my goal if I turned around, anything else will be equivalent to extra steps.
So I will apply what I found and continue on with my app, should be on the App Store by mid July.

Android Sensor Data Collection is not working properly

I developed a Data collector which collects data from Accelerometer, Gyroscope, Magnetometer and it worked fine for a while. Then I added Linear Acceleration to it as well (After 4 months, this week). Now both the version are behaving very strangely. Sometime they log the data perfectly when I do some physical activities like walking etc. However, sometimes it doesn't update sensors values and just repeat old values i.e each sensor value is updated lets after 5 seconds, 2 sec etc randomly and I need a sampling rate of 50 samples per second. I experimented with 10-15 participants and all my data was invalid because of this. The strange things is that the same app has worked perfectly before. I can't find any problem in it. I am placing some of the snapshots here. May be if someone can point to any bug or something ?
The buffered Writter:
FileWriter fow;
BufferedWriter bow;
extfile = new File(extfilepath, message + ".csv");
fow = new FileWriter(extfile);
bow = new BufferedWriter(fow);
This bow.writer is then being used in timertask thread to log data every 20 milliseconds.
Can anyone please comment or help me with this ? This weird behavior of this app is beyond my understanding.

Check that you have a wake lock acquired if your application goes to background. I've used PowerManager.PARTIAL_WAKE_LOCK successfully in a data collection application.
When your display turns off, your application is at least paused (and system might even stop it). The partial wake lock "Ensures that the CPU is running; the screen and keyboard backlight will be allowed to go off." So reading between the lines it means that otherwise your CPU might go to sleep for small periods of time in order to save power.

Did you forget to paste in:
else if (event.sensor.getType() == Sensor.TYPE_LINEAR_ACCELERATION){} ?
Are you using the accelerometer data, then subtracting gravity?
OK. What's your code look like to call the timer?? Something like this?
Timer updateTimer = new Timer("linear accel");
updateTimer.scheduleAtFixedRate(new TimerTask() {
public void run() {
updateGUI();
}
}, 0, 100);
}
private void updateGUI() {
runOnUiThread(new Runnable() {
public void run() {} } ?

How to perform random and periodic sampling / data collection of sensors in Android?

Android supports many sensors and the API supports periodic sampling i.e. I can sample the sensor data at every specified sampling time. Ideally, the periodic sampling is useful, but often it is unnecessary to continuously read a sensor say the accelerometer when the phone is not vibrating for a long duration (e.g. during charging). I want to read the data of accelerometer randomly, where the sampling frequency is not fixed i.e. the duration between two samples is random. How do I do that ?
I tried the following code for reading the accelerometer data:
final public SensorEventListener listener=new SensorEventListener() {
public void onSensorChanged(SensorEvent e) {
Log.d(TAG, "in sensor changed");
if (e.sensor.getType()==Sensor.TYPE_ACCELEROMETER) {
double netForce=e.values[0]*e.values[0];
netForce+=e.values[1]*e.values[1];
netForce+=e.values[2]*e.values[2];
AccelX=e.values[0];
AccelY=e.values[1];
AccelZ=e.values[2];
message_acc= String.valueOf(AccelX)+","+String.valueOf(AccelY)+","+String.valueOf(AccelZ);
Log.d(TAG, message_acc);
//check if the phone is shaking
if (threshold<netForce) {
isShaking();
}
else {
isNotShaking();
}
}
}
This method is called in a thread, which is called periodically. In the above code, the API allows to collect the data when ever there is change in the data. My doubt is, is this approach a periodic data collection for a given sampling period? How do I deterministically specify the period of sampling ?
Secondly, If I want to perform random data collection, should it be done at the thread level, I mean, call/schedule the above method randomly? Any pointers to implement this or better ways to achieve this?