Google added restrictions of not broadcasting the Bluetooth switch states to the app when the app in the background. This effectively blocks the optimized Bluetooth beacon scanning in the background. How to get around this issue other than periodic job scheduling?
Any help is appreciated.
You have several options for background BLE Beacon Scanning on Android 8+
A regular background service. You can use these just like on Android 4.3-6.x, but you are generally limited to 10 minutes of running time in the background, after which time Android will kill your app and it won't be able to scan anymore.
Use a foreground service. These work much like Android background services except they display an even-present notification with an icon of your choice to indicate that your app is running in the background. With a foreground service, you can effectively scan for beacons in the background with no restrictions just like on Android 4.3-6.x.
Use Intent-based scans. If you simply need to know when a beacon appears or disappears, you can set up an Intent-based scan for BLE devices with a bluetooth packet filter that filters on the presence of the byte pattern of the beacon, or the absence of the byte pattern of the beacon. When the beacon appears or disappears, Android will send an Intent to a BroadcastReceiver in your app that will wake it up in the background and let it run for about 10 minutes before killing it. During this time you can keep scanning for beacons.
Use the job scheduler (also known as WorkManager). You can schedule a job to run at most every ~15 minutes in the background to do scanning. A job is generally limited to 10 minutes of running time in the background. Since start times vary by +/- 10 minutes, you'll have gaps of up to 0-15 minutes where you won't be scanning.
Play games with (3) and (4) to bend the rules. While this goes against the spirit of Android's restrictions, you can play games with the job scheduler by starting an immediate job, cancelling it before 10 minutes is up, then restarting it. You can do similar things with an intent based scan by simply triggering it over and over. Be forewarned, however, doing these things will drain the users' batteries, perhaps leading them to uninstall your app. This rule bending may be blocked in future Android releases.
You can read my blog post about the merits of these techniques here. The open source Android Beacon Library uses techniques 3 and 4 on Android 8+ devices by default, and also supports configuring a foreground service if you wish to choose option 2.
Related
is there any way to keep a BeaconTransmitter instance sending forever?
The Problem:
I want a RPi detect when I am coming home independent of Wifi running or the RPi even having access to the internet (e.g. it could then turn on all routers etc., when it detects the phone).
I have a minimalistic app on my android phone sending out an iBeacon running and the beacontools Library on the RPi detects it. This is working well, in principle. (Although I know it's supposed to be the other way around, but I again, I want the RPi to be able to do "stuff" without having access to the internet, what the phone cannot do).
However, the power management(?) on the Android phone seems to stop the transmission, the latest after a few hours (even if the App is "not optimized" for battery usage). I am not even sure if this is because of the battery optimization or something else.
Solution so far:
I am using a PeriodicWorkRequest that checks every 30 minutes (I read somewhere this is the minimum time), if the beacon is still running and relaunches it, if it is not.
This also works as expected, but of course, I keep coming home in the time slot when the beacon transmission has stopped and before it is kicked off again by the WorkManager. ;)
A cleaner way would be to "intercept" when the BeaconTransmitter is killed by PowerManagement/BatteryManager(??) or something. Is that possible?
Those covid tracking apps are using something (I hope) that ensures continuous transmission, but I know that Google/Apple had to implement something fundamental to allow those to work, and of course, I cannot use the covid-tracking itself because it is anonymous by design.
A precise answer will vary by phone manufacturer as many OEM's fork Android to kill long-running apps in the background. See here for a summary of challenges by OEM.
For vanilla Android (Pixel phones, many Nokia Android Once devices), Motorola and even most Samsung devices, it is sufficient to keep a foreground service active and have Location always runtime permission. My BeaconScope app uses this foreground service technique, and I typically see a transmitter keep going for weeks on a Pixel or Samsung device. Be sure you grant "Always" location permission to the app for this to work.
You can see the code needed to set up the foreground service in the Kotlin reference app for the Android Beacon Library. The comments indicate that the foreground service is useful for continuous beacon scanning, but it is equally true that it is useful for long-term beacon transmission.
Without the foreground service, the beacon transmission will only last about 10 minutes after the app is put to the background on vanilla Android, at which time the app will be killed.
The problem with a periodic work request or a job service is that they can run at most every 15 +/- 10 minutes on Android. So if the transmitter is killed every 10 minutes you will end up with gaps of up to 15 minutes without transmission. On Android 8+ you have no choice but to use a foreground service.
I was going through the table here http://www.davidgyoungtech.com/2017/08/07/beacon-detection-with-android-8 and I am confused as to how does the library achieve detection time of 5 seconds after app kill on Android platforms 5-7. This is what the link says - Apps on Android 4.3-7.x used long-running background services or alarms to periodically look for beacons in the background. As far as I know, alarms cannot be set for less than 15 minutes so how does the 5 second thing work?
My blog post's detection times table that you mention (reproduced below) says that detection times are 5 seconds on Android 5.0-7.x. (Clarification: it does not make this claim for 4.x.) This is possible on 5.x-7 by using a long-running filtered low-power scan using an API introduced in Android 5.0. Here's the code that sets that up.
What this does is puts the Bluetooth LE chip in a low power mode that automatically wakes up if any packet matching the expected filter patter is seen. After the wake up, the Android OS delivers the scan result to the library which then sends a callback to the application via didEnterRegion or didRangeBeaconsInRegion. This process typically takes just a few seconds after the packet appears.
These scanning times have nothing to do with Alarms. On Android 4.x-7.x, the AlarmManager is used for a completely different purpose -- to keep a long-running background service alive. An Alarm repeatedly reset for 5 minutes in the future, and as long as the service keeps running it never goes off. If the app is killed due to low memory or the user swiping it off the screen, this alarm will fire a BroadcastReceiver which triggers the long-running scanning service to start itself again.
To my knowledge there is nothing on Android 4.x-7.x that limits Alarms from running more than once every 15 minutes. You may be thinking of a limitation on Android 8+ that restricts periodic JobScheduler items to every 15 minutes or more.
Android Beacon Library Detection Times:
Before Android L Android Altbeacon library used background running services to scan BLE beacons. Default scan times are 1.1 seconds in a loop in the foreground and 10 seconds every 5 mins in the background. Also for background tasks alarm manager is used to wakeup the app.
I was looking at how similar setup works in Android Oreo given that long-running background services are not allowed. I was going through
http://www.davidgyoungtech.com/2017/08/07/beacon-detection-with-android-8
and it did clear something but I am still not completely clear on how scan works with the combination of bluetooth scanning API and job scheduler.
So how is low power scan is used in combination of job scheduler based periodic scheduling to monitor and range beacons? If someone could provide an example that would be great.
To be a bit more specific -
when would we go for passive new beacon detection vrs normal background scans?
Why is passive scan work only for new detections? Why can't we run this in a loop? And why is it run only between job scheduler scans?
Lets us say I have set scan period of 10 seconds and scan interval of 2 minutes. What techniques are used at what intervals to scan the beacons?
If job scheduler can schedule jobs with frequency more than 15 mins and it can run with max 10 mins time can we set scan interval to 10 mins and scan interval to 5 mins? That we would lose only 5 mins worth of scan due to OS limitation? Also, can we do a passive scan in these 5 mins? So we do not have any span of time when beacons are not scanned?
It is critical to understand how low power scans work at a low level:
A low power scan relies on offloading the Bluetooth LE pattern matching to a hardware filter on the Bluetooth chip. If it is matched by an incoming packet, the chip notifies the operating system, which sends an intent to the app that wakes it up to process the packet.
The above relies on a combination on of filtered scans delivered by intents (a new API in Android 8).
When a scheduled scan job ends with the Android Beacon Library, this intent based delivery is set up to be handled by the operating system and Bluetooth chip. The end result is that if a new beacon is detected, it will be handled by a broadcasted receiver very quickly. This can send a monitoring callback to the app within a few seconds instead of waiting 15 minutes for the next scheduled job.
This is all designed to work when the filter will not match a Bluetooth device that is already nearby. If one is already nearby, the Intent would be sent within 100ms or so, which is a very CPU inefficient way of getting constant updates, which defeats the whole point of low power APIs, and would cause the app to be running constantly and eligible for termination by Android 8.
The filter used matches any beacon. This is done to conserve filters as they are a limited hardware resource. For this reason, these scanning APIs are not used if a beacon is known to be present already, because it would immediately match a packet and defeat the purpose of the technique. Bottom line: if a scan job ends with a beacon visible, the low power scanning is not done.
You cannot set a scheduled job to execute any more often than 15 minutes. This is an operating system restriction. A 10 minute between scan period would work, but the library only delivers scan results at the end of the cycle, so this probably isn't lot helpful. It would be possible to modify the library to simply scan for the max 10 minutes on each scan job, but currently it does not do this, and as you note it would leave 5 minutes uncovered.
If you really want to do constant BLE scanning in the background on Android 8+, the OS-permitted way to do so is with a foreground service that displays an icon so the user knows it is running. Be careful about doing this in consumer apps, though, as battery drain will be very significant and users will probably uninstall the app after blaming it for dead batteries. For special use cases, however, this may be acceptable.
What I want to achieve
I am building an app for a shopping mall, with large amount of beacons installed not quite far away from each other (assume ~20m distance).
When a user walks in this mall, even without opening the app, the app needs to keep scan for beacons. When a beacon is detected, I will then query the server to ask if I need to and what local notification to push to user.
What I planned to do
My original plan was to create a Service, returns START_STICKY in onStartCommand() in order to make sure that the service will restart itself even the app is killed by a task manager.
This Service will keep scanning for beacons.
In fact, this is the existing approach of my colleague. According to this colleague, the service starts almost immediately after the app was killed by a task manager.
The problem
But soon I found that this approach is problematic.
The above approach has now 2 major problems:
It drains batteries because it keeps scanning;
It does not work on
Android 8 due to new limitations introduced in Android
8.
My next plan
Although I know that JobScheduler can be used to replace Service, which is also the existing approach of Android Beacon Library, perform a scan every 15-25 minutes absolutely cannot fulfill my requirements, where beacons are very close to each other and therefore beacons need to be scanned frequently.
Therefore I come up with another plan:
Use Android Beacon Library to run background detection
Once the
first beacon in filter list has been detected, start a foreground
service (which will not be killed even in Android 8) that
continuously scan for beacons
When all beacons in the filter has been exited, stop the above foreground service. Android Beacon
Library shall resume to its background detection state.
The intentions of this approach are:
Make advantage of Android Beacon Library's background detection which saves battery, according to their documentation
Step away of Android Beacon Library's handling of Android 8 due to its built-in limitation of long scanning interval
Scanning goes on even on Android 8 since I am going to use a foreground service
My major question
By reading documentations, I already know how to scan for beacons in the background.
But how do I make use of Android Beacon Library to scan in a foreground service?
Plus, is there any problem that you can discover in the above approach / Do you have any better suggestions to achieve such requirements?
My other question
In fact according to this post, the background service starts 5 minutes after an app is killed.
But by returning START_STICKY in onStartCommand() of a Service, it is restarted almost immediately.
Then, why will there be a 5 minutes delay, even in pre-Oreo?
This approach is sound. The Android Beacon Library 2.15+ natively supports foreground services as a scan mechanism for supporting cases like this on Android 8. See here for more info.
The tricky part is to switch back and forth between using Job Scheduler and a Service to do scanning. I have not tested this, but my suggestion would be to bind manually to the BeaconManager in a custom Application class.
Then :
On entering a region, stop monitoring and then unbind the BeaconManager.
Start a custom foreground service
In the foreground service, disable Scan Jobs, then bind to the BeaconManager and start ranging
Once no beacons have been ranged for a time, stop ranging, unbind from the BeaconManager, enable Scan Jobs, bind again and then start monitoring.
Finally, exit the foreground service
On the second question, yes, START_STICKY will very quickly restart a service on most platforms. The library uses a 5 minute timer with the AlarmManager as a backup, which will relaunch the service if a START_STICKY restart fails. Indeed, in typical use, the scanning service restarts much more quickly than five minutes.
Background:
Ideally I would like my Android device to be scanning for Bluetooth Low Energy devices all the time an the ability to start an application whenever a new device with specific properties appears.
So the broadcast packet in BLE will for instance enumerate a set of services provided by the broadcasting device. An app would then be able to register an interest for certain services and automatically be started when a device with this services comes into range.
From what I understand this is not how the Android BLE API works? So how can I get something similar?
Simplest possible example:
I have a BLE sensor that logs ambient temperature over time. Whenever my Android phone is close enough I want to connect and download all the data, sending it to some cloud storage solution. This app would not need any GUI (at least not after configuration is done). But how should it run in the background without draining the battery, but still give me a fairly good chance of connecting the device quickly once it is in range?
Question:
Do I need to set a timer and wake the app every once in a while and then manually start scanning? What kind of intervals should I then choose. How long can I leave the scanner running without adversely affecting the battery?
Possible solution:
This is what I've come up with so far.
A configuration activity to set the intervals and devices to scan for
The configuration activity will set up an WakefulBroadcastReceiver similar to the Scheduler example
When the receiver get's the onReceive event I start a BLE scan service (that I've written) as a wakeful service.
The scan service starts scanning (with a registered callback).
The service might get adv reports that it can act upon
After a timeout the service will stop the scanner and end the wakeful service.
This works, but I'm not sure it's the best way. I also don't know how small intervals I can have and still avoid destroying the battery life. What I would want is to start scanning every two minutes, scanning for 10-20 seconds. But I'm afraid that would be rather frequently to wake up the device?
This functionality has all been moved to the open source Android Beacon Library which will:
wake up/launch your app when iBeacons matching a desired pattern are detected
perform beacon scanning in the background even if the user has not launched your app yet
reduce the scan rate automatically in the background to 30 seconds every five minutes to save battery. (Timing configurable.)
Code examples are show here
If your BLE device is not a beacon you could still use this library to accomplish this by having your sensor also transmit as a beacon then after it is detected connect to the main service.