Before Android L Android Altbeacon library used background running services to scan BLE beacons. Default scan times are 1.1 seconds in a loop in the foreground and 10 seconds every 5 mins in the background. Also for background tasks alarm manager is used to wakeup the app.
I was looking at how similar setup works in Android Oreo given that long-running background services are not allowed. I was going through
http://www.davidgyoungtech.com/2017/08/07/beacon-detection-with-android-8
and it did clear something but I am still not completely clear on how scan works with the combination of bluetooth scanning API and job scheduler.
So how is low power scan is used in combination of job scheduler based periodic scheduling to monitor and range beacons? If someone could provide an example that would be great.
To be a bit more specific -
when would we go for passive new beacon detection vrs normal background scans?
Why is passive scan work only for new detections? Why can't we run this in a loop? And why is it run only between job scheduler scans?
Lets us say I have set scan period of 10 seconds and scan interval of 2 minutes. What techniques are used at what intervals to scan the beacons?
If job scheduler can schedule jobs with frequency more than 15 mins and it can run with max 10 mins time can we set scan interval to 10 mins and scan interval to 5 mins? That we would lose only 5 mins worth of scan due to OS limitation? Also, can we do a passive scan in these 5 mins? So we do not have any span of time when beacons are not scanned?
It is critical to understand how low power scans work at a low level:
A low power scan relies on offloading the Bluetooth LE pattern matching to a hardware filter on the Bluetooth chip. If it is matched by an incoming packet, the chip notifies the operating system, which sends an intent to the app that wakes it up to process the packet.
The above relies on a combination on of filtered scans delivered by intents (a new API in Android 8).
When a scheduled scan job ends with the Android Beacon Library, this intent based delivery is set up to be handled by the operating system and Bluetooth chip. The end result is that if a new beacon is detected, it will be handled by a broadcasted receiver very quickly. This can send a monitoring callback to the app within a few seconds instead of waiting 15 minutes for the next scheduled job.
This is all designed to work when the filter will not match a Bluetooth device that is already nearby. If one is already nearby, the Intent would be sent within 100ms or so, which is a very CPU inefficient way of getting constant updates, which defeats the whole point of low power APIs, and would cause the app to be running constantly and eligible for termination by Android 8.
The filter used matches any beacon. This is done to conserve filters as they are a limited hardware resource. For this reason, these scanning APIs are not used if a beacon is known to be present already, because it would immediately match a packet and defeat the purpose of the technique. Bottom line: if a scan job ends with a beacon visible, the low power scanning is not done.
You cannot set a scheduled job to execute any more often than 15 minutes. This is an operating system restriction. A 10 minute between scan period would work, but the library only delivers scan results at the end of the cycle, so this probably isn't lot helpful. It would be possible to modify the library to simply scan for the max 10 minutes on each scan job, but currently it does not do this, and as you note it would leave 5 minutes uncovered.
If you really want to do constant BLE scanning in the background on Android 8+, the OS-permitted way to do so is with a foreground service that displays an icon so the user knows it is running. Be careful about doing this in consumer apps, though, as battery drain will be very significant and users will probably uninstall the app after blaming it for dead batteries. For special use cases, however, this may be acceptable.
Related
My goal is to understand how AltBeacon behaves and to obtain the most efficient application regarding power consumption and beacon detection.
Currently I am analysing the library and I need some clarifications.
Environment:
Using custom beacons, not having power dependencies (on beacon HW), transmit rate (adv) is bigger than 1/s
Using android-beacon-library-reference with light modifications to match the custom beacons;
Using Android 10 OS
All the testing was made using only backgroundMode
Here are my questions:
At initialization (using RegionBootstrap), we reach ScanJobScheduler from two points
BeaconManager:startMonitoringBeaconsInRegion: schedules an immediate ScanJob to run in 50 millis
BeaconManager:setBackgroundMode: cancels the previous scheduled immediate ScanJob (because we are in background mode) and schedules a periodic ScanJob
Is the reason that immediate ScanJob get canceled because the 2nd call is made in less then 50 millis? and why the periodic ScanJob is scheduled twice?
After I kill the application, it takes ~10 minutes to start again, and after another 15 minutes for the 1st scan to start.
Can we do one "periodic scan job" immediately after the application starts and not have to wait for 15 minutes more in order for the 1st scheduled periodic ScanJob to start?
Low Power HW Filter only works after one ScanJob is completed
I've seen that the HW filter is installed after ScanJob Lifecycle is completed (createScanFiltersForBeaconParsers)
In the case where the application is killed, this means that the filter will be installed after the 1st periodic ScanJob completes (25 minutes)
Can we install the Low Power HW Filter immediately after initialization and not wait for the 1st ScanJob to complete?
If I modify the ADV TX power on the custom beacon to be lower, will the distance calculator still work properly?
Does it makes sense to only use background mode or should I switch to foreground? Will this achieve my goal?
Start service in foreground
Install HW filter (if point 3 is not an option, do an immediate scan in foreground to trigger it)
Scan in background until the HW filter delivers the intent
Start scanning in foreground after we detect the 1st beacon
Scan in foreground until we are out of the region in order for the HW filter to be installed again (startPassiveScanIfNeeded)
Switch to scanning in background and wait for the next wake-up
Is there any possible way to receive an Intent from BluetoothLEScanner when the beacon is not in range?
Thanks,
Vlad
A few answers:
The 50 second delay in starting the immediate scan job is designed to prevent a race condition that starts two immediate scan jobs at the same time. When you set background mode to true, it assumes you are changing the background mode so it cancels any immediate scan job in progress (designed only for foreground use) to switch to background operations where there is no scan job. The scheduling happening twice happens because the first BeaconManager:startMonitoringBeaconsInRegion and the BeaconManager:setBackgroundMode both trigger a "reschedule" process. You can avoid all of this churn by calling BeaconManager:setBackgroundMode: first, earlier in your init process. That way the library is already in background mode when you call BeaconManager:startMonitoringBeaconsInRegion and things will work more smoothly.
The way the Android Job scheduler works is that if you set a periodic job at the most frequent (~15 minute) interval, it doesn't run right away. So when the library starts back up in background mode, it won't do anything until the next interval starts. This interval might be quite soon, or it could be as long as 25 minutes. It would be possible to optimize this process by scheduling an immediate scan job if we determine the last one has not been run in a long time through some persisted timestamp. This is a reasonable optimization that could be added to the library by preparing a pull request if you are interested in working on one.
The library only sets up the low-power intent-delivered scan after a scan cycle completes as this is the normal course of operations. I agree that if the app is killed, it won't necessarily be set up as soon as possible when the app restarts. The easiest way to optimize this is to complete the optimization proposed in (2) above, which would solve this problem as a side effect.
If you don't want to harm the accuracy of distance estimates, you want the transmitter power of your hardware beacons to be as high as possible and you want the advertising rate to be as high as possible. Fewer packets per second give fewer statistical samples to work with and increases noise on the distance estimates. Lower output power increases the noise floor relative to the signal, and gives very little margin in detectable difference between RSSI at 1 meter vs. 10 meter. I understand that high transmitter power and high advertising rates significantly affect the lifespan of battery-powered beacons. However, IMO if you reduce the transmitter power enough to save battery or reduce the advertising rate below, say, 5 Hz, then you should probably give up on distance estimates.
How best to use the library really depends on your use case. If scans every ~15 minutes are good enough for your use case, you should probably stick with background mode. I am not sure if the hack you describe will work to make the library behave how you want without modifications. You could try, but it might end up taking more time than simply making the change as described in point (2) above.
In theory Android does have hardware-backed API support for quick callbacks associated with beacons disappearing. Unfortunately, however, Android's Doze feature makes this unreliable and worthless for most use cases. See my summary of work on this here
I was going through the table here http://www.davidgyoungtech.com/2017/08/07/beacon-detection-with-android-8 and I am confused as to how does the library achieve detection time of 5 seconds after app kill on Android platforms 5-7. This is what the link says - Apps on Android 4.3-7.x used long-running background services or alarms to periodically look for beacons in the background. As far as I know, alarms cannot be set for less than 15 minutes so how does the 5 second thing work?
My blog post's detection times table that you mention (reproduced below) says that detection times are 5 seconds on Android 5.0-7.x. (Clarification: it does not make this claim for 4.x.) This is possible on 5.x-7 by using a long-running filtered low-power scan using an API introduced in Android 5.0. Here's the code that sets that up.
What this does is puts the Bluetooth LE chip in a low power mode that automatically wakes up if any packet matching the expected filter patter is seen. After the wake up, the Android OS delivers the scan result to the library which then sends a callback to the application via didEnterRegion or didRangeBeaconsInRegion. This process typically takes just a few seconds after the packet appears.
These scanning times have nothing to do with Alarms. On Android 4.x-7.x, the AlarmManager is used for a completely different purpose -- to keep a long-running background service alive. An Alarm repeatedly reset for 5 minutes in the future, and as long as the service keeps running it never goes off. If the app is killed due to low memory or the user swiping it off the screen, this alarm will fire a BroadcastReceiver which triggers the long-running scanning service to start itself again.
To my knowledge there is nothing on Android 4.x-7.x that limits Alarms from running more than once every 15 minutes. You may be thinking of a limitation on Android 8+ that restricts periodic JobScheduler items to every 15 minutes or more.
Android Beacon Library Detection Times:
Google added restrictions of not broadcasting the Bluetooth switch states to the app when the app in the background. This effectively blocks the optimized Bluetooth beacon scanning in the background. How to get around this issue other than periodic job scheduling?
Any help is appreciated.
You have several options for background BLE Beacon Scanning on Android 8+
A regular background service. You can use these just like on Android 4.3-6.x, but you are generally limited to 10 minutes of running time in the background, after which time Android will kill your app and it won't be able to scan anymore.
Use a foreground service. These work much like Android background services except they display an even-present notification with an icon of your choice to indicate that your app is running in the background. With a foreground service, you can effectively scan for beacons in the background with no restrictions just like on Android 4.3-6.x.
Use Intent-based scans. If you simply need to know when a beacon appears or disappears, you can set up an Intent-based scan for BLE devices with a bluetooth packet filter that filters on the presence of the byte pattern of the beacon, or the absence of the byte pattern of the beacon. When the beacon appears or disappears, Android will send an Intent to a BroadcastReceiver in your app that will wake it up in the background and let it run for about 10 minutes before killing it. During this time you can keep scanning for beacons.
Use the job scheduler (also known as WorkManager). You can schedule a job to run at most every ~15 minutes in the background to do scanning. A job is generally limited to 10 minutes of running time in the background. Since start times vary by +/- 10 minutes, you'll have gaps of up to 0-15 minutes where you won't be scanning.
Play games with (3) and (4) to bend the rules. While this goes against the spirit of Android's restrictions, you can play games with the job scheduler by starting an immediate job, cancelling it before 10 minutes is up, then restarting it. You can do similar things with an intent based scan by simply triggering it over and over. Be forewarned, however, doing these things will drain the users' batteries, perhaps leading them to uninstall your app. This rule bending may be blocked in future Android releases.
You can read my blog post about the merits of these techniques here. The open source Android Beacon Library uses techniques 3 and 4 on Android 8+ devices by default, and also supports configuring a foreground service if you wish to choose option 2.
What I want to achieve
I am building an app for a shopping mall, with large amount of beacons installed not quite far away from each other (assume ~20m distance).
When a user walks in this mall, even without opening the app, the app needs to keep scan for beacons. When a beacon is detected, I will then query the server to ask if I need to and what local notification to push to user.
What I planned to do
My original plan was to create a Service, returns START_STICKY in onStartCommand() in order to make sure that the service will restart itself even the app is killed by a task manager.
This Service will keep scanning for beacons.
In fact, this is the existing approach of my colleague. According to this colleague, the service starts almost immediately after the app was killed by a task manager.
The problem
But soon I found that this approach is problematic.
The above approach has now 2 major problems:
It drains batteries because it keeps scanning;
It does not work on
Android 8 due to new limitations introduced in Android
8.
My next plan
Although I know that JobScheduler can be used to replace Service, which is also the existing approach of Android Beacon Library, perform a scan every 15-25 minutes absolutely cannot fulfill my requirements, where beacons are very close to each other and therefore beacons need to be scanned frequently.
Therefore I come up with another plan:
Use Android Beacon Library to run background detection
Once the
first beacon in filter list has been detected, start a foreground
service (which will not be killed even in Android 8) that
continuously scan for beacons
When all beacons in the filter has been exited, stop the above foreground service. Android Beacon
Library shall resume to its background detection state.
The intentions of this approach are:
Make advantage of Android Beacon Library's background detection which saves battery, according to their documentation
Step away of Android Beacon Library's handling of Android 8 due to its built-in limitation of long scanning interval
Scanning goes on even on Android 8 since I am going to use a foreground service
My major question
By reading documentations, I already know how to scan for beacons in the background.
But how do I make use of Android Beacon Library to scan in a foreground service?
Plus, is there any problem that you can discover in the above approach / Do you have any better suggestions to achieve such requirements?
My other question
In fact according to this post, the background service starts 5 minutes after an app is killed.
But by returning START_STICKY in onStartCommand() of a Service, it is restarted almost immediately.
Then, why will there be a 5 minutes delay, even in pre-Oreo?
This approach is sound. The Android Beacon Library 2.15+ natively supports foreground services as a scan mechanism for supporting cases like this on Android 8. See here for more info.
The tricky part is to switch back and forth between using Job Scheduler and a Service to do scanning. I have not tested this, but my suggestion would be to bind manually to the BeaconManager in a custom Application class.
Then :
On entering a region, stop monitoring and then unbind the BeaconManager.
Start a custom foreground service
In the foreground service, disable Scan Jobs, then bind to the BeaconManager and start ranging
Once no beacons have been ranged for a time, stop ranging, unbind from the BeaconManager, enable Scan Jobs, bind again and then start monitoring.
Finally, exit the foreground service
On the second question, yes, START_STICKY will very quickly restart a service on most platforms. The library uses a 5 minute timer with the AlarmManager as a backup, which will relaunch the service if a START_STICKY restart fails. Indeed, in typical use, the scanning service restarts much more quickly than five minutes.
Background:
Ideally I would like my Android device to be scanning for Bluetooth Low Energy devices all the time an the ability to start an application whenever a new device with specific properties appears.
So the broadcast packet in BLE will for instance enumerate a set of services provided by the broadcasting device. An app would then be able to register an interest for certain services and automatically be started when a device with this services comes into range.
From what I understand this is not how the Android BLE API works? So how can I get something similar?
Simplest possible example:
I have a BLE sensor that logs ambient temperature over time. Whenever my Android phone is close enough I want to connect and download all the data, sending it to some cloud storage solution. This app would not need any GUI (at least not after configuration is done). But how should it run in the background without draining the battery, but still give me a fairly good chance of connecting the device quickly once it is in range?
Question:
Do I need to set a timer and wake the app every once in a while and then manually start scanning? What kind of intervals should I then choose. How long can I leave the scanner running without adversely affecting the battery?
Possible solution:
This is what I've come up with so far.
A configuration activity to set the intervals and devices to scan for
The configuration activity will set up an WakefulBroadcastReceiver similar to the Scheduler example
When the receiver get's the onReceive event I start a BLE scan service (that I've written) as a wakeful service.
The scan service starts scanning (with a registered callback).
The service might get adv reports that it can act upon
After a timeout the service will stop the scanner and end the wakeful service.
This works, but I'm not sure it's the best way. I also don't know how small intervals I can have and still avoid destroying the battery life. What I would want is to start scanning every two minutes, scanning for 10-20 seconds. But I'm afraid that would be rather frequently to wake up the device?
This functionality has all been moved to the open source Android Beacon Library which will:
wake up/launch your app when iBeacons matching a desired pattern are detected
perform beacon scanning in the background even if the user has not launched your app yet
reduce the scan rate automatically in the background to 30 seconds every five minutes to save battery. (Timing configurable.)
Code examples are show here
If your BLE device is not a beacon you could still use this library to accomplish this by having your sensor also transmit as a beacon then after it is detected connect to the main service.