I've implemented Android client using Twilio SDK to do video calls. It works as expected but I found an edge case which I can not figure out how to fix. Here is essence of video call code:
val connectionOptions = ConnectOptions.Builder(accessToken)
.audioTracks(listOf(audioManager.getLocalAudioTrack()))
.roomName(roomId)
.build()
val roomListener = RoomListener()
Video.connect(androidContext, connectOptions, roomListener)
class RoomEventListener : Room.Listener {
override fun onParticipantDisconnected(room: Room, remoteParticipant: RemoteParticipant) {
// remove participant from the screen, unsubscribe from events
}
override fun onConnectFailure(room: Room, twilioException: TwilioException) {
exitScreenWithErrorMessage(R.string.video_consult_room_connection_error)
}
override fun onReconnected(room: Room) {
_shouldShowReconnectionActivity.value = false
}
override fun onParticipantConnected(room: Room, remoteParticipant: RemoteParticipant) {
onRemoteParticipantConnected(remoteParticipant)
}
override fun onConnected(room: Room) {
_shouldShowConnectionActivity.value = false
this#VideoCallViewModel.room = room
room.remoteParticipants.forEach { onRemoteParticipantConnected(it) }
determineMainParticipant()
onLocalParticipantConnected(room)
}
override fun onDisconnected(room: Room, twilioException: TwilioException?) {
exitVideoConsultScreen()
}
override fun onReconnecting(room: Room, twilioException: TwilioException) {
_shouldShowReconnectionActivity.value = true
}
}
Test case:
Bob joins video call using Android phone
Jane joins same video call from any device (iOS, web, Android)
When Jane loses connection (i.e. turn off internet)
Then Bob sees for 1-2 minutes reconnecting (programmatically "onReconnecting" callback triggered)
[Actual] And Bob disconnected from room (in logs I see Media connection failed or Media activity ceased with error code 53405)
[Expected] Bob stays at the room.
I'm not sure why under such conditions Android client has been disconnected (We tested it on different devices with Android 8/9).
Couple more details:
If Jane exits room using "End call" button (so room.disconnect() code from Twilio SDK has been called) then Bob stays in the room.
When Bob using iOS device (implementation of iOS and Android quite the same) then described use case passes.
We tried 5.0.1 and 5.1.0 version of com.twilio:video-android library.
I noticed Known Issue for Android Twilio Video library in Release notes and I'm not sure can it affects described use case or not:
Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. As a result, tracks published after a Room.State.RECONNECTED event might not be subscribed to by a RemoteParticipant.
I opened issue on twilio github repo https://github.com/twilio/video-quickstart-android/issues/454 - and this is expected behaviour for twilio video sdk 5.x+. Both for Android and iOS sdks.
Related
I am using the latest Phone Number Hint API library, provided by Google Play Services, for requesting phone number as hint. Here is my code :
implementation 'com.google.android.gms:play-services-auth:20.3.0'
implementation 'com.google.android.gms:play-services-auth-api-phone:18.0.1'
private val phoneNumberHintIntentResultLauncher =
registerForActivityResult(ActivityResultContracts.StartIntentSenderForResult()) { result ->
try {
val phoneNumber =
Identity.getSignInClient(requireContext()).getPhoneNumberFromIntent(result.data)
Log.d("PhoneNumberRetrieved", phoneNumber)
} catch (e: Exception) {
e.printStackTrace()
}
}
private fun requestPhoneNumberHint() {
val request = GetPhoneNumberHintIntentRequest.builder().build()
Identity.getSignInClient(requireContext())
.getPhoneNumberHintIntent(request)
.addOnSuccessListener { taskResult ->
phoneNumberHintIntentResultLauncher.launch(
IntentSenderRequest.Builder(
taskResult.intentSender
).build()
)
}
.addOnFailureListener {
it.printStackTrace()
}
}
Above code is working fine on devices with Android version less than 12. But it is not working properly on devices with Android 12. I have tried on two different devices and here are the results*:
Realme narzo 50a** - failureListner is invoked with this message
16: API is disabled
Samsung Galaxy M32 - failureListner is invoked with this message
16: No phone number is found on this device. Even though the device has valid numbers available.
* NOTE : The old deprecated Hint Request API is working properly on both devices giving expected results i.e. showing device phone numbers in a pop-up.
** IMPORTANT : Same code worked properly on this device on day 1 of my implementation i.e. it showed the available phone numbers on a bottom sheet. But on the next day, for unknown reasons, it started giving above mentioned result. Although I made no changes in my code.
If anyone else has faced this issue or have any information regarding this, please do share you thoughts.
I use AltBeacon library for simple ble scanning. Scanning for about 7 or 8 seconds then I stop it. tapping button for rescanning. the problem that I have had from the previous version of this library , when I scan then stop it, and disconnect beacon power and I rescan , rangenotifier or observer(another method to watch beacons) could see disconnected beacon! for first time after disconnected it is happened. after that works correctly and if I do the whole process again it is happens.
in the previous library version I had to bind and unbind each time.(not good approach but I had to do) .but in newer version there are no unbind or bind methods. most of methods and functions are deprecated.
I use scanner in fragment. even it is not matter if switch to another fragment. when I come back to scanning fragment again it finds disconnected beacon for fist time after beacon power disconnected. I'm not sure if this library is suitable for a simple bacon scanning. But it is very powerful and simplified some complex thing.
class ScanningFragment() : androidx.fragment.app.Fragment(){
lateinit var beaconManager:BeaconManager
lateinit var region:Region
val rangeNotifier =object:RangeNotifier{
override fun didRangeBeaconsInRegion(beacons: MutableCollection<Beacon>?, region: Region?) {
Log.d(TAG,"in didRangeBeacon")
if (beacons!!.size > 0) {
Log.d(TAG, "didRangeBeaconsInRegion called count: " + beacons.size + beacons.iterator().next().id1)
val firstBeacon = beacons.iterator().next()
}
override fun onCreate(savedInstanceState: Bundle?)
{
Log.d("lifecycl","it is oncreate ")
super.onCreate(savedInstanceState)
BeaconManager.setDebug(true)
beaconManager=BeaconManager.getInstanceForApplication(requireContext()).apply {
foregroundScanPeriod=7000L
foregroundBetweenScanPeriod=5000L
updateScanPeriods()
beaconParsers.clear()
beaconParsers.add(BeaconParser().setBeaconLayout("m:2-3=0215,i:4-8,i:4-19,i:20-21,i:22-23,p:24-24"))
region = Region("prefixRegion", Identifier.parse("0x0000000000"), null, null)
}
setupPermissions()
}
fun rangingButtonTapped() {
if (beaconManager.rangedRegions.size == 0) {
beaconManager.addRangeNotifier(rangeNotifier)
beaconManager.startRangingBeacons(region)
binding.insideviewmodel?.isScanning?.value = true
}
else {
beaconManager.stopRangingBeacons(region)
binding.BTNScan.run {
Handler(Looper.getMainLooper()).postDelayed({ stopAnimation() }, 1000)
Handler(Looper.getMainLooper()).postDelayed({ revertAnimation() }, 2000)
}
}
override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
binding.insideviewmodel?.isScanning?.observe(viewLifecycleOwner,Observer{
currentStatusScan->
if(currentStatusScan)
{
object : CountDownTimer(8500, 1000) {
override fun onTick(p0: Long) {
}
override fun onFinish() {
beaconManager.stopRangingBeacons(region)
beaconManager.removeRangeNotifier(rangeNotifier)
binding.insideviewmodel?.isScanning?.value = false
}
}.start()
}
})
}
}
}
it is a debug log for first time scanning.
https://jpst.it/2LVY4
it is a debug log for rescanning after disconnecting beacon power.
https://jpst.it/2LVZs
Profiler:
The second log line "after disconnecting beacon power" shows that the Android OS BLE scanner does indeed deliver an iBeacon detection at 12:53:31:
2022-02-24 12:53:31.117 23528-23528/ D/CycledLeScannerForLollipop: got record ... Processing pdu type FF: 0201041aff4c000215....
The library source code shows that this log line is issued immediately upon a callback from the operating system about a BLE advertisement detection. See here.
Clearly it is not possible for a Bluetooth scanner to detect an advertisement from a powered-off BLE device so there must be an alternate explanation. A few possibilities:
The BLE transmitter is not really powered off at (or slightly before) 12:53:31.117
The detected advertisement comes from a different transmitter
The callback from the Android OS is delayed, perhaps because the main thread on which is delivered was blocked by lots of CPU usage in the app.
Some flaw in the bluetooth stack or UI thread handling for the phone in question is delaying delivery of detections.
In order to figure out the cause I would suggest the following:
To eliminate a bluetooth stack flaw, test the same code on a different Android phone, preferably by a different manufacturer.
To eliminate the possibility of the UI thread being blocked, run this in the Android Studio profiler, or simply cut out as much code as possible that executes before the delay is seen.
To verify the transmitter is really off and that there are no other transmitters around, use a second phone with an off the shelf beacon scanner to monitor what devices are actually transmitting. Only perform your test when you confirm with a second device there are no other visible transmitters.
Everything works fine, for instance, on Samsung (Android 11), Huawei (Android 10), with the exception of at least Google Pixel 2 (Android 11), Google Pixel 5 (Android 11). There is also no problem with Wi-Fi on these devices.
There is a registration screen. The user enters the data and clicks on the "sign up" button.
Everything is fine until the user performs the following actions:
Enable the mobile network -> Click on the "sign up" button -> For example, the message "email is already in use" -> Disable the mobile network -> Click on the "sign up" button -> The suspended coroutine never continues (FirebaseNetwork exception is expected)
However, it works:
Enable the mobile network -> Disable the mobile network -> Click on the "sign up" button -> For example, the message "email is already in use" (everything is fine because the suspended coroutine has woken up)
Bottom line: Firebase does not throw a FirebaseNetwork or any exception and, as a result, the user interface "freezes" (I disable the form when the request is being processed) when the user submits the form with the mobile network enabled and then submits the form with the mobile network turned off.
private suspend fun handleResult(task: Task<AuthResult>) =
suspendCoroutine<AuthResult> { continuation ->
task.addOnSuccessListener { continuation.resume(it) }
.addOnFailureListener { continuation.resumeWithException(it) }
}
I solved the problem with this answer.
The code now looks like:
private suspend fun handleResult(task: Task<AuthResult>) =
withTimeout(5000L) {
suspendCancellableCoroutine<AuthResult> { continuation ->
task.addOnSuccessListener { continuation.resume(it) }
.addOnFailureListener { continuation.resumeWithException(it); }
}
}
Do I need to use the suspendCancellableCoroutine with timeout instead of suspendCoroutine always with Firebase to avoid these bugs on another devices?
I don't know if this is the cause, but it might help. There is already a suspend function that handles Google Tasks without you having to implement suspendCancellableCoroutine yourself. Their implementation is quite a bit more thorough than yours (it's about 30 lines of code) and maybe handles some edge cases that yours doesn't. It also optimizes results when the task is already finished by the time you call it, and it handles cancellation correctly, which yours does not.
The function is Task.await(). If it's not available to you, then add this dependency in your build.gradle:
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-play-services:1.6.0"
The problem is solved by combining await() (thank you, #Tenfour04) + withTimeout(). It looks like Firebase doesn't have a timeout for network authentication calls.
build.gradle to get suspending await() (replace "x.x.x" with the latest version):
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-play-services:x.x.x'
For example:
private val firebaseAuth: FirebaseAuth
suspend fun create(userInitial: UserInitial): AuthResult = withTimeout(5000L) {
firebaseAuth.createUserWithEmailAndPassword(
userInitial.email,
userInitial.password
)
.await()
}
withTimeout() throws a TimeoutCancellationException if the timeout was exceeded
I want to make an Android application that can record both incoming and outgoing calls in the background as a service in kotlin and at a particular time in the day, it sends all that recordings to a server by API. I had researched about it all I found is to use Device Policy Manager and Telephoney Manager but it is not much about it on the internet. So can you help me with any article, documentation, or tutorial?
There is no solution from Google as of now. Google has deprecated the feature of recording the calls in it's latest versions of Android OS. Earlier it was possible, I had tried various methods but I was only getting the silent audio when I had tried to record calls. When using Google's Phone application it only allows that application to use the microphone and other things it won't allow any other application to overpower and get that hardware access.
But there are actually two hacks to do that.
Build your own phone application like Truecaller and manage every call and other things from that application by doing this you can get access to managing calls on your device and you will also get the access to record the calls.
If your work is specific to any one mobile example like Samsung, OnePlus, etc. Then you can use any Truecaller or Google's Phone application which will store the recordings of the calls in file storage and then you can make a service to upload that call recording from that particular file location every night at 12 AM or something.
first create MyCallRecordReceiver class
class MyCallRecordReceiver(callRecord: CallRecord) : CallRecordReceiver(callRecord) {
override fun onIncomingCallReceived(context: Context, number: String?, start: Date) {
super.onIncomingCallReceived(context, number, start)
}
}
then in MainActivity
class MainActivity : AppCompatActivity() {
companion object {
private val TAG = MainActivity::class.java.simpleName
}
private lateinit var callRecord: CallRecord
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
callRecord = CallRecord.Builder(this)
.setLogEnable(true)
.setRecordFileName("CallRecorderTestFile")
.setRecordDirName("CallRecorderTest")
.setAudioSource(MediaRecorder.AudioSource.VOICE_COMMUNICATION)
.setShowSeed(true)
.build()
}
fun StartCallRecordClick(view: View) {
LogUtils.i(TAG, "StartCallRecordClick")
callRecord.startCallReceiver()
}
fun StopCallRecordClick(view: View) {
LogUtils.i(TAG, "StopCallRecordClick")
callRecord.stopCallReceiver()
}
}
In addition Add it as a dependency in your app's build.gradle file
allprojects {
repositories {
maven { url 'https://jitpack.io' }
}
}
and this
compile 'com.github.aykuttasil:CallRecorder:1.5.3'
I'm trying to incorporate Pepper's built in Android tablet more in DialogFlow interactions. Particularly, my goal is to open applications installed on the tablet itself for people to use while they're talking with Pepper. I'm aware there is a 'j-tablet-browser' app installed on Pepper's end that can let a person browse the tablet like an ordinary Android device, but I would like to take it one step further and directly launch an Android app, like Amazon's Alexa.
The best solution I can up with is:
User says specific utterance (e.g. "Pepper, open Alexa please")
DialogFlow launches the j-tablet-browser behavior
{
"speak": "Sure, just a second",
"action": "startApp",
"action_parameters": {
"appId": "j-tablet-browser/."
}
}
User navigates the Android menu manually to tap the Alexa icon
My ideal goal is to make the process seamless:
User says specific utterance (e.g. "Pepper, open Alexa please")
DialogFlow launches the Alexa app installed on the Android tablet
Does anyone have an idea how this could be done?
This is quite a broad question so I'll try and focus on the specifics for launching an app with a Dialogflow chatbot. If you don't already have a QiSDK Dialogflow chatbot running on Pepper, there is a good tutorial here which details the full process. If you already have a chatbot implemented I hope the below steps are general enough for you to apply to your project.
This chatbot only returns text results for Pepper to say, so you'll need to make some modifications to allow particular actions to be launched.
Modifying DialogflowDataSource
Step 2 on this page of the tutorial details how to send a text query to Dialogflow and get a text response. You'll want to modify it to return the full reponse object (including actions), not just the text. Define a new function called detectIntentFullResponse for example.
// Change this
return response.queryResult.fulfillmentText
// to this
return response.queryResult
Modifying DialogflowChatbot
Step 2 on this page shows how to implement a QiSDK Chatbot. Add some logic to check for actions in the replyTo function.
var response: DetectIntentResponse? = null
// ...
response = dataSource.detectIntentFullResponse(input, dialogflowSessionId, language)
// ...
return if (reponse.action != null) {
StandardReplyReaction(
ActionReaction(qiContext, response), ReplyPriority.NORMAL
)
} else if (reponse.answer != null) {
StandardReplyReaction(
SimpleSayReaction(qiContext, reponse.answer), ReplyPriority.NORMAL
)
} else {
StandardReplyReaction(
EmptyChatbotReaction(qiContext), ReplyPriority.FALLBACK
)
}
Now make a new Class, ActionReaction. Note that the below is incomplete, but should serve as an example of how you can determine which action to run (if you want others). Look at SimpleSayReaction for more implementation details.
class ActionReaction internal constructor(context: QiContext, private val response: DetectIntentResponse) :
BaseChatbotReaction(context) {
override fun runWith(speechEngine: SpeechEngine) {
if (response.action == "launch-app") {
var appID = response.parameters.app.toString()
// launch app at appID
}
}
}
As for launching the app, various approaches are detailed in other questions, such as here. It is possible to extend this approach to do other actions, such as running or retrieving online data.