Develop Reference

Can we do Speech Recognition and Video Capture in Android using camerax simultaneously?

I want to implement an app that can convert text to speech and record videos (with audio) simultaneously.
But when I am calling both function either one of them working (the recent one that has been called). Can anyone suggest some ways to implement these two together.
`
speechRecognizer = SpeechRecognizer.createSpeechRecognizer(activity)
val speechRecognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
speechRecognizerIntent.putExtra(
RecognizerIntent.EXTRA_LANGUAGE_MODEL,
RecognizerIntent.LANGUAGE_MODEL_FREE_FORM,
)
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_COMPLETE_SILENCE_LENGTH_MILLIS,10000)
// speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_MINIMUM_LENGTH_MILLIS,30000)
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault())
speechRecognizerIntent.putExtra("android.speech.extra.GET_AUDIO_FORMAT", "audio/MP3")
speechRecognizerIntent.putExtra("android.speech.extra.GET_AUDIO", true)
speechRecognizer.setRecognitionListener(object : RecognitionListener {
override fun onReadyForSpeech(bundle: Bundle?) {
speechRecognizer.startListening(speechRecognizerIntent)
}
override fun onBeginningOfSpeech() {}
override fun onRmsChanged(v: Float) {}
override fun onBufferReceived(bytes: ByteArray?) {}
override fun onEndOfSpeech() {
// changing the color of our mic icon to
// gray to indicate it is not listening
// #FF6D6A6A
}
override fun onError(i: Int) {}
override fun onResults(bundle: Bundle) {
}
override fun onPartialResults(bundle: Bundle) {
val result = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)
if (result != null) {
for (i in 0 until result.size){
text.add(result[0])
Log.d("Record",result[0])
//binding.tvtext.text = binding.tvtext.text.toString() + result[0]
}
}
}
override fun onEvent(i: Int, bundle: Bundle?) {}
})
speechRecognizer.startListening(speechRecognizerIntent)
}
`

move from callback to suspend function

I have this function :
fun createBiometricPrompt(
activity: AppCompatActivity,
processSuccess: (BiometricPrompt.AuthenticationResult) -> Unit,
callback: BiometricCallback
): BiometricPrompt {
var nbFailure = 0
val executor = ContextCompat.getMainExecutor(activity)
val callback = object : BiometricPrompt.AuthenticationCallback() {
override fun onAuthenticationError(errCode: Int, errString: CharSequence) {
super.onAuthenticationError(errCode, errString)
Log.d(TAG, "errCode is $errCode and errString is: $errString")
}
override fun onAuthenticationFailed() {
super.onAuthenticationFailed()
Log.d(TAG, "User biometric rejected.")
nbFailure++;
if(nbFailure == MAX_BIOMETRICS_FAILURE){
nbFailure = 0
callback.onFailure()
}
}
which is using a callback callback: BiometricCallback, it's only for now using it for notifying that a failure happened by sending callback.onFailure()
Callback is called in the calling Fragment by doing this:
Fragment.kt
biometricPrompt =
BiometricPromptUtils.createBiometricPrompt(requireActivity() as AppCompatActivity, ::encryptAndStoreServerToken, object: BiometricCallback {
override fun onFailure() {
biometricPrompt.cancelAuthentication()
viewModel.loginError.set("Error Biometrics")
}
})
The onFailure is used then to cancel the process and display an error.
The Callback interface is defined as :
interface BiometricCallback {
fun onFailure()
}
I have been asked to use a suspend function instead of a callback but I have no clue how to do it properly. Should I use LiveData ? If any idea, please help
Thanks

you can use suspendCancellableCoroutine or suspendCoroutine to convert any callback based Api in direct style to make it more kotlin friendly plus it provides more encapsulation you will be just returned with the result all complexity is handled inside the function only
suspend fun authenticate():BiomatricPrompt.AuthenticationResult? {
return suspendCancelableCoroutine { continuation ->
biometricPrompt = BiometricPrompt(this, executor,
object : BiometricPrompt.AuthenticationCallback() {
override fun onAuthenticationError(errorCode: Int,
errString: CharSequence) {
super.onAuthenticationError(errorCode, errString)
continuation.resume(null,null)
}
override fun onAuthenticationSucceeded(
result: BiometricPrompt.AuthenticationResult) {
super.onAuthenticationSucceeded(result)
continuation.resume(result,null)
}
override fun onAuthenticationFailed() {
super.onAuthenticationFailed()
continuation.resume(null,null)
}
})
promptInfo = BiometricPrompt.PromptInfo.Builder()
.setTitle("Biometric login for my app")
.setSubtitle("Log in using your biometric credential")
.setNegativeButtonText("Use account password")
.build()
biometricPrompt.authenticate(promptInfo)
}
}
usage
val authenticationResult = authenticate()
if(authenticationResult == null){
//authentication failed
}else{
//authenticated successfully
}
you can use custom object as result type to handle more use cases
more information

How to Initiate virtual assistance with voice in android?

I am trying to build in app app voice assistant. I am able to record voice using SpeachRecognition but have to manually click button to start recording. How can I make it work by saying key word "Hey 'app name'" like hey google. I want my app to list the key word to start recording when app is in foreground.
fun initVoiceInput() {
val speechRecognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
val speechRecognizer = SpeechRecognizer.createSpeechRecognizer(application)
speechRecognizerIntent.putExtra(
RecognizerIntent.EXTRA_LANGUAGE_MODEL,
RecognizerIntent.LANGUAGE_MODEL_FREE_FORM
)
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault())
speechRecognizer.setRecognitionListener(object : RecognitionListener {
override fun onReadyForSpeech(bundle: Bundle) {
val data = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)
data?.apply {
// TODO
}
}
override fun onBeginningOfSpeech() {}
override fun onRmsChanged(v: Float) {}
override fun onBufferReceived(bytes: ByteArray) {}
override fun onPartialResults(p0: Bundle?) {
// TODO("Not yet implemented")
}
override fun onEvent(p0: Int, p1: Bundle?) {
// TODO("Not yet implemented")
}
override fun onEndOfSpeech() {
// TODO
}
override fun onError(p0: Int) {
// TODO("Not yet implemented")
}
override fun onResults(bundle: Bundle) {
val data = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)
data?.apply {
this.first()?.let {
if (it.contentEquals("my app name")) {
// Perform task here
}
}
}
}
})
}
It allows recording for service call but ends after few seconds. normally around 1 mins.

Android pageKeyed datasource not getting called

I am trying to achieve paging in my app using android paging library.But I am stuck in one place. The dataSource is not getting created from the factory. Snippet below.
private fun getLivePagedListBuilder(queryString: String): LivePagedListBuilder<Int, News> {
val dataSourceFactory = object : DataSource.Factory<Int, News>() {
override fun create(): DataSource<Int, News> {
return NewsDataSource(queryString)
}
}
return LivePagedListBuilder(dataSourceFactory, config)
}
1. The create method is not getting called. So the return inside tht method not firing.
My DataSource
class NewsDataSource(val searchQuery: String) : PageKeyedDataSource<Int, News>() {
override fun loadInitial(params: LoadInitialParams<Int>, callback: LoadInitialCallback<Int, News>) {
api.searchNews(searchQuery, Constants.perPageLimit, 1)
.enqueue(object : Callback<NewsResponse> {
override fun onFailure(call: Call<NewsResponse>, t: Throwable) {
Log.d("TAG1", "Failure")
}
override fun onResponse(call: Call<NewsResponse>, response: Response<NewsResponse>) {
callback.onResult(response.body()?.news, response.body()?.page - 1, response.body()?.page + 1)
}
})
}
}
The API returns the page number and I am planning to load the next page when the scroll ends
What determines the datatype of the Key in DataSource.Factory<Int, News>()
I am really stuck on this :(

kotlin coroutine throws java.lang.IllegalStateException: Already resumed, but got value Location

I'm quite new to Kotlin coroutine and Android development in general.
While playing around to understand how it worked, I faced an error I can't seem to solve.
From a basic activity i try to connect to the googleApiClient. The permissions are ok.
I wish to use kotlin coroutines to get location updates from LocationManager in a direct style to use this Location object later on.
The first time I changed my position in the emulator it works fine, the second time I change my position, It crashes
with an exception like this:
FATAL EXCEPTION: main
Process: com.link_value.eventlv, PID: 32404
java.lang.IllegalStateException: Already resumed, but got value Location[gps 48.783000,2.516180 acc=20 et=+59m16s372ms alt=0.0 {Bundle[mParcelledData.dataSize=40]}]
at kotlinx.coroutines.experimental.AbstractContinuation.resumeImpl(AbstractContinuation.kt:79)
at kotlinx.coroutines.experimental.AbstractContinuation.resume(AbstractContinuation.kt:72)
at com.link_value.eventlv.View.Create.NewEventLvActivity$await$2$1.onLocationChanged(NewEventLvActivity.kt:100)
at android.location.LocationManager$ListenerTransport._handleMessage(LocationManager.java:297)
at android.location.LocationManager$ListenerTransport.-wrap0(LocationManager.java)
at android.location.LocationManager$ListenerTransport$1.handleMessage(LocationManager.java:242)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:154)
at android.app.ActivityThread.main(ActivityThread.java:6077)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:866)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:756)
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_new_event_lv)
askForUserLocation()
val locationManager = this.getSystemService(Context.LOCATION_SERVICE) as LocationManager
val presenter = CreateEventPresenterImpl(this#NewEventLvActivity)
googleApiClient = GoogleApiClient.Builder(this#NewEventLvActivity)
.enableAutoManage(this /* FragmentActivity */,
this /* OnConnectionFailedListener */)
.addApi(Places.GEO_DATA_API)
.addConnectionCallbacks(this)
.build()
}
override fun onConnected(p0: Bundle?) {
val locationManager = this.getSystemService(Context.LOCATION_SERVICE) as LocationManager
input_address.addTextChangedListener(object: TextWatcher{
override fun afterTextChanged(p0: Editable?) {
}
override fun beforeTextChanged(p0: CharSequence?, p1: Int, p2: Int, p3: Int) {
}
override fun onTextChanged(query: CharSequence?, p1: Int, p2: Int, p3: Int) {
if (query.toString().length >= 4) {
launch(UI) {
val locationUpdated = locationManager.await(LocationManager.GPS_PROVIDER)
input_name.text = Editable.Factory.getInstance().newEditable(locationUpdated.toString())
}
}
}
})
}
private suspend fun LocationManager.await(locationProvider: String): Location? = suspendCoroutine { cont ->
try {
requestLocationUpdates(locationProvider, 0, 0.toFloat(), object : LocationListener {
override fun onStatusChanged(p0: String?, p1: Int, p2: Bundle?) {
}
override fun onProviderEnabled(p0: String?) {
}
override fun onProviderDisabled(p0: String?) {
cont.resumeWithException(UnknownLocationException())
}
override fun onLocationChanged(location: Location?) {
cont.resume(location)
}
})
} catch (ex: SecurityException) {
cont.resumeWithException(ex)
}
}
It's as if Kotlin use the same Continuation. I don't know what I'm doing wrong and why it crashes the second time. Can someone enlighten me.
Thks in advance.

According to documentation, it's possible to resume Continuation only single time. Second resume will throw IllegalStateException with "Already resumed, but got $proposedUpdate" message. You can add additional checking continuation.isActive to prevent this exception. It's better to use Channels for multiple callbacks like location updates.

I had this issue recently with one of 3rd party APIs we use in our project. The problem was solely on the side of the API as the callback was called more than once in the underlying implementation and unsubscribe method wasn't immedeately removing callback from the API.
So, approach I picked was just simply to add a boolean flag. Although, it could've been solved in a bit more elegant way with Kotlin's Flow.mapLatest/collectLatest or so.
suspend fun singularContentUpdate(): Data =
suspendCancellableCoroutine { continuation ->
var resumed = false
val callback = object : Callback {
override fun onDataReady(d: Data) {
api.unsubscribe(this)
if (!resumed) {
continuation.resume(d)
resumed = true
}
}
}
continuation.invokeOnCancellation {
api.unsubscribe(callback)
}
api.subscribe(subscriber)
api.refresh()
}

You can use this function to prevent this exception.
import kotlinx.coroutines.CancellableContinuation
import kotlin.coroutines.Continuation
import kotlin.coroutines.resume
inline fun <T> Continuation<T>.safeResume(value: T, onExceptionCalled: () -> Unit) {
if (this is CancellableContinuation) {
if (isActive)
resume(value)
else
onExceptionCalled()
} else throw Exception("Must use suspendCancellableCoroutine instead of suspendCoroutine")
}
Notice that I'll throw an exception if you're using suspendCoroutine instead of suspendCancellableCoroutine.
Usage:
suspend fun getId() : Result<String> = suspendCancellableCoroutine { continuation ->
...
continuation.safeResume(Result.success(adjustID)) { Timber.e("Job has already done") }
}