I follow this question to do the offline speech on android.
I downloaded the language in google voice and it can work in offline.
The problem is that I want to know it's current running on offline or online speech, (just like Apple speech to text, there is an api to check for that) to display the speech stream in my app correctly
I wonder is there anyway to do that?
Here is my code:
val intentSpeech = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
intentSpeech.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, "en-US")
intentSpeech = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
intentSpeech.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true);
intentSpeech.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
RecognizerIntent.LANGUAGE_MODEL_FREE_FORM)
intentSpeech.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,
packageName)
val recognizer = SpeechRecognizer.createSpeechRecognizer(this)
recognizer.setRecognitionListener(this)
P/s: I can see the Read Along app by google works perfectly in offline or online mode.
I'm trying to do the same with the android speech api. Is it possible?
For offline speech to text, you can use Google's default STT model but it seems to be non-continuous.
private fun startSpeechToText() {
val speechRecognizer = SpeechRecognizer.createSpeechRecognizer(this)
val speechRecognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
speechRecognizerIntent.putExtra(
RecognizerIntent.EXTRA_LANGUAGE_MODEL,
RecognizerIntent.LANGUAGE_MODEL_FREE_FORM
)
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault())
speechRecognizer.setRecognitionListener(object : RecognitionListener {
override fun onReadyForSpeech(bundle: Bundle?) {}
override fun onBeginningOfSpeech() {}
override fun onRmsChanged(v: Float) {}
override fun onBufferReceived(bytes: ByteArray?) {}
override fun onEndOfSpeech() {}
override fun onError(i: Int) {}
override fun onResults(bundle: Bundle) {
val result = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)
if (result != null) {
// result[0] will give the output of speech
}
}
override fun onPartialResults(bundle: Bundle) {}
override fun onEvent(i: Int, bundle: Bundle?) {}
})
// starts listening ...
speechRecognizer.startListening(speechRecognizerIntent)
}
If you don't want to use google as it requires to download the offline model for speech to text.
The other option for offline STT is the Vosk API as it has pretrained models in english and other language for live STT.
https://github.com/alphacep/vosk-android-demo
Reference: https://www.geeksforgeeks.org/offline-speech-to-text-without-any-popup-dialog-in-android/
Related
Im building an app that translate audio in background on live. I display a overlay over apps and i want to record the audio to translate in real time. But in android 13 the audio recording stops after a few secods recording.
I guess that the problem is for the security of the user, but im looking for a second opinion
Im using the SpeechRecognizer library. But I receive recommendations
Added a piece of code
AndroidManifiest.xml
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<service
android:name=".feature.overlay.OverlayService"
android:exported="false"
android:foregroundServiceType="microphone"/>
OverlayService
override fun onCreate() {
super.onCreate()
windowManager = getSystemService(Context.WINDOW_SERVICE) as WindowManager
configureSpeechToText()
windowManager.addView(layoutText,params)
}
fun configureSpeechToText() {
speechRecognizer = SpeechRecognizer.createSpeechRecognizer(this)
speechRecognizerIntent.putExtra(
RecognizerIntent.EXTRA_LANGUAGE_MODEL,
RecognizerIntent.LANGUAGE_MODEL_FREE_FORM
)
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault())
speechRecognizer?.setRecognitionListener(object : RecognitionListener {
override fun onReadyForSpeech(bundle: Bundle) {}
override fun onBeginningOfSpeech() {}
override fun onRmsChanged(v: Float) {}
override fun onBufferReceived(bytes: ByteArray) {}
override fun onEndOfSpeech() {}
override fun onError(i: Int) {
val errorMessage = getErrorText(i)
Log.d("Error", "FAILED: $errorMessage")
}
override fun onResults(bundle: Bundle) {
val data = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)
dataVoiceText = data!![0]
run(dataVoiceText)//The function that translate the text
}
override fun onPartialResults(bundle: Bundle) {}
override fun onEvent(i: Int, bundle: Bundle) {}
})
handler.postDelayed(Runnable {//Run this code every few seconds to translate every few seconds
handler.postDelayed(runnable!!, delay.toLong())
speechRecognizer?.stopListening()
speechRecognizer?.startListening(speechRecognizerIntent)
}.also { runnable = it }, delay.toLong())
}
Im trying to record audio in background with a service, im expecting to it continously record the audio. but right now it is stoping
I want to implement an app that can convert text to speech and record videos (with audio) simultaneously.
But when I am calling both function either one of them working (the recent one that has been called). Can anyone suggest some ways to implement these two together.
`
speechRecognizer = SpeechRecognizer.createSpeechRecognizer(activity)
val speechRecognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
speechRecognizerIntent.putExtra(
RecognizerIntent.EXTRA_LANGUAGE_MODEL,
RecognizerIntent.LANGUAGE_MODEL_FREE_FORM,
)
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_COMPLETE_SILENCE_LENGTH_MILLIS,10000)
// speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_MINIMUM_LENGTH_MILLIS,30000)
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault())
speechRecognizerIntent.putExtra("android.speech.extra.GET_AUDIO_FORMAT", "audio/MP3")
speechRecognizerIntent.putExtra("android.speech.extra.GET_AUDIO", true)
speechRecognizer.setRecognitionListener(object : RecognitionListener {
override fun onReadyForSpeech(bundle: Bundle?) {
speechRecognizer.startListening(speechRecognizerIntent)
}
override fun onBeginningOfSpeech() {}
override fun onRmsChanged(v: Float) {}
override fun onBufferReceived(bytes: ByteArray?) {}
override fun onEndOfSpeech() {
// changing the color of our mic icon to
// gray to indicate it is not listening
// #FF6D6A6A
}
override fun onError(i: Int) {}
override fun onResults(bundle: Bundle) {
}
override fun onPartialResults(bundle: Bundle) {
val result = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)
if (result != null) {
for (i in 0 until result.size){
text.add(result[0])
Log.d("Record",result[0])
//binding.tvtext.text = binding.tvtext.text.toString() + result[0]
}
}
}
override fun onEvent(i: Int, bundle: Bundle?) {}
})
speechRecognizer.startListening(speechRecognizerIntent)
}
`
Following is my code to detect biometric functionality
class MainActivity : AppCompatActivity() {
private var executor: Executor? = null
private var biometricPrompt: BiometricPrompt? = null
private var promptInfo: BiometricPrompt.PromptInfo? = null
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
val bioMetricManager = BiometricManager.from(this)
when (bioMetricManager.canAuthenticate()) {
BiometricManager.BIOMETRIC_SUCCESS -> {
executor = ContextCompat.getMainExecutor(this)
promptInfo = BiometricPrompt.PromptInfo.Builder().setTitle("Biometric login for leaao")
.setSubtitle("Log in using biometric credentials").setDeviceCredentialAllowed(true).build()
biometricPrompt = BiometricPrompt(this,
executor!!, object : BiometricPrompt.AuthenticationCallback() {
override fun onAuthenticationError(errorCode: Int, errString: CharSequence) {
super.onAuthenticationError(errorCode, errString)
Log.i("here4","Authentication error: $errString")
}
override fun onAuthenticationSucceeded(result: BiometricPrompt.AuthenticationResult) {
super.onAuthenticationSucceeded(result)
Log.i("here5","Success")
}
override fun onAuthenticationFailed() {
super.onAuthenticationFailed()
Log.i("here6","Authentication failed: ")
}
})
biometricPrompt?.authenticate(promptInfo!!)
}
BiometricManager.BIOMETRIC_ERROR_NO_HARDWARE -> {
Log.i("here","No biometric features available on this device.")
}
BiometricManager.BIOMETRIC_ERROR_HW_UNAVAILABLE -> {
Log.i("here2","Biometric features are currently unavailable.")
}
BiometricManager.BIOMETRIC_ERROR_NONE_ENROLLED -> {
Log.i("here3","The user hasn't associated any biometric credentials with their account.")
}
}
}
}
I have added <uses-permission android:name="android.permission.USE_BIOMETRIC" /> to Manifest file and added implementation 'androidx.biometric:biometric:1.0.1' to gradle
Still my code fails to detect biometric prompt. I have enabled face recognition on my mobile but it does not work when i open my app. It goes to BIOMETRIC_ERROR_HW_UNAVAILABLE case in my code.
By default, the BiometricManager seeks BIOMETRIC_STRONG sensors on the device, unless you specify them in .setAllowedAutnenticators(). However, you should know that as of now, there are no devices with STRONG face/iris sensors. That means that only the fingerprint sensor is "seen" as STRONG and all of the face and iris sensors (with some exceptions, where they are not seen as a sensor at all) are "seen" as BIOMETRIC_WEAK. With "seen" I mean they were set as such by the manufacturer. So having said that, something like this should be able to fix your issue:
BiometricPrompt.PromptInfo.Builder()
.setTitle(...)
.setDescription(...)
.setAllowedAuthenticators(
BiometricManager.Authenticators.BIOMETRIC_STRONG or
BiometricManager.Authenticators.BIOMETRIC_WEAK
)
and when checking for available authentication:
when (BiometricManager.from(requireContext()).canAuthenticate(
BiometricManager.Authenticators.BIOMETRIC_STRONG
or BiometricManager.Authenticators.BIOMETRIC_WEAK
)) { ... }
I have a program here, This is a function that is called on a click of a button. I would like the speech recognition to keep recognizing speech until stoplistening() is called. I keep getting the error codes 6, 7 and every time I speak it stops listening after a few seconds of silence, I even defined the silence and input values:
listenerintent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_POSSIBLY_COMPLETE_SILENCE_LENGTH_MILLIS , 200000)
listenerintent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_MINIMUM_LENGTH_MILLIS , 5000000)
but the application still closes after 2 seconds of silence if something is said before the silence. If nothing is said then the speech-recognition listens for the maximum amount of time I defined above.
I would like to have the speech recognition to be active regardless of the silence after something is recognized. Any parameter I am missing that could solve this issue? or is this library incapable of going forever and must stop after something is recognized? Is there another end pointer I am overlooking?
var recogobj = SpeechRecognizer.createSpeechRecognizer(this)
var listenerintent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
listenerintent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_POSSIBLY_COMPLETE_SILENCE_LENGTH_MILLIS , 200000)
listenerintent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_MINIMUM_LENGTH_MILLIS , 5000000)
listenerintent.putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_POSSIBLY_COMPLETE_SILENCE_LENGTH_MILLIS, 5000000)
val recoglistener = object : RecognitionListener {
override fun onReadyForSpeech(params: Bundle?) {
}
override fun onRmsChanged(rmsdB: Float) {
}
override fun onBufferReceived(buffer: ByteArray?) {
}
override fun onPartialResults(partialResults: Bundle?) {
Toast.makeText(this#Recordinit , RecognizerIntent.EXTRA_PARTIAL_RESULTS , Toast.LENGTH_LONG).show()
var data = partialResults!!.getStringArray(RecognizerIntent.EXTRA_PARTIAL_RESULTS)
for (result in data.orEmpty()){
if (result == "test") {
recogobj.stopListening()
Toast.makeText(this#Recordinit, "ok ", Toast.LENGTH_SHORT).show()
}
}
}
override fun onEvent(eventType: Int, params: Bundle?) {
}
override fun onBeginningOfSpeech() {
}
override fun onEndOfSpeech() {
}
override fun onError(error: Int) {
if (error ==6 || error == 7){
recogobj.startListening(listenerintent)
}
else{
Toast.makeText(this#Recordinit ,"${error}" , Toast.LENGTH_LONG).show()
}
}
override fun onResults(results: Bundle?) {
}
}
recogobj.setRecognitionListener(recoglistener)
recogobj.startListening(listenerintent)
}
I am developing an android application which plays videos from our server uploaded by both iphone and android, and the problem is that the videos recorded by iphone is in .mov format, which is not supported by android. I searched a lot, but couldn't found any solution. Help me out. Thanks in advance :)
I will recomend you to change the format from the iphones to a compatible (even open will be better), because 2 differents formats of videos on server soon or later will be a headache.
If you could, reformat your videos with ffmpeg and save it all with the same format on server.
If you can't or it's really hard to achieve, you could try the ExoPlayer component from Google.
I have tried on an app like you, where devices (iPhone and Android) record videos and upload to the server.
Reformatting all this video on server side will be almost impossible so we endly make the decission to apply a solution on client side for legacy videos.
https://google.github.io/ExoPlayer/
The setup from ExoPlayer it's larger than VideoView, but it's simple to be done.
private var player: ExoPlayer = initPlayer()
private fun initPlayer(): ExoPlayer {
val bandwidthMeter = DefaultBandwidthMeter()
val videoTrackSelectionFactory = AdaptiveTrackSelection.Factory(bandwidthMeter)
val trackSelector = DefaultTrackSelector(videoTrackSelectionFactory)
return ExoPlayerFactory.newSimpleInstance(context, trackSelector)
}
fun setup() {
videoExo.setPlayer(player)
videoExo.useController = false
val dataSourceFactory = DefaultDataSourceFactory(
context,
Util.getUserAgent(context, context?.packageName), DefaultBandwidthMeter()
)
val videoSource = ExtractorMediaSource.Factory(dataSourceFactory).createMediaSource(videoUri)
player.addListener(object: Player.EventListener {
override fun onPlayerStateChanged(playWhenReady: Boolean, playbackState: Int) {
if (playbackState == Player.STATE_READY) {
startCallback()
}
}
override fun onPlaybackParametersChanged(playbackParameters: PlaybackParameters?) {}
override fun onSeekProcessed() {}
override fun onTracksChanged(trackGroups: TrackGroupArray?, trackSelections: TrackSelectionArray?) {}
override fun onPlayerError(error: ExoPlaybackException?) {}
override fun onLoadingChanged(isLoading: Boolean) {}
override fun onPositionDiscontinuity(reason: Int) {}
override fun onRepeatModeChanged(repeatMode: Int) {}
override fun onShuffleModeEnabledChanged(shuffleModeEnabled: Boolean) {}
override fun onTimelineChanged(timeline: Timeline?, manifest: Any?, reason: Int) {}
})
player.prepare(videoSource)
player.playWhenReady = true
player.repeatMode = REPEAT_MODE_ALL
}
Hope this helps