I have a fully functional custom android keyboard in which i have to add speech recognition. Here are the relevant parts of the implementation i have
public class CustomInputMethodService
extends InputMethodService
implements <random stuff> {
private SpeechRecognizer mSpeechRecognizer;
private RecognitionListener mSpeechlistener;
public void onCreate() {
super.onCreate();
mSpeechRecognizer = SpeechRecognizer.createSpeechRecognizer(this);
mSpeechlistener = new CustomRecognitionListener();
mSpeechRecognizer.setRecognitionListener(mSpeechlistener);
}
#Override
public void onPress(int primaryCode) {
if (primaryCode == KeyCodes.VOICE_INPUT) {
mSpeechRecognizer.startListening(getSpeechIntent());
}else if(..){
...
}
}
private Intent getSpeechIntent() {
Intent speechIntent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
speechIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
speechIntent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, this.getPackageName());
speechIntent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, false);
return speechIntent;
}
}
The relevant method of the CustomRecognitionListener is simply:
#Override
public void onResults(Bundle results) {
ArrayList<String> matches = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
Log.d(TAG, "onResults: ----> " + matches.get(0));
if(matches != null && matches.size() > 0) {
writeText(matches.get(0));
}
}
This code is working just fine. The twist here is that i want a similar behaviour to what happens on google keyboard when the uset taps the microphone key:
This would ideally by achieved by something like:
Intent voiceIntent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
voiceIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_WEB_SEARCH);
try {
startActivityForResult(voiceIntent, Constants.RESULT_SPEECH);
} catch (ActivityNotFoundException ex) {
DebugLog.e(TAG, "Not found excpetion onKeyDown: " + ex);
}
However, since the key listener is on and InputMethodService im not able to call startActivityForResult.
What is the ideal way to accomplish this? Should i simply start a new activity without a layout and have a callback to the inputMethodService? seems messy
Your screenshot shows "Google voice typing", which is an independent IME that is called by the Google Keyboard when its microphone button is pressed. So, your IME should do the same: replace itself with an IME that provides voice typing, and hope that there is a backlink to your IME once the voice typing is done.
The simplest implementation would be Switching among IME Subtypes, but you might want to have more control, e.g. start a specific IME with specific input parameters, etc. I'm not sure what is the best/standard way to achieve this extra control.
For an example of a voice typing IME you could look into (my app) Kõnele.
Simple implementation of the solution:
// on mic tap we call
public void startVoiceListening() {
InputMethodManager imeManager = (InputMethodManager) getApplicationContext().getSystemService(INPUT_METHOD_SERVICE);
String voiceExists = voiceExists(imeManager);
if (voiceExists != null) {
final IBinder token = getWindow().getWindow().getAttributes().token;
imeManager.setInputMethod(token,voiceExists);
}
}
private String voiceExists(InputMethodManager imeManager) {
List<InputMethodInfo> list = imeManager.getInputMethodList();
for (InputMethodInfo el : list) {
// do something to check whatever IME we want.
// in this case "com.google.android.googlequicksearchbox"
}
return null;
}
Once we no longer want to use the current IME just close it and it will fall back to the previous one
Related
I'd like to ask you for some help with Android TextToSpeech feature.
Basically, I'd like to develop a simple AI which speaks, asking a question then waits for an answer, and at last, based on answer asks another question and so on, until user pronounces a keyword which stops everything.
Now I know TextToSpeech has to be initialized before using speak method, and I'm trying to take this into account by using onActivityResult method.
Below some code:
Activity class:
public class MainActivity extends AppCompatActivity implements OnInitListener, Button.OnClickListener{
Button sayHello;
TextView textView;
private static final int CHECK_DATA = 0;
private static final Locale defaultLocale = Locale.UK; // British English
private static final String TAG = "TTS";
private TextToSpeech tts;
private boolean isInit = false;
sayIt Method: used to speak:
public void sayIt(String text, boolean flushQ){
if(isInit){
if(flushQ){
tts.speak(text, TextToSpeech.QUEUE_FLUSH, null, null);
} else {
tts.speak(text, TextToSpeech.QUEUE_ADD, null, null);
}
} else {
Log.i(TAG, "Failure: TTS instance not properly initialized");
}
}
TextToSpeech Listener:
#Override
public void onInit(int status){
if(status == TextToSpeech.SUCCESS){
isInit = true;
// Enable input text field and speak button now that we are initialized
sayHello.setEnabled(true);
// Set to a language locale after checking availability
Log.i(TAG, "available="+tts.isLanguageAvailable(Locale.UK));
tts.setLanguage(defaultLocale);
// Examples of voice controls. Set to defaults of 1.0.
tts.setPitch(1.0F);
tts.setSpeechRate(1.0F);
// Issue a greeting and instructions in the default language
tts.speak("Initialized!", TextToSpeech.QUEUE_FLUSH, null, Integer.toString(12));
} else {
isInit = false;
Log.i(TAG, "Failure: TTS instance not properly initialized");
}
}
Button Listener:
#Override
public void onClick(View v){
if(isInit)
sayIt("You clicked!", true);
}
onActivityResult Method:
// Create the TTS instance if TextToSpeech language data are installed on device. If not
// installed, attempt to install it on the device.
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
if (requestCode == CHECK_DATA) {
if (resultCode == TextToSpeech.Engine.CHECK_VOICE_DATA_PASS) {
// Success, so create the TTS instance. But can't use it to speak until
// the onInit(status) callback defined below runs, indicating initialization.
Log.i(TAG, "Success, let's talk");
tts = new TextToSpeech(this, this);
// Use static Locales method to list available locales on device
Locale[] locales = Locale.getAvailableLocales();
Log.i(TAG,"Locales Available on Device:");
for(int i=0; i<locales.length; i++){
String temp = "Locale "+i+": "+locales[i]+" Language="
+locales[i].getDisplayLanguage();
if(locales[i].getDisplayCountry() != "") {
temp += " Country="+locales[i].getDisplayCountry();
}
Log.i(TAG, temp);
}
} else {
// missing data, so install it on the device
Log.i(TAG, "Missing Data; Install it");
Intent installIntent = new Intent();
installIntent.setAction(TextToSpeech.Engine.ACTION_INSTALL_TTS_DATA);
startActivity(installIntent);
}
}
}
And, at last, onCreate Method:
#Override
public void onCreate(Bundle savedInstance){
super.onCreate(savedInstance);
setContentView(R.layout.activity_main);
sayHello = findViewById(R.id.sayBtn);
textView = findViewById(R.id.textView);
sayHello.setEnabled(false);
sayHello.setOnClickListener(this);
Intent checkIntent = new Intent();
checkIntent.setAction(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA);
startActivityForResult(checkIntent, CHECK_DATA);
/* THIS SPEAK DOES NOT WORK! */
sayIt("Speech from method!", true);
}
Issue is: Button successfully gets enabled when onInit method initialises TextToSpeech and successfully pronounces text.
My goal is to make the Activity speak from onCreate method, since at the moment it only works from onInit and onClick listeners, bot not in onCreate, even if I check for tts initialization using onActivityResult.
Basically I want the TextToSpeech to speak with no Buttons involved.
I know very similar questions were already posted, but none solved my problem. Have some idea?
Hope I've been clear, Thank you!
UPDATE: Log shows ERROR detected occurs in else branch of onInit method, where Log.i(TAG, "Failure: TTS instance not properly initialized"); line is.
SOLUTION:
The only thing to do here is to wait a little time in order to let TextToSpeech initialize for good.
A good way seems to be by using a delayed Handler as follows:
final Handler handler = new Handler();
handler.postDelayed(new Runnable() {
#Override
public void run() {
//Waiting for RobotTextToSpeech initialization for 1500ms
rtts.speak("This speak will work!");
rtts.speak("This other speak will work too!");
}
}, 1500);
}
By doing this, looks like TextToSpeech works well even in onCreate method, we just have to wait little time.
Hope this can help.
Is there a way to auto answer whatsapp video call using AccessibilityService in Android?
OR is there a way to stimulate a click on headset's/bluetooth's call answering button? How can i get the id of the answering button?? to perform a click with accessibility service
I know that starting from Android 8.0 Oreo we have ANSWER_PHONE_CALLS permission, but for my project i want to use an old device for remote monitoring.
Any help would be appreciated!
----- Update: Thanks to the answer of Mr. hemisphire and Mr. Kahbazi, the app is able to answer the call,but needs to be a system app to work! is there any way to make it work without being a system app? without the headset's button hack?
public class AnswerCall extends AccessibilityService {
#Override
public void onAccessibilityEvent( AccessibilityEvent event )
{
if(event.getEventType() == TYPE_WINDOW_CONTENT_CHANGED)
{
if(event.getPackageName().equals("com.whatsapp"))
{
Thread thread = new Thread() {
#Override
public void run() {
try {
while(true) {
Instrumentation inst = new Instrumentation();
inst.sendKeyDownUpSync(KeyEvent.KEYCODE_HEADSETHOOK);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
thread.start();
StringBuilder sb = new StringBuilder();
List<CharSequence> texts = event.getText();
if (!texts.isEmpty())
{
for (CharSequence s : event.getText()) {
sb.append(s);
}
if(sb.toString().equals("Incoming video call"))
Log.d( "onAccessibilityEvent", "whatsapp video call" );
}
}
}
}
#Override
public void onInterrupt() {
}
}
I don't think you can do what you want. Using the AccessibilityService you can know when the video call comes in:
#Override
public void onAccessibilityEvent( AccessibilityEvent event )
{
if(event.getEventType() == AccessibilityEvent.TYPE_NOTIFICATION_STATE_CHANGED)
{
if(event.getPackageName().equals("com.whatsapp"))
{
StringBuilder sb = new StringBuilder();
List<CharSequence> texts = event.getText();
if (!texts.isEmpty())
{
for (CharSequence s : event.getText())
{
sb.append(s);
}
if(sb.toString().equals("Incoming video call"))
{
Log.d( "onAccessibilityEvent", "whatsapp video call" );
}
}
}
}
}
However, I've never been able to answer the call programmatically. The question at How can incoming calls be answered programmatically in Android 5.0 (Lollipop)? does a great job of enumerating all possible options, but most require root and/or being a system app.
You can use sendKeyDownUpSync method from Instrumentation class.
Instrumentation inst = new Instrumentation();
inst.sendKeyDownUpSync(KeyEvent.KEYCODE_HEADSETHOOK);
if this code didn't work, try to use another KeyEvent to find the correct one.
You can see the list of KeyEvent from this link : https://developer.android.com/reference/android/view/KeyEvent.html
You can check more info in from here : Instrumentation
A classic way to achieve this is to observe notifications using the NotificationListenerService and act on the relevant action of the notification.
I'm having a problem getting the supported languages. I have seen a solution
that is to create a Broadcast receiver and fill the list with the languages.
public class LanguageChecker extends BroadcastReceiver
{
private List<String> supportedLanguages;
private String languagePreference;
#Override
public void onReceive(Context context, Intent intent)
{
//Log.d(Constants.Tag,"OnReceive");
Bundle results = getResultExtras(true);
if (results.containsKey(RecognizerIntent.EXTRA_LANGUAGE_PREFERENCE))
{
languagePreference =
results.getString(RecognizerIntent.EXTRA_LANGUAGE_PREFERENCE);
}
if (results.containsKey(RecognizerIntent.EXTRA_SUPPORTED_LANGUAGES))
{
supportedLanguages =
results.getStringArrayList(
RecognizerIntent.EXTRA_SUPPORTED_LANGUAGES);
}
}
public List<String> getSupportedLanguages() {
return supportedLanguages;
}
}
but the problem is that I need this supportedLanguages list to fill my spinner.
When I call the method getSupportedLanguages, I get null.
This is how I use the broadcast within onCreate:
try {
lang = new LanguageChecker();
Intent detailsIntent = new Intent(RecognizerIntent.ACTION_GET_LANGUAGE_DETAILS);
sendOrderedBroadcast(detailsIntent, null, lang, null, Activity.RESULT_OK, null, null);
supportedLanguagesForSpeech=lang.getSupportedLanguages();
Log.d(Constants.Tag, String.valueOf(supportedLanguagesForSpeech.size()));
}
catch (Exception e){
Log.d(Constants.Tag,e.getMessage());
}
You should solve it with a callback to make sure that supportedLanguages is assigned. You are getting null because you are not waiting onReceive to be called.
Here is my current solution to querying all the available speech recognition services for their supported languages:
https://github.com/Kaljurand/K6nele/blob/3e514edc87e07babb0be57fa31ab48be7e2226e7/app/src/ee/ioc/phon/android/speak/RecognitionServiceManager.java
You can see it in action in the Kõnele app (http://kaljurand.github.io/K6nele/about/), in the "Settings -> Recognition languages & services" list.
I'm making an app that takes commands from User and write it in real time. What would be the Best option for me to take? Third Party software like sphinx or should I use the built in (android speech recognition)?
Secondly I want it to write in real time, like when I speak it starts writing?
You should use the built in Android Speech recognition. Specifically, you will need to operate the SpeechRecognier API so that there is no popup dialog box.
Also, do not expect SpeechRecognizer to return anything within onPartialResults(). It rarely does.
You could try to use Sphinx, but it seems other developers have difficulty getting it to run on Android. That said, sphinx will be your only option if you want your app to run without an internet connection.
Here is a snipped of code you will need to use SpeechRecognizer:
public void recognizeDirectly(Intent recognizerIntent)
{
// SpeechRecognizer requires EXTRA_CALLING_PACKAGE, so add if it's not
// here
if (!recognizerIntent.hasExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE))
{
recognizerIntent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,
"com.dummy");
}
SpeechRecognizer recognizer = getSpeechRecognizer();
recognizer.startListening(recognizerIntent);
}
#Override
public void onResults(Bundle results)
{
Log.d(TAG, "full results");
receiveResults(results);
}
#Override
public void onPartialResults(Bundle partialResults)
{
Log.d(TAG, "partial results");
receiveResults(partialResults);
}
/**
* common method to process any results bundle from {#link SpeechRecognizer}
*/
private void receiveResults(Bundle results)
{
if ((results != null)
&& results.containsKey(SpeechRecognizer.RESULTS_RECOGNITION))
{
List<String> heard =
results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
float[] scores =
results.getFloatArray(SpeechRecognizer.CONFIDENCE_SCORES);
receiveWhatWasHeard(heard, scores);
}
}
#Override
public void onError(int errorCode)
{
recognitionFailure(errorCode);
}
/**
* stop the speech recognizer
*/
#Override
protected void onPause()
{
if (getSpeechRecognizer() != null)
{
getSpeechRecognizer().stopListening();
getSpeechRecognizer().cancel();
getSpeechRecognizer().destroy();
}
super.onPause();
}
/**
* lazy initialize the speech recognizer
*/
private SpeechRecognizer getSpeechRecognizer()
{
if (recognizer == null)
{
recognizer = SpeechRecognizer.createSpeechRecognizer(this);
recognizer.setRecognitionListener(this);
}
return recognizer;
}
// other unused methods from RecognitionListener...
#Override
public void onReadyForSpeech(Bundle params)
{
Log.d(TAG, "ready for speech " + params);
}
#Override
public void onEndOfSpeech()
{
}
gregm is right but the main "write in real time" part of the question wasn't answered. You need to add an extra to indicate that you are interested in getting parts of the result back:
Adding the extra to the intent works for me
intent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true);
Warning: Partial does not return only new stuff but also the previous one. So you need to implement a check for the differences by yourself...
How can you read data, i.e. convert simple text strings to voice (speech) in Android?
Is there an API where I can do something like this:
TextToVoice speaker = new TextToVoice();
speaker.Speak("Hello World");
Using the TTS is a little bit more complicated than you expect, but it's easy to write a wrapper that gives you the API you desire.
There are a number of issues you must overcome to get it work nicely.
They are:
Always set the UtteranceId (or else
OnUtteranceCompleted will not be
called)
setting OnUtteranceCompleted
listener (only after the speech
system is properly initialized)
public class TextSpeakerDemo implements OnInitListener
{
private TextToSpeech tts;
private Activity activity;
private static HashMap DUMMY_PARAMS = new HashMap();
static
{
DUMMY_PARAMS.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, "theUtId");
}
private ReentrantLock waitForInitLock = new ReentrantLock();
public TextSpeakerDemo(Activity parentActivity)
{
activity = parentActivity;
tts = new TextToSpeech(activity, this);
//don't do speak until initing
waitForInitLock.lock();
}
public void onInit(int version)
{ //unlock it so that speech will happen
waitForInitLock.unlock();
}
public void say(WhatToSay say)
{
say(say.toString());
}
public void say(String say)
{
tts.speak(say, TextToSpeech.QUEUE_FLUSH, null);
}
public void say(String say, OnUtteranceCompletedListener whenTextDone)
{
if (waitForInitLock.isLocked())
{
try
{
waitForInitLock.tryLock(180, TimeUnit.SECONDS);
}
catch (InterruptedException e)
{
Log.e("speaker", "interruped");
}
//unlock it here so that it is never locked again
waitForInitLock.unlock();
}
int result = tts.setOnUtteranceCompletedListener(whenTextDone);
if (result == TextToSpeech.ERROR)
{
Log.e("speaker", "failed to add utterance listener");
}
//note: here pass in the dummy params so onUtteranceCompleted gets called
tts.speak(say, TextToSpeech.QUEUE_FLUSH, DUMMY_PARAMS);
}
/**
* make sure to call this at the end
*/
public void done()
{
tts.shutdown();
}
}
Here you go . A tutorial on using the library The big downside is that it requires an SD card to store the voices.
A good working example of tts usage can be found in the "Pro Android 2 book". Have a look at their source code for chapter 15.
There are third-party text-to-speech engines. Rumor has it that Donut contains a text-to-speech engine, suggesting it will be available in future versions of Android. Beyond that, though, there is nothing built into Android for text-to-speech.
Donut has this: see the android.speech.tts package.