I'm making an app that takes commands from User and write it in real time. What would be the Best option for me to take? Third Party software like sphinx or should I use the built in (android speech recognition)?
Secondly I want it to write in real time, like when I speak it starts writing?
You should use the built in Android Speech recognition. Specifically, you will need to operate the SpeechRecognier API so that there is no popup dialog box.
Also, do not expect SpeechRecognizer to return anything within onPartialResults(). It rarely does.
You could try to use Sphinx, but it seems other developers have difficulty getting it to run on Android. That said, sphinx will be your only option if you want your app to run without an internet connection.
Here is a snipped of code you will need to use SpeechRecognizer:
public void recognizeDirectly(Intent recognizerIntent)
{
// SpeechRecognizer requires EXTRA_CALLING_PACKAGE, so add if it's not
// here
if (!recognizerIntent.hasExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE))
{
recognizerIntent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,
"com.dummy");
}
SpeechRecognizer recognizer = getSpeechRecognizer();
recognizer.startListening(recognizerIntent);
}
#Override
public void onResults(Bundle results)
{
Log.d(TAG, "full results");
receiveResults(results);
}
#Override
public void onPartialResults(Bundle partialResults)
{
Log.d(TAG, "partial results");
receiveResults(partialResults);
}
/**
* common method to process any results bundle from {#link SpeechRecognizer}
*/
private void receiveResults(Bundle results)
{
if ((results != null)
&& results.containsKey(SpeechRecognizer.RESULTS_RECOGNITION))
{
List<String> heard =
results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
float[] scores =
results.getFloatArray(SpeechRecognizer.CONFIDENCE_SCORES);
receiveWhatWasHeard(heard, scores);
}
}
#Override
public void onError(int errorCode)
{
recognitionFailure(errorCode);
}
/**
* stop the speech recognizer
*/
#Override
protected void onPause()
{
if (getSpeechRecognizer() != null)
{
getSpeechRecognizer().stopListening();
getSpeechRecognizer().cancel();
getSpeechRecognizer().destroy();
}
super.onPause();
}
/**
* lazy initialize the speech recognizer
*/
private SpeechRecognizer getSpeechRecognizer()
{
if (recognizer == null)
{
recognizer = SpeechRecognizer.createSpeechRecognizer(this);
recognizer.setRecognitionListener(this);
}
return recognizer;
}
// other unused methods from RecognitionListener...
#Override
public void onReadyForSpeech(Bundle params)
{
Log.d(TAG, "ready for speech " + params);
}
#Override
public void onEndOfSpeech()
{
}
gregm is right but the main "write in real time" part of the question wasn't answered. You need to add an extra to indicate that you are interested in getting parts of the result back:
Adding the extra to the intent works for me
intent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true);
Warning: Partial does not return only new stuff but also the previous one. So you need to implement a check for the differences by yourself...
Related
I've seen some posts that seem similar, but they're all fairly old and slightly different. I have an app whose codebase I have not touched in a long time. It makes use of the android SpeechRecognizer Service, and has a toggle for the EXTRA_PREFER_OFFLINE intent param. Previously this was working with no issues, but since I've recently dusted it off I noticed that offline functionality immediately returns with error code 7 NO_MATCH.
I have confirmed that offline language packs are installed, and wrote some stand alone code to test SpeechRecognizer outside my larger code base.
I've not been able to find any documented solutions for the NO_MATCH error, but surely it must occur elsewhere.
For context: This was previously working last year/earlier this year (I've seen people claim this wasn't possible after 2015/17) and on Android 8
Sample code snippet:
speechRecognizer = SpeechRecognizer.createSpeechRecognizer(getContext());
final Intent speechRecognizerIntent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true);
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_PREFER_OFFLINE, true);
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, "en-US");
speechRecognizer.setRecognitionListener(new RecognitionListener() {
#Override
public void onReadyForSpeech(Bundle bundle) {
listening = true;
label.setText(R.string.listening_true);
}
#Override
public void onBeginningOfSpeech() {
}
#Override
public void onRmsChanged(float v) {
}
#Override
public void onBufferReceived(byte[] bytes) {
}
#Override
public void onEndOfSpeech() {
label.setText(R.string.listening_false);
listening = false;
}
#Override
public void onError(int i) {
Log.e(TAG, "Error code " + i);
label.setText(R.string.listening_false);
listening = false;
}
#Override
public void onResults(Bundle bundle) {
ArrayList<String> data = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
if (data != null && data.size() > 0) {
resultsTextView.setText(data.get(0));
resultsTextView.setTextColor(getResources().getColor(R.color.colorAccent));
listening = false;
}
}
#Override
public void onPartialResults(Bundle bundle) {
ArrayList<String> data = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
if (data != null && data.size() > 0) {
Log.i(TAG, "how many partial results? " + data.size());
resultsTextView.setText(data.get(0));
resultsTextView.setTextColor(getResources().getColor(R.color.design_default_color_primary));
Log.i(TAG, "Partial results: " + data);
}
}
#Override
public void onEvent(int i, Bundle bundle) {
}
});
view.findViewById(R.id.button_mic).setOnClickListener(view1 -> {
if (!listening) {
speechRecognizer.startListening(speechRecognizerIntent);
} else {
speechRecognizer.stopListening();
}
});
Google app's voice input could work offline previously, but unfortunately, this feature has been removed since version 11 (released in March 2020).
Thus, to use this feature, we have to keep the version of Google app on device at 10.99.8 or older.
However, on some new device released after March 2020, the pre-installed version of Google app is already 11 or 12, so it is not a simple task to downgrade it to an older version.
So we need to use adb command to downgrade it from PC. This command will try to force downgrade the app for the 'current user': adb install -r -d --user 0 google.apk
The answer is that this is no loner supported. If you need a short term fix then go to your "Google", not "Google Play Services", app and uninstall updates from the hamburger menu. This seems to downgrade a lot of libraries, including the voice recognition stuff.
A possibility might be to manually bring and load your own libraries, but I haven't tested this personally.
I have a fully functional custom android keyboard in which i have to add speech recognition. Here are the relevant parts of the implementation i have
public class CustomInputMethodService
extends InputMethodService
implements <random stuff> {
private SpeechRecognizer mSpeechRecognizer;
private RecognitionListener mSpeechlistener;
public void onCreate() {
super.onCreate();
mSpeechRecognizer = SpeechRecognizer.createSpeechRecognizer(this);
mSpeechlistener = new CustomRecognitionListener();
mSpeechRecognizer.setRecognitionListener(mSpeechlistener);
}
#Override
public void onPress(int primaryCode) {
if (primaryCode == KeyCodes.VOICE_INPUT) {
mSpeechRecognizer.startListening(getSpeechIntent());
}else if(..){
...
}
}
private Intent getSpeechIntent() {
Intent speechIntent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
speechIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
speechIntent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, this.getPackageName());
speechIntent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, false);
return speechIntent;
}
}
The relevant method of the CustomRecognitionListener is simply:
#Override
public void onResults(Bundle results) {
ArrayList<String> matches = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
Log.d(TAG, "onResults: ----> " + matches.get(0));
if(matches != null && matches.size() > 0) {
writeText(matches.get(0));
}
}
This code is working just fine. The twist here is that i want a similar behaviour to what happens on google keyboard when the uset taps the microphone key:
This would ideally by achieved by something like:
Intent voiceIntent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
voiceIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_WEB_SEARCH);
try {
startActivityForResult(voiceIntent, Constants.RESULT_SPEECH);
} catch (ActivityNotFoundException ex) {
DebugLog.e(TAG, "Not found excpetion onKeyDown: " + ex);
}
However, since the key listener is on and InputMethodService im not able to call startActivityForResult.
What is the ideal way to accomplish this? Should i simply start a new activity without a layout and have a callback to the inputMethodService? seems messy
Your screenshot shows "Google voice typing", which is an independent IME that is called by the Google Keyboard when its microphone button is pressed. So, your IME should do the same: replace itself with an IME that provides voice typing, and hope that there is a backlink to your IME once the voice typing is done.
The simplest implementation would be Switching among IME Subtypes, but you might want to have more control, e.g. start a specific IME with specific input parameters, etc. I'm not sure what is the best/standard way to achieve this extra control.
For an example of a voice typing IME you could look into (my app) Kõnele.
Simple implementation of the solution:
// on mic tap we call
public void startVoiceListening() {
InputMethodManager imeManager = (InputMethodManager) getApplicationContext().getSystemService(INPUT_METHOD_SERVICE);
String voiceExists = voiceExists(imeManager);
if (voiceExists != null) {
final IBinder token = getWindow().getWindow().getAttributes().token;
imeManager.setInputMethod(token,voiceExists);
}
}
private String voiceExists(InputMethodManager imeManager) {
List<InputMethodInfo> list = imeManager.getInputMethodList();
for (InputMethodInfo el : list) {
// do something to check whatever IME we want.
// in this case "com.google.android.googlequicksearchbox"
}
return null;
}
Once we no longer want to use the current IME just close it and it will fall back to the previous one
I am using Codenameone and ZXing to read a QRCode. When I call the Scanner, my mobile opens the QRCode reader application and I get to read the QRCode except that when android takes me back to my app it goes through init then start statuses. Which moves me back to the login form of my application instead of continuing filling the form that I was in.
Any help on what to do to stay in the same form? Is there something I'm doing wrong? Thanks in advance.
EverproX.addMessage("Before Scan\n");
CodeScanner.getInstance().scanQRCode(new ScanResult() {
public void scanCompleted(String contents, String formatName, byte[] rawBytes) {
EverproX.addMessage("Scan Completed "+contents);
}
public void scanCanceled() {
EverproX.addMessage("Scan Cancelled");
}
public void scanError(int errorCode, String message) {
EverproX.addMessage("Scan Error "+errorCode+" "+message);
}
});
EverproX can be seen as a log class.
By analyzing our log we can say that as soon as we call the CodeScanner.getInstance().scanQRCode() the application is called for 'Destroy'. Then after the scanning is done it goes again through the init and start. It never goes into the scanComplete scanCanceled or scanError Callbacks.
Is it normal that the App is destroyed upon call of CodeScanner? Many thanks.
Inside your codenameone project, you should find a class named (for example MyApp.java) based on your app's name, modify the code to read something like similar to this:
public class MyApp {
private Form current;
public void init(Object context) {
// Pro users - uncomment this code to get crash reports sent to you automatically
Display.getInstance().addEdtErrorHandler(new ActionListener() {
public void actionPerformed(ActionEvent evt) {
evt.consume();
Log.p("Exception in AppName version " + Display.getInstance().getProperty("AppVersion", "Unknown"));
Log.p("OS " + Display.getInstance().getPlatformName());
Log.p("Error " + evt.getSource());
Log.p("Current Form " + Display.getInstance().getCurrent().getName());
Log.e((Throwable) evt.getSource());
Log.sendLog();
}
});
}
public void start() {
if (current != null) {
current.show();
return;
}
new StateMachine("/theme");
}
public void stop() {
current = Display.getInstance().getCurrent();
}
public void destroy() {
current = null;
}
}
I am attempting to write a RemotePlaybackClient sample app, in part because the one published by Google crashes aapt.
I can get RemotePlaybackClient to support play(), and it plays back a video on a Chromecast.
However, when I call stop(), to stop playback of the video, while the Chromecast does stop playback (showing a black screen with a cast icon centered), the SessionActionCallback that I pass into the stop() call does not get called with onResult():
private void stop() {
logToTranscript(getActivity().getString(R.string.stop_requested));
SessionActionCallback stopCB=new SessionActionCallback() {
#Override
public void onResult(Bundle data, String sessionId,
MediaSessionStatus sessionStatus) {
logToTranscript(getActivity().getString(R.string.stopped));
isPlaying=false;
isPaused=false;
getActivity().supportInvalidateOptionsMenu();
endSession();
}
};
client.stop(null, stopCB);
}
The same thing happens if I try pause() -- the SessionActionCallback passed to pause() is not invoked.
The sample code published by Google shows that these callbacks should be invoked, but, again, I can't get that to compile successfully.
Does anyone know under what circumstances the SessionActionCallback would not work, while the ItemActionCallback used with play() would work?
UPDATE
I have filed issue 66996 and issue 67032, the latter of which is specifically the problem I am seeing here, as I run into this same problem with the official sample app.
I beleive all Answer reside on how you make connection.
Because in google code ,code says that client which you had made shold not leave session and should not be null.
if (!mClient.hasSession()) {
// ignore if no session
return;
}
/*********Rest of the code would be unreachable***********/
#Override
public void pause() {
if (!mClient.hasSession()) {
// ignore if no session
return;
}
if (DEBUG) {
Log.d(TAG, "pause");
}
mClient.pause(null, new SessionActionCallback() {
#Override
public void onResult(Bundle data, String sessionId, MediaSessionStatus sessionStatus) {
logStatus("pause: succeeded", sessionId, sessionStatus, null, null);
if (mCallback != null) {
mCallback.onPlaylistChanged();
}
}
#Override
public void onError(String error, int code, Bundle data) {
logError("pause: failed", error, code);
}
});
}
How can you read data, i.e. convert simple text strings to voice (speech) in Android?
Is there an API where I can do something like this:
TextToVoice speaker = new TextToVoice();
speaker.Speak("Hello World");
Using the TTS is a little bit more complicated than you expect, but it's easy to write a wrapper that gives you the API you desire.
There are a number of issues you must overcome to get it work nicely.
They are:
Always set the UtteranceId (or else
OnUtteranceCompleted will not be
called)
setting OnUtteranceCompleted
listener (only after the speech
system is properly initialized)
public class TextSpeakerDemo implements OnInitListener
{
private TextToSpeech tts;
private Activity activity;
private static HashMap DUMMY_PARAMS = new HashMap();
static
{
DUMMY_PARAMS.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, "theUtId");
}
private ReentrantLock waitForInitLock = new ReentrantLock();
public TextSpeakerDemo(Activity parentActivity)
{
activity = parentActivity;
tts = new TextToSpeech(activity, this);
//don't do speak until initing
waitForInitLock.lock();
}
public void onInit(int version)
{ //unlock it so that speech will happen
waitForInitLock.unlock();
}
public void say(WhatToSay say)
{
say(say.toString());
}
public void say(String say)
{
tts.speak(say, TextToSpeech.QUEUE_FLUSH, null);
}
public void say(String say, OnUtteranceCompletedListener whenTextDone)
{
if (waitForInitLock.isLocked())
{
try
{
waitForInitLock.tryLock(180, TimeUnit.SECONDS);
}
catch (InterruptedException e)
{
Log.e("speaker", "interruped");
}
//unlock it here so that it is never locked again
waitForInitLock.unlock();
}
int result = tts.setOnUtteranceCompletedListener(whenTextDone);
if (result == TextToSpeech.ERROR)
{
Log.e("speaker", "failed to add utterance listener");
}
//note: here pass in the dummy params so onUtteranceCompleted gets called
tts.speak(say, TextToSpeech.QUEUE_FLUSH, DUMMY_PARAMS);
}
/**
* make sure to call this at the end
*/
public void done()
{
tts.shutdown();
}
}
Here you go . A tutorial on using the library The big downside is that it requires an SD card to store the voices.
A good working example of tts usage can be found in the "Pro Android 2 book". Have a look at their source code for chapter 15.
There are third-party text-to-speech engines. Rumor has it that Donut contains a text-to-speech engine, suggesting it will be available in future versions of Android. Beyond that, though, there is nothing built into Android for text-to-speech.
Donut has this: see the android.speech.tts package.