I am trying to get continuous speech recognition working using pocketsphinx. I tried following their tutorial (https://cmusphinx.github.io/wiki/tutorialandroid/), although to me it is somewhat vague and I could not get it to work. I am now trying a different approach: to start with pocket-sphinx-android-demo (https://github.com/cmusphinx/pocketsphinx-android-demo) then reduce its limitations (to be able to recognize for longer and more words). I have figured out where the output of the recognition goes, although the demo is still only able to recognize weather words (I removed the digit and phones demos). I have discovered that the activation phrase can have an infinite vocabulary, but I can't figure out what is limiting the vocabulary of the actual recognition. Here is the github link to my project if it may be helpful: https://github.com/Michaelszeng/pocket-sphinx_App_mk2/commits/master
Does anybody know what is limiting the demo recognizer's vocabulary or how I could remove that limitation?
Related
I would like to implement offline voice recognition in my app. But I want it for two purposes:
For a small set of commands (play, stop, previous, next and a couple of others);
For a list of a few hundred bird names.
To implement (1), it seems to me a bad idea (slower and resource consuming) to use the full voice recognition force of android. In my mind, it would be easier to tell my app to only interpret a few words. That is, to use my own dictionary, telling my app to "use only these 10 words".
To implement (2) is similar to (1), but with a few hundred instead of 10.
Does this makes sense, and if so is there an easy way to implement it? Is it worth it?
Thanks!
L.
You can implement your app using CMUSphinx on Android. CMUSphinx tutorial is here:
http://cmusphinx.sourceforge.net/wiki/tutorial
The language models to recognize limited set of words are described here
http://cmusphinx.sourceforge.net/wiki/tutoriallm
You can use keyword spotting mode to recognize few commands.
Pocketsphinx on Android is described here:
http://cmusphinx.sourceforge.net/wiki/tutorialandroid
The demonstration includes the way to switch recognition modes from 10 words to few hundred words as you intend.
I have an iOS and an Android app that do Speech to Text recognition using Nuance Mobile SDK. Both apps work perfectly when using a Nuance sandbox appID.
However, when if I upload a phrase file, the Nuance server always returns zero results, as I can verify in the "didFinishWithResults" methods on both android and ios.
This is the phrase file I upload as a custom vocabulary to Nuance:
<phrases>
<phrase>two on to</phrase>
<phrase>bet 1 3 for</phrase>
<phrase>...and some other phrases.</phrase>
</phrases>
My Nuance custom dictionary is set to:
Topic:WebSearch
Grammar recognition mode: YES (<== Apps work perfectly when set to NO)
Vocabulary Weight: 90
Nuance's documentation claims that:
It is important to note that custom vocabularies are different than the constrained speech
recognition grammars many developers are familiar with. Using constrained grammars results in
high accuracy of words that are in the grammar and low (or no) recognition of words or phrases
that are not in the grammar. With custom vocabularies our large language models are still used
and the vocabulary simply adjusts the recognition probabilities of that large model. As a result,
using a vocabulary will not change your results as much as a conventional grammar would. For
example, you can expect to still get word results that are not in the vocabulary or “out of
grammar.” The grammar recognition mode feature makes the vocabulary act much more like a
traditional recognition grammar but as we are still using the underlying language model even
then you may get “out of grammar” results.
.....So my question is, What am I doing wrong to always get zero results from Nuance ASR when the custom vocabulary's grammar recognition mode is set to YES?
Nuance's customer support is totally useless, they probably outsource a bunch of people with just an FAQ in hand that have no idea about anything.
I hope somebody can help me out on this one.
I am building a speech recognition android app that will act as a virtual personal assistant with tasks such as:
Make appointments/Reminders
Weather Info
General queries to Wolfram|Alpha / Wikipedia - (i.e Who directed Ghostbusters, whats the £-$ Exchange rate)
My question is wheather to use Pocketsphinx or the Google API?
Originally I set this up with "android.speech.RecognitionListener", worked great, however I want to implement Keyword spotting so the user doesn't need to have any interaction other than just speaking.
Apparently Google API doesn't support this, so I looked into using pocketsphinx for this, and still using google for the rest of the app (As I heard pocketsphinx is not as accurate?)
However the two don't get along as they can't both occupy the microphone at the same time.
Is there a nice way to switch between recognizers? (cant even import both to same project)
Should I just go with pocketshinx and deal with the lower accuracy?
Suggestions would be helpful
Cheers
For anybody who wants to implement a similar project, I have found a work around. It's abit hacky and not entirely clean, but it works.
Using the android speech recognizer with a toggle on/off switch like in many examples across the web, when onResults comes back, the string will be checked for said "hotword", if it is not present, discard the string, if it is, process it. Once the query has been processed and the text to speech is responding, programatically reclick the toggle button, ensuring constant listening.
Do the same on "onError" as well.
I did also have it onPartialResults as well, but it appeared to make the thread crash, not entirely sure why but once it was removed everything seems to work nicely.
You can use pocketsphinx only to recognize predefined set of commands due to really poor accuracy (you should prepare your own dictionary and language model). Also pocketsphinx can be used offline and it is a big cons for some project.
In other hand google is very accurate but it's not free and works only online.
Doing some research I have found some different speech to text API's for Android.
Pocket Sphinx
Android Native API
I have the following requirements:
Must be able to support offline speech recognition (I'm not sure
if the Android API can do this)
Must be able to detect and
respond immediately to every word said. I would rather this than
detecting an entire sentence. I could split the returned sentence
into and array though and get each word.
The detection needs to
be processing in the backgound (no popups or anything as the Android
API seems to do)
Can someone recommend an API that is capable of my requirements.
Pocketsphinx meets all your requirements. What you call the "Android Native API" is basically a set of interface definitions and it does not contain the notion of offline/online.
You can also implement these interfaces using Pocketsphinx, since it supports things like partial results, confidence scores, n-best results etc. This way the implementation becomes available to any Android app. Maybe somebody has done it already, but I'm not aware of it.
i have used the code provided in this link for the speech recognition. in emulator it is saying recognizer not present,so i installed it on mobile. when i click on speak button it is working. but when i speak some names "rajesh" it is showing some possible verbs and all but not the name. but i want to use the input to select a contact from the address book in order to make a call . so please tell me how to carry on in this direction. one more thing, every time i need to develop the code in eclipse then install it on mobile and then check for output. is there any alternative to edit and check the app code in the mobile from eclipse.
please provide me any possible links. i want to develop a call app for blind,if the voice recognition does not work, what else could be done to take input from the user.
Names are hard for Speech recognition. There are more possible names in the world than words in any dictionary, so being able to recognise any arbitrary name is hard. Though common names are easier.
Anyway, if you want to recognise a customized list of words/names, You might want to look at Dragon Mobile from Nuance. Here is a copy-and-paste from another similar question I answered:
If you use 3rd party Android recognition from Nuance (The people behind DragonDictate), it supports a "grammar mode" where you can somewhat restrict the phrases that will be recognised during recognition.
Importantly, if you add unusual names into a Custom Vocabulary, they SHOULD become recognizable (Complex pronunciation issues aside).
You can find information if you dig through:
http://dragonmobile.nuancemobiledeveloper.com ,
looking for 'Custom Vocabularies'. Grammar mode is essentially a special mode of custom vocabularies.
At the time of writing, there was a document here that makes some mention of grammar mode:
http://dragonmobile.nuancemobiledeveloper.com/downloads/custom_vocabulary/Guide_to_Custom_Vocabularies_v1.5.pdf - It only really becomes clear when you try to progress in their provisioning web GUI.
You have to set up an account, and jump through other hoops, but there is a free tier. This is the only potential way I have found to constrain a recognition vocabulary.
Well, short of running up PocketSphinx, but that is still described as a 'Research' 'PreAlpha'.
No, I don't work for Nuance. Not sure anyone does. They may have all been eaten by zombies. You would guess as much reading their support forums. They never reply.