I am researching ways to implement form filling via voice command given by user inside my application.I have searched two options but no one is seems useful and I am bit confused here.
First I tried with android voice to speech library integration.It gives me text but isn't smart enough to converse with as google assistance do.
Then second I tried to integrate google assistance with api.a. It provides the user conversation but it is like adding command to google assistance .It doesn't provide me voice to text data so that I can fill form and do further operation.
Please suggest me ways to implement.
You can use the SDK provided by Slang Labs which allows you to add a custom voice experience inside your app. You can create a "buddy" in their console and configure the kind-of intents/utterances you want to handle. Then integrate its SDK into your app, which takes care of all the voice-related functionality and you can register callbacks for the intents you have configured in the console to handle the app-specific actions.
(disclaimer: I am a co-founder of Slang Labs :-))
You wouldn't use Actions on Google through Dialogflow for your implementation but rather the Google Assistant SDK which is meant for devices.
However, in your case it may make sense to use Dialogflow's Android client. You would not need to pull all of the Google Assistant's capabilities and the voice interaction would be limited to your own application.
Related
I know this is broad question but I've spent counless hours searching for the right solution. Google provides at least 15 different types of Google Assistant connections all of them are different and I don't know which one to use for my project. The project is pretty simple - I would like to call dynamic url with parameters (webhook to my home server) from mobile Google Assistant.Example - I would say to my Android mobile phone
> Ok, Google set the TV volume to 50
and the assistant would call
GET "https://192.168.1.12/tv/volume/50
or
GET "https://192.168.1.12/?device=tv&action=volume&value=50
where 'tv', 'volume' and '50' are the dynamic parameters (not static) so I could also call
> Ok, Google set the TV channel to 132
I just want a link or a name of the Google dev console that I can use. I don't want to waste another several hours just to find out that another Google package is not suitable for my project. Have anyone done something similar?
PS
I know that I could achieve something similar with "Ok, Google let's talk to ...." but that is not my case.
also I CAN'T USE IFTTT
There's no one-click mechanism to do this directly in Assistant. The smart home platform allows you to configure a service that would capture commands such as Channel and Volume and let you handle those commands in the way you want, using a cloud backend and optionally the Local Home SDK.
This may be a bit more work than you want, and you may want to consider existing smart home platforms which may handle some of the backend work such as https://homeassistant.io which does have an Assistant integration already, though I'm less sure of whether it may work in your use case.
It's hard to answer your question without knowing your constraints. Are you a developer? Are you looking to make a commercial app? For non-commercial use you can use the Google Assistant Service to create custom commands that do whatever you want (including call your API). As a benefit, you don't have to say "hey Google" before your command.
Also, it would help to know why you can't use IFTTT, given that it seems to do exactly what you're asking for.
Can I build custom actions to leverage the Google Assistant SDK without needing to be online? I'd like to have the natural language parser to help me navigate my Android App, even when offline. That is, speak to it and say special words to navigate the screens and what not, using the voice instructions of the user.
This seems totally possible using the Assistant SDK on the internet, but I need the app to run on my android even outside of connectivity.
Google assistant uses google servers so I don't think it is possible, but there are some other options check out this link.
Android: Speech Recognition without using google server
I am new to speech recognition, android and i have a use case where i need to build an android app which takes commands(limited set of commands, less than 100) from users and executes some logic. I have googled a bit and found the following can be done
Use google cloud speech api
Use Android inbuilt speech to text capability (Is it different from google cloud speech api? If so how?). Also what are the pros and cons of using offline mode of android speech to text?
Use open source speech recognition libraries like Kaldi, CMU Sphinx(it looked like they need a lot of effort in collecting and training the data)
Can someone please suggest me which of the above might best suit my use case?
I have a limited set of commands and speed matters the most to me.
I am really confused and thus putting this question. Thanks in advance.
Use google cloud speech api
Very expensive since you have to pay for every request.
Use Android inbuilt speech to text capability (Is it different from google cloud speech api? If so how?). Also what are the pros and cons of using offline mode of android speech to text?
The inbuilt API is ok to use. It is different from cloud API and it is free. It does not work offline transparently for the user though. Bad side it is slow and you can not configure the vocabulary. So it will decode all words instead of some particular set of commands and often will confuse the required commands with other words in noise.
Use open source speech recognition libraries like Kaldi, CMU Sphinx(it looked like they need a lot of effort in collecting and training the data)
Proper development is always an effort.
My question is regarding custom voice commands for Android.
Is there a way to develop an app, not necessarily a UI app, that adds custom voice commands to any Android mobile?
For example, after installing AccuWeather on Android, every time I ask Google Now for the weather, it displays information fetched from the said app.
Did AccuWeather added new voice commands to my phone?
Can any app developed by anyone add custom voice commands to an
Android phone?
Can I write and install an app that, triggered by the voice command
"How is my portfolio?", fetches data from the stock market, returns it
and then be displayed in the same view, the Google Now view?
I Googled all of these concerns but found more questions than answers.
You can write an app that registers itself as a choice for the available system commands.
As of this writing, you can not register an app that registers a custom command. Google used to accept requests to register custom voice actions, but stopped. I don't know why they stopped, or if/when they'll allow customization again.
BTW there was a possibility with an XPOSED Module to add custom commands to google search ( and so you can use the voice recognition even with a smart watch connected ), but it stopped working since they updated the google search app.
if you are just doing this for personal use u can still use it with an old version of google search app and this xposed framework module and correct hooks ( they postet the correct hooks for some updated versions of google app later in the thread , just use the latest one...).
here you go:
https://forum.xda-developers.com/xposed/modules/mod-google-search-api-t2554173
i really hope google will bring out a better voice API than the actual is.
If I'm making an Android app (such as a messaging app), and I want the app to be opened up so that other developers can make extensions (i.e. connections to other messaging sites/protocols, additional features, custom UI), what options do I have available to me to do this?
The answer to this question makes it seem like if the other developer makes their own app, my app is able to pull data from it? Is this the direction I should be heading in?
I know such extensions are possible, because I've seen it with other apps, such as the file manager FX that has "add-ons" called "FX Plus" and "FX Root Access". So, basically, how can I also get add-ons/extensions for my app?
Options I am aware of:
Intent API can provide some functionality (You can give developers ability to send commands and data to your app). For example you can accept some kind of XML and build your UI according to it. It's slow process, but you can create some hacks to speed it up. documentation
Content provider you can give other developers ability to work with data of your app. Databases, photos, videos. You can share your's and you can accept new ones.
documentation
AIDL Service never used it myself but sounds promising. documentation