Is it possible to use text-based queries and responses in Assistant SDK in Android/iOS app?
Could responses be parsed as a text?
Is it possible to use certain Google account/user context while using Assistant SDK? Guess it could be useful for Assistant to know who exactly asks the questions. So it going to looks like a user asks Assistant, but actually the app does it.
Thanks.
1,2. Yes part of the API includes text input and text responses
3. Users must authenticate, so you will have a refresh token
3. You can register custom device actions in order trigger custom grammars that you've set.
Related
We are developing Native Mobile Application : Android platform, Ios Platform (Swift).
Instead of asking user to download and install https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2&hl=en_IN mobile app and then generate a QR code in our application to be scanned by this app, we are willing to do it api way. Means We want to call Google Authenticator API pass it QR image, secret and user is registered
Is this Ok and possible. Any one using it please suggest.
I'm not going to say that this is impossible, but integrating Google Authenticator would be a nightmare, and likely wouldn't do what you're expecting it to do. Allow me to explain.
Problem #1
For each instance of the app, you'll have to have a Unique Identifier to pair it to Google's Authenticator. In other words, you'll need to generate a new QR code for each device, pass that off to the system (Which doesn't exist without the app.) to pair the application. This is going to require a "Log-In" to properly store.
Check out this for a possible work-around to Problem #1: https://authenticatorapi.com/
Problem #2
If you're already logging in and the device is already being authenticated, what purpose does the Google Authenticator provide? Well, I'm assuming it allows you to prevent unauthorized access, possibly prevent more than one device. There are infinitely easier ways to manage this, location services are the first one that comes to mind for me.
Possible Solution (iOS)
This is absolutely my go-to method for handling authentication. Encourage or require your users to use 2FA (Two Factor Authentication) attached to SMS messaging. A simple line of code such as this will grab an SMS one-time code.
// Available as of iOS 12, make sure to check version.
self.verifyCodeTextField.textContentType = .oneTimeCode
Android SMS Retrieval
Android Auto-SMS-Retrieval https://developers.google.com/identity/sms-retriever/overview
I don't know it all, most certainly, but I hope this opens the floor for some discussion and brings forth a solution or explanation to your question.
Research Material
Google Authenticator available as a public service?
java API for google Authenticator
I am trying to control home appliances through my app. And I have a web server to take requests and make mqtt publishes with this parameter. And I want to make this requests with google assistant. But some parameters should come from my app and some of them come according to user says. For example:
"Turn on kitchen Light"
What google assistant should do:
http:/myserverIp/incoming/param=kitchenLight¶m=On¶m=userId
What my server should do:
mqtt publish: topic:home/userId/kitchenLight payload=1
I've read google assitant sdk, I watched google io-18/17 assistant talks but I couldn't find right answer. According to picture below I should use "Url template model" but I couldn't find out even where I should locate actions.xml in my android project!
Is it possible to pass data to google assistant from my app and making GET/POST request which is user based?
According to picture below I should use "Url template model" but I couldn't find out even where I should locate actions.xml in my android project!
I think this is the source of your confusion. If you are building a Smart Home Action for the Google Assistant - you're not building an Android project at all. You are building a cloud- or server-based webhook that will take JSON from the Assistant and handle it in some way that makes sense for your devices.
In your case, it sounds like one approach would be to have this webhook act as a sort of proxy to your existing service. So it would take the JSON from the Assistant and turn it into the URL query you are currently expecting.
I am learning about google actions and made a basic, google-keep like app which records notes for logged in users. it has a mobie client and a firebase server. i wish to know if assistant could be made to interact in such a manner that notes could be created via an assistant session without opening the mobile client at all.
For example:
User (on any screen): "hey google"
AI - "hi , how can i help?"
U - " place a note to call david in customNotter" ( suppose)
Ai - "am sorrry, customnotter does not exist on your mobile , download it from playstore http"//www.link..."
U - *downloads customNotter app , installs, logs himself in*
U - " place a note to call david in customNotter"
Ai - "Ok, your note is saved"
*saves note "call david" in customNotter (by somehow calling some code to interact with customNotter's server?)*
I can imagine a kotlin code that could be included in my app , which the assistant could trigger which could simply add the note to server, but cannot imagine how assistant could trigger that code with user's auth details and the note data?
PS: Also i wish to know if google home could also be integrated for such app. And if not , what are the key factors that would limit google home to not interact with my server ? like suppose am building a complete business model for CustomNotter , where there is a unified server and many clients like ios / android / web apps . can google home be able to act like another authenticated client, interacting directly with the server?
The best option is to use Actions on Google to directly make calls to your Firebase server and update the database that way. The hooks into Android apps are not deep at the moment. Android Actions were announced, but are not available right now.
The benefit of doing this is that your Action will work on any Assistant surface, beyond an Android phone, and will not require the user to install your app.
Since you can run Actions on Google on Firebase Functions, it is easy to incorporate into your Firebase database.
You're making a couple of assumptions here, but before I make different assumptions, I want to clarify something. You said "it has a mobile client and a firebase server". I assume the "firebase server" is what stores the user's notes?
If so, then you don't really need to have the app installed locally at all for the Assistant to work with it. You just need to make sure your mobile app and your Action both have the same user account reference, so they can update and read from the same records in the database. This would work anywhere the Assistant can run Actions - on a mobile device or on a speaker such as Google Home.
In this case, your conversation might look like this:
User: Hey Google, ask Custom Notter to remind me to call David.
Assistant: Getting Custom Notter
[Earcon, indicating your Action is running]
Action: Hi there. I've added a note to call David.
[Earcon, indicating the Action is done]
This is a very simple example of the conversation. Other conversation flows may also want to indicate if the user hasn't connected such an app to the account, or you may need to prompt them for permission to use their account, etc.
You will need to write server code to handle the processing - all Actions interact with a webhook running on a server in the cloud. You can write this in any language you wish, including Kotlin, and on any service you wish - however Google provides libraries and other support if you run it in node.js using Firebase Cloud Functions or on Google's Cloud.
All the details are available at https://developers.google.com/actions/
If you do have a mobile app, and you want to look into how to get the Assistant to launch things in that app, you can look into App Actions, which are coming soon for Android.
without getting into too many details, I would like to build an app capable of messaging between small groups of people. Basically, I need an API that will allow me to a) create a group message, and b) will allow me to later add people to it based on their selections from the application itself. I am relatively new to server side coding and am looking for some suggestions, tutorials, suggested apis (google hangouts vs facebook vs any other services that would be of use), etc. I already have google plus integration with the client side and server side authentication, so it seems like if something is possible with hangouts that would be the easiest route, but again I am just looking for anything really at this point.
The way I was kind of (hoping) it would work is that some messaging service provides the group chat and gives me an ID to it, which I could then have the clients request to be added to a particular chat and then I pass them back the ID to the chat and it begins an intent that takes them to the app itself that is hosting the chat between my users. Let me know if this is possible please, along with any suggestions!
You might want to try researching the ejabberd messaging server. It is probably going to be a bit heavy for your needs as well as difficult to learn (it is in Erlang) but I've found it extremely steady as a back-end for building a messaging server.
I've developed a couple other apps that did nothing more than preference management, some custom lists, and LAN communications but that's about it. I'm building a employee attendance app that allows the employee to register when they clock in, lunch out, lunch in, and clock out, all via the app. Once all four have been entered, I want it to send it via a Google Form to a Google spreadsheet that I can then use to calculate there hours and have a long on all of their information.
How would I go about integrating submitting a Google Form Response programmatically in Android?
Thanks in advance, -John
A Google Form is just a fancy UI for having users submit responses which get stored as a Google spreadsheet. That's it - there's nothing special about it. To accomplish this on Android you just need to setup your UI with the appropriate widgets to capture responses. Then save that data to a Google spreadsheet.
To work with the Google Spreadsheet API you can check out this guide from Google.