I am learning about google actions and made a basic, google-keep like app which records notes for logged in users. it has a mobie client and a firebase server. i wish to know if assistant could be made to interact in such a manner that notes could be created via an assistant session without opening the mobile client at all.
For example:
User (on any screen): "hey google"
AI - "hi , how can i help?"
U - " place a note to call david in customNotter" ( suppose)
Ai - "am sorrry, customnotter does not exist on your mobile , download it from playstore http"//www.link..."
U - *downloads customNotter app , installs, logs himself in*
U - " place a note to call david in customNotter"
Ai - "Ok, your note is saved"
*saves note "call david" in customNotter (by somehow calling some code to interact with customNotter's server?)*
I can imagine a kotlin code that could be included in my app , which the assistant could trigger which could simply add the note to server, but cannot imagine how assistant could trigger that code with user's auth details and the note data?
PS: Also i wish to know if google home could also be integrated for such app. And if not , what are the key factors that would limit google home to not interact with my server ? like suppose am building a complete business model for CustomNotter , where there is a unified server and many clients like ios / android / web apps . can google home be able to act like another authenticated client, interacting directly with the server?
The best option is to use Actions on Google to directly make calls to your Firebase server and update the database that way. The hooks into Android apps are not deep at the moment. Android Actions were announced, but are not available right now.
The benefit of doing this is that your Action will work on any Assistant surface, beyond an Android phone, and will not require the user to install your app.
Since you can run Actions on Google on Firebase Functions, it is easy to incorporate into your Firebase database.
You're making a couple of assumptions here, but before I make different assumptions, I want to clarify something. You said "it has a mobile client and a firebase server". I assume the "firebase server" is what stores the user's notes?
If so, then you don't really need to have the app installed locally at all for the Assistant to work with it. You just need to make sure your mobile app and your Action both have the same user account reference, so they can update and read from the same records in the database. This would work anywhere the Assistant can run Actions - on a mobile device or on a speaker such as Google Home.
In this case, your conversation might look like this:
User: Hey Google, ask Custom Notter to remind me to call David.
Assistant: Getting Custom Notter
[Earcon, indicating your Action is running]
Action: Hi there. I've added a note to call David.
[Earcon, indicating the Action is done]
This is a very simple example of the conversation. Other conversation flows may also want to indicate if the user hasn't connected such an app to the account, or you may need to prompt them for permission to use their account, etc.
You will need to write server code to handle the processing - all Actions interact with a webhook running on a server in the cloud. You can write this in any language you wish, including Kotlin, and on any service you wish - however Google provides libraries and other support if you run it in node.js using Firebase Cloud Functions or on Google's Cloud.
All the details are available at https://developers.google.com/actions/
If you do have a mobile app, and you want to look into how to get the Assistant to launch things in that app, you can look into App Actions, which are coming soon for Android.
Related
I've a made a really basic chat application(no need to see the code)(from the course of Angela Yu, flutter)it is integrated with google firebase which uses firebase_core, firebase_auth & cloud_firestore packages for authentication and uploading chat data in cloud Firestore.
I've installed the app through debug.APK( 1. would it matter if it was release.APK?) in my & my friend's phone both android ( let's take a neutral approach i.e the phone could be either IOS or Android).
What I aim is if I change my code( or we say update my app) I would want my app on both the phones to get updated( either automatically if possible or send a alert when user opens the app to update it), all this should be done with free of cost. 2. How can I achieve this in simplest manner?. I know there is a standard way for this if the app was in Google play store or App store but I don't want to upload it on stores.
My guess for this is that we integrate our app with a server so that every time the app starts it checks for a update with the server and ask the user to update and gets updated. If this is achievable can someone please guide me with these two methods mentioned below:
3(A). through free online server( can we use github or firebase?)
3(B). and can I make my own laptop as the server and code the app such that it doesn't need the server to be always online.( as in if my server is online it checks for update else it continues)
3(C). through paid servers ( just for information)
(4) Also please tell me why we chose cloud firestore over real time database for a chatting application?
(I don't want to create a web app)
I've no idea how would any of this be achievable like nothing like what server would be used or is this even possible. So please help me with a detailed explanation on all( 1,2,3(A),3(B),3(c) ). Any help would be really appreciated!
Since you already use Cloud Firestore, consider creating a document in there with a field that contains the version number of the latest app release.
In your application code you then read this document upon startup, and check it against the app's own version. If the version from the database is newer, tell the user to upgrade.
You could even include instructions on where/how to get the latest version in the database too. That can come in really handy of you suddenly have to change your release mechanism in between versions.
I am trying to control home appliances through my app. And I have a web server to take requests and make mqtt publishes with this parameter. And I want to make this requests with google assistant. But some parameters should come from my app and some of them come according to user says. For example:
"Turn on kitchen Light"
What google assistant should do:
http:/myserverIp/incoming/param=kitchenLight¶m=On¶m=userId
What my server should do:
mqtt publish: topic:home/userId/kitchenLight payload=1
I've read google assitant sdk, I watched google io-18/17 assistant talks but I couldn't find right answer. According to picture below I should use "Url template model" but I couldn't find out even where I should locate actions.xml in my android project!
Is it possible to pass data to google assistant from my app and making GET/POST request which is user based?
According to picture below I should use "Url template model" but I couldn't find out even where I should locate actions.xml in my android project!
I think this is the source of your confusion. If you are building a Smart Home Action for the Google Assistant - you're not building an Android project at all. You are building a cloud- or server-based webhook that will take JSON from the Assistant and handle it in some way that makes sense for your devices.
In your case, it sounds like one approach would be to have this webhook act as a sort of proxy to your existing service. So it would take the JSON from the Assistant and turn it into the URL query you are currently expecting.
I have a new project at hand where I need to make two android applications
1)Main Controller application (be installed in one device say its name be X)
2)Sub Child application (be installed in devices that need to be controlled by X )
So I want to get an idea/picture about how one android device like (2) can be managed fully(i.e. blocking applications or making them accessible for some interval / block calls / track location of user ) from another device like (1). So far I searched Google ,asked my seniors about the same but none was able to help me out.Found many scattered info (like need enterprise app to do this i.e use android management API or Device admin ) but not being able to get any clear idea about the work flow.
Thus now this the last place I can think of for helping me out with this work flow.
Reference
Update: Based on what #Fred said i.e. to use android management API (also based on the answers that he posted here) could we somehow have own custom web server and manage the interactions that EMM console does with all the child apps through our own API's(the same API's which we would call from the parent app.) ???
In other words instead of controlling form the EMM console, I would like to control those child application from another parent android application.
By FCM or Socket programming you can easily communicate with other app. Suppose 2nd app receive a call then 2nd app call a api of server about the information, then server knock 1st app by fcm or socket, then 1st app make a decision and submit the decision to server by API, And finally server submit the decision to 2nd app by FCM or socket.
You can check out the Flyve MDM which is an open source mobile device management solution including both agent app and web panel for management.
Is it possible to use text-based queries and responses in Assistant SDK in Android/iOS app?
Could responses be parsed as a text?
Is it possible to use certain Google account/user context while using Assistant SDK? Guess it could be useful for Assistant to know who exactly asks the questions. So it going to looks like a user asks Assistant, but actually the app does it.
Thanks.
1,2. Yes part of the API includes text input and text responses
3. Users must authenticate, so you will have a refresh token
3. You can register custom device actions in order trigger custom grammars that you've set.
I want to develop an app which will appear like a plugin when the user types something to a social network apps(comments or posts in facebook/tweets in twitter e.t.c). The app should appear as a handle so that the user will not be interrupted while using the social network. The app should be able to log the messages posted by the user and the user should be able to retrieve any logged texts from app so that he/she can reuse the same texts while posting.
Is it possible to implement such an app along with the social network app running? Kindly provide any links/ideas/design approaches....Waiting for your guidance..
Thanks,
Srooth
I don't think you'll be able to show a "handle" (unless you make a javascript plugin that runs on web browsers) but you can make the app I suppose. Here's one approach:
Create a web app that uses site-specific authentication and ask for permissions from all the social networks you plan to target.
Get permissions to read posts, comments, tweets etc using the site's API.
Periodically retrieve this stuff using the API from the sites and store them in your own server.
Write an android app that has a nice UI that retrieves this data for a particular user from your server (via your own REST API) and displays it in the mobile app.
This is a very high-level overview and the actual implementation is gonna be difficult and will take a while. But the basic approach should work. Good luck!