I want to create an android app which will "scan" other applications'layouts and find out if there are buttons etc.
Some apps, like whatsapp, doesn't allow third parts to move into the app, so I though that using accessibility could be the solution.
Now the problem is that I've never used Accessibility, so can someone of you can please show me how to "scan" an app's layout to find buttons?
Thanks a lot
What you're looking for is an Accessibility Service. Configuring an accessibility service is somewhat complicated. I have set up a repository of accessibility boilerplate code that sets up an AccessibilityService that logs the node heirarchy to LogCat, and has the default project settings activity set up as it's settings activity. Here are some of the highlights, as I dislike just posting github repos as answers. Note that I use a lot of my own libraries. CLog is a logging library and AndroidAccessibilityUtils wraps node infos with some common utility functions. You can find references to the dependencies in the build.gradle file on the github repo at the bottom. Here are some code highlights.
Your manifest.xml file is going to be significantly different from an Activity, and should contain an entry like this in your Application element:
<application .... >
...
<service
android:name="com.moba11y.basicaccessibilityservice.BasicAccessibilityService"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE" >
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/service_config" />
</service>
...
</application>
First you need to create a subclass of Accessibility Service:
public class BasicAccessibilityService extends AccessibilityService {
static {
CLog.initialize(BasicAccessibilityService.class.getSimpleName(), BuildConfig.DEBUG);
}
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {
CLog.d(event.toString());
switch (event.getEventType()) {
//On Gesture events print out the entire view heirarchy!
case AccessibilityEvent.TYPE_GESTURE_DETECTION_START:
CLog.d(A11yNodeInfo.wrap(getRootInActiveWindow()).toViewHeirarchy());
default: {
//If the event has a source, let's print it out separately.
if (event.getSource() != null) {
CLog.d(A11yNodeInfo.wrap(event.getSource()).toViewHeirarchy());
}
}
}
}
#Override
public void onInterrupt() {
CLog.e("Service Interrupted: Have never actually had this happen.");
}
}
That is the bulk of the highlights. You also should have a "service_config" XML with properties, as is referenced in the changes to the Manifest XML file. More details, and a reasonable starer point can be found in the open source repo on GitHub.
https://github.com/chriscm2006/Android-Accessibility-Service-Boilerplate
Related
Is it possible to NOT show the popup notification of OneSignal and instead create one programmatically?
Ok so after some research i found it out and is pretty straightforward.
here is the link to the offical doc on how to do it.
OneSignal Android Customizations
There are few step to achieve it:
1- Create a NotificationExtenderServiceClass
public class NotificationExtenderBareBonesExample extends NotificationExtenderService {
#Override
protected boolean onNotificationProcessing(OSNotificationReceivedResult receivedResult) {
// Read properties from result.
// Return true to stop the notification from displaying.
return true;
}
}
As you can see it's pretty clear, turning read to true will stop showing notifications.
But what the docs doesn't says is that FCM will stop working and instead the notifications will pass here.
2 - add the service to Manifest
<service
android:name=".NotificationExtenderBareBonesExample"
android:permission="android.permission.BIND_JOB_SERVICE"
android:exported="false">
<intent-filter>
<action android:name="com.onesignal.NotificationExtender" />
</intent-filter>
</service>
That's it.
Now, adding your logic inside NotificationExtender you will be able to freely manage your notifications.
So, I followed the official guide here https://developer.android.com/training/auto/start/index.html to create a very basic Android Auto Audio App. For the moment it does nothing, other then declaring what needs to be declared in the manifest and implementing empty onGetRoot() and onLoadChildren().
Problem is, that it is not being recognized by the Android Auto app.
Any idea where to get a working example? What could be wrong?
Manifest:
<service
android:name=".MyService"
android:enabled="true"
android:exported="true">
<intent-filter>
<action android:name=
"android.media.browse.MediaBrowserService"/>
</intent-filter>
</service>
automotive_app_desc.xml:
<automotiveApp>
<uses name="media" />
</automotiveApp>
service:
public class MyService extends MediaBrowserServiceCompat {
public static final String MEDIA_ID_ROOT = "__ROOT__";
#Override
public BrowserRoot onGetRoot(String clientPackageName, int clientUid,
Bundle rootHints) {
//TODO: check if the client is allow access
return new BrowserRoot(MEDIA_ID_ROOT, null);
}
#Override
public void onLoadChildren(final String parentMediaId,
final Result<List<MediaBrowserCompat.MediaItem>> result) {
// Assume for example that the music catalog is already loaded/cached.
List<MediaBrowserCompat.MediaItem> mediaItems = new ArrayList<>();
// Check if this is the root menu:
if (MEDIA_ID_ROOT.equals(parentMediaId)) {
// build the MediaItem objects for the top level,
// and put them in the mediaItems list
} else {
// examine the passed parentMediaId to see which submenu we're at,
// and put the children of that menu in the mediaItems list
}
result.sendResult(mediaItems);
}
You have to go to Android Auto settings, tap many times on the Version entry (the last one) to unlock Developer settings. Then tap on Developer settings menu item and enable Unknown sources. Restart Android Auto and if your app it's ok it will be listed. Worked for me
I didn't see this included in your snippet from the manifest, but double check that this line is also there.
<application>
...
<meta-data android:name="com.google.android.gms.car.application"
android:resource="#xml/automotive_app_desc"/>
...
</application>
I created a sample app matching everything you have (plus the line above), and it appears in Android Auto on the mobile device, as well as the Desktop Head Unit.
How can I program the Bixby button (KeyCode: 1082) on the Samsung Galaxy S8, that it will start my application instead of the Bixby launcher? The App All in one Gestures has already this function with custom keys, but how can I do this in Android programatically?
It seems Samsung doesn't approve of users doing this, and has apparently disabled this functionality in recent updates, at least in some places. Reports vary, but be warned that the example below might not work on every device, or at all in the near future. More details are available in the following article (off-site link to XDA Developers):
Samsung has Removed the Ability to Remap the Bixby Button on the Galaxy S8/S8+
All in one Gestures uses an AccessibilityService to accomplish this. Your app can do the same, but the user would have to explicitly enable your app as an Accessibility Service in the device Settings for it to work.
The Bixby button apparently emits simple KeyEvents with a keycode of 1082. Your AccessibilityService just needs to override the onKeyEvent() method, and check the keycode of the event passed in. For example:
public class BixbyInterceptService extends AccessibilityService {
private static final int KEYCODE_BIXBY = 1082;
#Override
protected boolean onKeyEvent(KeyEvent event) {
if (event.getKeyCode() == KEYCODE_BIXBY &&
event.getAction() == KeyEvent.ACTION_DOWN) {
// Do your thing here; startActivity(), Toast, Notification, etc.
Toast.makeText(this, "Bixby button pressed", 0).show();
// Return true to stop the event from propagating further.
return true;
}
return super.onKeyEvent(event);
}
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {}
#Override
public void onInterrupt() {}
}
You'll need to properly register your AccessibilityService in the manifest for it to be eligible to be enabled as such by the user. For example, between the <application> tags:
<service
android:name=".BixbyInterceptService"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/bixby_service_config" />
</service>
The resource attribute on the <meta-data> element above points to an XML file with the necessary settings for the Service. Under your project's res/ folder, create an xml/ folder if needed, and add this file there:
bixby_service_config.xml
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityFlags="flagRequestFilterKeyEvents"
android:canRequestFilterKeyEvents="true"
/>
After installation, you will need to enable your app in the Services under the Accessibility section in the device Settings.
I have been checking out and reading about Google Now on Tap (from http://developer.android.com/training/articles/assistant.html).
It was very interesting to find from that article that Now on Tap is based on Google's Assist API bundled with Marshmallow and it seems possible for us to develop our own assistant (the term Google used in the article to refer to app like Now on Tap) using the API.
However, the mentioned article only very briefly discusses how to use Assist API and I couldn't find any additional information about how to use it to develop a custom assistant even after spending a few days searching for it on the Internet. No documentation and no example.
I was wondering if any of you have experience with Assist API that you could share? Any help appreciated.
Thanks
You can definitely implement a personal assistant just like the Google Now on Tap using the Assist API starting Android 6.0. The official developer (http://developer.android.com/training/articles/assistant.html) guide tells exactly how you should implement it.
Some developers may wish to implement their own assistant. As shown in Figure 2, the active assistant app can be selected by the Android user. The assistant app must provide an implementation of VoiceInteractionSessionService and VoiceInteractionSession as shown in this example and it requires the BIND_VOICE_INTERACTION permission. It can then receive the text and view hierarchy represented as an instance of the AssistStructure in onHandleAssist(). The assistant receives the screenshot through onHandleScreenshot().
Commonsware has four demos for basic Assist API usage. The TapOffNow (https://github.com/commonsguy/cw-omnibus/tree/master/Assist/TapOffNow) should be enough to get you started.
You don't have to use the onHandleScreenshot() to get the relevant textual data, the AssistStructure in onHandleAssist() will give you a root ViewNode which usually contains all you can see on the screen.
You probably need to also implement some sorts of function to quickly locate the specific ViewNode that you want to focus on using recursive search on the children from this root ViewNode.
There is a complete example here but it's too complicated to start.
This is my example works on android 7.1.1
AndroidManifest.xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.eaydin79.voiceinteraction">
<application
android:allowBackup="true"
android:icon="#mipmap/ic_launcher"
android:label="#string/app_name"
android:roundIcon="#mipmap/ic_launcher_round"
android:theme="#style/AppTheme" >
<service
android:name="voiceInteractionService"
android:permission="android.permission.BIND_VOICE_INTERACTION" >
<meta-data
android:name="android.voice_interaction"
android:resource="#xml/interaction_service" />
<intent-filter>
<action android:name="android.service.voice.VoiceInteractionService" />
</intent-filter>
</service>
<service
android:name="voiceInteractionSessionService"
android:permission="android.permission.BIND_VOICE_INTERACTION" >
</service>
</application>
</manifest>
this is interaction_service.xml file stored in res\xml folder
<?xml version="1.0" encoding="utf-8"?>
<voice-interaction-service xmlns:android="http://schemas.android.com/apk/res/android"
android:sessionService="com.eaydin79.voiceinteraction.voiceInteractionSessionService"
android:recognitionService="com.eaydin79.voiceinteraction.voiceInteractionService"
android:supportsAssist="true" />
voiceInteractionService.java
package com.eaydin79.voiceinteraction;
import android.service.voice.VoiceInteractionService;
import android.service.voice.VoiceInteractionSession;
public class voiceInteractionService extends VoiceInteractionService {
#Override
public void onReady() {
super.onReady();
}
}
voiceInteractionSessionService.java
package com.eaydin79.voiceinteraction;
import android.os.Bundle;
import android.service.voice.VoiceInteractionSession;
import android.service.voice.VoiceInteractionSessionService;
public class voiceInteractionSessionService extends VoiceInteractionSessionService {
#Override
public VoiceInteractionSession onNewSession(Bundle bundle) {
return new voiceInteractionSession(this);
}
}
voiceInteractionSession.java
package com.eaydin79.voiceinteraction;
import android.app.VoiceInteractor;
import android.content.Context;
import android.os.Bundle;
import android.service.voice.VoiceInteractionSession;
import android.media.AudioManager;
public class voiceInteractionSession extends VoiceInteractionSession {
voiceInteractionSession(Context context) {
super(context);
}
#Override
public void onShow(Bundle args, int showFlags) {
super.onShow(args, showFlags);
//whatever you want to do when you hold the home button
//i am using it to show volume control slider
AudioManager audioManager = (AudioManager) getContext().getSystemService(Context.AUDIO_SERVICE);
if (audioManager != null) audioManager.adjustStreamVolume(AudioManager.STREAM_MUSIC, AudioManager.ADJUST_SAME, AudioManager.FLAG_SHOW_UI);
hide();
}
}
I am facing a problem in override the On Click Behavior in Appboy deeplink
Please find the following data
1- Register Appboy in BaseActivity which is the parent activity for all Application Activities
#Override
protected void onResume() {
AppboyInAppMessageManager.getInstance().registerInAppMessageManager(this);
Appboy.getInstance(this).requestInAppMessageRefresh();
}
#Override
protected void onPause() {
AppboyInAppMessageManager.getInstance().unregisterInAppMessageManager(this);
}
2- Add the receivers in Manifest File as following
<receiver android:name="com.forsale.forsale.appboy.AppboyGcmReceiver"
android:permission="com.google.android.c2dm.permission.SEND" >
<intent-filter>
<action android:name="com.google.android.c2dm.intent.RECEIVE" />
<action android:name="com.google.android.c2dm.intent.REGISTRATION" />
<category android:name="com.forsale.forsale" />
</intent-filter>
</receiver>
<receiver
android:name="com.forsale.forsale.appboy.AppBoyOpenReceiver"
android:exported="false" >
<intent-filter>
<action android:name="com.forsale.forsale.intent.APPBOY_PUSH_RECEIVED" />
<action android:name="com.forsale.forsale.intent.APPBOY_NOTIFICATION_OPENED" />
</intent-filter>
</receiver>
Know I can send in app message using app boy dashboard, and receive the message, but when I click the message it open appboy web activity with the link
I need to override this behaviour to be able to get the link that I sent in In app message and parse some parameters from it and direct the use to an activity inside my app
I have tried the following
remove default app boy web activity from manifest file /// the app crash
implement the IInAppMessageManagerListener /// the app stop receiving any messages
Please note that the application call the onReceive method when trying to register appboy and print the log (action = REGISTRATION, RegId = "..."), but it never lo any other actions like RECEIVE, or OPEN
public void onReceive(Context context, Intent intent) {
AppboyLogger.i("AMIRA", String.format("Amira %s", intent.toString()));
String action = intent.getAction();
AppboyLogger.i("AMIRA", String.format("Amira %s", action));
Bundle bundle = intent.getExtras();
for (String key : bundle.keySet()) {
Object value = bundle.get(key);
AppboyLogger.i("AMIRA", String.format("Amira %s", key + ":" + value.toString()));
}
}
The root of the problem is that we differentiate deep links and http links based on schema of the URI, so http (and some other schemes) links are detected as web links, and other formats are seen as deep links (see https://github.com/Appboy/appboy-android-sdk/blob/master/android-sdk-ui/src/com/appboy/ui/actions/ActionFactory.java).
We’ll consider how to instrument things for the use case you have, but in the meantime there’s a couple of ways you could solve the issue:
1) Create a deep link that is not also an http link. Everything should work if your link instead looks like, for example, forsale://mylink?a=b&2=3....etc.
2) Set a custom in-app message manager listener: https://documentation.appboy.com/Android/#in-app-message-customization. You can see an example of how we do this in our Droidboy sample app. In your case, you’d want to return defaults for everything but onInAppMessageButtonClicked and onInAppMessageClicked where you’d want to handle the link yourself if it’s of the format of your deep link. Your ticket indicates you’ve tried this, but I’d suggest starting with "the default one we create in the AppboyInAppMessageManager.java (#L608) in the Android SDK - and then just modifying the *clicked methods.
3) Download our UI code and modify the source. You could optionally download the Appboy Android SDK and modify the ActionFactory to handle your deep link in the way you want. Though, at the point you are going to do something like this, solution #2 is likely going to be a nicer one to implement and maintain.
Please let us know if one of these solutions works for you and if you have any other comments/questions.
Thanks,
Waciuma