I'm writing an acessibility service in Android which relies on getting the view id of the currently selected view, however on some devices (Nexus 6P 6.0.1, Samsung Galaxy S6 edge+ 5 + 6.0.1) I get no view id through and on others (HTC One M8 5.0.1) it comes through fine. Because it works fine on some devices, I'm sure there's not a problem with my code, however I've posted a minimal test case below.
Can anyone can help me get my service reporting ids across all devices?
The A11y service
public class ViewIdLoggingAccessibilityService extends AccessibilityService {
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {
AccessibilityNodeInfo source = event.getSource();
if (source == null) {
Log.d("onAccessibilityEvent", "source was null for: " + event);
} else {
String viewIdResourceName = source.getViewIdResourceName();
Log.d("onAccessibilityEvent", "viewid: " + viewIdResourceName);
}
}
#Override
public void onInterrupt() {
Log.d("!# onInterrupt", "called");
}
}
The AndroidManifest.xml
<manifest package="com.example.a11yservice"
xmlns:android="http://schemas.android.com/apk/res/android"
>
<application
android:allowBackup="true"
android:icon="#drawable/ic_launcher"
android:label="#string/app_name"
android:theme="#style/AppTheme"
>
<service
android:name=".ViewIdLoggingAccessibilityService"
android:label="#string/view_id_logging_service_label"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
>
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/serviceconfig"
/>
</service>
</application>
</manifest>
serviceconfig.xml
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityEventTypes="typeViewClicked|typeViewFocused"
android:accessibilityFeedbackType="feedbackGeneric"
android:accessibilityFlags="flagReportViewIds"
android:canRetrieveWindowContent="true"
android:notificationTimeout="100" />
It's known issue and was posted to AOSP issue tracker some time ago. You can check status of issue here: "Accessibility Issue: flagReportViewIds has no effect on real devices in Android M".
Unfortunately issue exists even on latest Android 7.0 but I found simple workaround. You can call refresh() method on AccessibilityNodeInfo object to refresh it and collect all missed data including id.
Something like that:
public void onAccessibilityEvent(AccessibilityEvent event) {
AccessibilityNodeInfo ani = event.getSource();
if (ani != null) {
ani.refresh(); // to fix issue with viewIdResName = null on Android 6+
}
}
I found that if the viewId isn't available, querying the focused element seems to work:
AccessibilityNodeInfo source = findFocus(AccessibilityNodeInfo.FOCUS_INPUT);
viewIdResourceName = source.getViewIdResourceName();
returns the correct view id
Related
I've implemented iOS and Android SDK, now I'm testing my app configuration in test environment. While with iOS everything seems working fine, I've noticed that with Android, the install event is correctly attributed to my quick link, any other events I'm sending (even thos I can correctly see them in the liveview) are not attributed to the quick link used to open the app.
Here my dependencies:
com.android.tools.build:gradle:3.1.1
com.google.gms:google-services:4.2.0
io.fabric.tools:gradle:1.31.0
com.android.support:appcompat-v7:28.0.0
Here the main lines of code:
#Override
protected void handleOnStart() {
super.handleOnStart();
Branch.enableDebugMode();
// Branch object initialization
Branch.getAutoInstance(this.getActivity().getApplication());
branchInstance = Branch.getInstance();
branchInstance.disableTracking(trackingDisabled);
branchInstance.initSession(new Branch.BranchReferralInitListener() {
#Override
public void onInitFinished(JSONObject referringParams, BranchError error) {
if (error != null) {
log("onInitFinished - " + error.getMessage());
} else {
log("onInitFinished invoked with " + referringParams.toString());
testEvent();
// Retrieve deeplink keys from 'referringParams' and evaluate the values to determine where to route the user
// Check '+clicked_branch_link' before deciding whether to use your Branch routing logic
}
}
}, getActivity().getIntent().getData(), getActivity());
}
#Override
public void onNewIntent(Intent intent) {
this.setIntent(intent);
}
private void testEvent() {
BranchEvent event = new BranchEvent(BRANCH_STANDARD_EVENT.VIEW_ITEM);
event.logEvent(this.getActivity());
}
And my AndroidManifest.xml looks like this:
<application android:launchMode="singleTask" .....>
<meta-data android:name="io.branch.sdk.BranchKey" android:value="#string/branchio_key_live" />
<meta-data android:name="io.branch.sdk.BranchKey.test" android:value="#string/branchio_key_test" />
<meta-data android:name="io.branch.sdk.TestMode" android:value="#bool/branchio_test_mode" />
<receiver android:enabled="#bool/branchio_track_referral_active" android:name="io.branch.referral.InstallListener" android:exported="true">
<intent-filter>
<action android:name="com.android.vending.INSTALL_REFERRER" />
</intent-filter>
</receiver>
</application>
It looks like your code is slightly different than what we recommend in our Android Docs, so I would recommend you conform your code as closely as possible to our code snippets as possible. For example, we recommend initializing in the onStart() method whereas you are doing yours in handleOnStart(). Here are the Android docs: https://docs.branch.io/apps/android/
If you do this and are still having difficulties, please send an email to support#branch.io and provide your App ID which is found in your Account Settings on your dashboard so we can investigate this further.
Is it possible to monitor touching edit text behavior in another android app? For example, when touching edit text in specific android app, paste some words automatically which have been set up before.
PS. if it is not possible, any way to receive soft keyboard pop up broadcast ?
It's possible to detect when an EditText is touched, even when the EditText belongs to another app. However, this can only be done through Accessibility Services.
First, you need to create a serviceconfig.xml file in your xml folder:
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:description="#string/accessibility_permission_desc"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFeedbackType="feedbackGeneric"
android:notificationTimeout="100"
android:canRetrieveWindowContent="true"
/>
Second, you need to create an AccessibilityService, like so:
public class MyAccessibilityService extends AccessibilityService {
...
#Override
public void onAccessibilityEvent(AccessibilityEvent accessibilityEvent) {
switch(accessibilityEvent.getEventType()) {
case AccessibilityEvent.TYPE_VIEW_CLICKED:
case AccessibilityEvent.TYPE_VIEW_FOCUSED:
try {
Class className = Class.forName(accessibilityEvent.getClassName().toString());
if (EditText.class.isAssignableFrom(className)) {
// An EditText was Clicked or Focused
// Use other methods from the accessibilityEvent to do what
// you need to do
}
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
break;
}
}
...
}
Third, you need to add the service to the AndroidManifest.xml:
<service android:name=".MyAccessibilityService"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/serviceconfig" />
</service>
Finally, you need to enable the AccessibilityService through the Accessibility Settings. You can go directly to there through:
startActivity(new Intent(android.provider.Settings.ACTION_ACCESSIBILITY_SETTINGS));
Edit:
You can't get the EditText through an AccessibilityEvent. Instead, you need to get an AccessibilityNodeInfo and perform typical EditText features through this AccessibilityNodeInfo.
To get the AccessibilityNodeInfo, you'll need to call this code:
AccessibilityNodeInfo nodeInfo = accessibilityEvent.getSource()==null ? null : accessibilityEvent.getSource();
Then, you can perform specific actions like setText() like so:
if (nodeInfo != null) {
nodeInfo.refresh();
Bundle bundle = new Bundle();
bundle.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, newString);
nodeInfo.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, bundle);
}
Notice how I'm using performAction() onto the AccessibilityNodeInfo and using the action AccessibilityNodeInfo.ACTION_SET_TEXT and passing in a bundle with the key AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE.
This is almost the same as calling setText() on the EditText View.
If you don't want to setText() and want to just directly paste text into the EditText, you can use:
if (nodeInfo != null) {
nodeInfo.refresh();
nodeInfo.performAction(AccessibilityNodeInfo.ACTION_PASTE);
}
Background
Back a few years ago, I asked how TeamViewer allows the user to control the device without normal interaction with the device. I was told it's a special "backdoor" that manufacturers allow specifically for this app, and only possible using root priviledge for other apps.
Seeing that an app like "Airplane Mode Shortcut" allows to toggle airplane mode, by automatic navigation to its screen and toggling the switch, it made me realize this situation has changed.
The problem
It is said in the docs:
Starting with Android 4.0 (API Level 14), accessibility services can
act on behalf of users, including changing the input focus and
selecting (activating) user interface elements. In Android 4.1 (API
Level 16) the range of actions has been expanded to include scrolling
lists and interacting with text fields. Accessibility services can
also take global actions, such as navigating to the Home screen,
pressing the Back button, opening the notifications screen and recent
applications list. Android 4.1 also includes a new type of focus,
Accessibilty Focus, which makes all visible elements selectable by an
accessibility service.
These new capabilities make it possible for developers of
accessibility services to create alternative navigation modes such as
gesture navigation, and give users with disabilities improved control
of their Android devices.
But there is no more information about how to use it.
Only samples I've found are at the bottom, but those are very old and a part of the apiDemos bundle.
The question
How do I make a service that can query, focus, click, enter text, and perform other UI related operations?
By implementing AccessibilityService (https://developer.android.com/training/accessibility/service.html) you get access to that features.
You can either inspect or perform action on the element lastly interacted by user or inspect whole application which currently active.
Intercept user events by implementing onAccessibilityEvent(AccessibilityEvent event), here you can retrieve virtual view (representing original view) with event.getSource() and then inspect it with getClassName() or getText() or anything you find in the documentation.
Inspect whole application by calling getRootInActiveWindow() and iterate throught tree of virtaul views with getRootInActiveWindow().getChild(index).
Both getRootInActiveWindow() and event.getSource() return AccessibilityNodeInfo, on which you can invoke performAction(action) and do something like Click, Set Text, etc..
Example: Play Store
Search for 'facebook' app and open it's page on play store, once you opened the play store app.
#Override
public void onAccessibilityEvent(final AccessibilityEvent event) {
AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
//Inspect app elements if ready
if (rootInActiveWindow != null) {
//Search bar is covered with textview which need to be clicked
List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
if (searchBarIdle.size() > 0) {
AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
//Check is search bar is visible
List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
if (searchBars.size() > 0) {
AccessibilityNodeInfo searchBar = searchBars.get(0);
//Check is searchbar have the required text, if not set the text
if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
Bundle args = new Bundle();
args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
} else {
//There is no way to press Enter to perform search, so find corresponding suggestion and click
List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
for (AccessibilityNodeInfo suggestion : searchSuggestions) {
if(suggestion.getText().toString().equals("Facebook")) {
//We found textview, but its not clickable, so we should perform the click on the parent
AccessibilityNodeInfo clickableParent = suggestion.getParent();
clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
}
}
}
}
EDIT: full code below:
MyAccessibilityService
public class MyAccessibilityService extends AccessibilityService {
#Override
public void onCreate() {
super.onCreate();
Log.d("MyAccessibilityService", "onCreate");
}
#Override
public void onAccessibilityEvent(final AccessibilityEvent event) {
Log.d("MyAccessibilityService", "onAccessibilityEvent");
AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
//Inspect app elements if ready
if (rootInActiveWindow != null) {
//Search bar is covered with textview which need to be clicked
List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
if (searchBarIdle.size() > 0) {
AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
//Check is search bar is visible
List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
if (searchBars.size() > 0) {
AccessibilityNodeInfo searchBar = searchBars.get(0);
//Check is searchbar have the required text, if not set the text
if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
Bundle args = new Bundle();
args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
} else {
//There is no way to press Enter to perform search, so find corresponding suggestion and click
List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
for (AccessibilityNodeInfo suggestion : searchSuggestions) {
if (suggestion.getText().toString().equals("Facebook")) {
//We found textview, but its not clickable, so we should perform the click on the parent
AccessibilityNodeInfo clickableParent = suggestion.getParent();
clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
}
}
}
}
#Override
public void onInterrupt() {
}
}
AndroidManifest.xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.findfacebookapp">
<application
android:allowBackup="true"
android:icon="#mipmap/ic_launcher"
android:label="#string/app_name"
android:supportsRtl="true"
android:theme="#style/AppTheme">
<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
</intent-filter>
</activity>
<service
android:name=".MyAccessibilityService"
android:label="#string/accessibility_service_label"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService"/>
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/accessibility_service_config"/>
</service>
</application>
</manifest>
res/xml/accessibility_service_config.xml
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFeedbackType="feedbackAllMask"
android:accessibilityFlags="flagDefault"
android:canRequestEnhancedWebAccessibility="true"
android:canRetrieveWindowContent="true"
android:description="#string/app_name"
android:notificationTimeout="100"/>
MainActivity
public class MainActivity extends AppCompatActivity {
public void onEnableAccClick(View view) {
startActivityForResult(new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS), 1);
}
}
I have a little app that I made that have worked for months, but suddenly I get a
android.os.NetworkOnMainThreadException
I havent changed any code, this is the line that crashes
try {
endPoint = InetAddress.getByName(target.getToIp());
port = target.getPort();
socket = new DatagramSocket();
}
catch(Exception e) {
}
Note that I'm not sending anything I'm just creating the instances, it breaks already on the InetAddress row.
It wored as late as last night, all I have done on the phone is run update on Google play, it updated to latest version of Google Maps, but that cant have anything todo with it?
It works in the emulator, only my HTC One X that breaks
Full code here
https://github.com/AndersMalmgren/FreePIE/tree/master/Lib/Android/FreePIE%20Android%20IMU
edit:
Manifest
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.freepie.android.imu"
android:versionCode="1"
android:versionName="1.0">
<uses-sdk
android:minSdkVersion="9"
android:targetSdkVersion="15" />
<uses-permission android:name="android.permission.INTERNET" />
<application
android:icon="#drawable/ic_launcher"
android:label="#string/app_name"
android:theme="#style/AppTheme">
<activity
android:name=".MainActivity"
android:label="#string/title_activity_main"
android:screenOrientation="portrait">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
</manifest>
Tried to move the problem code to the thread instead, now it wont break, but the socket is set to null.
running = true;
worker = new Thread(new Runnable() {
public void run(){
try {
endPoint = InetAddress.getByName(target.getToIp());
port = target.getPort();
socket = new DatagramSocket();
}
catch(Exception e) {
Exception err = e;
}
while(running) {
try {
sync.await();
} catch(Exception e) {}
Send();
}
try {
socket.disconnect();
}
catch(Exception e) {}
}
});
worker.start();
update:
Solution was to add some failback if user types a DNS instead of a IP
https://github.com/AndersMalmgren/FreePIE/commit/e8017e02a7893d9df41e4ed67a037f016b6a7d39
You will get NetworkOnMainThreadException on Android 4.0+ by default if you attempt to do network I/O on the main application thread. getByName() does a DNS lookup if you provide a domain name (instead of dotted-IP notation), and that will trigger NetworkOnMainThreadException.
You must put your code in AsyncTask like this:
new AsyncTask<Void,Void,Void>(){
doInBackground(){
endPoint = InetAddress.getByName(target.getToIp());
port = target.getPort();
socket = new DatagramSocket();
}
}.execute();
Note this code is not a working one, please read more about AsyncTasks here
Use Network on main thread is not allowed from the Android 4.0. If you run the code at Android 2.3 or other, it will not throw any exception.
Ok, I have been working on a Android application for a while. It's basicaly a RSS reciever. I use RSS to check the news, weather forecast, TV schedule and Horoscope.
Everything was fine while I was testing it on Android 2.3.3, 2.3.6 and even on 2.1; but when I sent it to my friend who has Android 4.0 it did not work. I was suspicious so I tested it on emulator. Nor 4.0, nor 4.1 Emulators can run it. Here I provide Manifest and one class. If anybody knows how to fix this, that will be great.
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="XXX.XXXXX.XXX"
android:versionCode="1"
android:versionName="1.0" >
<uses-sdk
android:minSdkVersion="7"
android:targetSdkVersion="14" />
<application
android:icon="#drawable/ic_launcher"
android:label="#string/app_name"
android:theme="#style/AppTheme" >
<activity
android:name=".Vodic"
android:label="#string/title_activity_pocetna" >
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<activity
android:name=".Pomoc"
android:label="#string/m_tel_vodi_za_odlu_ne"
></activity>
<activity
android:name=".Pomocna"
android:label="#string/m_tel_vodi_za_odlu_ne"
></activity>
.
.
.
<activity
android:name=".TvRasporedTreca"
android:label="#string/m_tel_vodi_za_odlu_ne"
></activity>
</application>
<uses-permission android:name="android.permission.INTERNET" />
<!-- Needed to check when the network connection changes -->
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>
</manifest>
And here's one class:
public class Vodic extends Activity {
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_pocetna);
Button tv = (Button)findViewById(R.id.tv);
Button vijesti = (Button)findViewById(R.id.vijesti);
Button horoskop = (Button)findViewById(R.id.horoskop);
Button vremenska_prognoza = (Button)findViewById(R.id.vremenska_prognoza);
Button o_aplikaciji = (Button)findViewById(R.id.o_aplikaciji);
Button pomoc = (Button)findViewById(R.id.pomoc);
tv.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View arg0) {
Intent fy=new Intent(getApplicationContext(), TV_Raspored.class);
startActivity(fy);
}
});
.
.
.
pomoc.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View arg0) {
Intent fy=new Intent(getApplicationContext(), Pomoc.class);
startActivity(fy);
}
});
}
public boolean isOnline() {
ConnectivityManager cm = (ConnectivityManager) getSystemService(Context.CONNECTIVITY_SERVICE);
NetworkInfo netInfo = cm.getActiveNetworkInfo();
if(netInfo != null && netInfo.isConnected()) {
return true;
}
return false;
}
#Override
public boolean onCreateOptionsMenu(Menu menu) {
getMenuInflater().inflate(R.menu.activity_pocetna, menu);
return true;
}
}
So, to sum up. It works on 2.1 and 2.3.3 but doesn't on 4.0 and 4.1 . I have no Idea what's wrong.. so if anybody knows, please help..
From Android 3.x and onward Google has added some checks that generate errors if your application employs very bad practices.
Without a log it is hard to confirm, but the most likely scenario is that you are making network operations in the core of your activities, generating a NetworkOnUIThread exception.
It is NOT a bug in the Android framework. The problem is that your activities run in the main thread, ie the UI thread.
So when you make a network call in an Activity, the whole UI freezes until you get an answer (or a timeout); which is something that you absolutely want to avoid.
An easy solution to this issue is to use AsyncTask : it is an easy to implement built-in solutions to make operations in a secondary thread.