Develop Reference

How can I use AccessibilityService to retrieve content of other apps running..?

I'm working in Accessibility Service. i need to get the app name using Accessibility Service. i have studied the documentation of the Accessibility Service in developer's of Android website. but there is no mention about getting the app name using Accessibility.
I also want to extract text from "TextViews" of the other apps(Activities) running in background. How i can do this..

I'm assuming you know how to implement an AccessibilityService.
Retrieving window content:
First register for TYPE_WINDOW_CONTENT_CHANGED events.
#Override
public void onAccessibilityEvent(AccessibilityEvent accessibilityEvent){
int eventType = accessibilityEvent.getEventType();
switch (eventType) {
case AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED:
AccessibilityNodeInfo rootNode = getRootInActiveWindow();
ArrayList<AccessibilityNodeInfo> textViewNodes = new ArrayList<AccessibilityNodeInfo>();
findChildViews(rootNode);
for(AccessibilityNodeInfo mNode : textViewNodes){
if(mNode.getText()==null){
return;
}
String tv1Text = mNode.getText().toString();
//do whatever you want with the text content...
}
break;
}
}
Method findChildViews() :
private void findChildViews(AccessibilityNodeInfo parentView) {
if (parentView == null || parentView.getClassName() == null || ) {
return;
}
if (childCount == 0 && (parentView.getClassName().toString().contentEquals("android.widget.TextView"))) {
textViewNodes.add(parentView);
} else {
for (int i = 0; i < childCount; i++) {
findChildViews(parentView.getChild(i));
}
}
}
}
As far as i know, there's no assured way to get the app name, but you can try fetching the text content from the event object you get from TYPE_WINDOW_STATE_CHANGED events.
Try dumping accessibilityEvent.toString() & you'll know what i'm talking about.
Also, accessibilityEvent.getPackageName() is a simple way to get package name of that app; in case you find it useful!

previous answer is missing definition of childCount
int childCount = parentView.getChildCount();

Android AccessibilityService: How to record and replay view clicks?

I am trying to implement an AccessibilityService that records the user's actions (only click events at this point) and stores them, such that they can be replayed at a later point in time.
For this I register for AccessibilityEvent.TYPE_VIEW_CLICKED events and check if I could somehow recover them later on (when all I have is a reference to the window / root node of the activity) by using two strategies:
Get the clicked view's id and look for this id in the root node's tree
Get the clicked view's text and look for this text in the root node's tree
I have tested this in various applications and different parts of the Android system and the results have been very confusing. About half of the views were not recoverable by any of the two strategies, and some views were sometimes reported as being recoverable and sometimes not. I found out that the latter was due to a race condition, since the accessibility service runs in a different process than the application to which the clicked views belong.
My question now is whether there is a better way to get a handle to a view in an accessibility and find this view again in a later execution of the application.
Below you find the code of my AccessiblityService class:
public class RecorderService extends AccessibilityService {
private static final String TAG = "RecorderService";
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {
switch (event.getEventType()) {
case AccessibilityEvent.TYPE_VIEW_CLICKED:
AccessibilityNodeInfo node = event.getSource();
if (node == null) {
Log.i(TAG, "node is null");
return;
}
AccessibilityNodeInfo root = getRootInActiveWindow();
if (root == null) {
Log.i(TAG, "root is null");
return;
}
// Strategy #1: locate node via its id
String id = node.getViewIdResourceName();
if (id == null) {
Log.i(TAG, "id is null");
} else {
List<AccessibilityNodeInfo> rootNodes = root.findAccessibilityNodeInfosByViewId(id);
if (rootNodes.size() == 1) {
Log.i(TAG, "success (via id)");
return;
} else {
Log.i(TAG, "multiple nodes with that id");
}
}
// Strategy #2: locate node via its text
CharSequence text = node.getText();
if (text == null) {
Log.i(TAG, "text is null");
} else {
List<AccessibilityNodeInfo> rootNodes = root.findAccessibilityNodeInfosByText(text.toString());
if (rootNodes.size() == 1) {
Log.i(TAG, "success (via text)");
return;
}
}
Log.i(TAG, "failed, node was not recoverable");
}
}
#Override
protected boolean onKeyEvent(KeyEvent event) {
Log.i("Key", event.getKeyCode() + "");
return true;
// return super.onKeyEvent(event);
}
#Override
public void onInterrupt() {
}
}
I am developing this on SDK Version 21 (Lollipop) and testing it on a HTC Nexus M8 and a Samsung Galaxy Note2, both showing similar results.

What's the difference between "getApplicationLabel" and "loadLabel"?

Background
I'm working on my app that is an alternative to the app manager (link here), and wish to optimize it a bit.
As it turns out, one of the slowest things on the app is its bootup, and the main reason for this is getting the app name . I intend on caching it, but I also wish to optimize the way it's being queried, if possible.
The problem
Android has two ways to get the app name: PackageManager.getApplicationLabel and ApplicationInfo.loadLabel .
both have about the same description, but I'm not sure which one should be used.
Not only that, but looking at the code of "ApplicationInfo.loadLabel" , it looks something like this:
public CharSequence loadLabel(PackageManager pm) {
if (nonLocalizedLabel != null) {
return nonLocalizedLabel;
}
if (labelRes != 0) {
CharSequence label = pm.getText(packageName, labelRes, getApplicationInfo());
if (label != null) {
return label.toString().trim();
}
}
if (name != null) {
return name;
}
return packageName;
}
I can't find the code of "PackageManager.getApplicationLabel", as it's abstract.
The question
Is there any difference between the two?
If there is no difference, why do we have 2 very similar methods to get the same app name? I mean, I can use either of them only if I have both applicationInfo object and the PackageManager object, but that's enough to use any of the methods...
If there is difference, which of them is better in terms of speed?

The source of 'PackageManager.getApplicationLabel' is available in 'ApplicationPackageManager.java'. It is as follows;
#Override
public CharSequence getApplicationLabel(ApplicationInfo info) {
return info.loadLabel(this);
}
ApplicationPackageManager.java
I see in AppUtils.java the same wrapping is done as follows;
/** Returns the label for a given package. */
public static CharSequence getApplicationLabel(
PackageManager packageManager, String packageName) {
try {
final ApplicationInfo appInfo =
packageManager.getApplicationInfo(
packageName,
PackageManager.MATCH_DISABLED_COMPONENTS
| PackageManager.MATCH_ANY_USER);
return appInfo.loadLabel(packageManager);
} catch (PackageManager.NameNotFoundException e) {
Log.w(TAG, "Unable to find info for package: " + packageName);
}
return null;
}

Accessibility function implementation problems in Android

I'm developing application that views books. There is a screen (Activity) which shows a book. It has custom view, something similar to ViewSwitcher and every page is a bitmap that is rendered by a custom View.
Now I should implement accessibility function - book should be read by the phone (audio).
I've read Accessibility section here https://developer.android.com/guide/topics/ui/accessibility/index.html but it is not clear enough.
I use SupportLibrary for accessibility management and now I have this code in ViewGroup (which manages book pages). Code 1:
private class EditionPagesViewSwitcherAccessibilityDelegate extends AccessibilityDelegateCompat {
private int mPageCount;
private double[] mPageRange;
#Override
public void onInitializeAccessibilityEvent(final View host, final AccessibilityEvent event) {
super.onInitializeAccessibilityEvent(host, event);
event.setClassName(EditionPagesViewSwitcher.class.getName());
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH) {
event.setScrollable(canScroll());
}
if (event.getEventType() == AccessibilityEventCompat.TYPE_VIEW_SCROLLED && updatePageValues()) {
event.setItemCount(mPageCount);
// we use +1 because of user friendly numbers (from 1 not 0)
event.setFromIndex((int) (mPageRange[0] + 1));
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH) {
event.setToIndex((int) (mPageRange[1] + 1));
}
}
}
#Override
public void onInitializeAccessibilityNodeInfo(final View host, final AccessibilityNodeInfoCompat info) {
super.onInitializeAccessibilityNodeInfo(host, info);
info.setClassName(EditionPagesViewSwitcher.class.getName());
info.setScrollable(canScroll());
info.setLongClickable(true);
if (canScrollForward()) {
info.addAction(AccessibilityNodeInfoCompat.ACTION_SCROLL_FORWARD);
}
if (canScrollBackward()) {
info.addAction(AccessibilityNodeInfoCompat.ACTION_SCROLL_BACKWARD);
}
}
#Override
public boolean performAccessibilityAction(final View host, final int action, final Bundle args) {
if (super.performAccessibilityAction(host, action, args)) {
return true;
}
switch (action) {
case AccessibilityNodeInfoCompat.ACTION_SCROLL_FORWARD: {
if (canScrollForward()) {
showNext();
return true;
}
}
return false;
case AccessibilityNodeInfoCompat.ACTION_SCROLL_BACKWARD: {
if (canScrollBackward()) {
showPrevious();
return true;
}
}
return false;
}
return false;
}
Here is code from page view Code 2:
#Override
public void onInitializeAccessibilityEvent(final View host, final AccessibilityEvent event) {
super.onInitializeAccessibilityEvent(host, event);
event.setClassName(EditionPageView.class.getName());
if (hasText()) {
event.getText().add(getPageRangeText());
final String trimText = mSurfaceUpdateData.getPageText().trim();
if (trimText.length() > MAX_TEXT_LENGTH) {
event.getText().add(trimText.substring(0, MAX_TEXT_LENGTH));
// event.getText().add(trimText.substring(MAX_TEXT_LENGTH, trimText.length()));
}
else {
event.getText().add(trimText);
}
}
}
#Override
public void onInitializeAccessibilityNodeInfo(final View host, final AccessibilityNodeInfoCompat info) {
super.onInitializeAccessibilityNodeInfo(host, info);
info.setClassName(EditionPageView.class.getName());
}
Because page text data loads asynchronous first time accessibility don't have any text while executes onInitializeAccessibilityEvent code. And then when data have been loaded I fire AccessibilityEvent.TYPE_VIEW_SELECTED and AccessibilityEvent.TYPE_VIEW_TEXT_CHANGED events. Then onInitializeAccessibilityEvent executes again and phone "read" book text.
So my questions:
Is my Accessibility implementation right? May be it is design wrong? Because I didn't find any good tutorial about this feature.
Why I need to use SDK versions checks in Support implementations in Code 1? Why support implementation doesn't handle it correctly?
Is firing TYPE_VIEW_SELECTED and TYPE_VIEW_TEXT_CHANGED really needed? Or may be some other code should be implemented?
The main question. In Code 2 there is commented code line. This code statement substring text to be less then MAX_TEXT_LENGTH (it's 3800) because if text is bigger nothing is played. Nothing. Is it accessibility restriction? Any other text that is less then this value is played well.
Does anyone know where I can find any good tutorial? (yes I saw samples).
Does anyone have any custom realizations to look through?
UPDATED
Well. Here is some answers:
As I can see TYPE_VIEW_SELECTED and TYPE_VIEW_TEXT_CHANGED events are not needed if you don't want this text to be read as soon as you get it.
On Nexus 7 all large text is played well (text up to 8000 symbols), so this issue doesn't reproduce on it, but on Samsung Galaxy Tab 10.1 (Android 4.0.4) and Genymotion emulator of Tab 10.1 with Android 4.3 does. And this is strange...

4.. According to the documentation of String.substring()
The first argument you pass is the start index in the original string, the second argument is the end index in the original string.
Example:
String text = "Hello";
partOfText = text.substring(2,text.length() - 1);
partOfText equals to "llo" (the first char is index 0)
So by putting your constant MAX_TEXT_LENGTH as a first argument, it would start at index 3800 to take out the substring.
http://developer.android.com/reference/java/lang/String.html#substring(int)

You are right MAX_TEXT_LENGTH is 3800.
About your doubt,
this code:
event.getText().add(trimText.substring(MAX_TEXT_LENGTH, trimText.length()));
}
you are trying to substring "trimText" from MAX_TEXT_LENGTH to trimText.length() !
Supposing that trimText = "STACK", trimText.length() = 5, then trimText.substring(3800,5) is going to be ?
At first, this doesn't have sense, using correctly would be like this:
trimText.substring(0,2) = "ST";

Is it possible to find out if an Android application runs as part of an instrumentation test

Is there a runtime check for an application to find out if it runs as part of an instrumentation test?
Background: Our application performs a database sync when starting. But that should happen only when started regularly. It especially interferes with the instrumentation tests testing the db sync. Not surprisingly.
And with all the other tests it's just a waste of CPU cycles.

A much simpler solution is check for a class that would only be present in a test classpath, works with JUnit 4 (unlike the solution using ActivityUnitTestCase) and doesn't require to send custom intents to your Activities / Services (which might not even be possible in some cases)
private boolean isTesting() {
try {
Class.forName("com.company.SomeTestClass");
return true;
} catch (ClassNotFoundException e) {
return false;
}
}

Since API Level 11, the ActivityManager.isRunningInTestHarness() method is available. This might do what you want.

If you are using Robolectric, you can do something like this:
public boolean isUnitTest() {
String device = Build.DEVICE;
String product = Build.PRODUCT;
if (device == null) {
device = "";
}
if (product == null) {
product = "";
}
return device.equals("robolectric") && product.equals("robolectric");
}

If you're using ActivityUnitTestCase, you could set a custom Application object with setApplication, and have a flag in there to switch database sync on or off? There's an example of using a custom Application object on my blog:
http://www.paulbutcher.com/2011/03/mock-objects-on-android-with-borachio-part-3/

You can pass an intent extra to your activity indicating it's under test.
1) In your test, pass "testMode" extra to your activity:
public void setUp() throws Exception {
super.setUp();
Intent activityIntent = new Intent();
activityIntent.putExtra("testMode", true);
setActivityIntent(activityIntent);
}
2) In your activity, check for testMode:
Bundle extras = getIntent().getExtras();
if (extras != null && extras.getBoolean("testMode")) {
// disable your database sync
}

d= (◕‿↼ ) Great answer, but if some library developer (like me) wants to know if the Host (or App using the library) is being tested, then try:
import android.content.pm.ApplicationInfo;
// ...
private static int wasTestRun = 0xDEAD;
/**
* Should only be used to speed up testing (no behavior change).
* #return true in tests, if Gradle has the right dependencies.
*/
public static boolean isTestRun(#NonNull Context context) {
if (wasTestRun != 0xDEAD) {
return wasTestRun != 0;
}
// Ignore release builds (as App may be using JUnit by mistake).
if (isDebuggable(context)) {
try {
Class.forName("org.junit.runner.Runner");
wasTestRun = 1;
return true;
} catch (ClassNotFoundException ignored) {
}
}
wasTestRun = 0;
return false;
}
public static boolean isDebuggable(#Nullable Context context) {
return context != null && (context.getApplicationContext()
.getApplicationInfo().flags & ApplicationInfo.FLAG_DEBUGGABLE) != 0;
}
Note that I am not using any AtomicBoolean or other helpers, as it is already pretty fast (and locking may just bring the speed down).

You can try this
if (isRunningTest == null) {
isRunningTest = false;
StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();
List<StackTraceElement> list = Arrays.asList(stackTrace);
for (StackTraceElement element : list) {
if (element.getClassName().startsWith("androidx.test.runner.MonitoringInstrumentation")) {
isRunningTest = true;
break;
}
}
}

This work for me because no actual device is running
public static boolean isUnitTest() {
return Build.BRAND.startsWith(Build.UNKNOWN) && Build.DEVICE.startsWith(Build.UNKNOWN) && Build.DEVICE.startsWith(Build.UNKNOWN) && Build.PRODUCT.startsWith(Build.UNKNOWN);
}