I'm working in Accessibility Service. i need to get the app name using Accessibility Service. i have studied the documentation of the Accessibility Service in developer's of Android website. but there is no mention about getting the app name using Accessibility.
I also want to extract text from "TextViews" of the other apps(Activities) running in background. How i can do this..
I'm assuming you know how to implement an AccessibilityService.
Retrieving window content:
First register for TYPE_WINDOW_CONTENT_CHANGED events.
#Override
public void onAccessibilityEvent(AccessibilityEvent accessibilityEvent){
int eventType = accessibilityEvent.getEventType();
switch (eventType) {
case AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED:
AccessibilityNodeInfo rootNode = getRootInActiveWindow();
ArrayList<AccessibilityNodeInfo> textViewNodes = new ArrayList<AccessibilityNodeInfo>();
findChildViews(rootNode);
for(AccessibilityNodeInfo mNode : textViewNodes){
if(mNode.getText()==null){
return;
}
String tv1Text = mNode.getText().toString();
//do whatever you want with the text content...
}
break;
}
}
Method findChildViews() :
private void findChildViews(AccessibilityNodeInfo parentView) {
if (parentView == null || parentView.getClassName() == null || ) {
return;
}
if (childCount == 0 && (parentView.getClassName().toString().contentEquals("android.widget.TextView"))) {
textViewNodes.add(parentView);
} else {
for (int i = 0; i < childCount; i++) {
findChildViews(parentView.getChild(i));
}
}
}
}
As far as i know, there's no assured way to get the app name, but you can try fetching the text content from the event object you get from TYPE_WINDOW_STATE_CHANGED events.
Try dumping accessibilityEvent.toString() & you'll know what i'm talking about.
Also, accessibilityEvent.getPackageName() is a simple way to get package name of that app; in case you find it useful!
previous answer is missing definition of childCount
int childCount = parentView.getChildCount();
Related
I want to navigate Angular single page application without reloading the webpage. On button click I'm calling following code to navigate.
String fullUrl = "https://example.com/page1";
String hashUrl = "/page1";
public void goBackInWebView(String fullUrl, String hashUrl) {
WebBackForwardList history = webview.copyBackForwardList();
int index;
int currentIndex = history.getCurrentIndex();
for (int i = 0; i < history.getSize(); i++) {
index = i - currentIndex;
if ((webview.canGoBackOrForward(1 + i)) || (webview.canGoBackOrForward(1 - i))) {
if (history.getItemAtIndex(i).getUrl().endsWith(hashUrl)) {
webview.goBackOrForward(index);
break;
} else if (i == history.getSize()) {
webview.loadUrl(fullUrl);
break;
}
} else {
//OUTER ELSE
webview.loadUrl(fullUrl);
break;
}
}
}
If the webpage is not saved in the history it will call webview.loadUrl(fullUrl); else it will load the page using webview.goBackOrForward(index);.
but is above code everytime OUTER ELSE is callling.
First of all canGoBackOrForward is used for Getting whether the page can go back or forward the given number of steps. as you mentioned in question your web page is single page so you don't need call this method. I think bellow code can solve your problem.
public void goBackInWebView(String fullUrl, String hashUrl) {
WebBackForwardList history = webview.copyBackForwardList();
if (history.getSize() == 0) {
webview.loadUrl(fullUrl);
}else {
for (int i = 0; i < history.getSize(); i++) {
if (history.getItemAtIndex(i).getUrl().endsWith(hashUrl)) {
webview.goBackOrForward(i);
break;
} else {
webview.loadUrl(fullUrl);
}
}
}
}
I find it difficult to see much relation to Android and WebView, because the problem is SPA.
Better use hashtag # navigation for SPA, because this won't request anything server-side.
The point simply is, that the back button does not matter the least, while never navigating.
One just needs to bind JavaScript, which runs whenever window.location.hash changes.
I am developing an application like VoiceAccess app. Using accessibility service I am able to perform all clicks which are on top activity(3rd party applications). But I am facing issue with ListItem clicks. I am trying this code for FaceBook app. below is my code. Can any one help me on this.
public class MyService extends AccessibilityService {
/**/
private SharedPreferences S_PREF;
private SharedPreferences.Editor editor;
private static final String TAG = MyService.class
.getSimpleName();
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {
clickPerform(getRootInActiveWindow(), 0);
}
public void clickPerform(AccessibilityNodeInfo nodeInfo, final int depth) {
if (nodeInfo == null) return;
List<AccessibilityNodeInfo> list = nodeInfo
.findAccessibilityNodeInfosByViewId("com.facebook.katana:id/bookmarks_tab");
for (AccessibilityNodeInfo node : list) {
Log.i(TAG, "ViewID-: bookmarks_tab " + node.getChild(0));
if (S_PREF.getBoolean("fb_menu", false)) {
editor.putBoolean("fb_menu", false);
node.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
List<AccessibilityNodeInfo> list2 = nodeInfo
.findAccessibilityNodeInfosByViewId("com.facebook.katana:id/bookmarks_list");
for (AccessibilityNodeInfo node2 : list2) {
if (node2.getChild(0) != null)
if (S_PREF.getBoolean("fb_scroll_down", false)) {
editor.putBoolean("fb_scroll_down", false);
node2.performAction(AccessibilityNodeInfo.ACTION_SCROLL_FORWARD);
}
}
editor.commit();
for (int i = 0; i < nodeInfo.getChildCount(); ++i) {
clickPerform(nodeInfo.getChild(i), depth+1);
}
}
#Override
public void onInterrupt() {
}
protected void onServiceConnected() {
S_PREF = getSharedPreferences("S_PREF", Context.MODE_PRIVATE);
editor = S_PREF.edit();
AccessibilityServiceInfo info = new AccessibilityServiceInfo();
info.flags = AccessibilityServiceInfo.DEFAULT;
info.eventTypes = AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED | AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED;
info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC;
setServiceInfo(info);
Toast.makeText(getApplicationContext(), "onServiceConnected", Toast.LENGTH_SHORT).show();
}
}
Below is the VoiceAccess screen.
When user says any number, then particular item click will be performed.
I am able to get 7,8,9,10 events but from 11 onwards I am not getting list items individually. My code returning listview id only.
Thanks in advance....
#steveenzoleko
I got solution for this problem by myself. Here
AccessibilityNodeInfo always doesn't return complete list or list size. It returns VIEWs. It may be TextView, Button etc... Every View has multiple methods like getText(), getContentDescription(), getClassName(), getChildCount(), getViewIdResourceName() etc... here Voice Access app detecting all views and giving them some numbers. For lists, using getChildCount() method in for loop we can getChildViews/listItems. Ex:
AccessibilityNodeInfo node = getRootInActiveWindow();
if(node != null) {
for(int i = 0; i < node.getChildCount(); i++){
AccessibilityNodeInfo childNode = node.getChild(i);
if(childNode != null){
Log.i("childNode", "-----getText->"+childNode.getText()+"---getContentDescription-->"+childNode.getContentDescription() );
}
}
}
use getText() and getContentDescription() methods to know text of textview, button, checkbox.
getClassName() method retuns the view type like TextView, Button, ImageView, CheckBox etc...
Maybe you can use dispatchGesture from AccessibilityService API 7.0.
I'm working on PJSIP Android app and facing a problem with call hold. While the caller is call to the receiver when caller is put call on hold, receiver how can identify is remote server call on hold? Which event is occurs in receiver hand?
According to pjsip docs:
virtual void onCallMediaState(OnCallMediaStateParam &prm)
Notify application when media state in the call has changed.
Normal application would need to implement this callback, e.g. to connect the call’s media to sound device.
This method is in Call class in pjsip java:
CallMediaInfo class contains pjsua_call_media_status.
Using pjsua_call_media_status we can know whether current call media status is active or is on call hold.
#Override
public void onCallMediaState(OnCallMediaStateParam prm)
{
CallInfo ci;
try {
ci = getInfo();
} catch (Exception e) {
return;
}
CallMediaInfoVector cmiv = ci.getMedia();
CallMediaInfo callMediaInfo = null;
for (int i = 0; i < cmiv.size(); i++) {
CallMediaInfo cmi = cmiv.get(i);
if (cmi.getType() == pjmedia_type.PJMEDIA_TYPE_AUDIO &&
(cmi.getStatus() ==
pjsua_call_media_status.PJSUA_CALL_MEDIA_REMOTE_HOLD ||
cmi.getStatus() ==
pjsua_call_media_status.PJSUA_CALL_MEDIA_LOCAL_HOLD) )
{
//set ur call status as hold on your TextView
}else if (cmi.getType() == pjmedia_type.PJMEDIA_TYPE_AUDIO &&
(cmi.getStatus() ==
pjsua_call_media_status.PJSUA_CALL_MEDIA_ACTIVE))
{
//set ur call status as connected on your TextView
}
}
}
I am developing a floating window, which will be shown on apps I am observing using AccessibilityService.
But I don't want the window shown on other apps.
So I firstly configure the android:packageNames property of <accessibility-service>. Then find some solutions to check whether user is using apps I configured, like getRunningTasks, getRunningAppProcesses and UsageStatsManager. But they all have some shortcomings.
The question is:
Is there any way to check whether apps I configured using Accessibility is foreground currently?
OR:
Is there any way to make the window shown on apps I configured using Accessiblity and not shown on other apps?
Thanks a lot!
Thanks to idea of #ataulm !
I find a perfect way of checking whether my apps are in foreground:
set accessibility's packageName to all
set accessibility's EventTypes to typeAllMask
Java:
public void onAccessibilityEvent(AccessibilityEvent event) {
if (event == null || event.getSource() == null) {
return;
}
AccessibilityNodeInfo source = event.getSource();
String className = source.getClassName().toString();
switch (event.getEventType()) {
case AccessibilityEvent.TYPE_VIEW_SCROLLED:
Log.i(TAG, "onAccessibilityEvent: scrolled");
break;
case AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED:
setForegroundApp(event.getPackageName());
break;
}
}
public static String sForegroundPackageName;
static final List<String> supportedPackagesName = blablablabla;
public static void setForegroundApp(CharSequence packageName) {
if (packageName != null) {
sForegroundPackageName = packageName.toString();
}
}
// use this to check.
public static boolean isForeground() {
return supportedPackagesName.contains(sForegroundPackageName);
}
I'm trying to access the textual content of another app which is probably built using a non-native(js+html) based framework.
Hence, I figured I'll try to access the data from an accessibility node corresponding to a WebView element.
However, I'm unable to grab textual/html data using the usual methods since methods like getText() work only if it is a native android element such as a TextView, Button etc.
public class MyAccessibilityService extends AccessibilityService {
#Override
public void onAccessibilityEvent(AccessibilityEvent accessibilityEvent) {
AccessibilityNodeInfo accessibilityNodeInfo = accessibilityEvent.getSource();
if (accessibilityNodeInfo == null) {
return;
}
int childCount = accessibilityNodeInfo.getChildCount();
for (int i = 0; i < childCount; i++) {
AccessibilityNodeInfo accessibilityNodeInfoChild = accessibilityNodeInfo.getChild(i);
myRecursiveFunc(accessibilityNodeInfoChild);
}
}
#Override
public void onInterrupt() {
}
private void myRecursiveFunc(AccessibilityNodeInfo accessibilityNodeInfoChild) {
if (accessibilityNodeInfoChild.getChildCount() > 0) {
for (int j = 0; j < accessibilityNodeInfoChild.getChildCount(); j++) {
AccessibilityNodeInfo child = accessibilityNodeInfoChild.getChild(j);
if (child != null) {
myRecursiveFunc(child);
}
}
} else {
if ("android.webkit.WebView".equals(accessibilityNodeInfoChild.getClassName())) {
//===========This is a WebView's AccessibilityNodeInfo !!!!
//===========How to get HTML data from nodeinfo object here ??
}
}
}
}
Does your service have FLAG_REQUEST_ENHANCED_WEB_ACCESSIBILITY?
final AccessibilityServiceInfo info = getServiceInfo();
info.flags |= AccessibilityServiceInfo.FLAG_REQUEST_ENHANCED_WEB_ACCESSIBILITY;
setServiceInfo(info);
Android's default AccessibilityService (TalkBack) requires this to be able to screen read the text in WebViews. From examining the source, it appears that there's some script injection and some slightly hackiness in order to be able to get the textual content of the HTML elements
You should check out their source:
TalkBackService
WebInterfaceUtils
ProcessorWebContent