I was curious, I seen this app the other day that allowed it to open other apps and set certain functions up for you automatically. I have came to realize that it must be using an on screen click function of some sort, but I can't seem to find any documentation for something like this. For example if we know the on screen text from the other app is "Ready", is there a way to read that text and maybe do something like:
protected void processText(String text)
{
if (text.contains("Ready"))
// click the ready text
}
I have done this using AccessibilityService. It will only work fine on API level >= 16 though.
You need to extend AccessibilityService. For instance this class will get text of USSD responses and dismiss the dialog.
// ....
public class UssdAccessibilityService extends AccessibilityService {
public UssdAccessibilityService() {
}
#TargetApi(16)
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {
if (!"com.android.phone".equalsIgnoreCase((String)event.getPackageName())){
// In this example we are only interested in events comming
// from "com.android.phone" package
event.recycle();
return;
}
String className = (String)event.getClassName();
if (className == null || (!className.contains("AlertDialog") && !className.contains("AlertDialog"))){
// Class is not an USSD dialog
event.recycle();
return;
}
AccessibilityNodeInfo source = event.getSource();
if (source == null) {
// getSource() is annotated #Nullable, so we do this to be
// safe just in case
event.recycle();
return;
}
AccessibilityNodeInfo acceptButton = null;
String ussdText = null;
int childCount = source.getChildCount();
for (int i = 0; i < childCount; i++){
AccessibilityNodeInfo current = source.getChild(i);
if (current == null)
continue;
String currentText = (String)current.getText();
if (current.isClickable()){
// In the case of USSD dialogs, there is only one clickable.
// May be necessary to do more robust search in other scenarios
acceptButton = current;
continue;
}
ussdText = currentText;
current.recycle();
}
if (ussdText!= null) {
if (acceptButton != null)
acceptButton.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
source.recycle();
event.recycle();
}
// ....
}
You must declare the accessibility service in the manifest under <application>
<service
android:name=".UssdAccessibilityService"
android:enabled="true"
android:label="Read USSD codes and dismiss"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/accessibility_service_config" />
</service>
Under res/xml create accessibility_service_config.xml
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:description="#string/accessibility_service_description"
android:packageNames="com.android.phone,com.ats.android.activationcodebot"
android:accessibilityEventTypes="typeWindowStateChanged|typeWindowContentChanged"
android:accessibilityFlags="flagDefault"
android:accessibilityFeedbackType="feedbackSpoken"
android:notificationTimeout="100"
android:canRetrieveWindowContent="true"
/>
Of course you have to adapt this code to your own needs.
Finally you will have to enable the accessibility service manually on Settings > Accessibility in Android (or ask the user to do it).
Read more ... Developing an Accessibility Service
Related
Is it possible to monitor touching edit text behavior in another android app? For example, when touching edit text in specific android app, paste some words automatically which have been set up before.
PS. if it is not possible, any way to receive soft keyboard pop up broadcast ?
It's possible to detect when an EditText is touched, even when the EditText belongs to another app. However, this can only be done through Accessibility Services.
First, you need to create a serviceconfig.xml file in your xml folder:
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:description="#string/accessibility_permission_desc"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFeedbackType="feedbackGeneric"
android:notificationTimeout="100"
android:canRetrieveWindowContent="true"
/>
Second, you need to create an AccessibilityService, like so:
public class MyAccessibilityService extends AccessibilityService {
...
#Override
public void onAccessibilityEvent(AccessibilityEvent accessibilityEvent) {
switch(accessibilityEvent.getEventType()) {
case AccessibilityEvent.TYPE_VIEW_CLICKED:
case AccessibilityEvent.TYPE_VIEW_FOCUSED:
try {
Class className = Class.forName(accessibilityEvent.getClassName().toString());
if (EditText.class.isAssignableFrom(className)) {
// An EditText was Clicked or Focused
// Use other methods from the accessibilityEvent to do what
// you need to do
}
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
break;
}
}
...
}
Third, you need to add the service to the AndroidManifest.xml:
<service android:name=".MyAccessibilityService"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/serviceconfig" />
</service>
Finally, you need to enable the AccessibilityService through the Accessibility Settings. You can go directly to there through:
startActivity(new Intent(android.provider.Settings.ACTION_ACCESSIBILITY_SETTINGS));
Edit:
You can't get the EditText through an AccessibilityEvent. Instead, you need to get an AccessibilityNodeInfo and perform typical EditText features through this AccessibilityNodeInfo.
To get the AccessibilityNodeInfo, you'll need to call this code:
AccessibilityNodeInfo nodeInfo = accessibilityEvent.getSource()==null ? null : accessibilityEvent.getSource();
Then, you can perform specific actions like setText() like so:
if (nodeInfo != null) {
nodeInfo.refresh();
Bundle bundle = new Bundle();
bundle.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, newString);
nodeInfo.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, bundle);
}
Notice how I'm using performAction() onto the AccessibilityNodeInfo and using the action AccessibilityNodeInfo.ACTION_SET_TEXT and passing in a bundle with the key AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE.
This is almost the same as calling setText() on the EditText View.
If you don't want to setText() and want to just directly paste text into the EditText, you can use:
if (nodeInfo != null) {
nodeInfo.refresh();
nodeInfo.performAction(AccessibilityNodeInfo.ACTION_PASTE);
}
I'm trying to send messages via Whatsapp programmatically, the code works except the user needs to click the send button. I need the app to do everything (all user interactions). One way to do to it as follows .
Go to Menu Button > Settings > Chats. and check the "Enter is send option"
Here's the code I'm using:
protected void sendwts(){
String smsNumber = "2126123456789"; // E164 format without '+' sign
Intent sendIntent = new Intent(Intent.ACTION_SEND);
// Intent sendIntent = new Intent(Intent.ACTION_SENDTO);
sendIntent.setType("text/plain");
sendIntent.putExtra(Intent.EXTRA_TEXT, "test \n");
sendIntent.putExtra("jid", smsNumber + "#s.whatsapp.net"); //phone number without "+" prefix
sendIntent.setPackage("com.whatsapp");
startActivity(sendIntent);
}
Thank you
You can do that only using the Accessibility API of Android.
The idea is quite simple, you'll actually make Android perform the click on Whatsapp's send button.
So the flow will be:
Send a regular message (with the intent you're currently using) with a suffix at the end of your message content such as "Sent by MY_APP".
Once the text there, your accessibility service will be notified that the EditText of whatsapp is filled.
If the suffix is present on the EditText of whatsapp, Your accessibility service will click on the send button. (this is to avoid performing actions as the user types in naturally a regular message).
Here's an example (which you'll have tweak if you wanna make it more restrictive):
public class WhatsappAccessibilityService extends AccessibilityService {
#Override
public void onAccessibilityEvent (AccessibilityEvent event) {
if (getRootInActiveWindow () == null) {
return;
}
AccessibilityNodeInfoCompat rootInActiveWindow = AccessibilityNodeInfoCompat.wrap (getRootInActiveWindow ());
// Whatsapp Message EditText id
List<AccessibilityNodeInfoCompat> messageNodeList = rootInActiveWindow.findAccessibilityNodeInfosByViewId ("com.whatsapp:id/entry");
if (messageNodeList == null || messageNodeList.isEmpty ()) {
return;
}
// check if the whatsapp message EditText field is filled with text and ending with your suffix (explanation above)
AccessibilityNodeInfoCompat messageField = messageNodeList.get (0);
if (messageField.getText () == null || messageField.getText ().length () == 0
|| !messageField.getText ().toString ().endsWith (getApplicationContext ().getString (R.string.whatsapp_suffix))) { // So your service doesn't process any message, but the ones ending your apps suffix
return;
}
// Whatsapp send button id
List<AccessibilityNodeInfoCompat> sendMessageNodeInfoList = rootInActiveWindow.findAccessibilityNodeInfosByViewId ("com.whatsapp:id/send");
if (sendMessageNodeInfoList == null || sendMessageNodeInfoList.isEmpty ()) {
return;
}
AccessibilityNodeInfoCompat sendMessageButton = sendMessageNodeInfoList.get (0);
if (!sendMessageButton.isVisibleToUser ()) {
return;
}
// Now fire a click on the send button
sendMessageButton.performAction (AccessibilityNodeInfo.ACTION_CLICK);
// Now go back to your app by clicking on the Android back button twice:
// First one to leave the conversation screen
// Second one to leave whatsapp
try {
Thread.sleep (500); // hack for certain devices in which the immediate back click is too fast to handle
performGlobalAction (GLOBAL_ACTION_BACK);
Thread.sleep (500); // same hack as above
} catch (InterruptedException ignored) {}
performGlobalAction (GLOBAL_ACTION_BACK);
}
}
Then create its definition in res -> xml -> whatsapp_service.xml:
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityEventTypes="typeWindowContentChanged"
android:packageNames="com.whatsapp"
android:accessibilityFeedbackType="feedbackSpoken"
android:notificationTimeout="100"
android:canRetrieveWindowContent="true"/>
Then declare it in your manifest:
<service
android:name=".services.WhatsappAccessibilityService"
android:label="Accessibility Service"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/whatsapp_service"/>
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService"/>
</intent-filter>
</service>
And last thing, is to check if the accessibility services are enabled for your app or not, and redirect the user to the settings if not:
private boolean isAccessibilityOn (Context context, Class<? extends AccessibilityService> clazz) {
int accessibilityEnabled = 0;
final String service = context.getPackageName () + "/" + clazz.getCanonicalName ();
try {
accessibilityEnabled = Settings.Secure.getInt (context.getApplicationContext ().getContentResolver (), Settings.Secure.ACCESSIBILITY_ENABLED);
} catch (Settings.SettingNotFoundException ignored) { }
TextUtils.SimpleStringSplitter colonSplitter = new TextUtils.SimpleStringSplitter (":");
if (accessibilityEnabled == 1) {
String settingValue = Settings.Secure.getString (context.getApplicationContext ().getContentResolver (), Settings.Secure.ENABLED_ACCESSIBILITY_SERVICES);
if (settingValue != null) {
colonSplitter.setString (settingValue);
while (colonSplitter.hasNext ()) {
String accessibilityService = colonSplitter.next ();
if (accessibilityService.equalsIgnoreCase (service)) {
return true;
}
}
}
}
return false;
}
which you'll call with:
if (!isAccessibilityOn (context, WhatsappAccessibilityService.class)) {
Intent intent = new Intent (Settings.ACTION_ACCESSIBILITY_SETTINGS);
context.startActivity (intent);
}
This is purely on the technical aspect of the solution.
Now, the ethical question of "should you do that?", I believe the answer is quite clear:
Except if you are targeting people with disabilities (which is the very purpose of the Accessibility API), you should probably NOT do that.
Background
Back a few years ago, I asked how TeamViewer allows the user to control the device without normal interaction with the device. I was told it's a special "backdoor" that manufacturers allow specifically for this app, and only possible using root priviledge for other apps.
Seeing that an app like "Airplane Mode Shortcut" allows to toggle airplane mode, by automatic navigation to its screen and toggling the switch, it made me realize this situation has changed.
The problem
It is said in the docs:
Starting with Android 4.0 (API Level 14), accessibility services can
act on behalf of users, including changing the input focus and
selecting (activating) user interface elements. In Android 4.1 (API
Level 16) the range of actions has been expanded to include scrolling
lists and interacting with text fields. Accessibility services can
also take global actions, such as navigating to the Home screen,
pressing the Back button, opening the notifications screen and recent
applications list. Android 4.1 also includes a new type of focus,
Accessibilty Focus, which makes all visible elements selectable by an
accessibility service.
These new capabilities make it possible for developers of
accessibility services to create alternative navigation modes such as
gesture navigation, and give users with disabilities improved control
of their Android devices.
But there is no more information about how to use it.
Only samples I've found are at the bottom, but those are very old and a part of the apiDemos bundle.
The question
How do I make a service that can query, focus, click, enter text, and perform other UI related operations?
By implementing AccessibilityService (https://developer.android.com/training/accessibility/service.html) you get access to that features.
You can either inspect or perform action on the element lastly interacted by user or inspect whole application which currently active.
Intercept user events by implementing onAccessibilityEvent(AccessibilityEvent event), here you can retrieve virtual view (representing original view) with event.getSource() and then inspect it with getClassName() or getText() or anything you find in the documentation.
Inspect whole application by calling getRootInActiveWindow() and iterate throught tree of virtaul views with getRootInActiveWindow().getChild(index).
Both getRootInActiveWindow() and event.getSource() return AccessibilityNodeInfo, on which you can invoke performAction(action) and do something like Click, Set Text, etc..
Example: Play Store
Search for 'facebook' app and open it's page on play store, once you opened the play store app.
#Override
public void onAccessibilityEvent(final AccessibilityEvent event) {
AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
//Inspect app elements if ready
if (rootInActiveWindow != null) {
//Search bar is covered with textview which need to be clicked
List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
if (searchBarIdle.size() > 0) {
AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
//Check is search bar is visible
List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
if (searchBars.size() > 0) {
AccessibilityNodeInfo searchBar = searchBars.get(0);
//Check is searchbar have the required text, if not set the text
if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
Bundle args = new Bundle();
args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
} else {
//There is no way to press Enter to perform search, so find corresponding suggestion and click
List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
for (AccessibilityNodeInfo suggestion : searchSuggestions) {
if(suggestion.getText().toString().equals("Facebook")) {
//We found textview, but its not clickable, so we should perform the click on the parent
AccessibilityNodeInfo clickableParent = suggestion.getParent();
clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
}
}
}
}
EDIT: full code below:
MyAccessibilityService
public class MyAccessibilityService extends AccessibilityService {
#Override
public void onCreate() {
super.onCreate();
Log.d("MyAccessibilityService", "onCreate");
}
#Override
public void onAccessibilityEvent(final AccessibilityEvent event) {
Log.d("MyAccessibilityService", "onAccessibilityEvent");
AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
//Inspect app elements if ready
if (rootInActiveWindow != null) {
//Search bar is covered with textview which need to be clicked
List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
if (searchBarIdle.size() > 0) {
AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
//Check is search bar is visible
List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
if (searchBars.size() > 0) {
AccessibilityNodeInfo searchBar = searchBars.get(0);
//Check is searchbar have the required text, if not set the text
if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
Bundle args = new Bundle();
args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
} else {
//There is no way to press Enter to perform search, so find corresponding suggestion and click
List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
for (AccessibilityNodeInfo suggestion : searchSuggestions) {
if (suggestion.getText().toString().equals("Facebook")) {
//We found textview, but its not clickable, so we should perform the click on the parent
AccessibilityNodeInfo clickableParent = suggestion.getParent();
clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
}
}
}
}
#Override
public void onInterrupt() {
}
}
AndroidManifest.xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.findfacebookapp">
<application
android:allowBackup="true"
android:icon="#mipmap/ic_launcher"
android:label="#string/app_name"
android:supportsRtl="true"
android:theme="#style/AppTheme">
<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
</intent-filter>
</activity>
<service
android:name=".MyAccessibilityService"
android:label="#string/accessibility_service_label"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService"/>
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/accessibility_service_config"/>
</service>
</application>
</manifest>
res/xml/accessibility_service_config.xml
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFeedbackType="feedbackAllMask"
android:accessibilityFlags="flagDefault"
android:canRequestEnhancedWebAccessibility="true"
android:canRetrieveWindowContent="true"
android:description="#string/app_name"
android:notificationTimeout="100"/>
MainActivity
public class MainActivity extends AppCompatActivity {
public void onEnableAccClick(View view) {
startActivityForResult(new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS), 1);
}
}
im making android voice assistant app...that run service in background for recognizing the voice command .
i want to take picture in default system camera app when the user say's the word "selfie".i already know how to work with voice command but the problem is i cant make the camera app take picture ...
i tried some way but wont helped
1st i tried to simulate android camera key event
Intent intent1 = new Intent("android.intent.action.CAMERA_BUTTON");
intent1.putExtra("android.intent.extra.KEY_EVENT", new KeyEvent(0,
KeyEvent.KEYCODE_CAMERA));
sendOrderedBroadcast(intent1, null);
intent1 = new Intent("android.intent.action.CAMERA_BUTTON");
intent1.putExtra("android.intent.extra.KEY_EVENT", new KeyEvent(1,
KeyEvent.KEYCODE_CAMERA));
sendOrderedBroadcast(intent1, null);
this one open camera but wont take picture in phone's without physical camera key
2nd i tried to inject key event "enter" ... like bluetooth remote shutter ...
KeyEvent eventDown = new KeyEvent(KeyEvent.ACTION_DOWN, KeyEvent.KEYCODE_ENTER);
KeyEvent eventUp = new KeyEvent(KeyEvent.ACTION_UP, KeyEvent.KEYCODE_ENTER);
dispatchKeyEvent(eventDown);
dispatchKeyEvent(eventUp);
but in this one i faced 2 problem 1st this code cant be use in service 2nd its impossible to inject event to other app since that only system app could do this
now the question is how can i fix this problem?
is it possible or not?
i read some thing on web that appium could do this but its online & i want my app working off line
note that : adding camera permission & inject event permission wont help and i don't want to use camera api because i want to take pic in default system camera app.
Yes, Its possible After 2 days investigation I find the solution.
Requirement : Open system camera app and click pic.
Step 1:
Add Camera permission in manifest file:
<uses-permission android:name="android.permission.CAMERA"/>
<uses-feature
android:name="android.hardware.camera"
android:required="false" />
<uses-feature
android:name="android.hardware.camera.front"
android:required="false" />
Step 2: Create one service which extends AccessibilityService
<service
android:name=".AccessTest"
android:enabled="true"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
android:exported="true">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/accessibility_service_config"/>
</service>
Step 3: Start service when you required
Intent mailAccessabilityIntent = new Intent(getApplicationContext(), AccessTest.class);
startService(mailAccessabilityIntent);
Step 4: Add accessibility file.
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFeedbackType="feedbackAllMask"
android:accessibilityFlags="flagEnableAccessibilityVolume"
android:canRetrieveWindowContent="true"
android:notificationTimeout="100"
android:packageNames="com.google.android.GoogleCamera"
android:settingsActivity="com.mobiliya.cameraautoclick.MainActivity" />
Step 5: Write service class where you want to handle camera related listener.
public class AccessTest extends AccessibilityService {
private final static String TAG = "Yogesh";
#Override
public void onCreate() {
super.onCreate();
Log.d("Yogesh","I am started");
}
#Override
protected void onServiceConnected() {
super.onServiceConnected();
Log.d(TAG, "onServiceConnected");
}
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {
Log.d(TAG, "ACC::onAccessibilityEvent: " + event.getEventType());
//TYPE_WINDOW_STATE_CHANGED == 32
if (AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED == event
.getEventType()) {
AccessibilityNodeInfo nodeInfo = event.getSource();
if (nodeInfo == null) {
return;
}
Intent takePictureIntent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
String x = takePictureIntent.resolveActivity(getPackageManager()).getPackageName();
Log.d("Yogesh","Package name " + x);
List<AccessibilityNodeInfo> list1 = nodeInfo.findAccessibilityNodeInfosByText("Switch to front camera");
for (AccessibilityNodeInfo node : list1) {
Log.i(TAG, "ACC::onAccessibilityEvent: click " + node);
node.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
final List<AccessibilityNodeInfo> list = nodeInfo.findAccessibilityNodeInfosByText("Take photo");
final android.os.Handler handler = new android.os.Handler();
handler.postDelayed(new Runnable() {
#Override
public void run() {
for (AccessibilityNodeInfo node : list) {
Log.i(TAG, "ACC::onAccessibilityEvent: click " + node);
node.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
handler.postDelayed(this,5000);
}
},10000);
for (AccessibilityNodeInfo node : list) {
Log.i(TAG, "ACC::onAccessibilityEvent: click " + node);
node.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
Log.d(TAG,"Access " + getAllChildNodeText(nodeInfo).toString());
}
}
private List<CharSequence> getAllChildNodeText(AccessibilityNodeInfo infoCompat) {
List<CharSequence> contents = new ArrayList<>();
if (infoCompat == null)
return contents;
if (infoCompat.getContentDescription() != null) {
contents.add(infoCompat.getContentDescription().toString().isEmpty() ? "unlabelled" : infoCompat.getContentDescription());
} else if (infoCompat.getText() != null) {
contents.add(infoCompat.getText().toString().isEmpty() ? "unlabelled" : infoCompat.getText());
} else {
getTextInChildren(infoCompat, contents);
}
if (infoCompat.isClickable()) {
if (infoCompat.getClassName().toString().contains(Button.class.getSimpleName())) {
if (contents.size() == 0) {
contents.add("Unlabelled button");
} else {
contents.add("button");
}
}
contents.add("Double tap to activate");
}
return contents;
}
private void getTextInChildren(AccessibilityNodeInfo nodeInfoCompat, List<CharSequence> contents) {
if (nodeInfoCompat == null)
return;
if (!nodeInfoCompat.isScrollable()) {
if (nodeInfoCompat.getContentDescription() != null) {
contents.add(nodeInfoCompat.getContentDescription());
} else if (nodeInfoCompat.getText() != null) {
contents.add(nodeInfoCompat.getText());
}
if (nodeInfoCompat.getChildCount() > 0) {
for (int i = 0; i < nodeInfoCompat.getChildCount(); i++) {
if (nodeInfoCompat.getChild(i) != null) {
getTextInChildren(nodeInfoCompat.getChild(i), contents);
}
}
}
}
}
#Override
public void onInterrupt() {
}
}
Here getAllChildNodeText() return all text button which is clickable, Google default application have Take Photo text so for this view you can perform action.
Added handler for capture pic every 10 seconds for more clarification.
If you want to track multiple camera app then remove below line and use Java code for set package more details See -Accessibility Service
android:packageNames="com.google.android.GoogleCamera"
I uploaded working example -> https://github.com/ycrathi/cameraautoclick
Note: In above GitHub repo have multiple unwanted code, which I tried.
This solution is not global for all app. You can find some famous app like google camera and find text and then perform click action package wise.
you can use 3rd-party "fake camera" apps such as:
Image2Camera
Fake Camera by New Horizon Apps
Fake Camera - donate version by Vaclav Balak
alternatively you can use:
ICS emulator - that supports camera
alternatively you can use:
In your AVD advanced settings, you should be able to set front and back cameras to Webcam() or Emulated
I am making an application to call multiple numbers.
In that app
When I call to 1 person and if the call is answered by the user then
the loop should be stopped.
But If the call is rejected then the call should be on next number and
loop should be couninue.
My problem is I cant detect whether the call is rejected or answered. when I had search on net some people says it is not possible to detect the call is answered or rejected.
Is it really not possible to detect the call in android If it is possible then how can I do that?
I think you can check outgoing call time of last call in PhoneStateListener class' onCallStateChanged method. Fetch the data if state is idle that is TelephonyManager.CALL_STATE_IDLE.
Something like this:
Cursor mCallCursor = context.getContentResolver().query(android.provider.CallLog.Calls.CONTENT_URI,null,null,null,null);
int duration = mCallCursor.getColumnIndex( CallLog.Calls.DURATION);
while(mCallCursor.moveToFirst())
{
Toast.makeText(context, mCallCursor.getString(duration), Toast.LENGTH_LONG).show();
}
You can find more about that here. I haven't tested the above code. But something like that should work.
You can check if time's 00:00, then call next number of loop. Else you can stop calling.
Hope this helps you.
below is a code of detecting outgoing call by accessibility events -
Add a class which extends AccessibilityService in your projects -
public class CallDetection extends AccessibilityService {
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {
acquireLock(this);
Log.d("myaccess","after lock");
if (event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED) {
Log.d("myaccess","in window changed");
AccessibilityNodeInfo info = event.getSource();
if (info != null && info.getText() != null) {
String duration = info.getText().toString();
String zeroSeconds = String.format("%02d:%02d", new Object[]{Integer.valueOf(0), Integer.valueOf(0)});
String firstSecond = String.format("%02d:%02d", new Object[]{Integer.valueOf(0), Integer.valueOf(1)});
Log.d("myaccess","after calculation - "+ zeroSeconds + " --- "+ firstSecond + " --- " + duration);
if (zeroSeconds.equals(duration) || firstSecond.equals(duration)) {
Toast.makeText(getApplicationContext(),"Call answered",Toast.LENGTH_SHORT).show();
// Your Code goes here
}
info.recycle();
}
}
}
#Override
protected void onServiceConnected() {
super.onServiceConnected();
Toast.makeText(this,"Service connected",Toast.LENGTH_SHORT).show();
AccessibilityServiceInfo info = new AccessibilityServiceInfo();
info.eventTypes = AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED;
info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC;
info.notificationTimeout = 0;
info.packageNames = null;
setServiceInfo(info);
}
#Override
public void onInterrupt() {
}
}
But to get the function event.getSource() working you have to specify some of your service configuration through xml, so create a xml folder in your project and add a xml file called serviceconfig.xml (you can give any name you want.
The content of serviceconfig is below -
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:description="#string/callDetection"
android:accessibilityEventTypes="typeWindowContentChanged"
android:notificationTimeout="100"
android:canRetrieveWindowContent="true"
/>
You can find more about serviceconfig in Here
Now add your service in you Manifest file like this -
<service android:name=".CallDetection"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
android:label="#string/callDetection">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="#xml/serviceconfig" />
</service>
And youre done, just run the app and go to Accessibility settings in your phone, you will find an option named as detection (or whatever name you have given as your service description), switch that on to give accesibility permissions for you app.
Now you will see a toast when call is answered.
you can Code any code you want in there, also you can call a callback function in your activity
Most important - Dont call your call window(android dialer window) untill the call is answered, otherwise this will not work.
Note - As android doesn't provide any solution to detect if the call is answered or not, this is the best alternative i have made, hope it works for you.