I have a Service that can draw a Canvas on top of all applications using a SYSTEM_ALERT_WINDOW which contains a custom View.
My Service and custom View both implement View.OnTouchListener
The onTouch(View v, MotionEvent ev){} method of the Service returns true while the custom View returns false. This allows me to display my custom View on top of all apps and still interact with the underlying Activity. This part works.
I also want the Service to simulate a touch event in the specified coordinates in the current Activity that is underneath the SYSTEM_ALERT_WINDOW.
I was hoping I could call View.dispatchTouchEvent(...) on the custom View and because onTouch(...) returns false, the touch event would get passed on to the underlying Activity. It does not work.
How can I simulate touch events in any Activity in this situation?
I have used code from the following sources...
Draw a canvas on top of all applications:
http://www.piwai.info/chatheads-basics/
Draw an Android canvas on top of all applications?
Passing touches to underlying app:
How can I interact with elements behind a translucent Android app?
Simulating touches:
How to simulate a touch event in Android?
AFAIK, you can't, for obvious security reasons, except probably on rooted devices. Even the accessibility APIs don't support this AFAICT. Allowing arbitrary apps to fake input to other apps would be a major security hole.
If your objective is to do this during testing, instrumentation testing allows you to simulate touch events in your own app.
Related
It is common knowledge that we can observe or query the view hierarchy of any app with an AccessibilityService:
Create your own accessibility service.
It is also possible to perform actions on behalf of the user:
Developing an Accessibility Service for Android.
My question is, can we modify the view hierarchy of a foreground app with an AccessibilityService?
I have already referred the following questions:
How do I add and remove a layout programmatically from an accessibility service?
Get view of AccessibilityNodeInfo to create overlay.
What they're doing is using the WindowManager and the SYSTEM_ALERT_WINDOW permission to overlay a view on top of the app in the foreground. The problem with this approach is that if the user presses BACK or HOME, the app is dismissed, but the view remains visible on the screen, even after the app is gone. The view is on TOP of the view hierarchy, and not a part of it.
Is there a way to add / modify the AccessibilityNodeInfo objects?
Are these the same as a View or ViewGroup?
Are we allowed to add views or modify existing views with an AccessibilityService?
My requirement is to display a small view within the app itself.
It has to be part of the view hierarchy of the app, so that it stays
or goes with the app. For this I need to modify the view hierarchy of
the app, namely the AccessibilityNodeInfo objects retrieved from the
service.
What I want is something similar to addView(), but add the View to the view hierarchy of the app itself, not on top of it.
How can we do this? Is this possible?
UPDATE:
Apps that support Custom Views for Accessibility
No, you can't modify the view hierarchy of another app because it exists in a separate process.
This is similar to not being able to modify accessibility nodes from within an accessibility service.
1) You can exploit draw over other apps permission. That solution will allow you only to draw overlays over another apps and not to change another apps behavior.
2) You can exploit instrumentation test mechanism. If you have enough information about the app(app id, activity name) and enough privileges (Run an instrument test from within app and wait for result), or root privileges. Here is an example:
#RunWith(AndroidJUnit4::class)
class InjectView {
#get:Rule
val activityRule = ActivityTestRule<MainActivity>(MainActivity::class.java)
#Test
fun injectView() {
val rootLayout = activityRule.activity.findViewById<ViewGroup>(android.R.id.content)
activityRule.runOnUiThread {
rootLayout.addView(TextView(activityRule.activity).apply {
text = "Injected View"
})
}
Thread.sleep(10_000)
}
}
You can still go with the approach of drawing on top of the app, not as part of its view hierarchy (which is impossible) - using SYSTEM_ALERT_WINDOW permission.
In order to know when the app is dismissed and dismiss your own overlay - listen to the accessibility event TYPE_WINDOW_STATE_CHANGED and check that there's a package change.
You can also go further with listening to TYPE_WINDOW_CONTENT_CHANGED and determining that there was some layout update.
More accessibility events might come in handy or fine-tuning your overlay's accuracy.
In short - as long as you have the appropriate information to know about the underlying app's layout and when things happen, you can draw on top as if it's part of the app.
Might be tricky and require some calculations since you're not able to just push views into the hierarchy, but totally doable.
I am working on a background service that runs to give sound feedback device-wide whenever the user is touching the screen. I tried to create an overlay with a custom view to listen to touches and pass them down to the views below in a similar direction to what was attempted in this question but my overlay seemed to be either consuming all the touch events or if I returned false only the down touch was detected.
I was wondering if there was perhaps an accessibility route I could take? Basically I need a way to get the information of any current touches on the screen (preferably multitouch) I don't need to alter the touch event or anything, just need the information.
I have an Android Wear app in which I use a transparent overlay view to recieve touch events. I need to be able to receive all touch events that happen anywhere on the screen, and also be able to see their coordinates.
All research I have found says that I cannot recieve all touch events and pass them on. However, Wear Mini Launcher and Swipify are both doing something like this.
Here is where I have looked:
Android overlay to grab ALL touch, and pass them on?
https://stackoverflow.com/questions/19356834/overlay-view-which-intercepts-some-events
Android - Intercept and pass on all touch events
Android : Multi touch and TYPE_SYSTEM_OVERLAY
Creating a system overlay window (always on top)
Getting the View that is receiving all the touch events
Android Jelly Bean service that receives all touch events
TYPE_SYSTEM_OVERLAY in ICS
Overlay App that reacts only on some touch events
This must be possible to do somehow. Maybe an overlay view is not the right way to approach this? Any pointers?
I had quite a bit of trouble with this as well using the same links that you provided. But then I found this: http://www.piwai.info/chatheads-basics/
It's basically the same as the other solutions, except that the arguments to the WindowManager LayoutParams are TYPE_PHONE and FLAG_NOT_FOCUSABLE.
Not only did this solution work for me (I was testing on an LG Urbane), the onTouchEvent callback had all the actions (down, move, and up), which the other solutions said you wouldn't get.
Since all the touch events are passed through the view, in theory we can use gesture detectors to get callbacks for things like double taps and flings. I haven't tried this yet, though.
Regardless, the above link should get you past the initial problems you're facing.
I am trying to get multiple touch events on a system overlay view, but am only receiving the MotionEvent.ACTION_OUTSIDE event.
Is there any possible way of getting multiple touch events on a system overlay?
Any examples or links would be very helpful.
To create an overlay view, when setting up the LayoutParams you need to set the type to TYPE_SYSTEM_OVERLAY and use the flag FLAG_WATCH_OUTSIDE_TOUCH. This presents a problem because as the Android documentation states:
you will not receive the full down/move/up gesture, only the location of the first down as an ACTION_OUTSIDE.
In order to receive the full array of touch events you need to use the TYPE_SYSTEM_ALERT type, but this causes the overlay to take over the screen and stop interaction with other elements. The solution is to use both TYPE_SYSTEM_OVERLAY and TYPE_SYSTEM_ALERT and switch between them by changing the type of the LayoutParams as needed.
This is accomplished by:
Watch for the ACTION_OUTSIDE motion event.
When it occurs, test if it occured within the overlay.
If it did, switch the LayoutParams type to TYPE_SYSTEM_ALERT
Once the interaction with the overlay is complete, switch back to TYPE_SYSTEM_OVERLAY
Repeat
The one thing to keep in mind is that the ACTION_OUTSIDE motion event is always passed on to the rest of the elements on the screen. So, for example, if the overlay is on top of a button, that button will also receive the motion event and there is no way to stop it.
Also make sure you add the SYSTEM_ALERT_WINDOW permission to the mainifest file.
I've posted the complete solution here:
http://www.jawsware.mobi/code_OverlayView/ (UPDATED)
It includes all the source code and a link to download the entire example project.
Update for Android 4.0 - 1/3/2013
To create an overlay view, when setting up the LayoutParams DON'T set the type to TYPE_SYSTEM_OVERLAY.
Instead set it to TYPE_PHONE.
Use the following flags:
FLAG_NOT_TOUCH_MODAL
FLAG_WATCH_OUTSIDE_TOUCH
FLAG_NOT_TOUCH_MODAL << I found this one to be quite important. Without it, focus is given to the overlay and soft-key (home, menu, etc.) presses are not passed to the activity below.
Also, the previous link (above) has been updated to reflect this update.
Starting from Android 4.x, Android team Android team fixed a potential
security problem by adding a new function adjustWindowParamsLw() in which it
will add FLAG_NOT_FOCUSABLE, FLAG_NOT_TOUCHABLE and remove FLAG_WATCH_OUTSIDE_TOUCH flags for TYPE_SYSTEM_OVERLAY window.
That's TYPE_SYSTEM_OVERLAY window won't receive any touch event on ICS platform.
Updated: 2016/5/4
TYPE_TOAST cannot receive touch events on Android 4.0~4.3. It's the same as TYPE_SYSTEM_OVERLAY. On Android 4.4+, TYPE_TOAST removed from the excluded list, so you can use TYPE_TOAST to receive touch events on Android 4.4+ devices.
Hello i make appliction like Drum Studio. I need make support for touch more one button at once. For example: User touch button1(AND HOLD IT!) it play sound, but user still hold its, user click another button and it play sound too. How make it programmatically? By default onTouch or onClick use, if 1 button pressed, another buttons not react before button isn't released.
Use the onKeyDown() events to trigger each response and onKeyUp() events to finish them.
http://developer.android.com/reference/android/view/KeyEvent.html
http://developer.android.com/reference/android/view/KeyEvent.Callback.html
For touchs, you can use the OnTouchListener
http://developer.android.com/reference/android/view/View.OnTouchListener.html
and determine the kind of touch by the MotionEvent:
http://developer.android.com/reference/android/view/MotionEvent.html
To clarify my response: To make a sound while holding one button, you can start the sound when the ACTION_DOWN action in the MotionEvent event is triggered and stop it with the ACTION_UP action of the same event.
Create a class extending View where you're holding all your buttons.
Override the onTouchEvent vor that class.
Switch between the different actions.
Use event.getX() and event.getY() to figure out where the touch event was originated.
Check which button is located at that position
Play the sound accordingly.
You cannot touch two separate Views at the same time, even if multi-touch is supported on your device. If you touch Button1 first, all subsequent MotionEvents (including ones with separate pointer IDs) are bound to that View.
Unless you're using a dedicated game engine like AndEngine that has this sort of functionality built in, your only option is to capture all the MotionEvents yourself (most easily with an empty ImageView covering your entire screen) and route them to the appropriate Views based on their screen coordinates. This can get tricky, especially with multiple touch pointers, but the Android UI framework does not natively support the kind of multi-touch you require.
You definitely want to be using Android 2.0 or higher - previous versions don't support multitouch.
Check the pid in your onTouch - 0 is a single touch, 1 is the second.