I have made an event injector for Android based on code from the AndroidScreencast project. I'm trying to simulate the Xperia Play touchpad on other devices. My code works great when I'm simulating normal touchscreen events, but does something unexpected when I try to actually simulate the touchpad. Here's my code:
MotionEvent.PointerCoords[] coords = { new MotionEvent.PointerCoords() };
coords[0].x = 200;
coords[0].y = 200;
int[] ptrs = { 0 };
MotionEvent event = MotionEvent.obtain(SystemClock.uptimeMillis(), SystemClock.uptimeMillis(), action, 1, ptrs, coords, 0, 1, 1, 0, 0, InputDevice.SOURCE_TOUCHPAD, 0);
windowManager.injectPointerEvent(event, false);
Here's the strange part: the event is sent, but in my little testing app, it shows up as SOURCE_TOUCHSCREEN!
Am I doing something wrong, or is this some Android quirk where it converts non-supported sources?
Related
Path clickPath = new Path();
clickPath.moveTo(x, y);
GestureDescription.StrokeDescription clickStroke = new GestureDescription.StrokeDescription(clickPath, 0, 1);
GestureDescription.Builder clickBuilder = new GestureDescription.Builder();
clickBuilder.addStroke(clickStroke);
dispatchGesture(clickBuilder.build(), null, null);
With this code I can perform clicks anywhere on the screen. Is there any way to perform touch and hold gesture using AccessibilityService?
Is there any way to perform touch and hold gesture using AccessibilityService?
I think that you need decide if the gesture willContinue or not. Then, based in your code i suggest change:
GestureDescription.StrokeDescription clickStroke = new GestureDescription.StrokeDescription(clickPath, 0, 1);
To:
GestureDescription.StrokeDescription clickStroke = new GestureDescription.StrokeDescription(clickPath, 0, 1, true);
Simply, add true to last parameter of StrokeDescription. PS: this works only from Android 8+.
want to trigger on two fingers down event for a few milliseconds.
I am able to trigger simple touch event by using this code:
objImageView.getLocationOnScreen(coords);
int xa = coords[0];
int ys = coords[1];
objImageView.dispatchTouchEvent(MotionEvent.obtain(0, 0, MotionEvent.ACTION_DOWN, xa, ys, 0.5f, 5, 0, 1, 1, 0, 0));
objImageView.dispatchTouchEvent(MotionEvent.obtain(0, 0, MotionEvent.ACTION_MOVE, xa, ys, 0.5f, 5, 0, 1, 1, 0, 0));
objImageView.dispatchTouchEvent(MotionEvent.obtain(0, 0, MotionEvent.ACTION_UP, xa, ys, 0.5f, 5, 0, 1, 1, 0, 0));
Now if I try to do the same for ACTION_POINTER_DOWN nothing is happening. Here is my code:
int[] coords = new int[2];
parentLayout.getLocationOnScreen(coords);
int xa = coords[0];
int ys = coords[1];
parentLayout.dispatchTouchEvent(MotionEvent.obtain(0, 0, MotionEvent.ACTION_POINTER_DOWN, xa, ys, 0.5f, 5, 0, 1, 1, 0, 0));
parentLayout.dispatchTouchEvent(MotionEvent.obtain(0, 0, MotionEvent.ACTION_MOVE, xa, ys, 0.5f, 5, 0, 1, 1, 0, 0));
parentLayout.dispatchTouchEvent(MotionEvent.obtain(0, 0, MotionEvent.ACTION_POINTER_UP, xa, ys, 0.5f, 5, 0, 1, 1, 0, 0));
I want to trigger the same event which called when user places the two fingers down and hold fingers there for a few milliseconds.
To simulate "two fingers" you will need to create a motion event that includes more than one pointer, as you do now.
To do that, use the MotionEvent obtain variation that supports multiple pointers as is documented here.
MotionEvent obtain (long downTime,
long eventTime,
int action,
int pointerCount,
PointerProperties[] pointerProperties,
PointerCoords[] pointerCoords,
int metaState,
int buttonState,
float xPrecision,
float yPrecision,
int deviceId,
int edgeFlags,
int source,
int flags)
First you should dispatch an event for the first finger (exactly like you did) and when you are about to simulate MotionEvent.ACTION_POINTER_DOWNyou should use the above obtain variation.
Fields are pretty self explanatory, make sure the params are:
pointerCount = 2
pointerProperties = [firstFingerPointerProperties, secondFingerPointerProperties]
pointerCoords = [firstFingerPointerCoords, secondFingerPointerCoords]
Documentation said that: "A gesture starts with a motion event with ACTION_DOWN that provides the location of the first pointer down. As each additional pointer that goes down or up, the framework will generate a motion event with ACTION_POINTER_DOWN or ACTION_POINTER_UP accordingly." so You need trigger simple touch event at first. And take a look at this question.
You should generate a sequence like this: ACTION_DOWN -> ACTION_POINTER_DOWN -> optionally small delay -> ACTION_POINTER_UP -> ACTION_UP.
You should also set pointer index (and other metadata) correctly for second pointer, because otherwise the gesture detection code may be unable to tell that ACTION_POINTER_DOWN for secondary press is describing a secondary finger, not primary.
Use this method to fill in all necessary information for ACTION_POINTER_DOWN describing second press. Make sure to pass correct pointerProperties!
This is an old thread, but since, none of the answers were complete, adding a bit more info here for future folks. Like the author I wanted simulate two touches, and even though my sequence wa:
ACTION_DOWN(0), PointerCount = 1
ACTION_POINTER_DOWN(5), PointerCount = 2
ACTION_POINTER_UP(6), PointerCount = 2
ACTION_UP(1), PointerCount = 1
It was simply not working, I only then understood, that you need to shift action by pointer count.
if ((action == ACTION_POINTER_DOWN || action == ACTION_POINTER_UP) && pointerCount > 1)
action |= 256 << (pointerCount - 2);
In other words, the action values become
ACTION_DOWN(0), PointerCount = 1
ACTION_POINTER_DOWN(261), PointerCount = 2
ACTION_POINTER_UP(262), PointerCount = 2
ACTION_UP(1), PointerCount = 1
Only then it started working for me
I was going through some of the awesome Android libraries and I found the Android BrokenView. It's quite simple to use and uses the touch events to show the broken animation. Here's the code that we're required to add in order to enable a View to behave as a BrokenView:
brokenView = BrokenView.add2Window(context);
listener = new BrokenTouchListener.Builder(brokenView).build();
view.setOnTouchListener(listener);
The problem that I've is that I want to perform the broken animation programmatically i.e, without any actual touch events. I tried looking at the source code but couldn't figure out how to achieve that as most of the methods are protected.
Any ideas on how can I do that?
I've achieved this using a motion event. It works, but I'd prefer to call createAnimator() directly, as I don't want the user to be able to trigger the smash via a touch event.
long downTime = SystemClock.uptimeMillis();
long eventTime = SystemClock.uptimeMillis()+1000;
int[] coords = new int[2];
targetView.getLocationOnScreen(coords);
float x = coords[0] + targetView.getWidth()/2;
float y = coords[1] + targetView.getHeight()/2;
MotionEvent motionEvent = MotionEvent.obtain(
downTime,
eventTime,
MotionEvent.ACTION_DOWN,
x,
y,
0
);
mBrokenTouchListener = new BrokenTouchListener.Builder(mBrokenView)
.setEnableArea(mRootView)
.build();
mBrokenTouchListener.onTouch(targetView, motionEvent);
What I'd like to do is this:
Point point = new Point((int)x, (int)y);
BrokenConfig config = new BrokenConfig();
brokenView.createAnimator(mFLTargetView, point, config);
brokenView.setEnable(true);
// or
brokenView.start();
but BrokenConfig is package private, and there's no start() method.
HTH
I am working on a real android device through mac , executing java commands from eclipse, everything works great except the swipe command, here is my code :
wd = new AndroidDriver<MobileElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
wd.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
//enter user name (works)
wd.findElement(By.xpath("//android.widget.LinearLayout[1]/android.widget.FrameLayout[1]/android.widget.LinearLayout[1]/android.widget.ScrollView[1]/android.widget.LinearLayout[1]/android.widget.EditText[1]")).sendKeys(username);
//enter password(works)
wd.findElement(By.xpath("//android.widget.LinearLayout[1]/android.widget.FrameLayout[1]/android.widget.LinearLayout[1]/android.widget.ScrollView[1]/android.widget.LinearLayout[1]/android.widget.EditText[2]")).sendKeys(password);
//click login(works)
wd.findElement(By.xpath("//android.widget.LinearLayout[1]/android.widget.FrameLayout[1]/android.widget.LinearLayout[1]/android.widget.ScrollView[1]/android.widget.LinearLayout[1]/android.widget.Button[1]")).click();
//try to swipe a list (does not work)
wd.swipe(300, 300, 1, 300, 1500);
Adding to the comment, two things :
1. Make sure the (x,y) coordinates specified by you actually exists on the screen.
2. Reduce the duration to perform the swipe, try this
wd.swipe(300, 300, 1, 300, 10);
3. One crucial thing, definition of the method you are trying to use :
public void swipe(int startx, int starty, int endx, int endy, int duration) {
TouchAction touchAction = new TouchAction(this);
// appium converts press-wait-moveto-release to a swipe action
touchAction.press(startx, starty).waitAction(duration)
.moveTo(endx, endy).release();
touchAction.perform();
}
Looking at this and your inputs :
startx = 300 ; starty = 300 ; endx = 1 ; endy = 300 you seem to be doing a swipe left at height 300, which we shall assume is what you are expecting and if not, the coordinates need to be changed.
For anyone facing a similar problem here is how it got solved,
I simply added a random assertion on the list , since I discovered that the swipe function gets executed instantly unlike the other functions which waits for the previous task to finish before executing (probably a bug).
tldr: I added this before the swipe
String firstName = wd.findElementByName("aerobics 101").getText();
Assert.assertEquals("aerobics 101", firstName);
Example:
[1] wd.findElement(By.xpath("//android.widget.LinearLayout[1]/android.widget.FrameLayout[1]/android.widget.LinearLayout[1]/android.widget.ScrollView[1]/android.widget.LinearLayout[1]/android.widget.EditText[2]")).sendKeys(password);
The system waits 30 seconds between [1] and [2]
[2]
wd.findElement(By.xpath("//android.widget.LinearLayout[1]/android.widget.FrameLayout[1]/android.widget.LinearLayout[1]/android.widget.ScrollView[1]/android.widget.LinearLayout[1]/android.widget.Button[1]")).click();
No wait between [2] and [3] !!!!!!!
[3]
wd.swipe(300, 300, 1, 300, 1500);
First I would take the dimensions of the screen:
Dimension dimension = driver.manage().window().getSize();
After you have this, take the x and y values by using the dimension instance.
int middleX = dimension.getWidth() / 2;
int middleY = dimension.getHeight() / 2;
int quarterY = dimension.getHeight() / 4;
driver.swipe(middleX, middleY, middleX, quarterY , duration );
In the duration I would use 800 or 1000 which is slow enough.
Find 2 corner(left and the right one) elements by name on the current screen and use this method to swipe:
public void swipeScreen(String name, String move) {
WebElement longPress = driver.findElementByName(name);
WebElement moveTo = driver.findElementByName(move);
TouchAction action = new TouchAction(driver).longPress(longPress, 20000).moveTo(moveTo).release();
action.perform();
}
I am trying to implement an opengl picking system I read about and have hit an issue with glReadPixels. Basically, every node in the scene gets a unique color and when a new touch happens, it renders the scene with nothing but the nodes painted with their unique color ids. I am trying to check the input coordinate with a list of stored color ids.
I can not get glReadPixels to work right. It always returns 0 0 0 for pixel values. i would really appreciate any help with getting the correct pixel values from it. thanks
here is the relevant code
private void handleEvent(MotionEvent event) {
int x = (int) event.getX();
int y = (int) event.getY();
int action = event.getAction();
int actionCode = action & MotionEvent.ACTION_MASK;
if (actionCode == 0) {
// Paint with colorID color
mSettings.picking(true);
dumpEvent(event);
final GL10 gl = mSettings.getGL();
ByteBuffer PixelBuffer = ByteBuffer.allocateDirect(4);
PixelBuffer.order(ByteOrder.nativeOrder());
gl. glPixelStorei(gl.GL_UNPACK_ALIGNMENT, 1);
gl.glReadPixels(x, y, 1, 1, GL10.GL_RGBA, GL10.GL_UNSIGNED_BYTE, PixelBuffer);
byte b[] = new byte[4];
PixelBuffer.get(b);
String key = "" + b[0] + b[1] + b[2];
// Check for selection
mRenderer.processSelection(event, new SGColorI(pixel[0], pixel[1], pixel[2], pixel[3]));
log.pl("GL on touchdown", key);
} else if (actionCode == 2) {
mSettings.picking(false);
}
}
allocateDirect "just doesn't work" in this case. Use allocate.
It seemed very strange to me, but allocate vs allocateDirect was the only difference I came to.
Also this post in google groups helped me a lot.
btw, this discovery was made on emulator (several different versions), not the real device.
The use of glReadPixels for the classical pick problem is overkilled. Remember that glReadPixels is a blocking function, while openGL works mainly in an asynchronous way. Using glReadPixels means that all the commands in the driver queue have to be processed and this could be a huge waste of time.