I am trying to get the (x, y) coordinate of the touch through the pointer location option in Developer Options and I use these coordinates to tap on the screen using sendevent. Here is my script that does the sendevent.
tap.sh
sendevent /dev/input/event0 3 57 2421
sendevent /dev/input/event0 3 58 232
sendevent /dev/input/event0 3 53 $1
sendevent /dev/input/event0 3 54 $2
sendevent /dev/input/event0 0 0 0
sendevent /dev/input/event0 3 57 4294967295
sendevent /dev/input/event0 0 0 0
I call the script from adb shell sh tap.sh <x> <y> but it is not tapping on the right coordinate. Instead it is tapping at a different location.
Also when I tap on the screen and check the result in getevent adb shell getevent. I find that the coordinates that is shown on the pointer location and the getevent are different.
Why are they different and how do I solve this issue?
PS: The devices I tried are Nexus 7, Nexus 10.
The X and Y co-ordinates obtained from the getevent and the ones obtained from the pointer location in developer options are not the same. They are mapped using a formula.
displayX = (x - minX) * displayWidth / (maxX - minX + 1)
displayY = (y - minY) * displayHeight / (maxY - minY + 1)
Source: Touch Devices
Turn on developer options and enable Pointer Location and you can see the x and y coordinates on top of the screen when you tap on the screen use those coordinates to send tap events.
Are you aware that getevent (in my experience, this possibly varies between devices) shows base 16 values?
(side note: getevent -l is often easier to read as it prints a string representation of the event types)
i.e. if getevent -l says
/dev/input/event1: EV_ABS ABS_MT_POSITION_X 000001cb
/dev/input/event1: EV_ABS ABS_MT_POSITION_Y 00000376
the position of the touch is (459, 886) actually
however it appears that sendevent is not following suit in requiring hex values if your code works at all, as your (such as) 53 and 54 work where I would have used
0035 and 0036.
Edit:
Having tried the original code on a Nexus 5 (correct device file substituted in), I have found that no touch event is generated (nor when the hexadecimal equivalent is substituted, for experimental rigor), nor from reusing values captured (and converted) from getevent. Previously, I have had better experience converting the events with a Python script based on the C one here, and writing the output directly to the device file.
Edit 2:
This question here suggests that the initial code should work.
Related
I know about the input tap x y shell command, however, I'm trying to understand how to
perform a click using the sendevent command. I been able to achieve it with the following command:
sendevent /dev/input/event5 3 53 X &&
sendevent /dev/input/event5 3 54 Y &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0
Where X and Y is the position that will be clicked, I'm testing it on the android emulator BlueStacks 5 which the Display Resolution set to 1920x1080.
The code is working and the click is fired, however, I couldn't understand how to convert the position where I want to be clicked to the sendevent XY position.
If I send using ADB:
sendevent /dev/input/event5 3 53 2000 &&
sendevent /dev/input/event5 3 54 2000 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0
It clicks somewhere around x75 y75, how this calc is done? i mean screen xy -> sendevent xy?
How to replicate:
First enable BlueStacks 5 adb in the window: Settings -> Advanced -> Android debug bridge
Open a cmd window and run cd C:\Program Files\BlueStacks_nxt assuming BlueStacks where installed in the default path.
Execute the commands:
hd-adb.exe connect 127.0.0.1:X where X is the port shown in the window where you enabled the ADB.
hd-adb.exe -s 127.0.0.1:X shell
Now we are on the shell, execute a new command: getevent -p
and search for:
... /dev/input/event5
name: "BlueStacks Virtual Touch"
On my emulator the input event for touch is event5 on yours it can be different, replace it according.
Now you can simulate a click with the code below changing XY to the position where you want to be clicked:
sendevent /dev/input/event5 3 53 X &&
sendevent /dev/input/event5 3 54 Y &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0
I'm trying to figure out how to convert the emulator screen position to the sendevent position.
For example, if you want to perform a click at x200 y200, using sendevent what the value needed?
How to calculate it?
thanks for the very precise instructions to reproduce:
I enabled
Settings -> Advanced -> Input debugging -> Show visual feedback for taps
and
Settings -> Advanced -> Input debugging -> Show pointer location for current touch data
when I hold down click, I can see: X: Y:
I collected these x coordinate points, my max X: is 1600.0 so my width is 1600.0
(8000, 390.6)
(16000, 781.2)
(32000, 1562.5)
then Excel: X Y (scatter) chart, add trendline, click trendline and click big + sign -> Chart Elements -> Trendline : (tick that) and ▶, More Options... -> Trendline Options -> Display Equation on chart
, click on the formula, then Label Options -> Category: Number, Decimal Places: 10
y = 0.0488294643x - 0.0500000000
1600 = 0.0488294643x - 0.0500000000
32768.1252075501 = X
I round to 32768 because 32768 is a magic number, close to Int16's 32767
edit: after Nathan's comment, it's 32767 (32768 doesn't even move the cursor)
so the formula is : (W: width, H: height)
32767*X/W
32767*Y/H
for your (X=200, Y=200, W=1980, H=1080)
32767*200/1920
32767*200/1080
3413.22916666667
6067.96296296296
I used this code to test: it doesn't do a tap, it holds down without releasing
sendevent /dev/input/event5 3 57 0
sendevent /dev/input/event5 3 53 3413.22916666667
sendevent /dev/input/event5 3 54 6067.96296296296
sendevent /dev/input/event5 3 48 5
sendevent /dev/input/event5 3 58 50
sendevent /dev/input/event5 0 2 0
sendevent /dev/input/event5 0 0 0
https://ktnr74.blogspot.com/2013/06/emulating-touchscreen-interaction-with.html#:~:text=ABS_MT_TRACKING_ID%20(57)%20%2D%20ID,end%20of%20report
I'm using the code above to simulate a 'swipe' using sendevent:
sendevent /dev/input/event0 3 53 300 ;First position X
sendevent /dev/input/event0 3 54 600 ;First position Y
sendevent /dev/input/event0 3 48 5
sendevent /dev/input/event0 3 58 50
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0
sendevent /dev/input/event0 3 53 300 ;Second position X
sendevent /dev/input/event0 3 54 400 ;Second position Y
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0
However, it does swipe instantly without any delay.
I'm trying to figure how to specify the duration of the swipe, like you can do using adb shell input:
input [touchscreen|touchpad|touchnavigation] swipe <x1> <y1> <x2> <y2> [duration(ms)]
shell input swipe 300 400 300 200 2000
This produces a swipe with a duration of 2 seconds.
I have tried to add a
sleep 2 before the ;Second position but it does result in a pause before the swipe instead of a swipe with 2 seconds of duration.
With duration I mean, the time slowly swapping from position 1 to position 2.
The problem with this is that sending events through sendevent takes some time. I made a python script (You can take whatever you need from there) that interpolates points between the given ones. It also waits some time between points.
This is the lineal interpolation code:
def lerp(p1:tuple, p2:tuple, points:int) -> list:
output = []
header = [_p2 - _p1 for _p1, _p2 in zip(p1, p2)]
for p in range(points + 1):
percent = p / points
output.append((p1[0] + percent * header[0], p1[1] + percent * header[1]))
return output
The time problem appears when using multiples points. Using a path with 10 interpolated points with no time between them already takes 1.29 seconds and a 100 points one, 11.45.
If you compare the sendevent and input commands' source code you can clearly understand their goals; the former covers basic command line's input events whereas the latter covers more flexible and complex input scenarios.
To get an insight on how the swipe duration has been implemented (on input command) you can focus directly on the sendSwipe method: it sends multiple basic input events, leveraging the InputManager, in a timespan defined by the duration parameter
final long endTime = down + duration;
The function injectMotionEvent used by sendSwipe doesn't have any concept of "duration".
That said, I think the command you're looking for, as of today, doesn't exist and I believe you can still rely on console prompt like
input swipe 300 400 300 200 2000
that can be invoked after using
adb shell
One cannot set the duration on the low level, but one can record analog input and then play it back. This permits for more flexible and complex scenarios ...because the events are countless.
Run adb shell to open a shell.
Where ...
getevent --help shows all available options.
getevent -p shows all recordable devices.
getevent -lp /dev/input/event1 shows BTN_TOUCH event data format.
getevent /dev/input/event1 logs input events for device focaltech_ts.
getevent -l /dev/input/event1 is human-readable (useless for automation).
To record:
cd sdcard/Download
getevent /dev/input/event1 >> ./swipe.log
download swipe.log with the Android device explorer.
Where 0003 means coordinate, and 0x35 is the X-axis and 0x36 is the Y-axis:
0003 0035 000001a8
0003 0036 000005cb
This log can the be played back by a shell script loop, with sendevent.
sendevent --help shows the expected parameters: DEVICE TYPE CODE VALUE.
cat ./swipe.log | while read line
do
adb shell sendevent /dev/input/event1 $line
done
When delaying the execution with sleep, the lines with 0000 0000 00000000 might suit best.
Alike this one can also automate GPIO buttons, which maybe be quite specific on certain devices. UiObject2.swipe() might also just generate linear-interpolation coordinates and play them back. It generally does not matter, if they're generated or recorded - the only difference is that the one movement is perfectly straight and the other one obviously isn't.
The MonkeyDevice.drag currently can takes only 2 sets of co-ordinates which means I can move from (x1,y1) to (x2,y2). Is there a way I could make the monkey runner simulate a drag from (x1,y1) to (x3,y3) and then to (x2,y2) without breaking the same drag function?
Monkeyrunner is kind of deprecated.
You can use uiautomator instead. More details can be found here:
http://developer.android.com/tools/testing/testing_ui.html
This tool has a method for what you seek, a custom swipe using an array of points (x, y coordinates). More on that here:
http://developer.android.com/tools/help/uiautomator/UiDevice.html#swipe(android.graphics.Point[], int)
I would use Get/Send Event.
Get an event, e.g. touch screen:
issue command: adb shell getevent | grep event2 > CaptureEvent.txt
do the touchscreen activity
stop the recording with CTRL+C
Convert the event:
e.g.: /dev/input/event1 3 47 0
Send an event:
issue commands, e.g.: adb shell sendevent /dev/input/event1 3 47 0
I'm struggling with calibration of a touchscreen on Android plataform.
It is an USB Single-Touch Touchscreen from vendor 0dfc and product 0001 as checked with dmesg:
<6>[ 4118.091541] input: USB Touchscreen 0dfc:0001 as /devices/platform/usb20_host/usb2/2-1/2-1.3/2-1.3:1.0/input/input23
I'm pushing the Vendor_0dfc_Product_0001.idc file /data/system/devices/idc/ (following the documentation from android source - IDC
I got the touch device with all requirements for single touch events:
root#android:/ # getevent -il /dev/input/event3
add device 1: /dev/input/event3
bus: 0003
vendor 0dfc
product 0001
version 0202
name: "USB Touchscreen 0dfc:0001"
location: "usb-usb20_host-1.3/input0"
id: ""
version: 1.0.1
events:
KEY (0001): BTN_TOUCH
ABS (0003): ABS_X : value 540, min 0, max 32767, fuzz 0, flat 0, resolution 0
ABS_Y : value 289, min 0, max 32767, fuzz 0, flat 0, resolution 0
input props:
<none>
I also enabled the Pointer Location option from Developer options (Android settings) in order to debug this stage of calibration.
Setup 1
touch.deviceType = touchScreen
With this setup (1) all the gestures on the touchscreen take place at the up-left corner - just a few pixels left/right/up/down no matter the gesture (swipe). All the touchscreen get events. All the gestures are reversed - when swipe left the pointer goes right; when swipe up, the pointer goes down.
Setup 2
touch.deviceType = pointer
touch.gestureMode = pointer
With this setup (2), as expected, it shows a pointer, placed at the position from the last pointer device left (mouse). All the gestures on the touchscreen (no matter the swipe size) keep beaving like setup 1 - move only a few pixels with each swipe event, and with reversed axis.
Setup 3
touch.deviceType = pointer
touch.gestureMode = spots
With this setup (3) the result is the same as setup 2. I just did that to prove that the IDC file is being interpreted correctly.
At this stage, as you can check by now, I have a working IDC file (setup 1) requiring calibration for this touch device.
I tried a lot of combinations from other IDC files (internet samples) and from android source - IDC - ANY OTHER PROPERTY TOOK EFFECT (NOT A SINGLE ONE) - raw.*, output.*, touch.size.*
Does anyone knows how to calibrate properly a touch screen in Android that could guide me in this process?
Same here,
but my calibration app did't do anything.
After a while, reading /system/etc/init.sh i found the following:
mkdir -p /data/misc/tscal
touch /data/misc/tscal/pointercal
chown 1000.1000 /data/misc/tscal /data/misc/tscal/*
chmod 775 /data/misc/tscal
chmod 664 /data/misc/tscal/pointercal
Just run those commands manually, reboot, and start the calibration app
I am doing a generic automation script.
I need to send complex swipe events to the android screen without specifically having access to the focused application(s)
Best way I figured so far is to use adb, create a file with sendevent commands, push it on the device and run it from there. Even that, it is painfully slow (much slower compared to if I record it with getevent and pipe it back in).
I managed to optimize the file since I figured out that each sendevent block does not specifically require both X and Y, but it is still a few orders of magnitude slower
Example of part of the file (I'm trying on a HTC One):
sendevent /dev/input/event5 3 57 49
sendevent /dev/input/event5 3 53 942
sendevent /dev/input/event5 3 54 2747
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 53 1207
sendevent /dev/input/event5 3 54 2483
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 53 1472
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 54 2218
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 53 1207
sendevent /dev/input/event5 3 54 2483
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 53 1472
So my focus is to optimize the speed of single long-complex swipes, not of multiple small ones.
Anyone know of a better way to do this?
So, Chris Stratton's idea worked in principle (re-piping the cat-ed output generates the same swipe successfully), but I can't be able to create my own code to pipe it back in. I'm guessing it's something to do with the separators between send event commands... but I still can't get it to work
I used a modification of the sendevent.c file to get a file with triples per line and output to another file. Do you happen to know what could be the issue? Conversion looks good ...
SOLLUTION: I managed to solve it, mostly thanks to the answers bellow. Here is a C script that takes a file with HEX values and outputs the appropriate binary file.
Usage: (for me the touch driver file is /dev/input/event5 - HTC One - for other devices it might be a different file !!!)
$> adb shell getevent > tmp.in
$> ./sendevent tmp.in tmp.out
$> adb shell push tmp.out /mnt/sdcard/
$> adb shell "cd /mnt/sdcard/ && cat tmp.out > /dev/input/event5"
and the source:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <errno.h>
typedef uint32_t __u32;
typedef uint16_t __u16;
typedef __signed__ int __s32;
__attribute__((aligned(1),packed)) struct input_event {
__u32 time_dummy_1;
__u32 time_dummy_2;
__u16 type;
__u16 code;
__s32 value;
};
int convert (char * str) {
return (int) strtol(str, NULL, 16);
}
#define S_ALL (S_IRUSR | S_IWUSR | S_IXUSR | S_IRGRP | S_IWGRP | S_IXGRP | S_IROTH | S_IWOTH | S_IXOTH)
int main (int argc, char *argv[]) {
int i;
int fd;
int ret;
if(argc < 3) {
fprintf(stderr, "use: %s in-file out-file\n", argv[0]);
return 1;
}
fd = open(argv[2], O_CREAT | O_WRONLY, S_ALL);
if(fd < 0) {
fprintf(stderr, "could not open %s, %s\n", argv[2], strerror(errno));
return 1;
}
FILE * fd_in = fopen(argv[1], "r");
if (fd_in == NULL) {
fprintf(stderr, "Can't open input file: %s\n", argv[1]);
return 1;
}
struct input_event event;
char type[32];
char code[32];
char value[32];
int count = 0;
while (fscanf(fd_in, "%s %s %s", type, code, value) != EOF) {
memset(&event, 0, sizeof(event));
// printf("%d) %s %s %s\n", ++count, type, code, value);
event.type = convert(type);
event.code = convert(code);
event.value = convert(value);
memset(type, 0, sizeof(type));
memset(code, 0, sizeof(code));
memset(value, 0, sizeof(value));
ret = write(fd, &event, sizeof(event));
if(ret < sizeof(event)) {
fprintf(stderr, "write event failed, %s\n", strerror(errno));
return -1;
}
}
return 0;
}
Please note that this answer pertains to circa-2013 versions of Android and may not apply to current ones. Jellybean was contemporary at the time, Kitkat came out a few weeks after the question was asked
Your delay is likely a result of inefficiently having to repeatedly launch a new sendevent process, parse the textual event record, and open the device node - for each individual event. If you instead do everything from within a single process, opening the device file only once, it will be much more efficient.
If we look at the source for sendevent in toolbox contemporary with the date of the question (for example, https://android.googlesource.com/platform/system/core/+/jb-release/toolbox/sendevent.c ) we see that the core of what it is doing is encoding the events into binary records
struct input_event {
struct timeval time;
__u16 type;
__u16 code;
__s32 value;
};
and writing them to the appropriate device
memset(&event, 0, sizeof(event));
event.type = atoi(argv[2]);
event.code = atoi(argv[3]);
event.value = atoi(argv[4]);
ret = write(fd, &event, sizeof(event));
Provided that you are executing something as the shell userid or another in the input unix group, you should be able to accomplish the same thing that sendevent does from your own custom program, or using other command line tools like cat, thus efficiently pushing a binary file of event records.
For example
adb shell
cd /mnt/sdcard
cat /dev/input/event2 > events
Do a few touch screen events, then ctrl-C to kill cat
Now you can play back the captured file of binary events:
cat events > /dev/input/event2
(Note: sendevent is zeroing the timeval part of each record; recording and playback may not do that; you'll have to see, and if it matters zero those portions of each record from the file before you write it back)
If you just want to produce linear swipes, you can use input swipe command on shell.
$ adb shell input
usage: input ...
input text <string>
input keyevent <key code number or name>
input [touchscreen|touchpad|touchnavigation] tap <x> <y>
input [touchscreen|touchpad|touchnavigation] swipe <x1> <y1> <x2> <y2> [duration(ms)]
input trackball press
input trackball roll <dx> <dy>
Command below draws a nice line for me in a drawing application
$ adb shell input swipe 300 300 500 1000
and a quicker one
$ adb shell input touchscreen swipe 300 300 500 1000 100