I am doing a generic automation script.
I need to send complex swipe events to the android screen without specifically having access to the focused application(s)
Best way I figured so far is to use adb, create a file with sendevent commands, push it on the device and run it from there. Even that, it is painfully slow (much slower compared to if I record it with getevent and pipe it back in).
I managed to optimize the file since I figured out that each sendevent block does not specifically require both X and Y, but it is still a few orders of magnitude slower
Example of part of the file (I'm trying on a HTC One):
sendevent /dev/input/event5 3 57 49
sendevent /dev/input/event5 3 53 942
sendevent /dev/input/event5 3 54 2747
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 53 1207
sendevent /dev/input/event5 3 54 2483
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 53 1472
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 54 2218
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 53 1207
sendevent /dev/input/event5 3 54 2483
sendevent /dev/input/event5 0 0 0
sendevent /dev/input/event5 3 53 1472
So my focus is to optimize the speed of single long-complex swipes, not of multiple small ones.
Anyone know of a better way to do this?
So, Chris Stratton's idea worked in principle (re-piping the cat-ed output generates the same swipe successfully), but I can't be able to create my own code to pipe it back in. I'm guessing it's something to do with the separators between send event commands... but I still can't get it to work
I used a modification of the sendevent.c file to get a file with triples per line and output to another file. Do you happen to know what could be the issue? Conversion looks good ...
SOLLUTION: I managed to solve it, mostly thanks to the answers bellow. Here is a C script that takes a file with HEX values and outputs the appropriate binary file.
Usage: (for me the touch driver file is /dev/input/event5 - HTC One - for other devices it might be a different file !!!)
$> adb shell getevent > tmp.in
$> ./sendevent tmp.in tmp.out
$> adb shell push tmp.out /mnt/sdcard/
$> adb shell "cd /mnt/sdcard/ && cat tmp.out > /dev/input/event5"
and the source:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <errno.h>
typedef uint32_t __u32;
typedef uint16_t __u16;
typedef __signed__ int __s32;
__attribute__((aligned(1),packed)) struct input_event {
__u32 time_dummy_1;
__u32 time_dummy_2;
__u16 type;
__u16 code;
__s32 value;
};
int convert (char * str) {
return (int) strtol(str, NULL, 16);
}
#define S_ALL (S_IRUSR | S_IWUSR | S_IXUSR | S_IRGRP | S_IWGRP | S_IXGRP | S_IROTH | S_IWOTH | S_IXOTH)
int main (int argc, char *argv[]) {
int i;
int fd;
int ret;
if(argc < 3) {
fprintf(stderr, "use: %s in-file out-file\n", argv[0]);
return 1;
}
fd = open(argv[2], O_CREAT | O_WRONLY, S_ALL);
if(fd < 0) {
fprintf(stderr, "could not open %s, %s\n", argv[2], strerror(errno));
return 1;
}
FILE * fd_in = fopen(argv[1], "r");
if (fd_in == NULL) {
fprintf(stderr, "Can't open input file: %s\n", argv[1]);
return 1;
}
struct input_event event;
char type[32];
char code[32];
char value[32];
int count = 0;
while (fscanf(fd_in, "%s %s %s", type, code, value) != EOF) {
memset(&event, 0, sizeof(event));
// printf("%d) %s %s %s\n", ++count, type, code, value);
event.type = convert(type);
event.code = convert(code);
event.value = convert(value);
memset(type, 0, sizeof(type));
memset(code, 0, sizeof(code));
memset(value, 0, sizeof(value));
ret = write(fd, &event, sizeof(event));
if(ret < sizeof(event)) {
fprintf(stderr, "write event failed, %s\n", strerror(errno));
return -1;
}
}
return 0;
}
Please note that this answer pertains to circa-2013 versions of Android and may not apply to current ones. Jellybean was contemporary at the time, Kitkat came out a few weeks after the question was asked
Your delay is likely a result of inefficiently having to repeatedly launch a new sendevent process, parse the textual event record, and open the device node - for each individual event. If you instead do everything from within a single process, opening the device file only once, it will be much more efficient.
If we look at the source for sendevent in toolbox contemporary with the date of the question (for example, https://android.googlesource.com/platform/system/core/+/jb-release/toolbox/sendevent.c ) we see that the core of what it is doing is encoding the events into binary records
struct input_event {
struct timeval time;
__u16 type;
__u16 code;
__s32 value;
};
and writing them to the appropriate device
memset(&event, 0, sizeof(event));
event.type = atoi(argv[2]);
event.code = atoi(argv[3]);
event.value = atoi(argv[4]);
ret = write(fd, &event, sizeof(event));
Provided that you are executing something as the shell userid or another in the input unix group, you should be able to accomplish the same thing that sendevent does from your own custom program, or using other command line tools like cat, thus efficiently pushing a binary file of event records.
For example
adb shell
cd /mnt/sdcard
cat /dev/input/event2 > events
Do a few touch screen events, then ctrl-C to kill cat
Now you can play back the captured file of binary events:
cat events > /dev/input/event2
(Note: sendevent is zeroing the timeval part of each record; recording and playback may not do that; you'll have to see, and if it matters zero those portions of each record from the file before you write it back)
If you just want to produce linear swipes, you can use input swipe command on shell.
$ adb shell input
usage: input ...
input text <string>
input keyevent <key code number or name>
input [touchscreen|touchpad|touchnavigation] tap <x> <y>
input [touchscreen|touchpad|touchnavigation] swipe <x1> <y1> <x2> <y2> [duration(ms)]
input trackball press
input trackball roll <dx> <dy>
Command below draws a nice line for me in a drawing application
$ adb shell input swipe 300 300 500 1000
and a quicker one
$ adb shell input touchscreen swipe 300 300 500 1000 100
Related
I know about the input tap x y shell command, however, I'm trying to understand how to
perform a click using the sendevent command. I been able to achieve it with the following command:
sendevent /dev/input/event5 3 53 X &&
sendevent /dev/input/event5 3 54 Y &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0
Where X and Y is the position that will be clicked, I'm testing it on the android emulator BlueStacks 5 which the Display Resolution set to 1920x1080.
The code is working and the click is fired, however, I couldn't understand how to convert the position where I want to be clicked to the sendevent XY position.
If I send using ADB:
sendevent /dev/input/event5 3 53 2000 &&
sendevent /dev/input/event5 3 54 2000 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0
It clicks somewhere around x75 y75, how this calc is done? i mean screen xy -> sendevent xy?
How to replicate:
First enable BlueStacks 5 adb in the window: Settings -> Advanced -> Android debug bridge
Open a cmd window and run cd C:\Program Files\BlueStacks_nxt assuming BlueStacks where installed in the default path.
Execute the commands:
hd-adb.exe connect 127.0.0.1:X where X is the port shown in the window where you enabled the ADB.
hd-adb.exe -s 127.0.0.1:X shell
Now we are on the shell, execute a new command: getevent -p
and search for:
... /dev/input/event5
name: "BlueStacks Virtual Touch"
On my emulator the input event for touch is event5 on yours it can be different, replace it according.
Now you can simulate a click with the code below changing XY to the position where you want to be clicked:
sendevent /dev/input/event5 3 53 X &&
sendevent /dev/input/event5 3 54 Y &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0
I'm trying to figure out how to convert the emulator screen position to the sendevent position.
For example, if you want to perform a click at x200 y200, using sendevent what the value needed?
How to calculate it?
thanks for the very precise instructions to reproduce:
I enabled
Settings -> Advanced -> Input debugging -> Show visual feedback for taps
and
Settings -> Advanced -> Input debugging -> Show pointer location for current touch data
when I hold down click, I can see: X: Y:
I collected these x coordinate points, my max X: is 1600.0 so my width is 1600.0
(8000, 390.6)
(16000, 781.2)
(32000, 1562.5)
then Excel: X Y (scatter) chart, add trendline, click trendline and click big + sign -> Chart Elements -> Trendline : (tick that) and ▶, More Options... -> Trendline Options -> Display Equation on chart
, click on the formula, then Label Options -> Category: Number, Decimal Places: 10
y = 0.0488294643x - 0.0500000000
1600 = 0.0488294643x - 0.0500000000
32768.1252075501 = X
I round to 32768 because 32768 is a magic number, close to Int16's 32767
edit: after Nathan's comment, it's 32767 (32768 doesn't even move the cursor)
so the formula is : (W: width, H: height)
32767*X/W
32767*Y/H
for your (X=200, Y=200, W=1980, H=1080)
32767*200/1920
32767*200/1080
3413.22916666667
6067.96296296296
I used this code to test: it doesn't do a tap, it holds down without releasing
sendevent /dev/input/event5 3 57 0
sendevent /dev/input/event5 3 53 3413.22916666667
sendevent /dev/input/event5 3 54 6067.96296296296
sendevent /dev/input/event5 3 48 5
sendevent /dev/input/event5 3 58 50
sendevent /dev/input/event5 0 2 0
sendevent /dev/input/event5 0 0 0
https://ktnr74.blogspot.com/2013/06/emulating-touchscreen-interaction-with.html#:~:text=ABS_MT_TRACKING_ID%20(57)%20%2D%20ID,end%20of%20report
I'm using the code above to simulate a 'swipe' using sendevent:
sendevent /dev/input/event0 3 53 300 ;First position X
sendevent /dev/input/event0 3 54 600 ;First position Y
sendevent /dev/input/event0 3 48 5
sendevent /dev/input/event0 3 58 50
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0
sendevent /dev/input/event0 3 53 300 ;Second position X
sendevent /dev/input/event0 3 54 400 ;Second position Y
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0
However, it does swipe instantly without any delay.
I'm trying to figure how to specify the duration of the swipe, like you can do using adb shell input:
input [touchscreen|touchpad|touchnavigation] swipe <x1> <y1> <x2> <y2> [duration(ms)]
shell input swipe 300 400 300 200 2000
This produces a swipe with a duration of 2 seconds.
I have tried to add a
sleep 2 before the ;Second position but it does result in a pause before the swipe instead of a swipe with 2 seconds of duration.
With duration I mean, the time slowly swapping from position 1 to position 2.
The problem with this is that sending events through sendevent takes some time. I made a python script (You can take whatever you need from there) that interpolates points between the given ones. It also waits some time between points.
This is the lineal interpolation code:
def lerp(p1:tuple, p2:tuple, points:int) -> list:
output = []
header = [_p2 - _p1 for _p1, _p2 in zip(p1, p2)]
for p in range(points + 1):
percent = p / points
output.append((p1[0] + percent * header[0], p1[1] + percent * header[1]))
return output
The time problem appears when using multiples points. Using a path with 10 interpolated points with no time between them already takes 1.29 seconds and a 100 points one, 11.45.
If you compare the sendevent and input commands' source code you can clearly understand their goals; the former covers basic command line's input events whereas the latter covers more flexible and complex input scenarios.
To get an insight on how the swipe duration has been implemented (on input command) you can focus directly on the sendSwipe method: it sends multiple basic input events, leveraging the InputManager, in a timespan defined by the duration parameter
final long endTime = down + duration;
The function injectMotionEvent used by sendSwipe doesn't have any concept of "duration".
That said, I think the command you're looking for, as of today, doesn't exist and I believe you can still rely on console prompt like
input swipe 300 400 300 200 2000
that can be invoked after using
adb shell
One cannot set the duration on the low level, but one can record analog input and then play it back. This permits for more flexible and complex scenarios ...because the events are countless.
Run adb shell to open a shell.
Where ...
getevent --help shows all available options.
getevent -p shows all recordable devices.
getevent -lp /dev/input/event1 shows BTN_TOUCH event data format.
getevent /dev/input/event1 logs input events for device focaltech_ts.
getevent -l /dev/input/event1 is human-readable (useless for automation).
To record:
cd sdcard/Download
getevent /dev/input/event1 >> ./swipe.log
download swipe.log with the Android device explorer.
Where 0003 means coordinate, and 0x35 is the X-axis and 0x36 is the Y-axis:
0003 0035 000001a8
0003 0036 000005cb
This log can the be played back by a shell script loop, with sendevent.
sendevent --help shows the expected parameters: DEVICE TYPE CODE VALUE.
cat ./swipe.log | while read line
do
adb shell sendevent /dev/input/event1 $line
done
When delaying the execution with sleep, the lines with 0000 0000 00000000 might suit best.
Alike this one can also automate GPIO buttons, which maybe be quite specific on certain devices. UiObject2.swipe() might also just generate linear-interpolation coordinates and play them back. It generally does not matter, if they're generated or recorded - the only difference is that the one movement is perfectly straight and the other one obviously isn't.
I'm developing a standalone kiosk using an android tablet(iBall running on 4.2.2).Its has the chinese MTK in it.
Suppose when there is no power,then eventually the tab's battery will drain out and have no juice left in it.When the power comes back I want the tab to automatically bootup without any manual intervention.I read online that if we modify the code present in the battery animation file we can achieve this.For the same,I replaced the original code of the battery charging animation file called 'ipod' located at '/system/bin' with :
#!/system/bin/sh
/system/bin/reboot
However,when my tab was shutdown and docked it didnt boot-up,instead it was just stuck at the charging logo.When I replaced the above code with:
/system/bin/reboot
my tab did boot-up when it was shutdown and docked.This means my code was getting stuck at '#!/system/bin/sh' . What could be the reason?
Also,while booting up the tab using the above process I want to boot it up after a delay,for which I used
sleep 20
/system/bin/reboot
but there was no delay in the bootup process(irrespective of the value of sleep that I give)
How do I create this delay ?
PS: I gave 777 permission to the file; owner-root; group-shell.
Kindly assist.Many thanks !
Found the reason my commands were not executing.
The reason was because,I was editing on my notepad++ on windows,until I came across this answer on SO -
" Make sure your text editor is not putting a /r /n and only a /n for every new line. This is typical if you are writing the script on windows.Use notepad++ (windows) and go to edit|EOL convention|UNIX then save it. "
So I changed my convention as per the above answer and ran my code and got the desired result.
I also had the need to make a device (alcatel9002x) autoboot on charger plugged, and come up with a solution replacing the ipod file. It runs the original ipod binary and at the same time, simulate pressing on the power button.
I see the charging animation, but after, it does a "normal" power up.
You probably can do this with a script but I did with with a .c binary.
Solution:
move /sbin/ipod to /sbin/ipod.backup
And use this code as the new binary /sbin/ipod
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <unistd.h>
void run_cmd(char *s)
{
FILE *handle;
char buf[64];
size_t readn;
handle = popen(s, "r");
if (handle == NULL) {
return;
}
while ((readn = fread(buf, 1, sizeof(buf), handle)) > 0) {
fwrite(buf, 1, readn, stdout);
}
pclose(handle);
}
int main(void)
{
FILE *handle;
char buf[64];
size_t readn;
pid_t pid = fork();
if (pid == 0)
{
for(int i=0;i<10;i++)
{
run_cmd("/system/bin/sendevent /dev/input/event0 0001 116 1");
run_cmd("/system/bin/sendevent /dev/input/event0 0000 0000 00000000");
run_cmd("/system/bin/sleep 2");
run_cmd("/system/bin/sendevent /dev/input/event0 0001 116 00000000");
run_cmd("/system/bin/sendevent /dev/input/event0 0000 0000 00000000");
sleep(1);
}
}
else
{
run_cmd("/system/bin/sleep 2");
run_cmd("/system/bin/ipod.backup");
}
return 0;
}
I've compiled it with arm-linux-androideabi-gcc.
As I said, you probably can do this in a bash script, but this is how it worked for me.
Don't forget to chmod 667 and chown root:shell the binaries.
In Sony Xperia GO, the file name is "chargemon" and only by renaming it, the task is done (smartphone will be restart after plug in). (maybe other Sony brand smartphones be the same or similar)
I am trying to get the (x, y) coordinate of the touch through the pointer location option in Developer Options and I use these coordinates to tap on the screen using sendevent. Here is my script that does the sendevent.
tap.sh
sendevent /dev/input/event0 3 57 2421
sendevent /dev/input/event0 3 58 232
sendevent /dev/input/event0 3 53 $1
sendevent /dev/input/event0 3 54 $2
sendevent /dev/input/event0 0 0 0
sendevent /dev/input/event0 3 57 4294967295
sendevent /dev/input/event0 0 0 0
I call the script from adb shell sh tap.sh <x> <y> but it is not tapping on the right coordinate. Instead it is tapping at a different location.
Also when I tap on the screen and check the result in getevent adb shell getevent. I find that the coordinates that is shown on the pointer location and the getevent are different.
Why are they different and how do I solve this issue?
PS: The devices I tried are Nexus 7, Nexus 10.
The X and Y co-ordinates obtained from the getevent and the ones obtained from the pointer location in developer options are not the same. They are mapped using a formula.
displayX = (x - minX) * displayWidth / (maxX - minX + 1)
displayY = (y - minY) * displayHeight / (maxY - minY + 1)
Source: Touch Devices
Turn on developer options and enable Pointer Location and you can see the x and y coordinates on top of the screen when you tap on the screen use those coordinates to send tap events.
Are you aware that getevent (in my experience, this possibly varies between devices) shows base 16 values?
(side note: getevent -l is often easier to read as it prints a string representation of the event types)
i.e. if getevent -l says
/dev/input/event1: EV_ABS ABS_MT_POSITION_X 000001cb
/dev/input/event1: EV_ABS ABS_MT_POSITION_Y 00000376
the position of the touch is (459, 886) actually
however it appears that sendevent is not following suit in requiring hex values if your code works at all, as your (such as) 53 and 54 work where I would have used
0035 and 0036.
Edit:
Having tried the original code on a Nexus 5 (correct device file substituted in), I have found that no touch event is generated (nor when the hexadecimal equivalent is substituted, for experimental rigor), nor from reusing values captured (and converted) from getevent. Previously, I have had better experience converting the events with a Python script based on the C one here, and writing the output directly to the device file.
Edit 2:
This question here suggests that the initial code should work.
I have tried following code snippet to execute the batch of command for sendevent to click the coordinate 44,129 on the emulator. But it is not showing any result. But if i am giving the same batch of command to the shell prompt it is able to click the mentioned coordinate succesfully.
String[] cmmandemulatorarr = {"/system/bin/sendevent /dev/input/event0 3 0 44",
"/system/bin/sendevent /dev/input/event0 3 1 129",
"/system/bin/sendevent /dev/input/event0 1 330 1",
"/system/bin/sendevent /dev/input/event0 0 0 0",
"/system/bin/sendevent /dev/input/event0 1 330 0",
"/system/bin/sendevent /dev/input/event0 0 0 0", };
for (int i = 0; i < cmmandemulatorarr.length; i++) {
Process process =
Runtime.getRuntime().exec(cmmandemulatorarr[i]);
BufferedReader reader = new BufferedReader(
new InputStreamReader(process.getInputStream()));
int read;
char[] buffer = new char[4096];
StringBuffer output = new StringBuffer();
while ((read = reader.read(buffer)) > 0) {
output.append(buffer, 0, read);
}
reader.close();
}
} catch (IOException e) {
throw new RuntimeException(e);
}
Is there is anything i am missing here or i have to try something else to get click event on some coordinate through the code.
Note :: I am not getting any exception in the log while executing the code which seems that command is executed successfully.
Regards
Pinu
But it is not showing any result.
This is a good thing.
But if i am giving the same batch of command to the shell prompt it is able to click the mentioned coordinate succesfully.
The shell runs with root-level privileges. Your SDK application does not, unless you root your device and arrange to execute your code that way.
Bear in mind that not all devices will have a /system/bin/sendevent command and it can be removed at any time. This is not part of the Android SDK.
i have to try something else to get click event on some coordinate through the code.
This is not possible from the Android SDK for ordinary devices, for obvious security reasons.
Here you have answer. You need to find touch event id with getevent. Sendevent use decimal space, getevent use hex. This code from first "answer" works on 7.0.