Is there are way to fake mouse and keyboard input on Android?
I developed an application for controlling computers with a smartphone client and since smart TVs become more interesting for people I would also be very interested in providing a server for Android.
I already discovered monkeyrunner but I'm not sure whether this is really what I want since it is a development tool. A native way would be far more attractive.
Has anyone already experimented with this? I use Qt so a C++ way would be great. I have also seen someone faking key inputs per command line is there a way to execute command line commands in an Android application?
Is there are way to fake mouse and keyboard input on Android?
Only via custom firmware, a rooted device, or via the Android SDK tools. Apps cannot attack other apps via "fake mouse and keyboard input" for obvious security reasons.
I have also seen someone faking key inputs per command line is there a way to execute command line commands in an Android application?
Fortunately, that will not work, unless you run as root.
Related
Coming from a desktop and web background, I am familiar with tools that allow me to record mouse movements and keystrokes for Functional test.
What for Android apps? Is there any similar tool or tools? I assume with a device connected to your PC/Mac (emulator), but also simply on the device itself.
For the later, which is what I'm trying to accomplish, I would like to be able to record movements on the screen (just as in the eclipse emulator), i.e. swiping up, down, button on press, on release, etc., and simply have it run on the device itself without being connected to a computer, similar to being able to record a script (functional) and run it over and over, having this installed as an app on the device to allow me to record and run functional scripts as an overlay on top of an app in development?
Like being able to run something like MonkeyTalk directly on the device.
Think of being able to record and run Selenium functional scripts directly on and from the device.
Is there an overlay type of idea that would work? I've heard there are some security risks in this.
Looking for a solution please.
I suppose Espresso Test Recorder is exactly what you are looking for.
Is it possible to simulate a USB or Bluetooth device connected to Android?
I would like to make an app which is able to simulate an HID device locally. What I mean by this is: the app should make Android believe that an USB/Bluetooth HID device is connected. This would allow my app to inject touch events globally, I hope. Is there anyway to simulate a (virtual) device? Note that I don't really care about the device, I just want to use the built-in support for HID input.
I know a lot of people already asked about touch event injections but this approach seems to be a little bit different. I do not want to use this for testing purpose, so InstrumentationTestCase and the like won't help.
Rooting the device might be an option, although I can also imagine to ask my endusers to install a specifically signed app manually (according to https://stackoverflow.com/a/16737083/2923406 and https://stackoverflow.com/a/22902808/2923406 this works, but it would be device specific).
I am aware of:
Simulate a mouse input on android - This guy seems to do it in his own app, although his questions wasn't answered ;)
http://www.pocketmagic.net/2013/01/programmatically-injecting-events-on-android-part-2/#.U58SqfldVHV - Needs Root. Did anyone succeeded in using this? Also, the solution seems very device-specific again.
Bluetooth + simulating mouse - That's not what I want. The phone itself should not be an HID device, but use the (virtually created) one itself. Same for this:
https://stackoverflow.com/a/8174973/2923406
Note that I do not want to turn my phone into an HID device of any kind.
Simulating is a broad term. When I created my Bluetooth app I "simulated" a Bluetooth connection in two ways.
Way 1: Use a serial port UART converter and hook it up to a Bluetooth module transmitter.
Then you can use a terminal program like CoolTerm, to send your data.
Like so. In this case I coded in a string to send on successful connection with the device however you can make a infinite loop for testing purposes effectively not requiring your phone to be turned into a HID device.
Way 2 (not easy): Use your computers bluetooth in a server/client relashionship model.
This is harder to do. What you can do is convert your pc/mac into a server and the Android phone into a client or vice-versa. For this you will need to write external code which will need to be compiled separately on a jvm(java virtual machine). The procedure to do this can be found here. If you are using a Linux machine you have to separately download the Bluez module. I have not tried this on any other operating sytem other than Ubuntu, and it was a pain to get functional.
Hopefully that helped.
Yes, it's quite easy using the AOA2 protocol check this & this links for details ( you'll need to switch your device to the Accessory mode )
This may be possible (or at least be easier) using the Robolectric library, which simulates a full Android device locally. Although it is intended primarily for testing, the fact that it simulates a whole device locally - including Bluetooth and USB - means adding to it may be an easier approach.
In other words, you may be able to modify the classes it uses to simulate these abilities locally (i.e. in the IDE itself without an emulator or device) in order to simulate them on the device itself. After all, it does provide full simulations of these functions. You could simply change these Bluetooth and USB simulating classes to load onto the device itself rather than onto the local Robolectric test "device."
This is just an idea though - I can't confirm this will work - it just might be a good place for you to start.
I am currently working on a research project which involves people playing games on an Android device. I am hoping to be able to write an android app that records the accelerometer and touch events. I would like my users to be able to play games such as angry birds, whilst I record their touching data.
I understand that this type of data collection is possible from inside the app, but is it possible from outside the app? (perhaps via an app running in the background?)
If this is not possible, are there alternatives? (I believe I could theoretically go into the android OS source code and make this happen?)
I understand that this type of data collection is possible from inside the app, but is it possible from outside the app? (perhaps via an app running in the background?)
For touch events, no, for obvious privacy and security reasons. It used to be possible (research the term "tapjacking"), but current versions of Android finally blocked this behavior.
For accelerometer events, you can record those, because they are the same for all apps simultaneously. Bear in mind that your sampling rate may not exactly match that of the app being used.
I believe I could theoretically go into the android OS source code and make this happen?
Yes, though you would then need to turn that modified Android into a ROM mod and install it on devices.
Apps are not allowed to do this on a secured device (ie, consumer phone/tablet that has not been 'rooted').
However, depending on your needs it may be possible with the development tools connected to a computer. If you can do some moderately annoying setup before each controlled-circumstances trial, I believe you can do it on some stock devices by using the USB cable to switch ADB into wifi mode, then monitoring input events over a wireless ADB session. See
http://source.android.com/tech/input/getevent.html
That document seems to imply that 'su' is needed (which is odd as the official 'su' isn't usable by non-root users), however it works without on many stock devices.
You'd probably also want to be running logcat to figure out what application is in the foreground.
I would like to write some sort of macro app that presses keys. Would this be possible on IOS and/or Android?
Yes, for android, have you tried MonkeyRunner
The monkeyrunner tool provides an API for writing programs that
control an Android device or emulator from outside of Android code.
With monkeyrunner, you can write a Python program that installs an
Android application or test package, runs it, sends keystrokes to it,
takes screenshots of its user interface, and stores screenshots on the
workstation.
I am aiming to find out different security concerns that can occur on Android devices .One of them is the Keyboard Interception
Following are my findings
a) If i am not using a third party IME (keyboard), i am sure that no application can intercept the keys that i have entered.
b)If i am using a third party IME app, this app can get whatever i have typed and may forward it to some other app/ upload it to the net etc.
Both above points are valid for a Non Rooted as well as a Rooted phone.
If i am using the standard keyboard (Android or that provided by firmware), it is impossible to intercept key events unless the firmware is modified
Links i refereed are One and Two
What will happen if i plug in an external keyboard like we do on a Galaxy tab 2. When we type using this keyboard on the EditText on Activity at top, is it guaranteed that no one else can intercept the key presses(Rooted/Non rooted phones and without Firmware change)? I hope that the IME security rules given in Android Documentation here under Security Section apply for external keyboards too.
I recently found the DoMobile ShareKeyboard app on Market, it enables input through Computer keyboard using Wifi/GPRS on an Android device. Here obviously user is using a 3rd party IME so its completely insecure in cases where security is a major concern.
Please correct me if i am wrong in the points i mentioned.
When we type using this keyboard on the EditText on Activity at top, is it guaranteed that no one else can intercept the key presses(Rooted/Non rooted phones and without Firmware change)?
Yes, if the OS is the one handling the keyboard.
a)You can't be sure. The original keyboard came with your device was made by your device manufacturer but might be modded until it got in your hands(by operators, resellers, rom modders, or just your boss)- its incredibly easy.
From my personal experience, firmware developers usually dont remember to clean debug information from their IME app, and all your keys(and even touch input) is printed in the logger.
b)Actually this might just be a more secure solution. you get it from Google Market, so you know nobody touched it in the way. These apps are also made by people who's expertise is building IME and therefore they'll know better how to improve its security and performance.
If your device is rooted you have no chance protecting your privacy at all, apps which will use root permissions will be able to read your keys from the lowest level possible - like /dev/input/eventX kernel input device not event talk about reading the Android key dispatcher which is also easy.
External keyboard is not different from virtual keyboard, its keys can be intercepted via the active IME app.
If your security is important to you, use official firmware, dont root your device, and use a good proven IME app from the Market which have security considered.