Simulate Touch Controls Through Code - android

I'm trying to make it possible to navigate through my Google Glass application by using head gestures. I'm able to recognize head gestures like looking to the right left and up. They each have their own method for what to do when this gesture is recognized
Now I need to simulate the corresponding touch gestures inside each method. So it will think I'm swiping to the left or right which will allow me to navigate through the cards with the head gestures.
Does anyone have any idea on how to actually achieve this?
Edit
I created a quick hello world application to play with. I added my headgesture code and started trying to get the keys working.
I added the following to my onCreate()
Instrumentation instr = new Instrumentation();
Then I added the following lines to each respective headgesture method.
Headgesture upwards should correspond with tapping the touchpadinst.sendKeyDownUpSync(KeyEvent.KEYCODE_DPAD_CENTER)
Headgesture to the left should correspond with swiping left on the touchpad inst.sendKeyDownUpSync(KeyEvent.KEYCODE_DPAD_LEFT);
Headgesture to the right should correspond with swiping right on the touchpadinst.sendKeyDownUpSync(KeyEvent.KEYCODE_DPAD_RIGHT);
They are responding accordingly now, however I'm getting an exception saying:
java.lang.RuntimeException: This method can not be called from the main application thread

The Solution
In the end I went a different direction then the one I mentioned in my edit above.
I found out that it is possible to call touch controls in the shell by using
adb shell input keyevent <keycode here>
I then found a way to use this in android, I have the following class named issueKey
public class issueKey {
public void issueKey(int keyCode)
{
try {
java.lang.Process p = java.lang.Runtime.getRuntime().exec("input keyevent " + Integer.toString(keyCode) + "\n");
} catch (Exception e) {
Log.wtf("IssueKeyError", e.getMessage());
}
}
}
Then I simply call the class and pass the keycode for the corresponding gesture
mIssueKey.issueKey(4);//functions as swipe down
Here is the list of keycodes that I tested for anyone that is interested.
Keys for each respective button/gesture
4: Swipe Down
21: Swipe Left
22: Swipe Right
23: Tap
24: Volume Up
25: Volume Down
26: Lock/Unlock Screen
27: Camera Button
However, what I'm wondering now is. What would be best practice, getting the solution I metioned in my edit to work by using a asyncTask or is the solution I'm currently using better.

Using the Instrumentation class would work if you use a separate thread to call the sendKeyDownUpSync method from.
This can be done using the following steps:
Create and start a thread from your activity
In the run method, use the Looper class and create a Handler as explained here
Every time you want to call sendKeyDownUpSync, post a Runnable instance to the Handler, which calls sendKeyDownUpSync in its run method.
A similar code sample (not from me) is available here

Related

UiAutomator - UiDevice can't findObject by selector (package name and resource ID)

I'm unable to find an element (UiObject2) using UiAutomator within my androidTest. I obtained UiDevice instance and try to find the object with this:
MY_UI_DEVICE.findObject(By.res(CURRENT_PACKAGE, id));
CURRENT_PACKAGE is the package of my app MY_UI_DEVICE.getCurrentPackageName(). I tried also with this one:
MY_UI_DEVICE.wait(Until.findObject(By.res(CURRENT_PACKAGE, id)), 10000);
I can see the app is waiting for 10 seconds on the right screen (where the desired object persists), but after timeout it fails to find it and test fails. It always fails on emulator (API 23), but rarely works good on a real device (API 25).
When I debug the code I could see that manually I could obtain the right element by calling sequence of getChild(index) methods on AccessibilityNodeInfo but in the runtime it still fails even the app is waiting on the right screen where I expect the specific element.
I was playing with different UiDevice's functions, but none of the helped and I'm out of ideas, so any help will be appreciated.
There were 2 issues with my tests:
The first problem was in getting / initialising UiDevice instance in static block (as a static field in util class). I moved it into #Beforeand it helped to resolve the issue partially.
Another problem was occurring while searching for an element using a package name obtained from the UiDevice. I replaced getting package with InstrumentationRegistry.getTargetContext().getPackageName(); as it's done in google samples.
Make sure that your Test method is throwing the UiObjectNotFoundException. I had this issue with UiObject2 as well until I started forcing the error throw
#Test
public void clockTest() throws UiObjectNotFoundException, InterruptedException {
mDevice.click(1146,37); //click on clock top right corner
Thread.sleep(1500);//wait 1.5 seconds for screen to load
mDevice.click(1138,135);//clicks in shell
Thread.sleep(1500);//wait 1.5s for screen to load
UiObject2 dTSettingsButton = mDevice.findObject(By.text("Date & Time Settings"));
//assertNotNull(dTSettingsButton);//find and assert the settings button
dTSettingsButton.clickAndWait(Until.newWindow(), LAUNCH_TIMEOUT);//clicks the settings button
UiObject2 timeFormatButton = mDevice.findObject(By.text("Select Time Format"));
assertNotNull(timeFormatButton);//find and assert timeformat button
timeFormatButton.clickAndWait(Until.newWindow(), LAUNCH_TIMEOUT);//click timeformat button
UiObject2 twelveHourButton = mDevice.findObject(By.res("com.REDACTED.settings:id/first_btn"));
assertNotNull(twelveHourButton);//find and assert twelvehour button
twelveHourButton.clickAndWait(Until.newWindow(), LAUNCH_TIMEOUT);//click twelvehour button
}
Try use UiSelector methods. That worked for me much better than By selectors

Solution for OnPointerExit() Triggering on Touch Release On Android?

I have created some custom radial menu buttons for my Android game. The radial menu displays when the game object with the menu is touched. I'm using mouse events to activate the menu, which works in Unity and also when built to Android. When the menu is active, you can mouse or slide over a menu item to select it. If you then release on that menu item, it will pass the selection to the radial menu, which then takes the appropriate action.
The following works in Unity:
public void OnPointerEnter (PointerEventData eventData)
{
myMenu.selected = this;
Debug.Log ("Menu selection is now: " + this.action.ToString ());
defaultColor = circle.color;
circle.color = Color.white;
}
public void OnPointerExit (PointerEventData eventData)
{
myMenu.selected = null;
Debug.Log ("Menu selection has been nulled out.");
circle.color = defaultColor;
}
However, it does not work correctly when built to Android. Via some debug testing, I've determined that in Unity, if I activate the menu and mouse over a menu item, then release the mouse, myMenu.selected is correctly assigned. However, on Android only, lifting my finger over the menu item processes a final OnPointerExit, which nulls it out, meaning that menu never gets a proper selection. The mouse does not behave this way--it doesn't treat the pointer as having exited when the mouse button is released.
I can "solve" this problem by switching everything to touch, but then I cannot test using the Unity player and my mouse. So far, everything via mouse also worked correctly via touch. This is the first issue I've run into. Is there any clean solution for this? Or is the best solution to use macros to determine if I'm in the editor and have separate mouse and touch code for each?
It depends on the behavior you trying to implement.
Touch input does not have the notion of "hover" so you should probably be using the IPointerClickHandler interface and/or the IPointerDownHandler and IPointerUpHandler interfaces instead.
I recommend separating the hover logic vs touch/click clearly by using the preprocessor directive #if UNITY_STANDALONE around your hover-code.

simulate and dispatch touch event

I have loaded a HTML page (say it has a text and a button at the end of the page to share the text) inside a WebView.
The thing is i want to simulate the touch event of that share button and dispatch it to webview. That is (once the user spends some time in reading the text i would like to simulate the share button click (onClick method should be called) in an automated fashion so that even if user doesn't click share i have made it mandatory to be shared.
That's the logic. I refereed to some questions reg touch events in Stack Overflow and found that dispatchTouchEvent and MotionEvent should be used but not clear how to use them and apply my logic.
It would be really helpful if someone could explain its usage with respect to the above quoted idea in a detailed manner.
This is possible if you know the x,y coordinates where to click.
in this java example (not mine) it shows that given a rectangle of a DOM element, it will simulate a click:
The advantage here is that you can simulate the click even if the user didnt initiate an action (which is restricted by the browser itself)
https://github.com/46cl/cordova-android-focus-plugin/blob/master/src/android/Focus.java
This is a cordova example. See here for an example on how to call it from a regular android program:
Android Webview: Execution of javascript from java method called from javascript
runOnUiThread(new Runnable() {
public void run() {
final long uMillis = SystemClock.uptimeMillis();
webView.dispatchTouchEvent(MotionEvent.obtain(uMillis, uMillis,
MotionEvent.ACTION_DOWN, centerLeft, centerTop, 0));
webView.dispatchTouchEvent(MotionEvent.obtain(uMillis, uMillis,
MotionEvent.ACTION_UP, centerLeft, centerTop, 0));
}
});
you would need to dispatch the call from your javascript passing the rect to your java code:
here is an example (of a cordova app) calling the java method from javascript:
https://github.com/46cl/cordova-android-focus-plugin/blob/master/www/focus.js

JQuery Mobile swipe event only firing every other swipe

I have set up a swipeleft event in my app to move between fields of a form. All of the fields are dynamically generated, so I'm not swapping between pages, I'm clearing and re-generating all the DOM elements. The problem is the swipe event only fires every other time I swipe on the page or if I touch or tap anything on the page.
Here's the code that sets up the events:
$(document).delegate("#scorePage", "pageshow", function() {
$.event.special.swipe.scrollSupressionThreshold = 10;
$.event.special.swipe.horizontalDistanceThreshold = 30;
$.event.special.swipe.durationThreshold = 500;
$.event.special.swipe.verticalDistanceThreshold = 75;
$('#divFoo').on("swipeleft", swipeLeftHandler);
$('#divFoo').on("swiperight", swipeRightHandler);
tableCreate(traits[0].keyboardID);
});
For context, tableCreate is putting a dynamically generated table into divFoo that contains information a user can pick from. Here's the event code itself:
function swipeLeftHandler() {
$("#divFoo").empty();
traitIndex++;
tableCreate(traits[traitIndex].keyboardID);
}
Why is my swipe event only firing every other time there is a swipe on the page?
Primarily testing on Android right now, if that makes a difference.
Edit I'm using JQuery Mobile version 1.4.4
I figured out a way around this problem by simply rolling my own implementation of these events. There's some sample code on how to do something similar here:
https://snipt.net/blackdynamo/swipe-up-and-down-support-for-jquery-mobile/
If anyone else uses this code to fix my same problem, make sure to be aware that the article is implementing swipeup and swipedown so you will have to adapt it. In the end, I'm not entirely sure about the differences between this code and the actual implementations of swipeleft and swiperight, but this works consistently so I'm cutting my losses and going with it.

What DOM events are available to WebKit on Android?

I'm building a mobile web app targeting Android users. I need to know what DOM events are available to me. I have been able to make the following work, but not terribly reliably:
click
mouseover
mousedown
mouseup
change
I have not been able to get the following to work:
keypress
keydown
keyup
Does anyone know the full list of what is supported and in what contexts (e.g., is onchange only available to form inputs?)? I can't find a reference for this on The Googles.
Thanks!
Update: I asked the same question on the Android developers list. I will be doing some more testing and will post my results both here and there.
OK, this is interesting. My use case is that I have a series of links (A tags) on a screen in a WebKit view. To test what events area available, using jQuery 1.3.1, I attached every event listed on this page (even ones that don't make sense) to the links then used the up, down, and enter controls on the Android emulator and noted which events fired in which circumstances.
Here is the code I used to attach the events, with results to follow. Note, I'm using "live" event binding because for my application, the A tags are inserted dynamically.
$.each([
'blur',
'change',
'click',
'contextmenu',
'copy',
'cut',
'dblclick',
'error',
'focus',
'keydown',
'keypress',
'keyup',
'mousedown',
'mousemove',
'mouseout',
'mouseover',
'mouseup',
'mousewheel',
'paste',
'reset',
'resize',
'scroll',
'select',
'submit',
// W3C events
'DOMActivate',
'DOMAttrModified',
'DOMCharacterDataModified',
'DOMFocusIn',
'DOMFocusOut',
'DOMMouseScroll',
'DOMNodeInserted',
'DOMNodeRemoved',
'DOMSubtreeModified',
'textInput',
// Microsoft events
'activate',
'beforecopy',
'beforecut',
'beforepaste',
'deactivate',
'focusin',
'focusout',
'hashchange',
'mouseenter',
'mouseleave'
], function () {
$('a').live(this, function (evt) {
alert(evt.type);
});
});
Here's how it shook out:
On first page load with nothing highlighted (no ugly orange selection box around any item), using down button to select the first item, the following events fired (in order): mouseover, mouseenter, mousemove, DOMFocusIn
With an item selected, moving to the next item using the down button, the following events fired (in order): mouseout, mouseover, mousemove, DOMFocusOut, DOMFocusIn
With an item selected, clicking the "enter" button, the following events fired (in order): mousemove, mousedown, DOMFocusOut, mouseup, click, DOMActivate
This strikes me as a bunch of random garbage. And, who's that cheeky IE-only event (mouseenter) making a cameo, then taking the rest of the day off? Oh well, at least now I know what events to watch for.
It would be great if others want to take my test code and do a more thorough run through, perhaps using form elements, images, etc.
Since this is the second most popular Android + JavaScript post on SO (which is just a sad commentary on the state of web development targeting the Android platform), I thought it may be worthwhile including a link to pkk's touch event test results at http://www.quirksmode.org/mobile/tableTouch.html and also http://www.quirksmode.org/mobile/ in general.
As of Android 1.5, the same touch(start|move|end|cancel) events that the iPhone supports work in Android as well.
One problem I found was that touchmove ends get queued up. No workaround yet.

Categories

Resources