I want to use MonkeyRunner to test the compatibility of my android program for a list of devices with different screen resolutions. I need to click a view, but the view is not in the same position for different resolutions. How can I get the position of it or do something else to click it?
NEED your help!
I know it's a little late, but you can use hierarchyviewer in android sdk to get the view id.
Then, in your script, use this:
from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice
from com.android.monkeyrunner.easy import EasyMonkeyDevice, By
device = MonkeyRunner.waitForConnection()
easy_device = EasyMonkeyDevice(device)
# Start your android app
# touch the view by id
easy_device.touch(By.id('view_id'), MonkeyDevice.DOWN_AND_UP)
Later edit: thanks to dtmilano and AndroidViewClient, I was able to do the clicks on views as needed. The link is here: https://github.com/dtmilano/AndroidViewClient
Unfortunately this is not really possible with MonkeyRunner. One option is to use device.getProperty("display.width"),device.getProperty("display.height") and device.getProperty("display.density") and try to use those to somehow figure out where the view is. Another option would be to use a tool like Sikuli to try to click on the view.
Edit (one year after this answer was initially posted): It is now possible to do what you initially wanted with https://github.com/dtmilano/AndroidViewClient
Similar to what someoneHuman said above, I use two functions that convert tap coordinates based on the difference in the resolution of the device I originally wrote the script for and whatever device I am currently using.
First I get the current devices x and y pixel width.
CurrentDeviceX = float(device.getProperty("display.width"))
CurrentDeviceY = float(device.getProperty("display.height"))
Then I define a function for converting x and y coordinates. You can see that the functions below were written for a device that is 1280 x 800.
def transX(x):
''' (number) -> intsvd
TransX takes the x value supplied from the original device
and converts it to match the resolution of whatever device
is plugged in
'''
OriginalWidth = 1280;
#Get X dimensions of Current Device
XScale = (CurrentDeviceX)/(OriginalWidth)
x = XScale * x
return int(x)
def transY(y):
''' (number) -> int
TransY takes the y value supplied from the original device
and converts it to match the resolution of whatever device
is plugged in.
'''
OriginalHeight = 800;
#Get Y dimensions of Current Device
YScale = (CurrentDeviceY)/(OriginalHeight)
y = YScale * y
return int(y)
Then I can use these functions when creating tap events in my scripts.
example
device.touch(transX(737), transY(226), 'DOWN_AND_UP')
Please note that this approach is far from perfect, and it will only work if your app utilizes anchoring to adjust UI based on screen size. This is my quick and dirty approach to making scripts that work on multiple devices when UI IDs are unavailable.
Related
I was wondering if someone could give me a detailed explanation on how to run a game/app developed using Pygame on an Android phone. I recently finished programming PacMan and it works perfectly on my computer, but I think it would be awesome if I could get it running on my phone. I tried following the instructions at http://pygame.renpy.org/android-packaging.html, but every time i run "import android" on the IDLE I get an error saying it did not find the module. Could someone clearly explain how to set up the android module?
Also, in my program I used code such as if (event.key == K_UP or event.key == K_w): direction = UP. However there are no arrow keys on a phone. What code would I need to use to see if the user swiped the screen with their fingers from up -> down or left -> right, etc.
Any help would be great. Thanks <3
There is a pyGame subset for android. However this requires special reworking and changing the program. Hopefully it will not be to hard.
http://pygame.renpy.org/writing.html
http://pygame.renpy.org/index.html
However about your second question i am unable to awnser because I am Not yet experienced enough.
i think the pygame subset for android would be good but i dont trust its functionality, i use kivy as its cross platform
and if you ever decide to use the pygame subset for android your touch of flips on screen of an android device would be your mouse movement on the desktop so i ma saying treat the touch as the mouse good luck
There are some pretty good answers for your first part already so I won't answer that. (I came here looking into what to use for it too!)
However the second part of your question should be a lot easier.
Have a mouse object that on a mouse down event will save the coordinates of the touch to an MX and MY variable
Then when the mouse up event is triggered takes the new coordinates and calculates a vector using the MX and MY and this new point ie. The distance and angle of the swipe. Use trigonometry or the math module for the angle (research arctan2).
You can then use this in an if, elif, else to determine what quadrant the angle was and the distance to determine whether the swipe was valid if it's greater than a certain value.
I'm on mobile so unfortunately I can't give an example, however I'm certain you're apt to work out the solution with this guidance.
For your second question, there is an answer in another website.
https://amp.reddit.com/r/Python/comments/2ak14j/made_my_first_android_app_in_under_5_hours_using/
it says You can try code like this
if android:
android.map_key(android.KEYCODE_BACK, pygame.K_ESCAPE)
I hope it will help you.
I've runned pygame for android!!!!
Firstly, I'm debugged app using saving error to file.
I got error that on android it can be runned only under fullscreen.
I've created small app and it working:
import sys, os
andr = None
try:
import android
andr = True
except ImportError:
andr = False
try:
import pygame
import sys
import pygame
import random
import time
from pygame.locals import *
pygame.init()
fps = 1 / 3
width, height = 640, 480
screen = pygame.display.set_mode((width, height), FULLSCREEN if andr else 0)
width, height = pygame.display.get_surface().get_size()
while True:
screen.fill((random.randint(0, 255), random.randint(0, 255), random.randint(0, 255)))
for event in pygame.event.get():
if event.type == QUIT:
pygame.quit()
sys.exit()
pygame.display.flip()
time.sleep(fps)
except Exception as e:
open('error.txt', 'w').write(str(e))
Screenshot: https://i.stack.imgur.com/4qXPe.png
requirements in spec file:
requirements = python3,pygame
APK File size is only 12 MB!
pygame.event has multiple touch-screen events. Here are some useful ones:
FINGERMOTION: touch_id, finger_id, x, y, dx, dy
FINGERDOWN: touch_id, finger_id, x, y, dx, dy
FINGERUP: touch_id, finger_id, x, y, dx, dy
MULTIGESTURE: touch_id, x, y, pinched, rotated, num_fingers
I'd like to test android's behavior on all possible combinations of the following "inputs":
top activity's setRequestedOrientation() (15 possible values, not including SCREEN_ORIENTATION_BEHIND)
Settings.System.ACCELEROMETER_ROTATION (2 possible values)
Settings.System.USER_ROTATION (4 possible values)
device's physical orientation (queryable by OrientationEventListener) (4 possible quadrants)
Specifically, I want to see how the inputs affect the following "output":
getWindowManager().getDefaultDisplay().getRotation() (4 possible values)
So this will require testing at least all 15*2*4*4=480 possible input states.
Additionally, since rotation behavior is often dependent on the history
of the inputs (not just the current input values),
I want to test (at least) all possible transitions from one input state to an "adjacent" input state,
i.e. to an input state that differs from the given input state by one input parameter.
The number of such input state transitions is:
(number of input states) * (number of states adjacent to a given input state)
= (15*2*4*4) * ((15-1) + (2-1) + (4-1) + (4-1))
= 480 * 21
= 10080
Furthermore, sometimes output is dependent on the previous output as well as previous and current
input (e.g. SCREEN_ORIENTATION_LOCKED, SCREEN_ORIENTATION_SENSOR_LANDSCAPE).
The number of possible outputs for a given input state can be between 1 and 4,
so this multiplies the number of transitions that must be tested by up to 4:
10080 * 4 = 40320
That's a lot of transitions to test, so the testing would have to be programmatic/scripted.
Three out of the four input params are straightforward to control programmatically;
the one that's not straightforward to control is the device's physical orientation.
So, how would one go about scripting it? I can think of the following approaches.
Approach #1: Replace the (physical or emulated) device's accelerometer with a scriptable mock accelerometer
for the duration of the test. But, if I understand correctly, mock accelerometers do not exist for android.
Approach #2: Use the android emulator, and script pressing of the "rotate counterclockwise" and
"rotate clockwise" buttons using an interaction automation tool on the host machine (e.g. applescript / autohotkey / xdotool).
Any other ideas?
It turns out this is actually a duplicate of the following excellent question:
How can i simulate accelerometer in android emulator?
which has this excellent answer from #user1302884 :
Unfortunately, that question got no respect and was closed as off-topic (?!) so I won't mark this as a duplicate.
But here's the answer: no need for applescript/autohotkey/xdotool to drive the emulator's ui;
instead, telnet to the emulator and tell it which direction you want "up" to be.
telnet localhost 5554 # or whatever the port is
telnet> sensor # to get help on the sensor command
telnet> sensor get acceleration
acceleration = 0:9.81:0 # if in natural orientation
telnet> sensor get acceleration
acceleration = -9.81:0:0 # if rotated 90 degrees CW from natural orientation
telnet> sensor set orientation -1:1:0 # to set to 45 degrees CW from natural orientation
It would be nice if the emulated display would appear rotated by the specified number of degrees in response, but you can't have everything.
I'm developing in qt 5.3 on android device. I can't get the screen resolution.
With the old qt 5 version this code worked:
QScreen *screen = QApplication::screens().at(0);
largh=screen->availableGeometry().width();
alt =screen->availableGeometry().height();
However now it doesn't work (returns a screen size 00x00). Is there another way to do it? thanks
Size holds the pixel resolution
screen->size().width()
screen->size().height();
Whereas, availableSize holds the size excluding window manager reserved areas...
screen->availableSize().width()
screen->availableSize().height();
More info on the QScreen class.
for more information, screen availableSize is not ready at the very beginning, so you have to wait for it, here is the code:
Widget::Widget(QWidget *parent){
...
QScreen *screen = QApplication::screens().at(0);
connect(screen, SIGNAL(virtualGeometryChanged(QRect)), this,SLOT(getScreen(QRect)));
}
void Widget::getScreen(QRect rect)
{
int screenY = screen->availableSize().height();
int screenX = screen->availableSize().width();
this->setGeometry(0,0,screenX,screenY);
}
I found that there are several ways to obtain the device resolution, each outputs the same results and thankfully works across all Os-es supported by Qt...
1) My favorite is to write a static function using QDesktopWidget in a reference class and use it all across the code:
QRect const CGenericWidget::getScreenSize()
{
//Note: one might implement caching of the value to optimize processing speed. This however will result in erros if screen resolution is resized during execution
QDesktopWidget scr;
return scr.availableGeometry(scr.primaryScreen());
}
Then you can just call across your code the function like this:
qDebug() << CGenericWidget::getScreenSize();
It will return you a QRect const object that you can use to obtain the screen size without the top and bottom bars.
2) Another way to obtain the screen resolution that works just fine if your app is full screen is:
QWidget *activeWindow = QApplication::activeWindow();
m_sw = activeWindow->width();
m_sh = activeWindow->height();
3) And of course you have the option that Zeus recommended:
QScreen *screen = QApplication::screens().at(0);
largh=screen->availableSize().width();
alt =screen->availableSize().height();
I have recently started learning uiautomator for the UI testing of various Android devices. Currently I am testing on Galaxy S4.
I am looking for any class or method which can be used to automate the unlock pattern that user draws to unlock the phone. For example, I have letter N as a "draw pattern" to unlock the phone. How can I automate this unlock pattern in uiautomator?
Suppose you have letter "N" as unlock pattern, then first you have find the co-ordinates of each point of that N shape in your device. As you mentioned, the entire pattern lock will have 9 dots, you have to get (x,y) co-ordinates of 4 dots. To get the co-ordinates, you can use the
same method mentioned earlier in one of the answer.
Go to 'Settings' -> 'Developer Options'.
Under 'INPUT' section -> you will find a option 'Pointer Location' -> enable that option.
Once you get your 4 dots' co-ordinates, use swipe(Point[] segments, int segmentSteps) method of UiAutomator Framework.
The input for this method is the 4 set of co-ordinates that you got from your device screen as Point array. This will give continuous swipe through the points.
I have given a sample script below for your understanding.
import android.graphics.Point;
public void unlockpatternlock() throws UiObjectNotFoundException, Exception {
Point[] cordinates = new Point[4];
cordinates[0] = new Point(248,1520);
cordinates[1] = new Point(248,929);
cordinates[2] = new Point(796,1520);
cordinates[3] = new Point(796,929);
getUiDevice().wakeUp();
getUiDevice().swipe(cordinates, 10);
}
Above script would draw N shape smoothly. Remember input the co-ordinates according to your device screen.
This is the only way I know to do it, but it can be tedious trying to find your x and y coordinates.
UiDevice.getInstance().swipe(int startX, int startY, int endX, int endY, int steps)
The only probelm I see is to do an "N", you would need 3 of these swipe's. To unlock, it needs to be one continuous swipe.
Give it a show. Finding your x and y will be tough. I would go to my "apps home" page and look at apps (with the uiautomatorviewer) that are in relatively the same spot, find their coords, then go from there.
NOTE The int steps is how fast and "smooth" you want to swipe. I like to use 5 or 10. It seems pretty natural.
To find out the co-ordinates of the screen , you can follow this :
[1] Go to 'Settings' -> 'Developer Options'.
[2] Under 'INPUT' section -> you will find a option 'Pointer Location' -> enable that option.
After that if you touch anywhere on the screen -> you can view the exact screen coordinates of that point on top of the screen on your device.
And after you get the coordinates , you can use swipe method say like this -
UiDevice.getInstance().swipe(390, 1138, 719, 1128, 40);
method easily giving the exact co-ordinates where to drag from and till what point.
I have already used this and it works!
I was wondering if someone could give me a detailed explanation on how to run a game/app developed using Pygame on an Android phone. I recently finished programming PacMan and it works perfectly on my computer, but I think it would be awesome if I could get it running on my phone. I tried following the instructions at http://pygame.renpy.org/android-packaging.html, but every time i run "import android" on the IDLE I get an error saying it did not find the module. Could someone clearly explain how to set up the android module?
Also, in my program I used code such as if (event.key == K_UP or event.key == K_w): direction = UP. However there are no arrow keys on a phone. What code would I need to use to see if the user swiped the screen with their fingers from up -> down or left -> right, etc.
Any help would be great. Thanks <3
There is a pyGame subset for android. However this requires special reworking and changing the program. Hopefully it will not be to hard.
http://pygame.renpy.org/writing.html
http://pygame.renpy.org/index.html
However about your second question i am unable to awnser because I am Not yet experienced enough.
i think the pygame subset for android would be good but i dont trust its functionality, i use kivy as its cross platform
and if you ever decide to use the pygame subset for android your touch of flips on screen of an android device would be your mouse movement on the desktop so i ma saying treat the touch as the mouse good luck
There are some pretty good answers for your first part already so I won't answer that. (I came here looking into what to use for it too!)
However the second part of your question should be a lot easier.
Have a mouse object that on a mouse down event will save the coordinates of the touch to an MX and MY variable
Then when the mouse up event is triggered takes the new coordinates and calculates a vector using the MX and MY and this new point ie. The distance and angle of the swipe. Use trigonometry or the math module for the angle (research arctan2).
You can then use this in an if, elif, else to determine what quadrant the angle was and the distance to determine whether the swipe was valid if it's greater than a certain value.
I'm on mobile so unfortunately I can't give an example, however I'm certain you're apt to work out the solution with this guidance.
For your second question, there is an answer in another website.
https://amp.reddit.com/r/Python/comments/2ak14j/made_my_first_android_app_in_under_5_hours_using/
it says You can try code like this
if android:
android.map_key(android.KEYCODE_BACK, pygame.K_ESCAPE)
I hope it will help you.
I've runned pygame for android!!!!
Firstly, I'm debugged app using saving error to file.
I got error that on android it can be runned only under fullscreen.
I've created small app and it working:
import sys, os
andr = None
try:
import android
andr = True
except ImportError:
andr = False
try:
import pygame
import sys
import pygame
import random
import time
from pygame.locals import *
pygame.init()
fps = 1 / 3
width, height = 640, 480
screen = pygame.display.set_mode((width, height), FULLSCREEN if andr else 0)
width, height = pygame.display.get_surface().get_size()
while True:
screen.fill((random.randint(0, 255), random.randint(0, 255), random.randint(0, 255)))
for event in pygame.event.get():
if event.type == QUIT:
pygame.quit()
sys.exit()
pygame.display.flip()
time.sleep(fps)
except Exception as e:
open('error.txt', 'w').write(str(e))
Screenshot: https://i.stack.imgur.com/4qXPe.png
requirements in spec file:
requirements = python3,pygame
APK File size is only 12 MB!
pygame.event has multiple touch-screen events. Here are some useful ones:
FINGERMOTION: touch_id, finger_id, x, y, dx, dy
FINGERDOWN: touch_id, finger_id, x, y, dx, dy
FINGERUP: touch_id, finger_id, x, y, dx, dy
MULTIGESTURE: touch_id, x, y, pinched, rotated, num_fingers