Execute code if a test fails with py.test - android

I'm doing UI test automation on Android using Appium and py.test. I'd like to be able to save a bug report using adb after a test fails.
Is there a way to tell if a test fails in my test code so I can then run save the bug report in the teardown?
Originally, I was just going to save the bug report after each test, but it's a bit excessive adding 45 seconds to each test.

You can implement a pytest_runtest_logreport hook in your conftest.py like this:
def pytest_runtest_logreport(report):
if report.when == 'call' and report.failed:
# save bug report
For more information, see Woking with plugins and conftest files.

Related

Can I print an information message in JUnit tests?

I am running JUnit tests as part of my Android app tests. In this case it's just testing the responses of an API we rely on that may be changed frequently (in order to validate later tests)
When these tests pass the output window of Android studio shows no messages. We'd instead like to start each method with a message saying which endpoint it is testing (E.g. "Tests /oranges//segments") so that the message always shows as the first message for both passing and failing tests.
I tried the android logs Log.println(Log.ASSERT, "...", "...") but that did not show in the android studio output.
Is there a way to do this? The output window I refer to is the one displayed below
(When a test is marked as ignored, you can supply a message to display. I'd just like the same behaviour for my other tests)
The old fashioned Java styled System.out.println(); will work.
JUnit tests are not executed on a Android device (or emulator) but on Java Virtual Machine on PC's environment. You have two solution:
Just use some Logger (like log4j, JDK logger, etc) or simply use System.out.
Use Roboletric and its capability to define shadow classes to redefine Log class.
I suggest you the second way. Roboletric already have a shadow class for log. To redirect output into console
#Before
public void setUp() throws Exception {
ShadowLog.stream = System.out;
//you other setup here
}
Some references:
https://guides.codepath.com/android/Unit-Testing-with-Robolectric
https://github.com/robolectric/robolectric-samples
http://www.vogella.com/tutorials/Robolectric/article.html
I hope it helps.

Cucumber-JVM Android - Testing Native applications using JUnit

I am having trouble testing native applications (such as Contacts and Settings) using Cucumber-JVM for Android with JUnit. Initially, I got the following message:
"Test run failed: Permission Denial: starting instrumentation ComponentInfo???{com.test.contacts/android.test.InstrumentationTestRunner???} from pid=673, uid=673 not allowed because package com.test.contacts does not have a signature matching the target com.android.contacts"
To solve this problem, I signed the test application with the same keys of the Contacts native android application (shared.x509.pem and shared.pk8) and I also added the following line to the AndroidManifest.xml file (as suggested in How can I sign my application with the system signature key?):
“android:sharedUserId="android.uid.shared"
This seemed to solve the problem.
However, after this change, I only manage to run the first test from a test suite. When the second test is running, it gets lost in the getActivity() method from the class ActivityInstrumentationTestCase2, which my steps definitions class extends. More precisely, it doesn't get out of the mSync.wait() call in the method startActivitySync(Intent intent) from Instrumentation.java. The call to getActivity() calls launchActivityWithIntent(), from InstrumentationTestCase.java, which calls startActivitySync(Intent intent).
I found a similar issue on Why does getActivity() block during JUnit test when custom ImageView calls startAnimation(Animation)?, but the workaround described there doesn’t solve my problem.
My test application is really simple and it only checks the content of the buttons in the activity. I don’t have this problem if I use the same test application to test my own apps, only with native android applications such as Contacts and Settings.
Does anyone know something about this issue and could give me a light on how could I solve it?
Thanks in advance

Android Studio InstrumentationTestCase Configuration and Log Output

I'm new to Android development and just experimented with unit testing inside Android Studio. I have 2 questions,
each time I need to run tests I need to create a "Run/Debug Configurations" for the test class that derives from InstrumentationTestCase, and in this configuration I can only specify 1 test class and 1 test method at a time. Is there a way to break this limitation so I can run a bunch of test classes and their test methods altogether?
I have Log statements in my tests, but I could not find where it logged the output messages to, I checked logcat and event log but didn't find it there.
Thanks.
1) You should be able to right/control click either a test method, or a test class (suite) and run the test from the context menu.
After you run it once, the run config should be, permanently, in the list of runnable configurations, and available in all of the handy ways.
2) The most likely reason that your log messages are not showing up, is that Studio has this annoying habit of automatically installing a filter, on a run, so that it only shows log messages that are from the package of the app you are running. Since your test and your app are in different packages, it is probably filtering some of the messages away.

How can I catch kivy-client crash log and send it to my server?

I have problems when some hardware issues appears and my kivy app crashes. For example on Android or iOS. Regular users can't see the log, neither can I.
So, when my application starts, I want to create separate process and somehow look at the status of main application. In case of it's crash I'd like to send error log to my server. So, what is the best way to do this? Maybe another process is redundant and I can make it in more simple way? And how exactly I can catch crash log?...Thanks!
TLDR: Use Sentry
There is different kind of crash, and different kind-of tools.
Native crash: usually a segfault, a low level crash that you cannot really do anything. That's what you see on your Play store tab, native crash/art. None of the traceback will talk to you, as you'll see the C trace of your Python interpreter and all the others threads. The user can see a "The application XXX suddenly exited" or something like that. There is tools available to display nicer message in case of a native crash and send it somewhere else, but your application will never recover. The only thing you can do with such tools is to restart it.
Python crash: good news, you can catch them and have comprehensible traceback. I suggest you to look into Sentry. It's opensource, you can install sentry on your server, and when something bad happen in your app, you can send the full traceback to your sentry installation. Very useful.
The integration into Kivy is also very simple:
if __name__ == "__main__":
import traceback
from raven import Client
client = Client('requests+http://XXKEYXX#sentry.yourserver.com/sentry/1')
try:
YourApp().run()
except:
traceback.print_exc()
ident = client.get_ident(client.captureException())
print "Exception caught; reference is %s" % ident
Don't forget to have the INTERNET permission in Android. If there is no internet, it will fail twice on the console. But that's all.
Also, you might want to plug that into the Kivy's ExceptionManager. If the exception happen in the main loop, then you have the possibility to catch it and not quit the app (ignore the exception). Beware if you were doing something important :D

Android application spontaneously crashes

I have an android application that I have recently finished. So I have successfully finished each function and ensured that no errors are encountered.
When I have exported it to an apk file. It runs smoothly. It uses httpRequests to communicate with a remote mySQL server. But there are times that it spontaneously crashes?
For example, I have an activity that would receive input from the user then communicate with the server. It runs smoothly and quickly. But on next run it would crash. And when I restart the application it would run smoothly again even on the second run. I'm just going crazy from this. Are there any explanations for these?
Use ACRA to collect crash reports from your app. You don't need your own server, just a Google Docs form. The integration is described here:
https://github.com/ACRA/acra/wiki/BasicSetup#wiki-Setting-up_your_project
The advantage of using a crash report tool vs. logcat is that you don't need physical access to the device. There are also chances that you see crashes that you or your testers didn't even notice.
The first step to debugging your problem is to get a stack trace. If you can reproduce this on your own device, then right after you see a crash, connect it to your dev pc and run this command:
adb logcat > crash.log
Open the log file, copy and paste its contents on http://pastebin.com/ or similar and add the link to your question. Once we have that, we can try to figure out what's going on.

Categories

Resources