Develop Reference

Extent Report Issue Parallel testing

I have the following Reporting code:
public class Reporting {
private ExtentHtmlReporter extentHtmlReporter;
private static ThreadLocal<ExtentReports> extentReports = new ThreadLocal<>();
private static ThreadLocal<ExtentTest> extentTest = new ThreadLocal<>();
public synchronized ExtentTest createInstanceReport(String testCaseName) {
System.out.println(extentReports.get());
new File(Constants.userDir + "/Reports/").mkdirs();
// To generate report with name
extentHtmlReporter = new ExtentHtmlReporter(
Constants.userDir + "/Reports/" +
"ExecutionReport_" + new SimpleDateFormat(
Constants.date).format(new Date()) + ".html");
// Setting Document Title
extentHtmlReporter.config().setDocumentTitle("Demo");
// Setting Report Name
extentHtmlReporter.config().setReportName("Demo Automation");
// Setting Theme
extentHtmlReporter.config().setTheme(Theme.STANDARD);
// Setting Chart location
extentHtmlReporter.config().setTestViewChartLocation(ChartLocation.TOP);
// Setting Chart visibility
extentHtmlReporter.config().setChartVisibilityOnOpen(false);
// Setting Time stamp
extentHtmlReporter.config().setTimeStampFormat("yyyy-MM-dd HH:mm:ss");
// Setting append exist as true
extentHtmlReporter.setAppendExisting(true);
ExtentReports extentReports = new ExtentReports();
extentReports.attachReporter(extentHtmlReporter);
// Setting system info
extentReports.setSystemInfo("Name",
BaseTest.prop.getProperty(Constants.testerName));
extentReports.setSystemInfo("Environment",
BaseTest.prop.getProperty(Constants.environment));
extentReports.setSystemInfo("Browser",
BaseTest.prop.getProperty(Constants.browser));
Reporting.extentReports.set(extentReports); // Instead of using here extentReport thread like this, Can anyone suggest to use it directly
// Add test case name in report
ExtentTest extentTest = Reporting.extentTest.get();
extentTest = Reporting.extentReports.get().createTest(testCaseName);
Reporting.extentTest.set(extentTest);
// Assigning categories
extentTest.assignCategory(MultiFunction.getProp()
.getProperty(Constants.browser));
System.out.println(Reporting.extentReports.get());
System.out.println(Reporting.extentTest.get());
return extentTest;
}
public synchronized ExtentTest getExtentTest() {
return extentTest.get();
}
public synchronized ExtentReports getInstanceReport() {
return extentReports.get();
}
public synchronized void remove() {
extentReports.remove();
extentTest.remove();
}
}
I was trying parallel testing using TestNG (and will have to use Selenium grid and sauce in future). I execute 2 test cases then only one test case result is added in the report.
I have isolated the extentTest, extentReporter and WebDriver instances using threadPool.
Tried below with extentHtmlReporter instance:
1) Tried to make it static(no luck)
2) Tried to make it local (the same behaviour, getting only 1 test case result)
3) Tried as a non-static global variable ( no luck)
Could you suggest how to solve the above issue?
Please note: Only one report is generated. But when I tried to run parallel test cases in debug mode reports are generated for both the test case. I think because one test case gets over its killing some instance (when running in non-debug mode)
Also, I want to redesign the following place in my code:
For extentRpeort, I am using:
Reporting.extentReports.set(extentReports);
To add extentReport instance to my extentReport Thread.
Instead of adding like this I want to use it directly so as to reduce line of code.

If I understand correctly you have to generate Report from all executed TestNG cases.
However, from code which you shared, it is very visible that you will have some trouble with it. You are making a few critical mistakes and result are obvious:
For generating reports with TestNG I will suggest grabbing information about test execution from TestNG listener. Something like:
public final class TestNGListener extends TestListenerAdapter implements IInvokedMethodListener, ISuiteListener {
#Override
public void onStart(ITestContext context) {
Logger.info(buildMessage(Logger.PREFIX_TEST_STARTED, context.getName()));
}
#Override
public void onFinish(ITestContext context) {
Logger.info(buildMessage(Logger.PREFIX_TEST_FINISHED, context.getName()));
}
#Override
public void onTestStart(ITestResult result) {
Logger.info(buildMessage(Logger.PREFIX_METHOD_STARTED, getMethodName(result)));
}
#Override
public void onTestSuccess(ITestResult result) {
Logger.info(buildMessage(Logger.PREFIX_METHOD_SUCCESS, getMethodName(result)));
processTestResult(result);
}
#Override
public void onTestFailure(ITestResult result) {
Logger.info(buildMessage(Logger.PREFIX_METHOD_FAILED, getMethodName(result)));
}
You can't do everything in one method! You broke Single Responsibility Principle. Your createInstanceReport() is doing all jobs (setting report details, set system info, attach an executed test case to report) at one place. You have to redesign this logic to some logical separate operations. After redesigning your problem with the next line:
Reporting.extentReports.set(extentReports)
Could successfully disappear.
You have to consider a case, why you need to use exactly Extent, Reports Version 3. TestNG has test reports from the box. They are poor but they are presented out of the box. If you want just to improve it a little bit you could use ReportNG over TestNG.
It is quite easy to configure: Configuring ReportNG with TestNG for HTML Reports.
It isn't maintained, but it makes TestNG reports really eye candy and understandable.
Anyway, my suggestion is to use TestNGListener for getting info about test cases execution. And read more about good programming practice.

Work with TestNG/jUnit (or other runner framework that you are using) listener, here is a good example how to do it.
Do not put everything in a single class.
https://www.swtestacademy.com/extent-reports-version-3-reporting-testng/

The issue was with the flushing of extent report instance.
I was using ThreadLocal for storing extent report instance and was flushing the wrong instance.

Espresso / How can I move button state between activities in Espresso tests?

I have experienced next scenario:
each test in my test-project uses separate activity
test1 (main screen) belong to activity#1, there is a button on this screen which can change state from STATE#1 (OFF) to STATE#2 (ON)
tap on button STATE#1 (from test1) cause raise of another screen which belong to activity#2 (there is test2 starting), then on act.#2 user perform some actions which should change button state to STATE#2
but in my test button STATE#2 wouldn't be refreshed and passed to the previous the activity#1
Do I need to sync test data in specific way? If button's state (which is on activity#1) can be changed from another test activity (activity#2)
Here is example of what I'm doing:
First of all test after which button state should be changed
TEST2 - test that should change button1 state
#RunWith(AndroidJUnit4.class)
public class ChangeBtnState extends MyIdle
{
#Rule
public ActivityTestRule<Activity#2> EspressoTestRule#2 = new ActivityTestRule<>(Activity#2.class, false, true);
private MyIdle IdlingRecourseActivity#2;
#Before
public void SetUpTest2()
{
EspressoTestRule#2.getActivity().getSupportFragmentManager().beginTransaction();
IdlingRecourseActivity#2 = (ESP_idling) EspressoTestRule#2.getActivity().getIdlingResource();
Espresso.registerIdlingResources(IdlingRecourseActivity#2);
}
#Test
public void StartTestRule2()
{
// Just some actions on activity#2 after which button1 state should be from STATE#1 to STATE#2
ViewInteraction ButtonSendtoOFF = Espresso.onView(allOf(ViewMatchers.withId(android.R.id.SomebuttonONact#2))
.check(ViewAssertions.matches(ViewMatchers.isDisplayed()));
ButtonSendtoOFF.preform(click()); // here click
}
}
#After
public void unregistereSetUpTest2()
{
if (IdlingRecourseActivity#2 != null)
{
Espresso.unregisterIdlingResources(IdlingRecourseActivity#2);
}
}
}
Now checking does button state changed on activity#1
#RunWith(AndroidJUnit4.class)
public class CheckBtnState extends MyIdle
{
#Rule
public ActivityTestRule<Activity#1> EspressoTestRule#1 = new ActivityTestRule<>(Activity#1.class, false, true);
private MyIdle IdlingRecourseActivity#1;
#Before
public void SetUpTest1()
{
EspressoTestRule#1.getActivity().getSupportFragmentManager().beginTransaction();
IdlingRecourseActivity#1 = (ESP_idling) EspressoTestRule#1.getActivity().getIdlingResource();
Espresso.registerIdlingResources(IdlingRecourseActivity#1);
}
#Test
public void StartTestRule1()
{
// Check some actions FROM activity#2 to change **button1** from STATE#1 to STATE#2
ViewInteraction ButtonSTATE = Espresso.onView(allOf(ViewMatchers.withId(android.R.id.Button1), withText("OFF"))
.check(ViewAssertions.matches(ViewMatchers.isDisplayed()));
//ButtonSTATE.preform(click()); // here click
}
#After
public void unregistereSetUpTest1()
{
if (IdlingRecourseActivity#1 != null)
{
Espresso.unregisterIdlingResources(IdlingRecourseActivity#1);
}
}
}
BTW, I have look through a lot of topics and even found examples with intents, but it doesn't work for me, perhaps there is should be specific structure.
If you need extra info please add your question in comments.

The problem is that you are writing two tests such that the second test relies on the state of the first one. This is incorrect test design. Every test should be able to run completely independent of any other tests. You need to rethink what each of your tests are testing. What happens in the second activity that affects the state of the first one? Are you using startActivityForResult()? If so, your test for the second activity should verify that the expected result is set from the second activity. This test should not rely on anything in the first activity.

Correct way to use IdlingResource in Espresso Android

I'm writing UI tests with Espresso. App cooperates tightly with server, so in many cases, I need to wait for either value to be calculated, or data is got and displayed, etc. Espresso suggests using IdlingResource for this.
My IdlingResource classes look like this (simple and clear example).
public class IRViewVisible implements IdlingResource {
private View view;
private ResourceCallback callback;
public IRViewVisible(View view) {
this.view = view;
}
#Override
public String getName() {
return IRViewVisible.class.getName();
}
#Override
public boolean isIdleNow() {
if(view.getVisibility() == View.VISIBLE && callback != null) {
callback.onTransitionToIdle();
return true;
}
return false;
}
#Override
public void registerIdleTransitionCallback(ResourceCallback resourceCallback) {
this.callback = resourceCallback;
}
}
Please correct me if I'm wrong anywhere (as sometimes it seems to me that my IdlingResources do not work properly).
I register the idling resource in setUp() like this:
IRViewVisible ir = new IRViewVisible(View v);
Espresso.registerIdlingResources(ir).
Unregister it on tearDown().
I found this article (there is a section called "Register a component tied to an Activity instance") — I do not use his schema, but I checked hashcode of view that was set to IdlingResource after registering (in each method), and it's not the same view — all hashes are different.
Another question: One Test class (it's results) can't have any effect on another Test class, can it?

I'm guessing your problem stems from getName() returning the same name for all instances of IRViewVisible. This means you can only have one registered instance of it at a time - any subsequent registrations will fail (silently!).
You mention that you clear the IdlingResources at the end of each test, but if you are register multiple instances of it at once, you need to make sure each instance has a unique name. it's not clear from your question if you're registering multiple instances of IRViewVisible in a single test.
As to your final question: Yes, it is possible. Android doesn't completely shut down the Application between test runs - just the Activities. Common things which can cause problems:
Failing to clear persistent state (saved data).
Failing to clear global state - e.g. static variables/singletons
Not waiting for background threads to finish running.
As an aside, it's worth noting that you only call onTransitionToIdle() inside isIdleNow(). This works (thanks #Be_Negative for the heads up!) but it could slow down your tests a lot, since Espresso will only poll isIdleNow() every few seconds. If you call onTransitionToIdle() as soon as the view becomes visible, it should speed things up considerably.
I needed something similar to your IRViewVisible myself, here's my effort.

So the isIdleNow() method will never return true if you don't set a callback to the idlingResource?
I reckon it's better to refactor it like this:
#Override
public boolean isIdleNow() {
boolean idle = view.getVisibility() == View.VISIBLE;
if(idle && callback != null) {
callback.onTransitionToIdle();
}
return idle;
}

Well, first of all you shouldn't need to use Espresso IdlingResource to test server calls. If you use AsyncTasks in your server calls, Espresso will be able to know when to be idle and when not. If this is not enough: try to refactor your code in this way:
IRViewVisible idlingResource = new IRViewVisible(yourView);
IdlingPolicies.setMasterPolicyTimeout(waitingTime * 2, TimeUnit.MILLISECONDS);
IdlingPolicies.setIdlingResourceTimeout(waitingTime * 2, TimeUnit.MILLISECONDS);
// Now we wait
Espresso.registerIdlingResources(idlingResource);
// Stop and verify
// Clean up
Espresso.unregisterIdlingResources(idlingResource);
Hope to be helpful.

Espresso Android testing - using getActivity in a helper method (can't find method to import)

I'm testing an Android app with Espresso. I'm trying to write a helper function in a helper class that can check the value of a spinner with the following code:
public static void assertSpinner(MainActivity activity, int id, int val) {
Spinner s = (Spinner) activity.findViewById(id);
assertNotNull(s);
assertEquals(val, s.getSelectedItemPosition());
}
I can then call the helper from my test with:
assertSpinner(getActivity(),R.id.someSpinner,12);
Though it seems weird that every assertSpinner's first arg is getActivity(). I'd like to call getActivity() in the helper function instead so I don't need to pass it, but it seems that is only made available because the test extends ActivityInstrumentationTestCase2. Is there any way to get this value without having to pass it to each of my helpers, or does that not fit the Android way?

No, I don't think there's a clean/easy way to get the current Activity from outside of a test.
However, you could do this cleanly with a custom view matcher. Something like
static Matcher<View> withSelectedItemPosition(final int selectedItemPosition) {
return new BoundedMatcher<View, Spinner>(Spinner.class) {
#Override protected boolean matchesSafely(Spinner spinner) {
return spinner.getSelectedItemPosition() == selectedItemPosition;
}
#Override public void describeTo(Description description) {
description.appendText("with selected item position: ")
.appendValue(selectedItemPosition);
}
};
}
Then you could do
onView(withId(R.id.my_spinner)).check(matches(withSelectedItemPosition(5)));
It's a bit of extra code, but more idiomatic. Espresso really discourages you from interacting with the view hierarchy directly; ideally your tests should never call methods like findViewById.

Compensating for the shortcomings of AsyncTask with BetterAsyncTask (Droid-Fu)

After a lengthy and unsuccessful search on google, it turns out that there is hardly any information about a handy library called Droid-Fu https://github.com/kaeppler/droid-fu
After reading the introduction by the creator (http://brainflush.wordpress.com/2009/11/16/introducing-droid-fu-for-android-betteractivity-betterservice-and-betterasynctask/) or the API (http://kaeppler.github.com/droid-fu), I could not figure out how to define a new betterasynctask (what methods hold what information etc).
So if there is anyone out there that could provide me (and apperently others as well) with some useful source code or tutorials, I would greatly appreciate it!
(If you need the jar file of the project, let me know, I can send you a copy)
EDIT:
A good example can be found here! and here
Ok, here is some additional information I found in the source code:
A progress dialog is automatically shown. See useCustomDialog(), disableDialog()
If an Exception is thrown from inside doInBackground, this is now handled by the handleError method.
You should now longer override onPreExecute(), doInBackground() and onPostExecute(), instead you should use before(), doCheckedInBackground() and after() respectively.
Let's see what I can achieve from here on then...still looking for a working example though!
EDIT 2:
A couple of examples can be found here and here. I stick to it but I get an error. Only difference is that my AsyncTask is not defined within the activity, but a class of its own. Stepping through the code reveals that the error happens upon creation of the (AsyncTask built-in) Dialog.
This is my stacktrace:
coming in a minute

Droid-fu is a little outdated now, mainly due to the lack of Fragment support. But I'll give an example from an app I wrote that used it.
First, your activity class has to subclass BetterActivity (or BetterXXXActivity). In my code I was using a ListActivity so mine here subclasses BetterListActivity. I also define a subclass of BetterAsyncTask so I can extend some functionality.
public class DroidFuExample extends BetterListActivity {
private ExampleTask mTask;
private List<Stuff> mMainStuff;
private class ExampleTask extends BetterAsyncTask<Void, Void, Integer> {
private List<Stuff> mStuff;
private DroidFuExample mContext; // a context for lifecycle management
...
}
}
Now, my task needs doesn't take a parameter, uses an indeterminate dialog so no progress is published, and needs to return an Integer. Your needs may differ, and that affects the types used in the class definition.
Next step is to define what the task processes in the background. In my case I need to populate mStuff. In your task class, define either doInBackground() or doCheckedInBackground() (doChecked... can throw an exception if you want to catch it).
protected Integer doCheckedInBackground(Context context, Void... params)
throws Exception {
mStuff = // some long-running code (no longer on the UI thread)
return 1;
}
Finally, at the very least you need to do something with your result, like update a class variable or populate the UI or something. This is done in after:
protected void after(Context context, Integer integer) {
if (integer >= someAcceptablePositiveConstant) {
mMainStuff = mStuff;
doSomethingInTheUIWithMainStuff();
} else {
//gah!
}
}
As you mentioned there's more you can do with the class, such as define a before() override that does work on the UI thread before the task, or failed() / handleError() to handle unchecked/checked failures. This is just a simple example, hope it helps.

What a pain in the #ss! The error was well hidden inside the BetterActivityHelper class:
public static ProgressDialog createProgressDialog(final Activity activity,
int progressDialogTitleId, int progressDialogMsgId) {
ProgressDialog progressDialog = new ProgressDialog(activity);
if (progressDialogTitleId <= 0) {
progressDialogTitleId = activity.getResources().getIdentifier(
PROGRESS_DIALOG_TITLE_RESOURCE, "string", activity.getPackageName());
}
progressDialog.setTitle(progressDialogTitleId);
if (progressDialogMsgId <= 0) {
progressDialogMsgId = activity.getResources().getIdentifier(
PROGRESS_DIALOG_MESSAGE_RESOURCE, "string", activity.getPackageName());
}
progressDialog.setMessage(activity.getString(progressDialogMsgId));
progressDialog.setIndeterminate(true);
progressDialog.setOnKeyListener(new OnKeyListener() {
public boolean onKey(DialogInterface dialog, int keyCode, KeyEvent event) {
activity.onKeyDown(keyCode, event);
return false;
}
});
progressDialogTitleId and progressDialogMsgId expect a value inside the res/values/string.xml :
<!-- Droid-Fu Progressdialog -->
<string name="droidfu_progress_dialog_title">Some nasty dialog title</string>
<string name="droidfu_progress_dialog_message">Some funny message</string>
If they are not defined, a runtime exception will be thrown.
Unfortunately, even the best helper classes are almost useless if they are UNDOCUMENTED. Took me a couple of hours to figure out what went wrong. So again, NO "Thank you" to the developer. Sor