I have some problem running Robot Framework test suite with appium and pabot.
I'm running some tests suite on multiple android devices like this:
pabot --pabotlib --argumentfile1 phone1.dat --argumentfile2 phone2.dat -v LANG:fr --outputdir output --output basics.xml JustBasics_tests.robot
pabot --pabotlib --argumentfile1 phone1.dat --argumentfile2 phone2.dat -v LANG:fr --outputdir output --output another.xml LetsRunAnother_test.robot
I don't execute all suites in a single pabot instruction because I got some appium errors if I do like this ... errors like: WebDriverException: Message: A session is either terminated or not started pabot or WebDriverException: Message: A new session could not be created. Details: Problem getting session data for driver type AndroidDriver; does it implement 'get driverData'?
I want to make a report from another.xml and basics.xml to something like log.html and report.html (to work with Jenkins after).
My first problem is that basics.xml and another.xml files contain only one test execution from one of the two devices. If I want data from both devices of a test case execution I need to take the output.xml file which contains both test case execution... Of course output.xml is overwritten after each pabot command.
So I thought to just copy each output.xml and rename to outputX.xml and finally make a
rebot output1.xml output2.xml output3.xml ....
But rebot doesn't work like I expected : rebot just copying X times the first mobile test on the report so I have something like on my log.html:
JustBasics
Test_Results_for_phone1
Test_Results_for_phone1
LetsRunAnother
Test_Results_for_phone1
Test_Results_for_phone1
What am I doing wrong?
My second problem is that there are no screenshots in my log files, when something fails in test execution (error or crash of app for exemple). They all point to root directory from the log.html but images are in a path like pabots_results/X/TestSuiteName/appium-screenshoot-x.png
Here again what is wrong? Do I have to do something about images, or pabot should do it alone (in pabot.py nearly then end of file, there is a function named _copy_screenshots ....)
The disturbing thing about this story is that there are not plenty of documentation about pabot. And I don't know if I'm doing something wrong or if pabot has some bugs!
So thanks for reading me, and see you ;)!
UPDATE : To avoid all these problems, I wrote this little tool : https://github.com/bastienjalbert/aptr
For your screenshots issue, I'm assuming you're using Jenkins' Robot Framework plugin. What happens here is, that on post-execution of your tests, the plugin will move your output, report and log to the current build's workspace, to display it in the build's dashboard. The screenshots on the other hand, are by default, not moved.
In your job configuration, under post-build actions, you should have publish Robot Framework test results. Click the advanced button, to see all options, and you should have a field labeled as other files to copy
In this field you add the path that matches your selenium screenshots (probably just *.png)
Now, after your test execution, the plugin should move your screenshots in the same location as your log.html and they will be able to display in the log.
You can actually find this, and more information that may be relevant to you, in the plugin's page
I think you will want to use
rebot --merge output1.xml output2.xml output3.xml ....
From
rebot --help
(...)
Options
=======
-R --merge When combining results, merge outputs together
instead of putting them under a new top level suite.
Example: rebot --merge orig.xml rerun.xml
(...)
Related
I have an android app for which I wrote tests to take screenshots (note: I do not use the screengrab of fastlane). I then extract the files from the phone into a temp folder via adb. Now I want to upload those via a fastlane supply command (and nothing else should be uploaded). The task executes successfully but I can't see my screenshots when I check the Google Play Console, so I suspect that my directory structure is wrong.
I've tried various things but by comparing it to a screenshots of their setup doc I assume it has to be something like fastlane/metadata/en-US/screenshots
Or are there any requirements for the file names? Thanks in advance.
The easiest way to get the proper directory structure is to run fastlane supply init as described here.
If you want to have these metadata (which includes the screenshots) somewhere else, you can use the metadata_path parameter to change that.
And if you want to limit the upload to just screenshots, use the several skip_upload_ parameters that are available to turn off things.
I found the required structure here
So it's fastlane/metadata/android/en-US/images/phoneScreenshots.
I'm compiling in debug mode a certain .so in the Android-Native layer which by default outputs to the default logging file.
However, I do not want this data to be saved to the logs in the first place because it will overwhelm the logging file; I plan on streaming it off of the device.
POSSIBLE SOLUTIONS
Modify the .so code to output the debug info to NOT the logging file.
Definitely the most straightforward way but I'd prefer not to do this since this'll require modification of the .so. I agree this should be trivial modification but I have a requirement to modify the .so as little as possible.
Is there a way to create an alias for a file, pipe all writes to it through my app (like tee) and selectively allow writes to the real file?
Is there a built-in logcat filtering tool that can do this filtering for me with some regexs?
First of all, the first solution is indeed the trivial and the better one - since it remains in the code unlike the logcat filtering that might get deleted.
If the requirement is to modify the .so as little as possible, there is no real problem - you modify the code using #ifdef for debug only, so the code changes for debug mode, but the generated .so for release does not change. This will give you a very good control - but in case you have a lot of 'special' debug code, it will make your code a little ugly...
Edit:
If you want to write to you own log, you can run logcat using Runtime class, with your own parameters - look here: Using GREP command to filter logcat in Android.
Regarding the logcat filtering - you can use regex in the filtering in Android Studio. You have a 'regex' check box next to the input box. See here for more details: How to exclude certain messages by TAG name using Android adb logcat?
This might be a nit-picky thing, but in Xamarin when running an Android app, it dumps tons of lines in the console that start with [Mono]
Is there any way to disable these logs?
Thanks in advance
This can be done by changing the state of Monos execution environment on the device; which is just a set of environment variables that alters Monos behaviour (be it garbage collection, logging etc). In this case, to alter the logging behavior we need to modify the values stored in the environment variables MONO_LOG_LEVEL and MONO_LOG_MASK.
Xamarin.Android offers 2 mechanisms developers can use to change the execution environment:
1. Using adb shell setprop debug.mono.env. This can be done as a post build action.
2. Using an environment build file to change the execution environment state per project.
I prefer to use method 2 as it's easier to edit a text file than changing build actions. Do this using the steps outlined below.
Adding An Environment File
Add a plain text file called environment.txt to the root path of your Xamarin.Android project.
Right click on environment.txt and set its build action to AndroidEnvironment.
The environment file is series of key=value pairs seperated by newlines. For logging, we can set the following variables:
MONO_LOG_LEVEL
debug
info
message
warning
critical
error
MONO_LOG_MASK
asm
dll
cfg
all
type
gc
For example, we can ignore most messages by filtering MONO_LOG_LEVEL by error:
environment.txt
MONO_LOG_LEVEL=error
Background reading:
Android Environment
Mono - Logging Runtime Events
What I do is below. It's still imperfect as the window holds 10k logs, including the hidden ones, so mine eventually disappear. Additionally, I can't seem to copy from the window.
1) Use a tag of "AAA" for all my logs.
2) View the output in Tools>Android>Device Log
3) Sort alphabetically by Tag.
4) I find that the outputs beneath mine are still distracting. I can click the "filter" for the Tag and uncheck everything but mine. Annoyingly, I have to repeat this step periodically as new tags are not filtered by default.
I've spent almost a week on this now, trying to get FFmpeg "Angel"/"Happiness" to build for Android.
I've tried build scripts from all over the internet to no avail. I got closest was using this. As the author himself says the script doesn't work for newer versions of FFmpeg due to this bug, which has been dismissed on that ticket saying "I found a Makefile that does it." This was dis-heartening, being the only post on all of the vast Google world that was anywhere close to my problem.
So, question time:
Is there a way to get around the above bug? I'm trying to use the newest ffmpeg API, and "Love" is just giving me "undefined reference" errors while trying to use av_encode_video2(), and av_free_frame(). The code I was working on the lines of is at the ffmpeg git repo, under /doc/examples/decoding_encoding.c (the function starting on line 338).
Update: So they've done away with codec_names.sh in "Angel". Sorry didn't notice that before, but the problem persists in a different avatar now. With every build attempt the compiler throws a certain
start ndk-building...
/home/<user>/android-ndk/build/core/build-binary.mk:41: *** target file `clean' has both : and :: entries. Stop.
Say whatnow!?
Given the lack of any response at all, I'm assuming people who know their shit in this topic are really busy putting their skills to use with whatever they managed to compile. For the ones like me who scraped each corner of the web for an answer that makes any little sense, I have a.. more than decent workaround.
The Guardian Project, an awesome resource on github, has the perfect project set up for building an ffmpeg binary with all the settings of your choice. But just the one big problem of getting it to successfully build sans the "Unable to create executables" error.
So.. there's a way out there too. Less flexible, but it saves you from losing any more hair than I'm sure you (like me) already have. Head out here and profit.
From running the file command I noticed this binary was dynamically linked, that seemed weird, but it works.
Also, you'll have to run the chmod command before using it on the device (being a binary file and all). So pop it into your res/raw/ folder, load it up when needed and edit those videos like there's no tomorrow!
I started my emulator with ./emulator -trace profile -avd emulator_15. I then tracked down the trace files to ~/.android/avd/rodgers_emulator_15.avd/traces/profile, where there are six files: qtrace.bb, qtrace.exc, qtrace.insn, qtrace.method, qtrace.pid, qtrace.static. I can't figure out what to do with these files. I've tried both dmtracedump and traceview on all of the files, but none seem to generate any output I can do anything with.
How can I view the proportion of time taken by native method calls on Android?
You need to use tracedmdump to convert the output. This is a shell function defined in build/envsetup.sh in the full Android sources. If you're using the SDK, rather than building from a full tree, I'm not sure this will work.
(If you don't have the sources and want to take a peek at the tracedmdump function, you can see it here.)
If you used emulator -trace profile, you'd run tracedmdump profile. This will dig through various binaries to retrieve symbolic information and associate it with the trace data, generating an HTML summary and a traceview-compatible trace file.
It's worth noting that the VM will execute more slowly with profiling enabled (the interpreter has overhead on every method call and return, and it's running in the slower "debug" interpreter), while native code continues to run at full speed, so you have to be careful when drawing conclusions.
General comment: don't forget to use F9 or one of the method calls to start/stop the tracing -- the -trace flag just enables the feature.
In order to use those six files, there are other scripts in the same directory as that of dmtracedump such as read_pid, read_trace, profile_trace etc. U should first run post_trace on the trace directory containing the six files, then you can use any one of them to get profile info such as how often each basic block executes, the pids they belong to etc.