I have code that prints out the output of adb logcat into a wx.TextArea box, this all works great, click the button and logcat prints out, it prints out more when phone taps are made and everything works fine.
toolsDir and pkgName are both strings.
params = [toolsDir + "\\adb.exe", "logcat"]
p = Popen(params, stdout=subprocess.PIPE, bufsize=1)
for line in p.stdout:
self.progressBox.AppendText(line.decode('utf-8'))
However I have adapted this code to only print out the logs that are for a particular app, this is done using the windows 'findstr' function, adb logcat | findstr myApp. The code below initially works, but then stops, and doesn't display anything else whatever happens, button tapped, app closed etc.
Its like the buffer has reached the end and doesn't process any further events.
args = [toolsDir + '\\adb.exe', 'logcat']
args2 = ['findstr', pkgName]
process_adb = subprocess.Popen(args, stdout=subprocess.PIPE, shell=False)
process_fs = subprocess.Popen(args2, stdin=process_adb.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
for line in process_fs.stdout:
self.progressBox.AppendText(line.decode('utf-8'))
How come the bottom code, the one that filters the app name, stops printing out real time logs, but the top doesn't? I'm guessing its something to do with the piping of one command to another.
You can use Python to filter lines:
if pkgName in line:
BTW: it can be problem with two processes because probably second waits for EOF (end of file) or other signal (ie. closed pipe) but its stdin is open all the time waiting for new data from first process..
Check this: python-subprocess-interaction-why-does-my-process-work-with-popen-communicate
Related
PROBLEM:
Every time I run this python function in my monkeyrunner.py script, it opens a new background instance of (cmd, adb, and conhost). And so, in my automation script, if I have a loop that uses that 100 times, I'm going to see 100 of each cmd, adb, and conhost running in the background (I know this because I enter "ps" in powershell to get the list of processes.) The purpose of the function, if you're curious, is to look for logcat messages from the USB attached Android tablet, to see when processes are finished, so that the script knows when to command screen touches to move forward with automation testing.
action = "____"
waitTime = 1
def adb(logMessage, action):
start = time.time()
p = subprocess.Popen("adb logcat -v time", shell=True, cwd="C:\Users\<USERNAME>\AppData\Local\Android\sdk\platform-tools", stdout=subprocess.PIPE)
for line in p.stdout:
if logMessage in line:
print("Found message!")
break
pass
else:
continue
QUESTION:
How can I use "subprocess" to open adb WITHOUT opening a new instance each time? Is there a way to close the subprocess in the same function?
2 things.
adb logcat is a blocking call. It doesn't return unless you send it a SIGINT (ctrl +c). So inside a script you have to send it the "-d" flag. If you forget your script will keep waiting.
you can wait for a subprocess to complete with p.wait()
Therefore you can try this
action = "____"
waitTime = 1
def adb(logMessage, action):
start = time.time()
p = subprocess.Popen("adb logcat -d -v time", shell=True, cwd="C:\Users\<USERNAME>\AppData\Local\Android\sdk\platform-tools", stdout=subprocess.PIPE)
#p.wait()
for line in p.stdout:
if logMessage in line:
print("Found message!")
break
pass
else:
continue
This will ensure that every adb session launched to look at logact terminates properly and its output is looked at only after it returns
EDIT: You are absolutely right, p.wait() isn't needed and is actually wrong! because the logcat writes more than 4kb to stdout in yourcase. You could try to use commuinicate instead to help with that
I figured it out. To prevent a new session of adb from opening in the background per each time this function is called, all I had to do is place the "p" variable outside the function.... like this...
p = subprocess.Popen("adb logcat -v time", shell=True, cwd="C:\Users\<USERNAME>\AppData\Local\Android\sdk\platform-tools", stdout=subprocess.PIPE)
action = "____"
waitTime = 1
def adb(logMessage, action):
start = time.time()
for line in p.stdout:
if logMessage in line:
print("Found message!")
break
pass
else:
continue
There is still the issue of a single adb session that opens and does not close after the script is run. So, now, instead of 180 sessions (or more) opening, there is a single one. If I find out how to close the session I will update this ticket.
I want to create an AVD (Android Virtual Device) through command line in python. For that, I need to pass a string n to the stdin. I have tried the following
emulator_create = str(subprocess.check_output([android,'create', 'avd', '-n', emulator_name, '-t', target_id, '-b', abi],stdin=PIPE))
emulator_create.communicate("n")
but it raises the following error
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/home/fahim/Android/Sdk/tools/android', 'create', 'avd', '-n', 'samsung_1', '-t', '5', '-b', 'android-tv/x86']' returned non-zero exit status 1
Process finished with exit code 1
What can I do?
There's something not working with your example. subprocess.check_output() returns the output from the child process you want to execute, not a handle to this process. In other words you get a string object (or maybe a bytes object) which you cannot use to manipulate the child process.
Probably what happens is that your script, using subprocess.check_output(), will execute the child process and wait until it is finished before continuing. But since you are never able to communicate with it, it will finish with a non-zero return value which will raise the subprocess.CalledProcessError
Now, using grep as an example of a command that waits on the standard input to execute something (since I don't have an Android Virtual Device creator installed) you could do this:
#!/usr/bin/env python2.7
import subprocess
external_command = ['/bin/grep', 'StackOverflow']
input_to_send = '''Almost every body uses Facebook
You can also say that about Google
But you can find an answer on StackOverflow
Even if you're an old programmer
'''
child_process = subprocess.Popen(args=external_command,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
universal_newlines=True)
stdout_from_child, stderr_from_child = child_process.communicate(input_to_send)
print "Output from child process:", stdout_from_child
child_process.wait()
It will print "Output from child process: But you can find an answer on StackOverflow", which is the output from grep.
In this example, I have
Used the class subprocess.Popen to create an handle to the child process
Setting arguments stdin and stdout with the value subprocess.PIPE to enables us to communicate later on with this process.
Used its .communicate() method to send a string to its standard input. In the same step, I retrieved its standard output and standard error output.
Printed the standard output retrieved in the last step (just so to show that it is actually working)
Waited that this child process is finished
In Python 3.5, it's even simpler:
#!/usr/bin/env python3.5
import subprocess
external_command = ['/bin/grep', 'StackOverflow']
input_to_send = '''Almost every body uses Facebook
You can also say that about Google
But you can find an answer on StackOverflow
Even if you're an old programmer
'''
completed_process_result = subprocess.run(args=external_command,
input=input_to_send,
stdout=subprocess.PIPE,
universal_newlines=True)
print("Output from child process:", completed_process_result.stdout)
In this example, I have:
Used the module function subprocess.run() to execute the command.
The input argument is the string we send to the standard input of the child process
The return value is used later on to retreive the output of the child process
Now you have to adapt this code to your situation.
So I came across something strange that made me loose some time. I have been trying to print the content of an ArrayList containing string elements, sometimes, an element might contain an empty string, which is fine and absolutely my intention.
So I have something like this:
List<String> l = new ArrayList<String>();
//adding strings in l, sometimes it's an empty string
for (int i=0; i < l.size(); i++) {
Log.w("element in l : ", l.get(i));
}
So here, when the loop is gonna hit the empty string, logcat is simply NOT going to print it out BUT (and here is the root of my confusion), if you have a message following the one that failed to display, suddenly the failed message is going to show up as if it contained the new logcat message. For example if you try logging an empty string like this
Log.w(TAG, <empty string here>);
Logcat is going to output nothing at first, then, when it has a NEW message to display this is what it prints out (in this case the new message is some warning about AudioTrack):
08-21 17:06:02.265 13047-13047/company.myapp W/TAG﹕ [ 08-21 17:06:05.411 766: 937 W/AudioTrack ]
AUDIO_OUTPUT_FLAG_FAST denied by client
I'm interested in knowing how this happens, maybe it can help someone else not getting super confused like I did. I suppose trying to log an empty string triggers some kind of buffer that sits there until it gets something to print, is this a bug?
That is an interesting question. I just tried this in LogRabbit and am able to see the same result.
I took a quick browse through the android source and see that Log.W(...) ends up in native code and getting handled in logd_write.c
This basically writes the data to /dev/log/main (or one of the other logs)
You can get those logs like this:
adb pull /dev/log/events .
adb pull /dev/log/main .
adb pull /dev/log/radio .
adb pull /dev/log/system .
You will need to press cntl-C otherwise the copy will happen forever.
Looking in the raw log in /dev/log/main I see the message does get logged:
<8b>F×U^_<8c>^Y^U^Emfl_MessageList^#Before Empty^#^R^#^#^#!z^#^#!z^#^#
<8b>F×U^_<8c>^Y^U^Emfl_MessageList^#^#^]^#^#^#!z^#^#!z^#^#
<8b>F×U^_ <8c>^Y^U^Emfl_MessageList^#After Empty^#7^#^#^#^#^E^#^#^Z^E^#^#
That gets decoded by struct found in logger.h So I think this is a problem in adb. pull the source code from here: (looks like quite a few of undocumented commands there)
This is the primary function
static int logcat(TransportType transport, const char* serial, int argc, const char** argv) {
char* log_tags = getenv("ANDROID_LOG_TAGS");
std::string quoted = escape_arg(log_tags == nullptr ? "" : log_tags);
std::string cmd = "shell:export ANDROID_LOG_TAGS=\"" + quoted + "\"; exec logcat";
if (!strcmp(argv[0], "longcat")) {
cmd += " -v long";
}
--argc;
++argv;
while (argc-- > 0) {
cmd += " " + escape_arg(*argv++);
}
return send_shell_command(transport, serial, cmd);
}
Looking in there I see that all logcat does is basically this:
adb shell
> exec logcat
So I think the root of the problem is in logcat itself. Logcat.cpp calls into log_read.c
Based on my quick read through things what I think is happening is the message is not terminated properly. The empty message does not show up until another message is appended and the first message overruns and shows the second message because it has the appropriate termination.
I want to run any app (say Settings) after rebooting tablet. Can I use os.system or do I have to use other methods.
import os,time
for i in range(0,3):
os.system("adb reboot")
time.sleep(60)
Yes, you can use os.system to execute ADB commands. If you want to validate the command executed successfully, take a look at the check_output(...) function which is apart of the subprocess library. This code snipet is how I choose to implement the check_output function. For the full code look here.
def _run_command(self, cmd):
"""
Execute an adb command via the subprocess module. If the process exits with
a exit status of zero, the output is encapsulated into a ADBCommandResult and
returned. Otherwise, an ADBExecutionError is thrown.
"""
try:
output = check_output(cmd, stderr=subprocess.STDOUT)
return ADBCommandResult(0,output)
except CalledProcessError as e:
raise ADBProcessError(e.cmd, e.returncode, e.output)
To launch an application you can use the command am start -n yourpackagename/.activityname. To launch the Settings App, run adb shell am start -n com.android.settings/com.android.settings.Settings. This stackoverflow question shows you in detail the options you can use to start the application via a command line intent.
Other tips:
I created an ADB wrapper written in python along with a few other python utilities that may aid in what you are trying to accomplish. For example, instead of calling time.sleep(60) to wait for the reboot, you use adb to poll the status of the property sys.boot_completed and once the property is set the device has finished booting and you can launch any application. Below is a reference implementation you can use.
def wait_boot_complete(self, encryption='off'):
"""
When data at rest encryption is turned on, there needs to be a waiting period
during boot up for the user to enter the DAR password. This function will wait
till the password has been entered and the phone has finished booting up.
OR
Wait for the BOOT_COMPLETED intent to be broadcast by check the system
property 'sys.boot_completed'. A ADBProcessError is thrown if there is an
error communicating with the device.
This method assumes the phone will eventually reach the boot completed state.
A check is needed to see if the output length is zero because the property
is not initialized with a 0 value. It is created once the intent is broadcast.
"""
if encryption is 'on':
decrypted = None
target = 'trigger_restart_framework'
print 'waiting for framework restart'
while decrypted is None:
status = self.adb.adb_shell(self.serial, "getprop vold.decrypt")
if status.output.strip() == 'trigger_restart_framework':
decrypted = 'true'
#Wait for boot to complete. The boot completed intent is broadcast before
#boot is actually completed when encryption is enabled. So 'key' off the
#animation.
status = self.adb.adb_shell(self.serial, "getprop init.svc.bootanim").output.strip()
print 'wait for animation to start'
while status == 'stopped':
status = self.adb.adb_shell(self.serial, "getprop init.svc.bootanim").output.strip()
status = self.adb.adb_shell(self.serial, "getprop init.svc.bootanim").output.strip()
print 'waiting for animation to finish'
while status == 'running':
status = self.adb.adb_shell(self.serial, "getprop init.svc.bootanim").output.strip()
else:
boot = False
while(not boot):
self.adb.adb_wait_for_device(self.serial)
res = self.adb.adb_shell(self.serial, "getprop sys.boot_completed")
if len(res.output.strip()) != 0 and int(res.output.strip()) is 1:
boot = True
In an android app I am trying to get my application log messages and saving them to file
I am using the code below.
I am using a different TAG for each of my class and there are several of them.
doing logcat -d gives me all irrelevant messages..
putting my package name like
logcat -d myapp.com:I *:S
does not work the results are empty but if I do
logcat -d MYCLASS1TAG:I MYCLASS2TAG *:S
then it works, but I have many classes..
how can I just put my package name and get results ..??
try {
Process process = Runtime.getRuntime().exec("logcat -d");
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
while ((line = bufferedReader.readLine()) != null)
{
// write to my file here
}
} catch (IOException e) { }
I'm not sure how to do it from the command line, but the ADT plugin for Eclipse lets you filter by application.
edit: I was curious, so I looked through the ADT source to figure out how ADT does it. The long and short of it is that it uses the -v long option to include the PID with each message, and it keeps a Map from a PID to an app name. The relevant source code files are in the package com.android.ddmuilib.logcat.LogCatMessageParser and com.android.ddmuilib.logcat.LogCatPidToNameMapper.
So, one workaround I can think of is to call up a shell (adb shell), figure out your PID using ps, and then pipe the output of adb logcat to grep:
adb logcat -v long | grep <your PID>
It will be a bit of a pain since your PID will change every time you run the app, but that's how you can do it in a pinch.
edit: I just noticed that the long format actually prints the PID on one line and the message on the next line, so you may need to use something like awk instead of grep. I'll test a few things out and post a followup.
So as I understand it you are trying to run logcat on the device to grab and save to file. Unfortunately in the current command line the filterspec only supports the tag:priority format. You will need to run the output through your own line by line filter. You can probably automate this by figuring out your PID and then filtering lines based on that.