I'm writing a simple Bash script that simply the call of HadnBrakeCli for render videos.
I also implemented a simple queue option: the queue file just store the line-command it has to call to start a render.
So I wrote a while-loop to read one line at time, eval $line and repeat untill file ends.
if [[ ${QUEUE_MODE} = 'RUN' ]]; then
QUEUE_LEN=`cat ${CONFIG_DIR}/queue | wc -l`
QUEUE_POS='1'
printf "Queue lenght:\t ${QUEUE_LEN}\n"
while IFS= read line; do
echo "--Running render ${QUEUE_POS} on ${QUEUE_LEN}..."
echo "++" && echo "$line" && echo "++"
eval "${line}"
tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue"
echo "--Render ended"
QUEUE_POS=`expr $QUEUE_POS + 1`
done < "${CONFIG_DIR}/queue"
exit 0
The problem is that any command makes the loop to work fine (empty line, echo "test"...), but as soon a proper render is loaded, it is launched and finished correctly, but also the loops exists.
I am a newbie so I tried some minor changes to see what effect I got, but nothing change the result.
I commented the command tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue" or I added/removed IFS= in the while-loop or removed the -r in read command.
Sorry if the question is trivial, but I'm really missing some major part in how it works, so I have no idea even how to search for the solution.
I'll put a sample of a general render in the queue file.
HandBrakeCLI -i "/home/andrea/Videos/done/Rap dottor male e mini me.mp4" -o "/hdd/Render/Output/Rap dottor male e mini me.mkv" -e x265 -q 23 --encoder-preset faster --all-audio -E av_aac -6 dpl2 --all-subtitles -x pmode:pools='16' --verbose=0 2>/dev/null
HandBrakeCLI reads from standard input, which steals the rest of the queue file before read line can see it. My favorite solution to this is to pass the file over something other than standard input, like file descriptor #3:
...
while IFS= read line <&3; do # The <&3 makes it read from FD #3
...
done 3< "${CONFIG_DIR}/queue" # The 3< redirects the file into FD #3
Another way to avoid the problem is to redirect input to the HandBrakeCLI command:
...
eval "${line}" </dev/null
...
There's some more info about this in BashFAQ #89: I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!
Also, I'm not sure I trust the way you're using tail to remove lines from the queue file as they're executed. I'm not sure it's really wrong, it just looks fragile to me. Also, I'd recommend using lower- or mixed-case variable names, since there are a bunch of all-caps names with special meanings, and re-using one of them by mistake can have weird consequences. Finally, I'd recommend running your script through shellcheck.net, as it'll make some other good recommendations.
[BTW, this question is a duplicate of "Bash script do loop exiting early", but that doesn't have any upvoted or accepted answers.]
Related
Using adb shell to run commands on an android device, I get different results when running ls with or without a wildcard ( globbing, i.e * ).
When running ls without a wildcard, the last path is displayed properly. When running ls with a wildcard, the path is displayed with an : in the end of it for some reason. The actual file does not have a : in its path.
My issue is specifically with the last file: /data/data/com.kauf.wrapmyFaceFunphotoeditor/files/DV-com.com.kauf.wrapmyFaceFunphotoeditor-2020-05-17-17-44-30-DEBUG.txt:
it has an : in the end which isn't supposed to be there
Why does using a wildcard in ls add characters to the result path?
Edit, environment details: Windows 10 / Android 7, the code is running on sh. I've ran adb shell to get to this command prompt, and doing it in one line (i.e adb shell su -c ls ...) returns similar results, same for adb shell command ...; also clarified the question.
As described in Why you shouldn't parse the output of ls, ls's behavior is not always well-defined. It's generally safer to use NULs (if you don't have any control or knowledge of filenames) or newlines (if you have reason to be certain that filenames can't contain them) to directly delimit a list of values emitted by the shell. Consider, then:
# output is separated by NULs, which cannot possibly exist in filenames
printf '%s\0' /data/data/com.kauf.wrapmyfacefunphotoeditor/files/DV-*
...or...
# output is separated by newlines; beware of a file named DV-evil<newline>something-else
printf '%s\n' /data/data/com.kauf.wrapmyfacefunphotoeditor/files/DV-*
Note that if you're passing this through extra unescaping layers, it may be necessary to double up your backslashes -- if you see literal 0s or ns separating filenames in your output, that's evidence of same.
Note also that if no matching files exist, a glob will expand to itself, so you can get an output that contains only the literal string /data/data/com.kauf.wrapmyfacefunphotoeditor/files/DV-*; in bash this can be suppressed with shopt -s nullglob, but with /bin/sh (particularly the minimal busybox versions more likely to be available on Android) this may not be available. One way to work around this is with code similar to the following:
# set list of files into $1, $2, etc
set -- /data/data/com.kauf.wrapmyfacefunphotoeditor/files/DV-*
# exit immediately if $1 does not exist
if [ "$#" -le 1 ] && [ ! -e "$1" ]; then
exit
fi
# otherwise, print the list in our desired format
printf '%s\0' "$#"
I have a binary that repacks android kernel (not mkbootimg).
I'm making a script to automate the process.
I don't want any output on the screen. So I have redirected the output to a file named foo.log.
My current working folder is data/local/working/.
The command is - kernel_make "$zImage" "$ramdisk" "$cmdline" "$image" &> data/local/working/foo.log
What I've to do is, if the last line of the output (foo.log) is this -
"targed file $1 is patched sucesfully, enjoy new kernel"
Then return 0 and exit. Else return 1 and exit.
I'm trying to do with grep, but as I'm new to command line, do don't have any idea of doing it the right way.
Please be free to ask if anything more is required..
a common usage for that would be grep -q 'pattern', it doesn't generate any output, but you can check the return value of the command to decide if the pattern was found. See this example:
kent$ echo "foo"|grep -q bar
kent$ echo $?
1
kent$ echo "bar"|grep -q bar
kent$ echo $?
0
hope this is what you are looking for.
To extract the last line, you can use tail command.
In fact, you can also do it with awk in one shot:
..cmd with..output|awk 'END{if (!/pattern/)exit 1}'
then you can check the $?
I am seeking a way to measure launch times of general apps in Android, and happen to know that
adb logcat -b events | grep am_activity_launch_time
might be one answer.
But I failed to find out what duration the command above shows (i.e., from what event to what event). Any help?
You can use this command
cmd0=add logcat -c
to clear the logs and then execute the below command
cmd1 = "grep am_activity_launch_time Example.txt | grep com.example.example| cut -d ',' -f4-4 > Example.csv"
The values are then saved in a csv file.
Then you can the python commands
os.system(cmd0);
os.system(cmd2);
Then use matplotlib to plot the graph from the generated csv files.
axes=plt.gca()
axes.set_xlim([1,20])
axes.set_ylim([0,4000])
plt.title("Performance Analysis ")
plt.ylabel('Time in milliseconds')
plt.xlabel('Number of itirations (DEVICE:Samsung Galaxy Note 4)')
data1=np.genfromtxt("Example.csv)"
plt.plot(data1,label="Example",linewidth=2.0,color='m')
plt.legend(bbox_to_anchor=(1,1),loc=1,borderaxespad=0.)
plt.show()
Once this is done execute this file in the python cmd.
adb logcat -b all | egrep -n --color 'Displayed *'
12461:11-26 22:43:05.316 1110 1167 I ActivityTaskManager: Displayed mobi.goldendict.android.free/mobi.goldendict.android.GoldenDict: +1s494ms
You could measure app launches time in android by this.
I am logging the data coming from top and putting it into a circular set of files. I am not executing top for one set of data and then rerunning for the next set, but instead using a read time out to specify when to go from one log file to the next. This is primarily done this way to remove the startup CPU load cost every time top is executed. The shell script file's name is toplog.sh and looks similar to this:
#!/data/data/com.spartacusrex.spartacuside/files/system/bin/bash
date
echo " Logging started."
fileCmp()
{
test `ls -lc "$1" | sed -n 's/\([^ ]* *\)\{4\}\([0-9]*\).*$/\2/;p'` $2 $3
}
oldest()
{
ls -rc $1 2> /dev/null |head -1
}
file=`oldest /mnt/sdcard/toplog.\*.gz`
echo " Oldest file is $file"
if [ -z "$file" ]; then
x=0
else
file=${file%%.gz}
file=${file##*.}
x=$file
fi
echo " x=$x"
top -d 20 -b | \
while true; do
file=/mnt/sdcard/toplog.$x.gz
while read -t 5 line; do
echo "$line"
done | gzip -c > $file
if fileCmp "$file" -le 300; then
date
echo " Failure to write to file '$file'."
exit
fi
x=$((($x+1)%10))
sleep 14
done
I execute this using nohup so that when the shell dies, this process still runs, like so:
$ nohup ./toplog.sh
But there's a problem. top terminates when I exit the shell session that executed that command, and I'm not exactly sure why. Any ideas?
To clarify, I'm logging on a Android phone. The tools are limited in functionality (i.e. lack some of these switches) and is why I am using top as it contains the output I want.
Version of busybox I'm using is:
BusyBox 1.19.2 (2011-12-12 12:59:36 GMT)
Installed when I installed Terminal IDE.
BTW, this phone is not rooted. I'm trying to track down a failure when my phone responds as if the CPU has spiked and won't go down.
Edit:
Well, I found a workaround. But the reason is a bit hazy. I think it has to do with process management and smells of a bug in the busybox ver that I'm using that was missed during regression testing.
The workaround is to wrap top with a useless loop structure like this: while true; do top; done. Through testing, top never gets killed and never gets respawned, but by wrapping it up, it isn't killed.
Any insights on this?
going to sound stupid, but change your startup command from
nohup ./toplog.sh
to
nohup ./toplog.sh &
the & makes it run as a background process further removing it from the terminal stack.
Running the bash internal command "disown" on your script's process before logging off may prevent it from being signaled.
I'm having an issue with the time command.
I use /usr/bin/time so i can use the -f and -o modifiers.
My script is:
make clean
repo sync -j5
. ./platform_manifest/manual_add_kernel_manifest.sh \
&& . build/envsetup.sh \
&& /usr/bin/time -o log$day$month.log -f "Buildtime: %E" brunch aokp_mako-eng
The script is part of an automated build system for an android ROM.
When it gets to the time part, i get:
/usr/bin/time: cannot run brunch: No such file or directory
Command exited with non-zero status 127
Buildtime: 0:00.00
Brunch works fine with the regular time command, but I cannot route its output to a file, which is why I am using /usr/bin/time in the first place. If there is a way to do this, that is fine for me as I can trim off the real: header with | awk '{ print $2 }'.
Any help is appreciated!
Ubuntu 12.04, all updates installed
I don't know what brunch is, but the most likely reason that a non-builtin time program can't find it, even when the built-in time can, is that it's a shell function or a Bash alias.
Whether or not that's the case . . .
[…] I cannot route [the regular time command's] output to a file […]
You can write:
{ time brunch aokp_mako-eng ; } 2>"log$day$month.log"
to wrap time (and everything else) into a command-list whose STDERR has already been redirected to the file.