i'm writing a shell script for android and i need to get the last created directory, i would usually use
ls -t | head -1 but ls -t gives me the error "ls: Unknown option '-t'"
is there another shell command that can order the files by time-stamp or another way to do that in android? the busy box is more limited
It looks like stat is available in BusyBox, which means that you could do something like this:
stat -c '%Y %n' */ | sort -nr | cut -d' ' -f2-
This passes the names of all directories (paths ending in a slash) to stat, which prints the last modification time (seconds since the UNIX epoch) and the filename. These are sorted in reverse numerical order and then the time field is stripped from each line.
This assumes that your directory names don't contain newlines, otherwise the sorting would be messed up.
Related
I'm writing a simple Bash script that simply the call of HadnBrakeCli for render videos.
I also implemented a simple queue option: the queue file just store the line-command it has to call to start a render.
So I wrote a while-loop to read one line at time, eval $line and repeat untill file ends.
if [[ ${QUEUE_MODE} = 'RUN' ]]; then
QUEUE_LEN=`cat ${CONFIG_DIR}/queue | wc -l`
QUEUE_POS='1'
printf "Queue lenght:\t ${QUEUE_LEN}\n"
while IFS= read line; do
echo "--Running render ${QUEUE_POS} on ${QUEUE_LEN}..."
echo "++" && echo "$line" && echo "++"
eval "${line}"
tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue"
echo "--Render ended"
QUEUE_POS=`expr $QUEUE_POS + 1`
done < "${CONFIG_DIR}/queue"
exit 0
The problem is that any command makes the loop to work fine (empty line, echo "test"...), but as soon a proper render is loaded, it is launched and finished correctly, but also the loops exists.
I am a newbie so I tried some minor changes to see what effect I got, but nothing change the result.
I commented the command tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue" or I added/removed IFS= in the while-loop or removed the -r in read command.
Sorry if the question is trivial, but I'm really missing some major part in how it works, so I have no idea even how to search for the solution.
I'll put a sample of a general render in the queue file.
HandBrakeCLI -i "/home/andrea/Videos/done/Rap dottor male e mini me.mp4" -o "/hdd/Render/Output/Rap dottor male e mini me.mkv" -e x265 -q 23 --encoder-preset faster --all-audio -E av_aac -6 dpl2 --all-subtitles -x pmode:pools='16' --verbose=0 2>/dev/null
HandBrakeCLI reads from standard input, which steals the rest of the queue file before read line can see it. My favorite solution to this is to pass the file over something other than standard input, like file descriptor #3:
...
while IFS= read line <&3; do # The <&3 makes it read from FD #3
...
done 3< "${CONFIG_DIR}/queue" # The 3< redirects the file into FD #3
Another way to avoid the problem is to redirect input to the HandBrakeCLI command:
...
eval "${line}" </dev/null
...
There's some more info about this in BashFAQ #89: I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!
Also, I'm not sure I trust the way you're using tail to remove lines from the queue file as they're executed. I'm not sure it's really wrong, it just looks fragile to me. Also, I'd recommend using lower- or mixed-case variable names, since there are a bunch of all-caps names with special meanings, and re-using one of them by mistake can have weird consequences. Finally, I'd recommend running your script through shellcheck.net, as it'll make some other good recommendations.
[BTW, this question is a duplicate of "Bash script do loop exiting early", but that doesn't have any upvoted or accepted answers.]
Using adb shell to run commands on an android device, I get different results when running ls with or without a wildcard ( globbing, i.e * ).
When running ls without a wildcard, the last path is displayed properly. When running ls with a wildcard, the path is displayed with an : in the end of it for some reason. The actual file does not have a : in its path.
My issue is specifically with the last file: /data/data/com.kauf.wrapmyFaceFunphotoeditor/files/DV-com.com.kauf.wrapmyFaceFunphotoeditor-2020-05-17-17-44-30-DEBUG.txt:
it has an : in the end which isn't supposed to be there
Why does using a wildcard in ls add characters to the result path?
Edit, environment details: Windows 10 / Android 7, the code is running on sh. I've ran adb shell to get to this command prompt, and doing it in one line (i.e adb shell su -c ls ...) returns similar results, same for adb shell command ...; also clarified the question.
As described in Why you shouldn't parse the output of ls, ls's behavior is not always well-defined. It's generally safer to use NULs (if you don't have any control or knowledge of filenames) or newlines (if you have reason to be certain that filenames can't contain them) to directly delimit a list of values emitted by the shell. Consider, then:
# output is separated by NULs, which cannot possibly exist in filenames
printf '%s\0' /data/data/com.kauf.wrapmyfacefunphotoeditor/files/DV-*
...or...
# output is separated by newlines; beware of a file named DV-evil<newline>something-else
printf '%s\n' /data/data/com.kauf.wrapmyfacefunphotoeditor/files/DV-*
Note that if you're passing this through extra unescaping layers, it may be necessary to double up your backslashes -- if you see literal 0s or ns separating filenames in your output, that's evidence of same.
Note also that if no matching files exist, a glob will expand to itself, so you can get an output that contains only the literal string /data/data/com.kauf.wrapmyfacefunphotoeditor/files/DV-*; in bash this can be suppressed with shopt -s nullglob, but with /bin/sh (particularly the minimal busybox versions more likely to be available on Android) this may not be available. One way to work around this is with code similar to the following:
# set list of files into $1, $2, etc
set -- /data/data/com.kauf.wrapmyfacefunphotoeditor/files/DV-*
# exit immediately if $1 does not exist
if [ "$#" -le 1 ] && [ ! -e "$1" ]; then
exit
fi
# otherwise, print the list in our desired format
printf '%s\0' "$#"
myapps.txt - contains the list of all packages found through adb shell pm list packages > myapps.txt
package:com.flipkart.android
package:com.android.certinstaller
package:com.android.carrierconfig
package:com.reddit.frontpage
package:com.wapi.wapicertmanage
package:com.brave.browser
Following is the code I wrote in the batch script to copy all the apps in one go from android to PC using ADB.
Secondly, I'm splitting my string by colon(:) such that for example -
string1 contains package and
string2 contains com.google.android.youtube
#echo off
setlocal enabledelayedexpansion
for /f "tokens=1* delims=:" %%i in (myapps.txt) do (
echo j: %%j
set string2=%%j
adb shell pm path !string2! > tmp.txt
set /p new=< tmp.txt
#echo on
#echo new: !new!
#echo off
set "str=%new%"
set "string1=%str::=" & set "string3=%"
del tmp.txt REM delete file after reading from it.
REM creating new folder for each app
mkdir apps_%input%\%string2%
REM pulling app from Android to PC
adb pull %string3% apps_%input%\%string2%
set /a count+=1
echo Done !count!
)
Here's the following output after 2 executions of for loop I'm getting
j: com.flipkart.android
new: package:/data/app/com.flipkart.android-XOmoiAws7zOd07eM1nZIlg==/base.apk
A subdirectory or file apps_2\ already exists.
adb: error: failed to stat remote object 'apps_2\': No such file or directory
Done 1
j: com.android.certinstaller
new: package:/system/app/CertInstaller/CertInstaller.apk
A subdirectory or file apps_2\ already exists.
adb: error: failed to stat remote object 'apps_2\': No such file or directory
Done 2
Please help me why I'm getting this output. Also apps_2 don't exist before execution how it's prompting that it already exists.
But,the same thing is working perfectly in cmd prompt:
mkdir apps_2\com.flipkart.android
adb pull /data/app/com.flipkart.android-XOmoiAws7zOd07eM1nZIlg==/base.apk apps_2\com.flipkart.android
here's the output I received after execution
/data/app/com.flipkart.android-XOmoiAws7zOd07eM1nZIlg==/base.apk: 1 file pulled. 33.8 MB/s (12691794 bytes in 0.358s)
The batch file is not working as expected because of delayed expansion is used only for some, but not all environment variable references inside the FOR command block. Only the environment variable input can be referenced with %input% inside the command block as it is the only environment variable defined outside the command block and not modified inside the command block. All other environment variables are defined/modified inside the command block and referenced inside the command block. For that reason all environment variables except input must be referenced with using ! instead of %.
However, the usage of environment variables is not needed at all for this task.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
if not defined input set "input=1"
set "count=0"
if exist myapps.txt for /F "tokens=2 delims=:" %%I in (myapps.txt) do (
echo Getting path of app "%%I" ...
for /F "tokens=2 delims=:" %%J in ('adb.exe shell pm path "%%I" 2^>nul') do (
echo Path of app %%I is: "%%J"
rem Creating new folder for each app.
mkdir "apps_%input%\%%I" 2>nul
if exist "apps_%input%\%%I\" (
rem Pulling app from Android to PC.
echo Pulling app "%%I" to "apps_%input%\%%I" ...
adb.exe pull "%%J" "apps_%input%\%%I"
set /A count+=1
) else echo ERROR: Failed to create directory: "apps_%input%\%%I\"
)
)
if %count% == 1 (set "PluralS=") else set "PluralS=s"
echo Pulled %count% app%PluralS% from Android to PC.
endlocal
The outer FOR reads one line after the other from myapps.txt. Each line is split up into substrings using the colon as delimiter because of option delims=:. The first colon delimited substring is always package which is of no interest for this task. Therefore the option tokens=2 is used to assign the second substring like com.flipkart.android to the specified and case-sensitive loop variable I.
The inner FOR loop starts in background one more command process with %ComSpec% /c and the command line in the parentheses appended as additional argument. The output of adb to handle STDOUT of the background command process is captured by FOR and processed line by line after started cmd.exe closed itself after adb terminated itself.
The single line output by adb is again split up into substrings using colon as delimiter with assigning again just the second substring to specified loop variable J.
Next a subdirectory is created with application name as directory name with redirecting the error message output on directory already existing or failed to create from handle STDERR to device NUL to suppress it.
Then an existence check for the just created directory is made to verify if it really exists and if this is the case the application is pulled from Android device to PC.
The help output on running cmd /? in a command prompt window explains on last page that a file name (or any other argument string) containing a space or of these characters &()[]{}^=;!'+,`~ must be enclosed in double quotes. For that reason all argument strings referencing the value assigned currently to the loop variables I (app name) and J (app path) are enclosed in ".
All echo command lines and the last if condition can be removed as they are just for getting some progress information during execution of the batch file.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
echo /?
endlocal /?
for /?
if /?
mkdir /?
rem /?
set /?
setlocal /?
Read the Microsoft article about Using command redirection operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command line to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded adb command line with using a separate command process started in background.
Story
I take photos and record videos with my phone camera and keep all of them on my internal storage/sdcard. I periodically back them up on my PC, so I keep these camera photos on PC storage in sync with phone storage.
For years, I've been backing up my phone camera photos to my PC in the following way:
Plug in phone into PC and allow access to phone data
Browse phone storage → DCIM → Camera
Wait several minutes for the system to load a list of ALL photos
Copy only several latest photos which haven't been backed up yet
I figured that waiting several minutes for all photos to load is an unnecessary drag so I downloaded adb platform tools. I've added the folder bin to my Path environment variable (i.e. %USERPROFILE%\Tools\adb-platform-tools_r28.0.3) so that I can seamlessly use adb and not write its full path each time.
The script
I wrote the following script for Git Bash for Windows. It is also compatible with Unix if you change the $userprofile variable. Essentially, the script pulls camera photos between two dates from phone storage to PC.
# Attach device and start deamon process
adb devices
# Initialize needed variables
userprofile=$(echo "$USERPROFILE" | tr "\\" "/") # Windows adjustments
srcFolder="//storage/06CB-C9CE/DCIM/Camera" # Remote folder
dstFolder="$userprofile/Desktop/CameraPhotos" # Local folder
lsFile="$dstFolder/camera-ls.txt"
filenameRegex="2019061[5-9]_.*" # Date from 20190615 to 20190619
# Create dst folder if it doesn't exist
mkdir -p "$dstFolder"
# 1. List contents from src folder
# 2. Filter out file names matching regex
# 3. Write these file names line by line into a ls file
adb shell ls "$srcFolder" | grep -E "$filenameRegex" > "$lsFile"
# Pull files listed in ls file from src to dst folder
while read filename; do
if [ -z "$filename" ]; then continue; fi
adb pull "$srcFolder/$filename" "$dstFolder" # adb: error: ...
done < "$lsFile"
# Clean up
rm "$lsFile"
# Inform the user
echo "Done pulling files to $dstFolder"
The problem
When I run the script (bash adb-pull-camera-photos.sh), everything runs smoothly except for the adb pull command in the while-loop. It gives the following error:
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg
I am not sure why the output is broken. Sometimes when I resize the Git Bash window some of the text goes haywire. This is the actual error text:
adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg': No such file or directory
adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg': No such file or directory
adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg': No such file or directory
I am sure that these files exist in the specified directory on the phone. When I manually execute the failing command in bash, it succeeds with the following output:
$ adb pull "//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg" "C:/Users/User/Desktop/CameraPhotos/"
//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg: 1 file pulled. 15.4 MB/s (1854453 bytes in 0.115s)
The question
I can't figure out what's wrong with the script. I thought the Windows system might be causing a commotion, because I don't see the reason why the same code works when entered manually, but doesn't work when run in a script. How do I fix this error?
Additional info
Note that I had to use // in the beginning of an absolute path on Windows because Git Bash would interpret / as its own root directory (C:\Program Files\Git).
I've echoed all variables inside the script and got all the correct paths that otherwise work via manual method.
camera-ls.txt file contents
20190618_124656.jpg
20190618_204522.jpg
20190619_225739.jpg
Additional questions
Is it possible to navigate to external sdcard without using its name? I had to use /storage/06CB-C9CE/ because /sdcard/ navigates to internal storage.
Why does tr "\\" "/" give me this error: tr: warning: an unescaped backslash at end of string is not portable?
Windows batch script
Here's a .bat script that can be run by Windows Command Prompt or Windows PowerShell. No Git Bash required.
:: Start deamon of the device attached
adb devices
:: Pull camera files starting from date
set srcFolder=/storage/06CB-C9CE/DCIM/Camera
set dstFolder=%USERPROFILE%\Desktop\CameraPhotos
set lsFile=%USERPROFILE%\Desktop\CameraPhotos\camera-ls.txt
set dateRegex=2019061[5-9]_.*
mkdir %dstFolder%
adb shell ls %srcFolder% | adb shell grep %dateRegex% > %lsFile%
for /F "tokens=*" %%A in (%lsFile%) do adb pull %srcFolder%/%%A %dstFolder%
del %lsFile%
echo Done pulling files to %dstFolder%
Just edit the srcFolder to point to your phone camera folder,
plug a pattern into the dateRegex for matching the date interval and
save it as a file with .bat extension, i.e: adb-pull-camera-photos.bat.
Double-click the file and it will pull filtered photos into CameraPhotos folder on Desktop.
Keep in mind that you still need have adb for Windows on your PC.
The problem was with Windows line delimiters.
Easy fix
Just add the IFS=$'\r\n' above the loop so that the read command knows the actual line delimiter.
IFS=$'\r\n'
while read filename; do
if [ -z "$filename" ]; then continue; fi
adb pull "$srcFolder/$filename" "$dstFolder"
done < "$lsFile"
Explanation
I tried plugging the whole while-loop into the console and it failed with the same error:
$ bash adb-pull-camera-photos.sh
List of devices attached
9889db343047534336 device
tr: warning: an unescaped backslash at end of string is not portable
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg
Done pulling files to C:/Users/User/Desktop/CameraPhotos
This time I started investigating why the output was broken. I remembered that windows uses \r\n as newline, which means Carriage Return + Line Feed, (CR+LF), so some text must have been overwritten.
It was because of broken values stored inside the $filename variable.
This is the loop from the script:
while read filename; do
if [ -z "$filename" ]; then continue; fi
adb pull "$srcFolder/$filename" "$dstFolder"
done < "$lsFile"
Since each iteration of the while-loop reads a line from $lsFile in the following form:
exampleFilename.jpg\r\n
It misinterprets the newline symbols as part of the file name, so adb pull tries to read files with these whitespaces in their names, but fails and it additionally writes a broken output.
Adb Photo Sync
This might not be the answer but might be useful for others looking for android photo/files backup solution.
I use this script on my Windows with git bash. This can be easily used for Linux. A common issue with a long backup process is that it might get interrupted and you might have to restart the entire copy process from start.
This script saves you from this trouble. You can restart the script or interrupt in between but it will resume copy operation from the point it left.
Just change the rfolder => android folder, lfolder => local folder
#!/bin/sh
rfolder=sdcard/DCIM/Camera
lfolder=/f/mylocal/s8-backup/Camera
adb shell ls "$rfolder" > android.files
ls -1 "$lfolder" > local.files
rm -f update.files
touch update.files
while IFS= read -r q; do
# Remove non-printable characters (are not visible on console)
l=$(echo ${q} | sed 's/[^[:print:]]//')
# Populate files to update
if ! grep -q "$l" local.files; then
echo "$l" >> update.files
fi
done < android.files
script_dir=$(pwd)
cd $lfolder
while IFS= read -r q; do
# Remove non-printable characters (are not visible on console)
l=$(echo ${q} | sed 's/[^[:print:]]//')
echo "Get file: $l"
adb pull "$rfolder/$l"
done < "${script_dir}"/update.files
I am seeking a way to measure launch times of general apps in Android, and happen to know that
adb logcat -b events | grep am_activity_launch_time
might be one answer.
But I failed to find out what duration the command above shows (i.e., from what event to what event). Any help?
You can use this command
cmd0=add logcat -c
to clear the logs and then execute the below command
cmd1 = "grep am_activity_launch_time Example.txt | grep com.example.example| cut -d ',' -f4-4 > Example.csv"
The values are then saved in a csv file.
Then you can the python commands
os.system(cmd0);
os.system(cmd2);
Then use matplotlib to plot the graph from the generated csv files.
axes=plt.gca()
axes.set_xlim([1,20])
axes.set_ylim([0,4000])
plt.title("Performance Analysis ")
plt.ylabel('Time in milliseconds')
plt.xlabel('Number of itirations (DEVICE:Samsung Galaxy Note 4)')
data1=np.genfromtxt("Example.csv)"
plt.plot(data1,label="Example",linewidth=2.0,color='m')
plt.legend(bbox_to_anchor=(1,1),loc=1,borderaxespad=0.)
plt.show()
Once this is done execute this file in the python cmd.
adb logcat -b all | egrep -n --color 'Displayed *'
12461:11-26 22:43:05.316 1110 1167 I ActivityTaskManager: Displayed mobi.goldendict.android.free/mobi.goldendict.android.GoldenDict: +1s494ms
You could measure app launches time in android by this.