I'm trying to mount the userdata-qemu.img.qcow2 file created by the Android emulator. The following procedure does not work:
sudo qemu-nbd -c /dev/nbd0 ~/.android/avd/Pixel_C_API_27.avd/userdata-qemu.img.qcow2
First command runs well, but running
sudo qemu-nbd -c /dev/nbd0 ~/.android/avd/Pixel_C_API_27.avd/userdata-qemu.img.qcow2
results in this output:
Fehler: /dev/nbd0: unbekannte Partitionstabelle
Modell: Unbekannt (unknown)
Festplatte /dev/nbd0: 3146MB
Sektorgröße (logisch/physisch): 512B/512B
Partitionstabelle: unknown
Disk-Flags:
Basically it cannot recognize a partition table in the image file. You may wonder what's the output of
fdisk /dev/nbd0 -l
so here it is:
Medium /dev/nbd0: 3 GiB, 3145728000 Bytes, 6144000 Sektoren
Einheiten: sectors von 1 * 512 = 512 Bytes
Sektorengröße (logisch/physisch): 512 Bytes / 512 Bytes
I/O Größe (minimal/optimal): 512 Bytes / 512 Bytes
As you could expect already, mounting fails of course, since no partitions can be recognized if the partition table itself cannot be recognized either. Thanks for any help!
Edit: I've just found out that the problem must have to do with Android's userdata encryption. As I've never changed any password the encryption password of the emulator's userdata partition would have to be the default one "default_password". How can I decrypt the image to be able to mount it?
You can mount userdata-qemu.img.qcow2 using the following procedure
convert the image to a raw image
qemu-img convert -O raw userdata-qemu.img.qcow2 udata-raw.img
use losetup to setup a loopback device for mounting
sudo losetup -f -P userdata.img
use losetup -l to see what device was setup
losetup -l
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO
/dev/loop0 0 0 0 0 /path/to/userdata.img 0
mount the drive
mkdir /tmp/mnt
sudo mount /dev/loop0 /tmp/mnt
Then ls /tmp/mnt/ gives
app benchmarktest benchmarktest64 lost+found misc nativetest nativetest64
Related
I trying to get all data from my android phone. I got alot of files named: ".backup".
step 1:
cat userdata_20200107_174037.backup userdata_20200107_174037.backup1 userdata_20200107_174037.backup2 userdata_20200107_174037.backup3 userdata_20200107_174037.backup4 userdata_20200107_174037.backup5 userdata_20200107_174037.backup6 userdata_20200107_174037.backup7 userdata_20200107_174037.backup8 userdata_20200107_174037.backup9 userdata_20200107_174037.backup10 userdata_20200107_174037.backup11 userdata_20200107_174037.backup12 userdata_20200107_174037.backup13 > userdata.img
step 2 if this not works goto step3:
dd if=userdata.img bs=512 skip=1 | gunzip -c | tar xv
gzip: stdin: not in gzip format tar: This does not look like a tar
archive tar: Exiting with failure status due to previous errors
step 3:
simg2img userdata.img userdata_raw.img
Invalid sparse file format at header magi Failed to read sparse file
step 4:
sudo mount -t ext4 -o loop,rw userdata.img /media/phone/
mount: wrong fs type, bad option, bad superblock on /dev/loop8,
missing codepage or helper program, or other error In some cases
useful info is found in syslog - try dmesg | tail or so
i Follow the stack overflow here: https://android.stackexchange.com/questions/141805/how-to-extract-backup-files-created-by-android-stock-recovery
Is there a easy way to backup all data in android ? and how can i do it? hole data include system??
Note:
then i run (file userdata.img) it's data file
and not Android sparse image (file)
Story
I take photos and record videos with my phone camera and keep all of them on my internal storage/sdcard. I periodically back them up on my PC, so I keep these camera photos on PC storage in sync with phone storage.
For years, I've been backing up my phone camera photos to my PC in the following way:
Plug in phone into PC and allow access to phone data
Browse phone storage → DCIM → Camera
Wait several minutes for the system to load a list of ALL photos
Copy only several latest photos which haven't been backed up yet
I figured that waiting several minutes for all photos to load is an unnecessary drag so I downloaded adb platform tools. I've added the folder bin to my Path environment variable (i.e. %USERPROFILE%\Tools\adb-platform-tools_r28.0.3) so that I can seamlessly use adb and not write its full path each time.
The script
I wrote the following script for Git Bash for Windows. It is also compatible with Unix if you change the $userprofile variable. Essentially, the script pulls camera photos between two dates from phone storage to PC.
# Attach device and start deamon process
adb devices
# Initialize needed variables
userprofile=$(echo "$USERPROFILE" | tr "\\" "/") # Windows adjustments
srcFolder="//storage/06CB-C9CE/DCIM/Camera" # Remote folder
dstFolder="$userprofile/Desktop/CameraPhotos" # Local folder
lsFile="$dstFolder/camera-ls.txt"
filenameRegex="2019061[5-9]_.*" # Date from 20190615 to 20190619
# Create dst folder if it doesn't exist
mkdir -p "$dstFolder"
# 1. List contents from src folder
# 2. Filter out file names matching regex
# 3. Write these file names line by line into a ls file
adb shell ls "$srcFolder" | grep -E "$filenameRegex" > "$lsFile"
# Pull files listed in ls file from src to dst folder
while read filename; do
if [ -z "$filename" ]; then continue; fi
adb pull "$srcFolder/$filename" "$dstFolder" # adb: error: ...
done < "$lsFile"
# Clean up
rm "$lsFile"
# Inform the user
echo "Done pulling files to $dstFolder"
The problem
When I run the script (bash adb-pull-camera-photos.sh), everything runs smoothly except for the adb pull command in the while-loop. It gives the following error:
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg
I am not sure why the output is broken. Sometimes when I resize the Git Bash window some of the text goes haywire. This is the actual error text:
adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg': No such file or directory
adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg': No such file or directory
adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg': No such file or directory
I am sure that these files exist in the specified directory on the phone. When I manually execute the failing command in bash, it succeeds with the following output:
$ adb pull "//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg" "C:/Users/User/Desktop/CameraPhotos/"
//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg: 1 file pulled. 15.4 MB/s (1854453 bytes in 0.115s)
The question
I can't figure out what's wrong with the script. I thought the Windows system might be causing a commotion, because I don't see the reason why the same code works when entered manually, but doesn't work when run in a script. How do I fix this error?
Additional info
Note that I had to use // in the beginning of an absolute path on Windows because Git Bash would interpret / as its own root directory (C:\Program Files\Git).
I've echoed all variables inside the script and got all the correct paths that otherwise work via manual method.
camera-ls.txt file contents
20190618_124656.jpg
20190618_204522.jpg
20190619_225739.jpg
Additional questions
Is it possible to navigate to external sdcard without using its name? I had to use /storage/06CB-C9CE/ because /sdcard/ navigates to internal storage.
Why does tr "\\" "/" give me this error: tr: warning: an unescaped backslash at end of string is not portable?
Windows batch script
Here's a .bat script that can be run by Windows Command Prompt or Windows PowerShell. No Git Bash required.
:: Start deamon of the device attached
adb devices
:: Pull camera files starting from date
set srcFolder=/storage/06CB-C9CE/DCIM/Camera
set dstFolder=%USERPROFILE%\Desktop\CameraPhotos
set lsFile=%USERPROFILE%\Desktop\CameraPhotos\camera-ls.txt
set dateRegex=2019061[5-9]_.*
mkdir %dstFolder%
adb shell ls %srcFolder% | adb shell grep %dateRegex% > %lsFile%
for /F "tokens=*" %%A in (%lsFile%) do adb pull %srcFolder%/%%A %dstFolder%
del %lsFile%
echo Done pulling files to %dstFolder%
Just edit the srcFolder to point to your phone camera folder,
plug a pattern into the dateRegex for matching the date interval and
save it as a file with .bat extension, i.e: adb-pull-camera-photos.bat.
Double-click the file and it will pull filtered photos into CameraPhotos folder on Desktop.
Keep in mind that you still need have adb for Windows on your PC.
The problem was with Windows line delimiters.
Easy fix
Just add the IFS=$'\r\n' above the loop so that the read command knows the actual line delimiter.
IFS=$'\r\n'
while read filename; do
if [ -z "$filename" ]; then continue; fi
adb pull "$srcFolder/$filename" "$dstFolder"
done < "$lsFile"
Explanation
I tried plugging the whole while-loop into the console and it failed with the same error:
$ bash adb-pull-camera-photos.sh
List of devices attached
9889db343047534336 device
tr: warning: an unescaped backslash at end of string is not portable
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg
Done pulling files to C:/Users/User/Desktop/CameraPhotos
This time I started investigating why the output was broken. I remembered that windows uses \r\n as newline, which means Carriage Return + Line Feed, (CR+LF), so some text must have been overwritten.
It was because of broken values stored inside the $filename variable.
This is the loop from the script:
while read filename; do
if [ -z "$filename" ]; then continue; fi
adb pull "$srcFolder/$filename" "$dstFolder"
done < "$lsFile"
Since each iteration of the while-loop reads a line from $lsFile in the following form:
exampleFilename.jpg\r\n
It misinterprets the newline symbols as part of the file name, so adb pull tries to read files with these whitespaces in their names, but fails and it additionally writes a broken output.
Adb Photo Sync
This might not be the answer but might be useful for others looking for android photo/files backup solution.
I use this script on my Windows with git bash. This can be easily used for Linux. A common issue with a long backup process is that it might get interrupted and you might have to restart the entire copy process from start.
This script saves you from this trouble. You can restart the script or interrupt in between but it will resume copy operation from the point it left.
Just change the rfolder => android folder, lfolder => local folder
#!/bin/sh
rfolder=sdcard/DCIM/Camera
lfolder=/f/mylocal/s8-backup/Camera
adb shell ls "$rfolder" > android.files
ls -1 "$lfolder" > local.files
rm -f update.files
touch update.files
while IFS= read -r q; do
# Remove non-printable characters (are not visible on console)
l=$(echo ${q} | sed 's/[^[:print:]]//')
# Populate files to update
if ! grep -q "$l" local.files; then
echo "$l" >> update.files
fi
done < android.files
script_dir=$(pwd)
cd $lfolder
while IFS= read -r q; do
# Remove non-printable characters (are not visible on console)
l=$(echo ${q} | sed 's/[^[:print:]]//')
echo "Get file: $l"
adb pull "$rfolder/$l"
done < "${script_dir}"/update.files
I am working on one project requires obtaining the complete app package name inside kernel mode. I realized the package name is also the process name inside kernel. However, the task_struct->comm (process name) can only give me 15 characters long.
Also, fs/proc/base.c proc_get_cmdline() can return the full process name but it is private function. I try to export proc_get_cmdline() to public and invoke from my loadable kernel module, but it always crash when I invoke the public proc_get_cmdline().
Is there any way I can get the complete package name inside kernel? Like read from proc/pid/cmdline, read from mm_struct, etc. Appreciate any code example.
You are not supposed to call proc_pid_cmdline().
It is a non-public function in fs/proc/base.c:
static int proc_pid_cmdline(struct seq_file *m, struct pid_namespace *ns,
struct pid *pid, struct task_struct *task)
However, what it does is simple:
get_cmdline(task, m->buf, PAGE_SIZE);
That is not likely to return the full path though and it will not be possible to determine the full path in every case. The arg[0] value may be overwritten, the file could be deleted or moved, etc. A process may exec() in a way which obscures the original command line, and all kinds of other maladies.
A scan of my opensuse 12.3 system /proc/*/cmdline turns up all kinds of less-than-useful results:
/proc/1/cmdline
/sbin/init showopts
/proc/10/cmdline
/proc/11/cmdline
/proc/1163/cmdline
/sbin/dhclient6 -6 -cf /var/lib/dhcp6/dhclient6.eth0.conf -lf /var/lib/dhcp6/dhclient6.eth0.lease -pf /var/run/dhclient6.eth0.pid -q eth0
/proc/12/cmdline
/proc/13/cmdline
/proc/14/cmdline
/proc/15/cmdline
/proc/16/cmdline
/proc/17/cmdline
/proc/1710/cmdline
/sbin/dhcpcd --netconfig -L -E -HHH -c /etc/sysconfig/network/scripts/dhcpcd-hook -t 0 -h del1-dhp-32429 eth0
/proc/172/cmdline
/proc/185/cmdline
/proc/186/cmdline
/proc/187/cmdline
/proc/19/cmdline
/proc/2/cmdline
/proc/20/cmdline
/proc/21/cmdline
/proc/22/cmdline
/proc/23/cmdline
/proc/25/cmdline
/proc/254/cmdline
/proc/255/cmdline
/proc/26/cmdline
/proc/2671/cmdline
/usr/lib/upower/upowerd
/proc/2674/cmdline
/usr/lib/polkit-1/polkitd --no-debug
/proc/27/cmdline
/proc/2727/cmdline
/usr/lib/udisks2/udisksd --no-debug
/proc/28/cmdline
/proc/285/cmdline
/usr/lib/systemd/systemd-journald
/proc/286/cmdline
/proc/288/cmdline
/proc/29/cmdline
/proc/2913/cmdline
/usr/sbin/cron -n
/proc/2924/cmdline
/usr/sbin/sshd -D
/proc/3/cmdline
/proc/3023/cmdline
/usr/lib/postfix/master
/proc/3090/cmdline
pickup -l -t fifo -u
/proc/3091/cmdline
qmgr -l -t fifo -u
/proc/31/cmdline
/proc/311/cmdline
/usr/lib/systemd/systemd-udevd
/proc/3132/cmdline
/usr/lib/vmware/bin/vmware-vmblock-fuse -o subtype=vmware-vmblock,default_permissions,allow_other /var/run/vmblock-fuse
/proc/3168/cmdline
/usr/sbin/vmware-authdlauncher
/proc/32/cmdline
Works for me in openSUSE 12.3:
for I in /proc/*/cmdline; do echo $I; cat $I | tr '\000' ' '; echo; done
I have Nexus 5 with AOSP ROM 4.4.4.
I'm trying to "pull" boot.img from the device, unpack it, edit "init.rc", repack it to boot.img and flash it to the device.
I follow many guides but none of them have the same scenario as I have:
(http://droidcore.blogspot.co.il/2012/12/how-to-edit-initrc-in-android.html and http://www.digitalinternals.com/mobile/android-mmc-mmcblk-partition-layout/259/)
The device partition is mmcblck so from ls -l /dev/block/platform/msm_sdcc.1/by-name/ I got that the boot is /dev/block/mmcblk0p19:
lrwxrwxrwx root root 1971-02-28 21:30 boot ->
/dev/block/mmcblk0p19
I created the boot.img with the command: cat /dev/block/mmcblk0p19 > /mnt/sdcard/boot.img and pull this boot.img to my pc. This boot.img is ~23 MB.
I unmkbootimg the boot.img with the tool in http://droidcore.blogspot.co.il/2012/12/how-to-edit-initrc-in-android.html and got 2 files: initramfs.cpio.gz which is ~500 KB and kernel.gz which is ~8.4 MB. The details I got are:
Kernel size 8405280
Kernel address 0x8000
Ramdisk size 498992
Ramdisk address 0x2900000
Secondary size 0
Secondary address 0xf00000
Kernel tags address 0x2700000
Flash page size 2048
Board name is ""
Command line "console=ttyHSL0,115200,n8 androidboot.hardware=hammerhead
user_debug=31 maxcpus=2 msm_watchdog_v2.enable=1"
Extracting kernel.gz
... Extracting initramfs.cpio.gz ...
All done.
To recompile this image, use:
mkbooting --kernel kernel.gz --ramdisk initramfs.cpio.gz --base 0x26fff00 --cmdline 'console=ttyHSL0,115200,n8 androidboot.hardware=hammerhead user_debug=31 maxcpus=2 msm_watchdog_v2.enable=1' -o new_boot.img
I unpack initramfs.cpio.gz with the command: gunzip -c initramfs.cpio.gz | sudo sh -c 'cd ../ && cpio -i' (from http://www.wiki.xilinx.com/Build+and+Modify+a+Rootfs) and got many files, init.rc among them. I edited this file.
I repack initramfs.cpio.gz with the command: sh -c 'cd ../ && sudo find . | sudo cpio -H newc -o' | gzip -9 > new_initramfs.cpio.gz and got this file ~500KB.
Now I mkbootimg those 2 files with the command: ./mkbooting --kernel kernel.gz --ramdisk initramfs.cpio.gz --base 0x26fff00 --cmdline 'console=ttyHSL0,115200,n8 androidboot.hardware=hammerhead user_debug=31 maxcpus=2 msm_watchdog_v2.enable=1' -o new_boot.img and I got boot.img which is ~8.5 MB
Now flash the boot to the device and the device is stuck on the first screen (not boot up)
My questions are:
What's wrong?
Is that normal that my new boot.img is only 8.5 MB instead of 23 MB (the original)?
Thank you guys!
************* I FOUND THE PROBLEM *************
THE SOLUTION IS:
I used unmkbootimg from here: http://whiteboard.ping.se/Android/Unmkbootimg
After I run unmkbootimg I got this comment:
*** WARNING ****
This image is built using NON-standard mkbootimg!
OFF_KERNEL_ADDR is 0xFD908100
OFF_RAMDISK_ADDR is 0x00200100
OFF_SECOND_ADDR is 0xFE800100
Please modify mkbootimg.c using the above values to build your image.
so I follow the instructions here: https://gist.github.com/jberkel/1087757 and now it works!!!
I can't give the answer to the first question, but if I were you I'd skip step 4&5 and verify whether the unmkbootimg/mkbootimg process is correct. Only after it's verified, I till tackle with the cpio process, and finally the init.rc editing.
For the second question, google's official boot.img is 8.64MB, so I guess there is nothing wrong with the size.
Ubuntu's abootimg worked for me. I used the update (-u) option.
use ubuntu
I type file ramdisk.img and the Terminal display ramdisk.img: data,
I use cpio and get nothing extracted!
How can I extract such ramdisk.img?
It's highly possible that ramdisk.img actually is a compressed file ramdisk.cpio.gz. To extract it use this commands:
cd /Path/To/Folder/With/ramdisk.cpio.gz/File
mkdir temp_directory && cd temp_directory
gunzip -c ../ramdisk.img | cpio -idm
Check this blog post for more info: http://blog.djodjo.org/?p=536
you can mount images:
mkdir ~/img_mnt
mount -t udf filename.img ~/img_mnt -o loop
Just use p7-zip which is 7z just for POSIX.
7z e ramdisk.e RamdiskExtracted
I'm not sure about repacking and keeping metadata though.
I guess you could mount it as rw??