Develop Reference

Remotely debugging android NDK app using VSCode

I've tried a number of ways to get this to work but to no avail so far.
My requirements are as follows:
To be able to connect using visual studio code & gdb to a debug-enabled APK process running on an android device
The device should not have to be rooted
I expect full visual debugging of C++ NDK code on that device. Java code would be nice too but C++ is what I'm interested in.
I'm aware that the gdb server must run on the device. A python script in the NDK (ndk-gdb) can copy this to the device and execute it, which then launches its own shell which can be used to set breakpoints, which is useful in a pinch but nowhere near as intuitive as a full GUI.
So, given that the gdb-server is now running on the device, I should be able to connect to it.
I'm trying to use this 'launch.json' debug config in vs-code:
{
"name": "Debug App on Device",
"type": "cppdbg",
"request": "launch",
"cwd": "${workspaceRoot}",
"program":"${workspaceRoot}\\android-build\\DebugSys\\system\\bin\\app_process64",
"additionalSOLibSearchPath": "${workspaceRoot}\\android-build\\obj\\local\\arm64-v8a",
"miDebuggerServerAddress": "192.168.1.121:5039",
"setupCommands": [{
"text": "set solib-absolute-prefix ${workspaceRoot}/android-build/",
"ignoreFailures": false
}],
"windows": {
"miDebuggerPath": "C:\\Users\\luthe\\AppData\\Local\\Android\\Sdk\\ndk\\23.1.7779620\\prebuilt\\windows-x86_64\\bin\\gdb.exe",
"MIMode": "gdb"
}
},
But running it gives me this error:
(No connection could be made because the target machine actively refused it).
This probably isn't a firewall issue as I'm able to connect to another server I have running, launched by the app in question on port 8080.
Is there an error in my 'launch.json' or am I going about all of this in entirely the wrong way.
Also, would it make sense for my app to launch the gdb server instead or is it better/neutral to have ndk-gdb do it?

In then end, I eschewed lldb in favour of gdb, which ndk-debug can use (for now, they're depreciating it, even though it works and lldb clearly either does not work or requires some setting that it doesn't advertise to work) if you include the 'no_lldb' flag or set 'use_lldb' to 'False' in ndk-gdb.py.
However, if you want visual debugging, you have to not let ndk-dbg.py launch the local gdb client. I've done this by creating a minimal batch file that connects to the phone and this may be useful for people reading this in the future in order to have a clear overview of what's happening on the android device and on the local machine. This script assumes only one device is attached for clarity.
adb shell forward tcp:5039 localfilesystem:/data/user/0/yourorg.yourapp/debug_socket #forward port 5039 to a temporary file called 'debug_socket'. I believe this is called is a 'Unix domain socket'
adb shell run-as yourorg.yourapp rm /data/user/0/yourorg.yourapp/debug_socket #remove the old socket file
adb push libs\arm64-v8a\gdbserver /data/local/tmp/arm64-gdbserver #push the arm64-gdbserver to a temp dir on the device
adb shell run-as yourorg.yourapp "cp /data/local/tmp/arm64-gdbserver /data/user/0/yourorg.yourapp/arm64-gdbserver" #copy the gdbserver to a directory from which it can be run with the privilages of the app you wish to debug
adb shell run-as yourorg.yourapp chmod 700 /data/user/0/yourorg.yourapp/arm64-gdbserver #Ensure 'arm64-gdbserver' is executable
adb shell run-as yourorg.yourapp /data/user/0/yourorg.yourapp/arm64-gdbserver --once +/data/user/0/yourorg.yourapp/debug_socket --attach 1234 #Run gdb server on device, connecting to the societ and the process '1234' (replace with actual PID of your running process)
Where 'yourorg.yourapp' is the name of your org and app and 1234 is the PID of the process to be debugged on your android device.
In visual studio code, you will need to have C++ extensions and gdb debugger extensions installed.
This is the 'launch.json' script that launches gdb locally and connects to the android device. Note that symbols are needed on the client side but not on the android device, so they can stripped if needed.
{
"name": "Debug pilkapel on device",
"type": "cppdbg",
"request": "launch",
"cwd": "${workspaceRoot}",
"program":"${workspaceRoot}\\android-build\\DebugSys\\system\\bin\\app_process64",
"additionalSOLibSearchPath": "${workspaceRoot}\\android-build\\obj\\local\\arm64-v8a",
"miDebuggerServerAddress": "localhost:5039",
"setupCommands": [
{
"text": "set solib-absolute-prefix ${workspaceRoot}/android-build/",
"ignoreFailures": true
}
],
"windows":
{
"miDebuggerPath": "C:\\Users\\luthe\\AppData\\Local\\Android\\Sdk\\ndk\\23.1.7779620\\prebuilt\\windows-x86_64\\bin\\gdb.exe",
"MIMode": "gdb"
}
}
Also, compiler flags are important for this!
These are the compiler flags I used for the debug build of the library I'm working on:
LOCAL_CPPFLAGS := $(LOCAL_CPPFLAGS) -O0 -g -ggdb -gdwarf64
LOCAL_CPPFLAGS += -gfull -fstandalone-debug -Wl -gno-split-dwarf
LOCAL_CPPFLAGS += -fno-unique-internal-linkage-names -fno-direct-access-external-data
LOCAL_CPPFLAGS += -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes
LOCAL_CPPFLAGS += -Wformat -Werror=format-security -fno-limit-debug-info -fPIC
Some of these may be unnecessary for the task at hand, I did really quite a lot of iteration using lldb but nothing worked.
For gdb, I believe it needs -gdwarf-5 as the debugging format (which I gather is the default). When I specified -gdwarf-4 as part of my experimentations, the gdb server ran but breakpoints did not work.

Ok, so I still can't debug but I can now at least run the debug server and connect to it. It seems there's a problem with my symbol file for my C++ .so library which I'm trying to resolve here
The process I've used to run this on the device is this:
adb shell forward tcp:5039 localfilesystem:/data/user/0/totga.anthracite/debug_socket
adb shell run-as totga.anthracite rm /data/user/0/totga.anthracite/debug_socket
adb shell run-as totga.anthracite /data/user/0/totga.anthracite/arm64-lldb-server gdbserver unix:///data/user/0/totga.anthracite/debug_socket --attach 1234
where '1234' is the process ID on the device
and 'totga.anthracite' is my package name. You need to copy the 'arm64-lldb-server' to the data directory indicated (I'd run this step beforehand) and also remember to delete the 'debug_socket' file - that was messing things up before.
On the VSCode side, add something like this to 'launch.json'
{
"name": "Debug android app on device",
"type": "cppdbg",
"request": "launch",
"cwd": "${workspaceRoot}",
"program":"${workspaceRoot}\\android-build\\DebugSys\\system\\bin\\app_process64",
"additionalSOLibSearchPath": "${workspaceRoot}\\android-build\\obj\\local\\arm64-v8a",
"miDebuggerServerAddress": "localhost:5039",
"setupCommands": [
{
"text": "set solib-absolute-prefix ${workspaceRoot}/android-build/",
"ignoreFailures": true
}
],
"windows": {
"miDebuggerPath": "${ANDROID_NDK_HOME}/prebuilt/windows-x86_64\\bin\\gdb.exe",
"MIMode": "gdb"
}
}
This should connect but, if like me you have no symbols, you still can't debug. Hopefully once that issue is resolved, this will work and I'll be able to debug native C++ code on an android device.

Concatenate variables as commands in bash

I want to declare character variables and then write those variables one after the other in order to form a command. Example:
#!/system/bin/sh
tt=e;rr=c;uu=h;yy=o;
zz=i;ll=f;pp=n;cc=t
x=29
$zz$ll [ "$x"-eq 29 ]
$cc$uu$tt$pp
$tt$rr$uu$yy "yes"
$ll$zz
This code should read:
if [ "$x" -eq 29 ]
then
echo "yes"
fi
This works for the "echo" command but won't work for "if".
Always getting errors: if not found, then not found, fi not found.
I've tired surrounding with quotes and braces.
This is being done on android.

It turns out that i can achieve the desired outcome by utilizing the fact that echo will work regardless. so I echo the entire contents of the shell script in question (test.sh) and run commands in another bash instance reading from stdin.
Modified code now is
tt="e";rr="c";uu="h";yy="o";_1="i";ll="f";pp="n";cc="t"
x=29
"$1" "
$_1$ll [ "$x" -${tt}q 29 ]
$cc$uu$tt$pp
$tt$rr$uu$yy \"yes this file ran without error\"
$ll$_1
"
to run this:
/system/test.sh echo | sh -

Trying to replace supersu with magisk in customrom.zip

What if I want Magisk to be flashed with my custom Rom
but the Rom has update-binary which is a shell script and the updater-script is just a dummy file, how to make?

I found a way :
first add this in update-binary
exec sh META-INF/com/google/android/updater-script "$#"
and then add this in updater-script
package_extract_dir("META-INF/ADD-ONS/magisk", "/tmp/magisk");
run_program("/sbin/busybox", "unzip", "/tmp/magisk/magisk.zip", "META-INF/com/google/android/*", "-d", "/tmp/magisk");
run_program("/sbin/busybox", "sh", "/tmp/magisk/META-INF/com/google/android/update-binary", "dummy", "1", "/tmp/magisk/magisk.zip");
delete_recursive("/tmp/magisk");
and don't forget to add magisk.zip in the main Rom zip.

AWS Device Farm - Failed upload when using the CLI

I am trying to upload an apk to device farm using a bash script, I have used the create-upload endpoint to get a pre-signed url for the put request, and then I am using curl to PUT the apk to the end point.
RESPONSE=$(aws devicefarm create-upload --project-arn %deviceFarmProjectARN% --name platforms/android/build/outputs/apk/android-release-unsigned.apk --type ANDROID_APP --content-type application/octet-stream)
APK_ARN=$(echo "$RESPONSE" | grep arn:aws:devicefarm | gawk 'match($0, "(arn:aws:devicefarm(.*))\"", ary) {print ary[1]}')
APK_PUT_URL=$(echo "$RESPONSE" | gawk 'match($0, "\"url\": \"(.*)\"", ary) {print ary[1]}')
echo "$APK_ARN" >> apkARN.txt
curl $APK_PUT_URL -X PUT --upload-file platforms/android/build/outputs/apk/android-release-unsigned.apk -H "Content-Type:application/octet-stream"
The file upload seems to work, but when I check the status using the returned ARN the status is failed, and the reason is "Invalid application uploaded"
"upload": {
"status": "FAILED",
"contentType": "application/octet-stream",
"name": "platforms/android/build/outputs/apk/android-release-unsigned.apk",
"created": 1437582538.139,
"type": "ANDROID_APP",
"arn": "arn:aws:devicefarm:us-west-2:208791684493:upload:a9153182-8e03-4bc3-a5db-bc02034f1331/190ac7a5-bdc4-46f2-aac2-46bb994f2cf3",
"metadata": "{\"errorMessage\":\"Invalid application uploaded.\"}"
}
However when I upload exactly the same file through the GUI, on the web console, the application is fine, and I can run tests against it.

the upload name only accepts word characters. The upload is failing because of the forward slashes in the name.

shell script in android gives [: not found

I have this script which works on my linux machine
#!/bin/sh
c=1
if [ $c == 1 ]
then
echo c is 1
else
echo c is 0
fi
But when I use this in android as follows:
#!/system/bin/sh
c=1
if [ $c == 1 ]
then
echo c is 1
else
echo c is 0
fi
It gives an error like:
[: not found
EDIT
Is there any other logic to check the value of $c, whether it is 1 or 0 ?
Android shell have problem with [] in if so is there any other way to check the value of c ?

andriod shell sh is actually a link to busybox, and it is invoked as
busybox sh
you need setup [ applets manually
busybox ln -s /your_original_sh_path/busybox [
if you don't know where busybox is put, try list the /system/bin/sh which you give
ls /system/bin/sh
busybox which busybox

generally [ is an alias for test,
in Linux machine test is at
/usr/bin/test
and
if [ $c == 1 ]
is evaluated as
if test "$c" = 1
BUT here in android there is no test
so if with [] will not work in any case...
i will cross compile test for android and check it....!!!

Android does not provide a full UNIX environment, it is not a UNIX operating system. It has some similarities, much like how Windows also has some similarities to UNIX. Some Android devices and ROMs try to provide more of a UNIX-like environment that others, but you cannot rely on most of the standard shell scripting tools being installed if you are thinking about cross-device compatibility.
So for example, if you look at your GNU/Linux system, you can see that test and [ are actually programs. Try this: ls -l /usr/bin/[. Most Android installs do not include test or [. That means that if you want to try to do actual programming with Android's minimal shell environment, you have to use lots of odd tricks. You can install busybox to get a full UNIX shell environment, or you can even build busybox into your app. I do that when I need to include shell scripts in an app (for example, Lil' Debi and Commotion MeshTether).
Here's an example of writing a killall in Android's /system/bin/sh environment: http://en.androidwiki.com/wiki/Android_Shell_tips_and_tricks You can also use the various parameter expansions to create some logic, you can see an example of that in the Barnacle Wifi Tether scripts.

Use bash:
#!/system/bin/bash
or
#!/system/xbin/bash
You can check where your sh binary is pointing to on your Linux machine:
ls -l /bin/sh
Edit
BTW, use:
c=1
if [ $c -eq 1 ]
then
echo c is 1
else
echo c is 0
fi

Think you using the wrong arithmetic operator and there is a syntax error of a missing ";": try
[ $c -eq 1 ];
Also your location for Bash (sh) might be wrong at the top of your file:
#!/system/bin/sh

How about checking that the .sh file doesn't contain a carriage return before line feed.
Windows \r\n -> CR LF
Unix \n -> LF

use /system/bin/cmp for equality test.
if you need numerically test, substitute $(($c == 1)) with $c
#!/system/bin/sh
echo $c >/tmp/a
echo 1 >/tmp/b
if cmp /tmp/a /tmp/b
echo c is 1
else
echo c is 0
fi

I run into this issue also and found a solution (on another site)
if [[ $b -gt 0]]
then
echo 'Hooray it works'
else
echo 'still works'
fi