Develop Reference

How do I compile the msm android kernel using the AOSP toolchain?

I've been stuck at this for more than a week, so I'd appreciate any help.
What am I trying to do? (you can skip this part if you want)
I'm trying to do android kernel exploitation. That's not my current problem though. To learn kernel exploitation, I need to be able to get a version of android that is vulnerable to a certain vulnerability. So here's what I'm doing:
Going here to select a CVE that I want to learn how to exploit
After selecting a CVE, I need to select a build that is vulnerable to that CVE from here
Now that I have chosen a build (QQ3A.200805.001 in my case), I note its branch name (android-10.0.0_r41 in my case)
Now I need to be able to run this build in the android emulator (qemu) once as a production build, and once with KASAN+KCOV+debugging symbols for gdb
Before I begin, I'm following these guides:
https://source.android.com/devices/tech/debug/kasan-kcov
https://source.android.com/setup/build/building-kernels-deprecated
Google :)
My Setup:
I did this to download everything I need:
git clone https://android.googlesource.com/kernel/msm
cd msm
git fetch --all --tags --prune
git checkout remotes/origin/android-msm-coral-4.14-android10
cd ..
mkdir AOSP
cd AOSP
repo init -u https://android.googlesource.com/platform/manifest
repo sync -j`nproc`
repo init -b android-10.0.0_r41
repo sync -j`nproc`
cd ..
When I went to the AOSP directory and tried compiling with:
source ./build/envsetup.sh
lunch aosp_flame-userdebug # For the Pixel 4
m
It compiled in 4 hours, but it worked.
My First Issue:
I then created this bash script in the main folder to try to compile the msm kernel normally (without KASAN/KCOV):
###########
# Params: #
###########
ARCH=arm64
CONFIG=cuttlefish_defconfig
###########
# Script: #
###########
read -p "Compile with clang instead of gcc? [y/N] " USE_CLANG
if [ $USE_CLANG == y ] || [ $USE_CLANG == Y ]; then
COMPILER=clang
CC_PATH=$(pwd)/AOSP/prebuilts/clang/host/linux-x86/clang-r346389c/bin/
else
COMPILER=gcc
CC_PATH=$(pwd)/AOSP/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/
export CROSS_COMPILE=aarch64-linux-android-
fi
export ARCH=$ARCH
export PATH=$PATH:$CC_PATH
cd msm
make clean
make distclean
make $CONFIG
make CC=$COMPILER
When I run it with GCC, I get bombarded with depreciation warnings and I get an error:
Android GCC has been deprecated in favor of Clang, and will be removed from
Android in 2020-01 as per the deprecation plan in:
https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/+/master/GCC_4_9_DEPRECATION.md
...
Cannot use CONFIG_CC_STACKPROTECTOR_STRONG: -fstack-protector-strong not supported by compiler
When I run it with clang, I also get a ton of warnings and this error:
./arch/arm64/include/asm/stack_pointer.h:8:51: error: register 'sp' unsuitable for global register variables on this target
register unsigned long current_stack_pointer asm ("sp");
...
In file included from ./include/linux/kvm_host.h:14:
./include/linux/signal.h:81:11: warning: array index 3 is past the end of the array (which contains 1 element) [-Warray-bounds]
return (set->sig[3] | set->sig[2] |
Full outputs for the script can be found here (for gcc) and here (for clang). Basically my first issue is that I don't know why I'm getting these compilation errors. So any suggestions would be greatly appreciated.
My Second Issue:
Now, I am checking out the remotes/origin/android-msm-coral-4.14-android10 branch in the msm repo, but I think this isn't the way to go because I need the kernel version that corresponds to a certain build/branch (QQ3A.200805.001 or android-10.0.0_r41 in my case). I think I need to build the exact version that is precompiled in the AOSP repository.
For example, if I go to the AOSP/device/google/coral directory, and run git log, I get something like this:
commit 62d311ad7cfc3e76a5278634427596462069b44d (HEAD, tag: android-10.0.0_r41, tag: m/android-10.0.0_r41, aosp/android10-qpr3-release)
Merge: 7b91fff 0657c80
Author: android-build-team Robot <android-build-team-robot#google.com>
Date: Wed Jun 10 23:45:11 2020 +0000
Merge cherrypicks of [11827366, 11829660, 11829049, 11829662, 11829647, 11829300, 11826697, 11829719, 11829051, 11829663, 11829664, 11829838, 11829052, 11829472, 11829586] into qt-qpr3-release
Change-Id: Ic126de75c24133f8d43df7c9d8f09059a9ca8089
commit 0657c80f016720f70ed6f9618f928a30e71f2b79
Author: wenchangliu <wenchangliu#google.com>
Date: Tue Mar 24 18:11:18 2020 +0800
coral: Add seccomp policy for Codec2 process
Add seccomp policy for Codec2 process.
Bug: 149511958
Test: adb shell killall media.hwcodec
Change-Id: Iab64bd42ead0c5a27769a757007282c2dddf911d
(cherry picked from commit 3628a510a808a52c4ebf69958a8343928e5df3f2)
(cherry picked from commit 45a6093f3d085ca2a531c2607bd12dbc1eff6bf9)
commit 7b91fffc14afb41cf14772566272c72b37c36785
Merge: 2e7f1a3 e8c9484
Author: android-build-team Robot <android-build-team-robot#google.com>
Date: Wed Jun 10 00:37:23 2020 +0000
...
Link for this exact commit/tag can be found here. However, there is no android-10.0.0_r41 tag in the msm source repo, so I don't know which version of the msm repo corresponds to the prebuilt one in the AOSP directory. And.. I don't know where to start looking tbh.
Any help is greatly appreciated.

ndk-gdb cannot find gdb.setup but it is there under x86

Cocos2d-x 3.7.1 + ndk r10e project on Mac trying to debug in Eclipse. Days pass and the errors keep coming...
I can build and run using cocos compile and cocos run with -p android -m debug --ndk-mode NDK_DEBUG=1 from the terminal in proj.android.
My app runs Ok (eventually) on my x86 emulator no problem.
Typing 'ndk-gdb' in the terminal from ./proj.android and I get the error
ERROR: Could not find gdb.setup under ./libs/
BUT the file gdb.setup (and gdbserver & libcocos2dcpp.so) can be seen in ./proj.android/libs/x86 !?
My AndroidManifest.xml includes android:debuggable="true"
My jni/Application.mk includes APP_ABI := x86 (only)
If anyone can tell me how I can diagnose or fix this problem then I guess I owe them a beer (in Yorkshire btw)
UPDATE: I have tried running up GDB using the --verbose flag. The output shows
Using gdb setup init: ./libs/armeabi/gdb.setup
Using app out directory: ./obj/local/armeabi
Guess I need to change to settings/init file somewhere??

I ran into the same problem. It seems something changed in recent version of Android Build tools.
The trick is:
ERROR: Could not find gdb.setup under ./libs/
It expects gdb.setup is in the directory libs but NOT libs/armeabi
so the simple workground is to copy gdb.setup and gdbserver from "libs/cpu" to "libs" and it works like a magic!

This is a bug caused by adb from android sdk.Issue 191085: ndk-gdb issue - Could not find gdb.setup under ./libs/ (even though it's there)
Currently previewl channel's adb whose version is 1.0.35 still has the same problem.
Without revert adb to 1.0.31.A simple workaround would be modify ndk-gdb a little: change the line if [ $? = 0 ]; then which is below adb_var_shell BCFILES run-as $PACKAGE_NAME /system/bin/sh -c "ls lib/*.bc" to if [ $? = 1 ]; then

How to test android Goldfish Kernel 3.10 on Versatilepb Suported QEMU?

I'm trying to port Android on VersatilePB using [Android Goldfish 3.10 kernel](https://android.googlesource.com/kernel/goldfish/+/android-goldfish-3.10).
First, I configured kernel for versatile_defconfig and I successfully built it. I got .config file, renamed it as versatile_config and added Android base configuration to that using the following command
ARCH=arm scripts/kconfig/merge_config.sh versatile_config android/configs/android-base.cfg
Again, I built the kernel successfully. After that, I built the Android Google source code and I have taken ramdisk.img from google_src_v4.4/out/target/product/generic. After that, I tested using following command,
qemu-system-arm -M versatilepb -kernel ./arch/arm/boot/zImage -initrd ramdisk.img -append "console=ttyAMA0 root=/dev/ram0 rw" -serial stdio
but I'm getting the following error. Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
CPU: 0 PID: 1 Comm: init Not tainted 3.10.0 #15
[<c001271c>] (unwind_backtrace+0x0/0xe8) from [<c001129c>] (show_stack+0x10/0x14)
[<c001129c>] (show_stack+0x10/0x14) from [<c0368304>] (panic+0x8c/0x1d4)
[<c0368304>] (panic+0x8c/0x1d4) from [<c001fe94>] (do_exit+0x7f8/0x8d8)
[<c001fe94>] (do_exit+0x7f8/0x8d8) from [<c001ffe0>] (do_group_exit+0x40/0xd4)
[<c001ffe0>] (do_group_exit+0x40/0xd4) from [<c002a75c>] (get_signal_to_deliver+0x178/0x5c4)
[<c002a75c>] (get_signal_to_deliver+0x178/0x5c4) from [<c00109a4>] (do_signal+0xd0/0x434)
[<c00109a4>] (do_signal+0xd0/0x434) from [<c0010ea4>] (do_work_pending+0xa8/0xb8)
[] (do_work_pending+0xa8/0xb8) from [] (work_pending+0xc/0x20)

Somehow related to the "init()" function, where kernel is trying to create the very first process (in userspace) for the first time - this is also where LInux and Android differs. So did you use the QEMU from Android?:
https://github.com/android/platform_external_qemu
And see the discussion here:
https://balau82.wordpress.com/2010/03/22/compiling-linux-kernel-for-qemu-arm-emulator/
there are some hints here:
Run Android app in qemu-arm?
And here has commandline for emulation:
How to compile android goldfish 3.4 kernel and run on emulator

Same android cmake git project compiles well on one Ubuntu machine but does not compile on another Ubuntu machine even the entire project is the same

I'm developing an android cmake project that compiles succesfully on my 32-bit Ubuntu 12.04 machine at work (I will call it WorkMachine). When I want to compile it at home, I installed a 32-bit Ubuntu 12.04 virtual machine (I will call it HomeMachine), I git cloned the project on it, and when I start the compilation, it will block with an error.
CMAKE_CFLAGS are:
CMAKE_CXXFLAGS are:
-- Configuring done
-- Generating done
-- Build files have been written to: /home/raa/Dropbox/Demo/android_build
[ 0%] Built target bitstream
[ 0%] Built target distrat
[ 1%] Built target fisher
[ 1%] Built target gtest
[ 7%] Built target jpeg-8c
[ 7%] Built target map
[ 8%] Built target resampler
[ 9%] Built target timer
[ 92%] Built target fftw3f-3
[ 96%] Built target vlfeat
[ 96%] Built target extract_shared
[ 97%] Built target shared
[ 97%] Built target cssc_train
Linking CXX executable ../../bin/extract
/home/raa/Dropbox/Demo/libraries/fftw-3.3.3/api/configure.c:28: error: undefined reference to 'fftwf_dft_conf_standard'
collect2: error: ld returned 1 exit status
make[2]: *** [../bin/extract] Error 1
make[1]: *** [src/CMakeFiles/extract.dir/all] Error 2
make: *** [all] Error 2
Here I am reporting gcc version (the same on both computers)
gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-linux-gnu/4.6/lto-wrapper
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.6.3-1ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --enable-objc-gc --enable-targets=all --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu --target=i686-linux-gnu
Thread model: posix
gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)
But how is that possible?
It seems the problem is not due to some cmake cache or similar chaches, because if I git clone the project on another directory on the WorkMachine, the code compiles well too.
What can I check in order to resolve the problem? If you need additional details, please ask me.
Thank you for your time.
Riccardo

the solution to Riccardos problem is to change the commandline parameter of ar from r to q (see man page for difference, basically q does not check for duplicats)
do this via
SET(CMAKE_C_ARCHIVE_APPEND "<CMAKE_AR> q <TARGET> <LINK_FLAGS> <OBJECTS>")
you probably want to check if CMAKE_AR is "ar" (might be different on other systems)

whoa! Solved! It was really a weird behaviour from cmake: I had three files conf.c that must be merged into a static library. When cmake run the AR program to create the library, it executed a command that was like:
ar cr dir1/source1.c dir1/source2.c ... dft/conf.c rdft/conf.c
ar r reodft/conf.c dir2/source3.c
so, maybe due to the huge number of object files that were to be linked, cmake splitted ar commands in two command, BUT second command overwrited one of the two file conf.c that were added with the first command, so symbols defined with the first ar command were lost! This is clearly a bug in ar because even if the source file is the same, maybe the content may vary, thus symbols are not to be deleted. Thanks to all! Your suggestions brought me to the right solution!

Loading kernel module in Android kernel

I am listing my problem here.
I have a Google Nexus one a.k.a. "passion" phone with me. Fastboot and adb tools are installed in the phone. And the boot loader is unlocked.
My task: I have to add a linux kernel module to the Android kernel.
What I have done:
I followed the steps in http://source.android.com/source/initializing.html and downloaded the kernel for android-2.3.6_r1 (passion) and have built it. I am also able to flash it on the phone and the new android kernel also works fine. Now what I want is to modify the kernel and add my own kernel module and then flash it on the phone, so that the kernel on the phone is my modified kernel.
Now I have come across two approaches to do this.
1)
Cross Compile my kernel module with the android kernel and push it on the device with adb command. The Makefile I use in the kernel is as follows.
VERSION = 2
PATCHLEVEL = 3
SUBLEVEL = 6
EXTRAVERSION = -00054-g5f01537
obj-m += hello-1.o
KDIR=/home/apurva/android_dir
PWD := $(shell pwd)
all:
make -C $(KDIR) ARCH=arm CROSS_COMPILE=/home/apurva/android_dir/prebuilt/linux- x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi- SUBDIRS=$(PWD) modules
clean:
make -C $(KDIR) ARCH=arm CROSS_COMPILE=/home/apurva/android_dir/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi- SUBDIRS=$(PWD) clean
Now this is not able to generate new hello-1.ko. I do not know why, I guess there is some problem with the VERSION, PATCHLEVEL, SUBLEVEL and EXTRAVERSION values. Are these necessary? I tried these value from android-2.3.6_r1 also but still it does not work. I am not sure what is this EXTRAVERSION value?
I even tried with the hello-1.ko generated from the compiler in my ubuntu. I pushed this hello-1.ko into the emulator with the following adb command.
/root/bin/src/out/host/linux-x86/bin/adb shell mount
/root/bin/src/out/host/linux-x86/bin/adb push hello-1.ko /data
/root/bin/src/out/host/linux-x86/bin/adb insmod /data/hello-1.ko
But that hello-1.ko is not able to insmod and I get the following error.
insmod : Error in init_module() hello-1.ko function not implemented
Whereas the hello-1.c is quite simple:
#include <linux/module.h> /* Needed by all modules */
#include <linux/kernel.h> /* Needed for KERN_INFO */
int init_module(void)
{
printk(KERN_INFO "Hello world 1.\n");
return 0;
}
void cleanup_module(void)
{
printk(KERN_INFO "Goodbye world 1.\n");
}
2)
The second approach of doing this can be placing my source files of the kernel module in the kernel directory of android. May be in the system directory or somewhere else and ask the make to build these source files also along with the other source. But I am not sure where to ask the make process to do so. I tried to do so in main.mk and created a Android.mk file in the source directory of my source files but it did not work. May be this is a better solution but I could not find any help on this.
After doing this, my kernel modules should be able to control the wnic (Wireless Network Interface device) of the android phone. It should be able to put the wnic in sleep mode and then wake it up after receiving command from my kernel module. If you have some pointers on how to do this, that will be a help. I have found that on Android it is controlled through wpa_supplicant private driver. Commands, like:
wpa_cli driver powermode 0 - auto
wpa_cli driver powermode 1 - active
can do my task, but I am not sure since I have not tried. I have not reached that stage.
Please look into this and provide some help/guidance.
Thanks,
Apurva

Kernel modules (KO's) are much easier to work with than a static kernel - as long as the kernel has enabled them. The easiest way to tell is do an "adb shell lsmod". Second is to see if the kernel .config has enabled CONFIG_MODULES=y and CONFIG_MODULE_UNLOAD=y. Lots of info on the web about linux KO development.
Hummm, you're close but it looks like the makefile is screwy. First try to build the hello KO on your host for unit test, then build on your target. Here's a sample makefile I use on an OMAP36xx running gingerbread:
# Makefile for trivial android kernel module
obj-m += mod_hello.o
CROSS_COMPILE=/opt/distros/ARM/bin/arm-none-linux-gnueabi-
TARG_KDIR ?= /opt/android/dal/nook_kernel
HOST_KDIR=/lib/modules/$(shell uname -r)/build
# target creates:
# .<obj>.o: CC command line for the .o, including dependencies
# .<obj>.mod.o.cmd: CC command line for the mod.o, including dependencies
# .<obj>.ko.cmd: LD command line which links the .o and .mod.o to create the .ko
target:
#echo "Make module for target arm"
make -C $(TARG_KDIR) M=$(PWD) ARCH=arm CROSS_COMPILE=$(CROSS_COMPILE) modules
host:
#echo "Make module for host"
make -C $(HOST_KDIR) M=$(PWD) modules
clean:
#echo "cleaning target"
make -C $(TARG_KDIR) M=$(PWD) clean
#echo "cleaning host"
make -C $(HOST_KDIR) M=$(PWD) clean

First check in the .config if the module support is enabled. (CONFIG_MODULES=y and CONFIG_MODULE_UNLOAD=y) if not enable them using menuconfig.
Then place your module on the root of the kernel source and add this to the main makefile you find at the root
core-y := usr/ yourModule/
and this to the yourModule folders makefile
obj-m := yourModule.o