Develop Reference

Reducing AOSP kernel image size

I am trying to build an AOSP kernel, but I can't make the resulting boot.img small enough to flash and boot it. It contains the compressed kernel with appended dtbs and the initramfs. My intention is to fuzz-test drivers using Syzkaller. I'm currently using a Pixel XL (marlin). I use kernel branch android-msm-marlin-3.18-pie-qpr3 (tried related ones as well) and I'm trying various gcc-based toolchains. For AOSP I picked Pie (tag android-9.0.0_r46, build PQ3A.190801.002). However, please note that this is a general question, it's not fuzzing-, kernel- or device-specific.
My requirements are:
Keep vendor-introduced drivers (because they are what I want to fuzz)
Enable KASAN and KCOV (and their dependencies)
Enable CONFIG_DEBUG_INFO
Keep printk and the like
Things I tried and ideas I have:
Tuning kernel config - This helps to some extend, but it's just not enough. I'm already using for example CONFIG_CC_OPTIMIZE_FOR_SIZE=y, CONFIG_CORE_SMALL=y, CONFIG_NET_SMALL=y, CONFIG_KASAN_OUTLINE=y, CONFIG_TRIM_UNUSED_KSYMS=y, CONFIG_SLOB=y, ...
Compress kernel (and/or initramfs) with xz or lzma - The idea seems promising. I noticed the kernel build system for arm64 neither supports xz nor lzma out of the box (not sure why?), so I patched it myself to get it building. The build succeeds and the resulting Image.xzkern-dtb or Image.lzma-dtb seems okay, however when attempting to boot it (serial debugging logs):
[5710] partition_enable_wp: group 0 not defined
[5720] DTB offset is incorrect, kernel image does not have appended DTB
[5720] Device info 0x00000131/00010001/0x00010001/0, pmic 0x20009/0x455013/0x0/0x0
[5740] ERROR: Appended Device Tree Blob not found
[5740] panic (frame 0x83a86848):
...
I verified that the DTBs are present and the kernel is compressed as intended. My question here: Is it possible at all? Does the Android bootloader even support xz/lzma? I hardly found any info about that online. Is there any way to use xz or lzma compressed kernels to boot Android?
Resize the boot partition - It seems difficult but possible, not sure if worth attempting. Can you recommend any resources or tools for that?
Removing unnecessary features from initramfs
Link-Time Optimization - I could not get it to work with the AOSP kernel, no matter what gcc version I use. Any tips?
Building features as modules instead of built-in - The build won't succeed with any config I try, even when building only single features as module (with different compiler or linker errors depending on the config). Do you have any guides or tips?
I also followed the tutorial on building a Pixel kernel with KASAN and KCOV on the AOSP homepage (link). I had some problems with it, but eventually (using Linaro GCC 5.5.0, with some config changes and without CONFIG_DEBUG_INFO) I could get it building and booting. Nice as a proof-of-concept but not what I need, since I had to skip some required features.
I'm pretty much stuck and already spent some weeks on that problem. Are there any other options? I'd be happy to get it booting in any way. I'll happily provide any details and logs, but felt like it's already a very long post. Thank you a lot in advance!

I found a solution working for me. It's a close call though, probably not enough size reduction for different situations. Anyways, it might help others.
I'm using pigz instead of the default gzip tool, which offers an even higher compression level with the -11 switch. Increasing the number of optimization iterations with -I 45 further improved it (default is 15). This squeezes out the last bit that I need: The resulting Image.gz-dtb is about 2.5% smaller than before.
These are the changes I made to the kernel build system (assuming pigz will be in your PATH at build time):
diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
index b7cf2a498c19..7dfd438e997b 100644
--- a/arch/arm64/boot/Makefile
+++ b/arch/arm64/boot/Makefile
## -30,7 +30,8 ## $(obj)/Image: vmlinux FORCE
$(call if_changed,objcopy)
$(obj)/Image.gz: $(obj)/Image FORCE
- $(call if_changed,gzip)
+# $(call if_changed,gzip)
+ $(call if_changed,pigz)
$(obj)/Image.lz4: $(obj)/Image FORCE
$(call if_changed,lz4)
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index c2b437eb23d5..73d4581f7531 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
## -272,6 +272,11 ## quiet_cmd_gzip = GZIP $#
cmd_gzip = (cat $(filter-out FORCE,$^) | gzip -n -f -9 > $#) || \
(rm -f $# ; false)
+
+quiet_cmd_pigz = PIGZ $#
+cmd_pigz = (cat $(filter-out FORCE,$^) | pigz -n -f -11 -I 45 > $#) || \
+ (rm -f $# ; false)
+
# DTC
# ---------------------------------------------------------------------------
It flashes and boots fine, even boots in a reasonable time, as opposed to what I had before.

Missing Kernel DTB after creating signed Android images

I'm porting android to a display device, and have nearly completed this. The device use the Freescale/NXP i.MX6 Dual Lite Soc. The Android version used is Android 8.0.0, and the build is based on the Board Support Packages from NXP/Freescale (link below).
https://www.nxp.com/support/developer-resources/software-development-tools/i.mx-developer-resources/android-os-for-i.mx-applications-processors:IMXANDROID?tab=Design_Tools_Tab
The OS builds fine, and the images (u-boot, boot.img, system.img, vendor.img) resuling from the "make" process works perfercly fine on the device. So my last step is basically to sign the images, and this is where I struggle to get stuff working.
I am following the the guide found here:
https://source.android.com/devices/tech/ota/sign_builds
After completing the steps, I use the now signed images found in the "signed-img.zip" file to flash the device (using the NXP Manufacturing Tool, and not Fast Boot). However, the device now fails to boot the Kernel, giving me an error that the DTB is missing.
Hit any key to stop autoboot: 0
boota mmc0
kernel # 14008000 (8183104)
ramdisk # 15000000 (2036048)
## Booting Android Image at 0x12000000 ...
Kernel load addr 0x14008000 size 7992 KiB
Kernel command line: console=ttymxc0,115200 init=/init video=mxcfb0:dev=ldb video=mxcfb1:off video=mxcfb2:off video=mxcfb3:off vmalloc=128M androidboot.console=ttymxc0 consoleblank=0 ldo_active=on androidboot.hardware=sedevices cma=448M android.selinux=permissive android.dm_verify=disable androidboot.selinux=enforce androidboot.dm_verity=disable androidboot.storage_type=emmc loglevel=8 vt.global_cursor_default=0 buildvariant=userdebug androidboot.serialno=0b2861d4df668b47 androidboot.soc_type=imx6dl androidboot.storage_type=emmc
ERROR: Did not find a cmdline Flattened Device Tree
Could not find a valid device tree
resetting ...
I've narrowed the problem down to the very first step in the guide, where "make dist" is executed in the build directory. This produces a number of ZIP files in the "out/dist" folder, which is processed further in the following steps in the guide. I've tried flashing the device with the images produced in this step (found in the resulting "out/dist/*-img-*.zip" file), and this produces the exact same issue.
So my question is, what does really "make dist" do which cause the DTB to be missing in the "boot.img"? I would've expected it to use the already working "boot.img" found in "out/target/product//". But it instead seems to re-build this image, and in this case not include the DTB. As with so many other aspects of building Android from Source, the workings of "make dist" does not seem to be explained anywhere in the documentation.
I hope anyone with some experience in building Android from source knows something about this, because I seem to be royally stuck.
Just FYI; when I flash the "boot.img" produced after a normal "make", the output after U-boot is as follows:
Hit any key to stop autoboot: 0
boota mmc0
Error: blob decap job completed with errors 0x2000081A
In boota get fastboot lock status error. Set lock status
kernel # 14008000 (8183104)
ramdisk # 15000000 (2036754)
fdt # 14f00000 (40998)
## Booting Android Image at 0x12000000 ...
Kernel load addr 0x14008000 size 7992 KiB
Kernel command line: console=ttymxc0,115200 init=/init video=mxcfb0:dev=ldb video=mxcfb1:off video=mxcfb2:off video=mxcfb3:off vmalloc=128M androidboot.console=ttymxc0 consoleblank=0 ldo_active=on androidboot.hardware=sedevices cma=448M android.selinux=permissive android.dm_verify=disable androidboot.selinux=enforce androidboot.dm_verity=disable androidboot.storage_type=emmc loglevel=8 vt.global_cursor_default=0 buildvariant=userdebug androidboot.serialno=0b2861d4df668b47 androidboot.soc_type=imx6dl androidboot.storage_type=emmc
## Flattened Device Tree blob at 14f00000
Booting using the fdt blob at 0x14f00000
Loading Kernel Image ... OK
Using Device Tree in place at 14f00000, end 14f0d025
switch to ldo_bypass mode!
Starting kernel ...

It seem NXP/Freescale have modified the Build Scripts in their AOSP Board Support Package, and broken the DTB inclusion in the process. It may seem like they where happy when the output from the standard "make" process worked on their Dev Board, and never bothered checking if it was possible to create working signed Release images with it.
The mechanism to include a DTB in the dist package was in place (specifying the location of the DTB in BOARD_KERNEL_DTS), and was also used for their "make otapackage" target. However, that target only generates an unsigned (or signed with dev keys) OTA package, which can not be used with the signing scripts.
In order to make it work as it should, I had to make a small change in the main Makefile which specifies the location of the DTB defined in the BoardConfig.mk. The change was simple enough to make, but the hard part was figuring out where the problem was and how it was intended to work in the first place.
The following patch in "build/make/" fixes the issue, as long as a BoardConfig.mk ONLY specifies one DTB (which at least suits my needs):
diff --git a/core/Makefile b/core/Makefile
index a650565a1..92f3025a9 100644
--- a/core/Makefile
+++ b/core/Makefile
## -621,6 +621,19 ## ifdef INTERNAL_KERNEL_CMDLINE
INTERNAL_BOOTIMAGE_ARGS += --cmdline "$(INTERNAL_KERNEL_CMDLINE)"
endif
+# NOTE! This script has a defect which cause the kernel DTB to be left out when ever 'make dist'
+# is executed. The following addition "fixes" this by adding the first dtb specified in the
+# BoardConfig.mk file. (I would guess in most cases there is never more than one!)
+ifdef dist_goal
+ifndef BOARD_KERNEL_DTS
+ifdef TARGET_BOARD_DTS_CONFIG
+DTS_BOARD=$(word 2, $(subst :, ,$(word 1, $(TARGET_BOARD_DTS_CONFIG))))
+BOARD_KERNEL_DTS="$(KERNEL_OUT)/$(DTS_BOARD)"
+$(info FIXUP: Defining BOARD_KERNEL_DTS:=[$(BOARD_KERNEL_DTS)] for BOOT packaging)
+endif
+endif
+endif
+
INTERNAL_MKBOOTIMG_VERSION_ARGS := \
--os_version $(PLATFORM_VERSION) \
--os_patch_level $(PLATFORM_SECURITY_PATCH)

Custom included extras Android (AOSP) Compilation

For reasons, I want to compile the AOSP 4.3.3 tree with the 'user' (aosp_deb-user) build (and not the user-debug / eng builds).
However I would like to specify that I:
would like the su package included (system/extras)
possibly (but less importantly) remove some things I do not need in my testing (therefore speed compilation up) - such as chromium app / camera app / whatever.
Could anyone let me know how to do this?
I already attempted changing the build tag in the su 'Android.mk' to user (which was the old way of doing it) - but it now gives me an error stating I must request in my product packages, however i am unsure where this is.
thank you,

It's (mainly) the PRODUCT_PACKAGES variable that controls which modules are installed. That variable is set in the product makefiles, which form hierarchies of makefiles. The leaf file for a concrete product is usually device/vendorname/productname/productname.mk or similar, in your case device/asus/deb/aosp_deb.mk. In that file you'll find a couple of inclusions:
$(call inherit-product, device/asus/deb/device.mk)
$(call inherit-product, $(SRC_TARGET_DIR)/product/aosp_base.mk)
If you following the trail of inherit-product breadcrumbs you'll eventually encounter all PRODUCT_PACKAGES assignments, at least one of which will list the modules that you want to exclude. (The SRC_TARGET_DIR variable in the example above points to the build/target directory.)

QML device deployment

I have set the project to run QML with C++ having slots communication. Now I am stumbled with the message:
void QDeclarativeView::continueExecute()):
file:///data/data/org.qtproject.example.input/files/Input.qml: File not found
I searched around and checked a similar problem, thus I followed that suggestion, changing my project file as follows:
QT += qml quick sensors xml
QT += declarative
SOURCES += \
main.cpp
OTHER_FILES += \
Input.qml
HEADERS += \
Input.h
RESOURCES += \
Input.qrc
# Required for deployment (?) :
QML_FILES.source = qml
QML_FILES.target = .
DEPLOYMENTFOLDERS += QML_FILES
# Required for deployment too (?) :
include(qmlapplicationviewer/qmlapplicationviewer.pri)
qtcAddDeployment()
While adding this hack to the project file seems to work for Symbian and Harmattan, for Android the last two lines doesn't fix the problem, as the compiler gives the messages:
./input/qmlapplicationviewer/qmlapplicationviewer.pri:-1: error: No such file or directory
./input.pro:22: 'qtcAddDeployment' is not a recognized test function.
Even for Qt5 should we get worried about the target, even if when we defined the project we already set it for Android device?
So, could someone have any suggestion on how to get the QML deployed to Android device? Maybe some equivalent hack that will do the same magic that worked for Symbian but also could work for Android?

Try to downgrade your Android target from API 19 to 17. Apparently the API 19 is not working properly with Qt 5 for Android yet.
Using qtcAddDeployment and qmlapplicationviewer.pri might also solve your issue. But before going into that direction, you should have a close look in the article from Qt Blog on Android Deployment in Qt 5.2.

How to add a system service to the Android Framework

I want to add my own framework code that runs in the Android "system_server" (handles all the system level services). My service loads a JNI library that talks to a driver I have added to the kernel. The service is designed to allow an app to register a listener with it to get updates from the driver. I found a pretty good blog post (http://www.androidenea.com/2009/12/adding-system-server-to-android.html) that explains how to add a system service, but I cannot get it completely working.
First of all, the post mentions that an "appropriate" Android.mk file should be used to write the client/test application, but does not give an example of this. When I try to build it, I get build errors saying it can't find the service I have added. Could someone give an example of what this might look like?
Also, I'd like to implement this in the vendor directory (or device directory in froyo) rather than in the Android open source code. The blog post mentions that the proper place for this is the vendor directory, but does not give an example of what this should look like. Anyone know?
ANY additional information on implementing your own system service in Android would be helpful. Again my specific workflow is:
Android App -> System Service -> JNI (native) library -> Device Driver

Texas instruments has provided a nice example:
http://processors.wiki.ti.com/index.php/Android-Adding_SystemService
Additionally, the CyanogenMod SystemServer.java has also code for dynamically loading system services as defined in the array "config_vendorServices" in config.xml (see core/res/res/values/config.xml), which I believe can be overwritten in the vendor or device directories using the overlays. This is a CyanogenMod-specific addition, added in this commit:
https://github.com/CyanogenMod/android_frameworks_base/commit/88fff90131f54d45dc496c45127ac1d16ad257df

There are multiply way (or 6 ways, just to be explicit) of adding a system service.
What you tried (Android App -> System Service -> JNI (native) library -> Device Driver) is one of them. You might want check out this article for an in-depth explanation regarding system service implementation patterns.

Follow the below steps for writing your own System Service in android framework.
Write your Own Service/Manager with API's exposed by inheriting the stub.
Create an aidl file for your service to expose & include in build.
Add your service in System Server, you service will start along with all core services.
Register your service context in context impl file.
Use your service in application by calling getSystemService(Context of your service)
PS; if your service get some fatal exception, device will soft reboot, as your service is running under system service.

Here is an example of an Android.mk used to compile a JNItest.c located at system/extras/JNITest. The Android.mk is also inside system/extras/JNITest directory.
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_SRC_FILES:= JNITest1.c
LOCAL_MODULE:= JNITest
#LOCAL_FORCE_STATIC_EXECUTABLE := true
#LOCAL_STATIC_LIBRARIES := libc
LOCAL_SHARED_LIBRARIES := libc
#LOCAL_MODULE_PATH := $(TARGET_OUT_OPTIONAL_EXECUTABLES)
LOCAL_MODULE_TAGS := eng
include $(BUILD_EXECUTABLE)