Develop Reference

Reducing AOSP kernel image size

I am trying to build an AOSP kernel, but I can't make the resulting boot.img small enough to flash and boot it. It contains the compressed kernel with appended dtbs and the initramfs. My intention is to fuzz-test drivers using Syzkaller. I'm currently using a Pixel XL (marlin). I use kernel branch android-msm-marlin-3.18-pie-qpr3 (tried related ones as well) and I'm trying various gcc-based toolchains. For AOSP I picked Pie (tag android-9.0.0_r46, build PQ3A.190801.002). However, please note that this is a general question, it's not fuzzing-, kernel- or device-specific.
My requirements are:
Keep vendor-introduced drivers (because they are what I want to fuzz)
Enable KASAN and KCOV (and their dependencies)
Enable CONFIG_DEBUG_INFO
Keep printk and the like
Things I tried and ideas I have:
Tuning kernel config - This helps to some extend, but it's just not enough. I'm already using for example CONFIG_CC_OPTIMIZE_FOR_SIZE=y, CONFIG_CORE_SMALL=y, CONFIG_NET_SMALL=y, CONFIG_KASAN_OUTLINE=y, CONFIG_TRIM_UNUSED_KSYMS=y, CONFIG_SLOB=y, ...
Compress kernel (and/or initramfs) with xz or lzma - The idea seems promising. I noticed the kernel build system for arm64 neither supports xz nor lzma out of the box (not sure why?), so I patched it myself to get it building. The build succeeds and the resulting Image.xzkern-dtb or Image.lzma-dtb seems okay, however when attempting to boot it (serial debugging logs):
[5710] partition_enable_wp: group 0 not defined
[5720] DTB offset is incorrect, kernel image does not have appended DTB
[5720] Device info 0x00000131/00010001/0x00010001/0, pmic 0x20009/0x455013/0x0/0x0
[5740] ERROR: Appended Device Tree Blob not found
[5740] panic (frame 0x83a86848):
...
I verified that the DTBs are present and the kernel is compressed as intended. My question here: Is it possible at all? Does the Android bootloader even support xz/lzma? I hardly found any info about that online. Is there any way to use xz or lzma compressed kernels to boot Android?
Resize the boot partition - It seems difficult but possible, not sure if worth attempting. Can you recommend any resources or tools for that?
Removing unnecessary features from initramfs
Link-Time Optimization - I could not get it to work with the AOSP kernel, no matter what gcc version I use. Any tips?
Building features as modules instead of built-in - The build won't succeed with any config I try, even when building only single features as module (with different compiler or linker errors depending on the config). Do you have any guides or tips?
I also followed the tutorial on building a Pixel kernel with KASAN and KCOV on the AOSP homepage (link). I had some problems with it, but eventually (using Linaro GCC 5.5.0, with some config changes and without CONFIG_DEBUG_INFO) I could get it building and booting. Nice as a proof-of-concept but not what I need, since I had to skip some required features.
I'm pretty much stuck and already spent some weeks on that problem. Are there any other options? I'd be happy to get it booting in any way. I'll happily provide any details and logs, but felt like it's already a very long post. Thank you a lot in advance!

I found a solution working for me. It's a close call though, probably not enough size reduction for different situations. Anyways, it might help others.
I'm using pigz instead of the default gzip tool, which offers an even higher compression level with the -11 switch. Increasing the number of optimization iterations with -I 45 further improved it (default is 15). This squeezes out the last bit that I need: The resulting Image.gz-dtb is about 2.5% smaller than before.
These are the changes I made to the kernel build system (assuming pigz will be in your PATH at build time):
diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
index b7cf2a498c19..7dfd438e997b 100644
--- a/arch/arm64/boot/Makefile
+++ b/arch/arm64/boot/Makefile
## -30,7 +30,8 ## $(obj)/Image: vmlinux FORCE
$(call if_changed,objcopy)
$(obj)/Image.gz: $(obj)/Image FORCE
- $(call if_changed,gzip)
+# $(call if_changed,gzip)
+ $(call if_changed,pigz)
$(obj)/Image.lz4: $(obj)/Image FORCE
$(call if_changed,lz4)
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index c2b437eb23d5..73d4581f7531 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
## -272,6 +272,11 ## quiet_cmd_gzip = GZIP $#
cmd_gzip = (cat $(filter-out FORCE,$^) | gzip -n -f -9 > $#) || \
(rm -f $# ; false)
+
+quiet_cmd_pigz = PIGZ $#
+cmd_pigz = (cat $(filter-out FORCE,$^) | pigz -n -f -11 -I 45 > $#) || \
+ (rm -f $# ; false)
+
# DTC
# ---------------------------------------------------------------------------
It flashes and boots fine, even boots in a reasonable time, as opposed to what I had before.

Linux Kernel Dynamic Debug settings - persist across a reset

With Dynamic Debugging enabled in the Linux Kernel, I am able to control which prints show up by writing to /sys/kernel/debug/dynamic_debug/control. However, It seems that a reboot resets this file to its default settings (all prints disabled).
Is there a way to make the settings persist across a reset? I have a print which is emitted early in the boot process, and I am not able to get it to print, because the reboot disables this print in /sys/kernel/debug/dynamic_debug/control.

You should be able to add dyndbg='your query here' to the kernel command line using the bootloader configuration or manual options entry feature. This will vary with the bootloader although grub is the most common one
If you are using grub as a bootloader you would have to add it to the kernel command line in /boot/grub/grub.conf for the particular kernel you are launching at bootup.
A blog article about this can be found here. In particular it gives this example:
dyndbg='module xhci_hcd +p'
The article also discusses modifying grub as well.
This kernel howto about dynamic debugging has useful information about dyndbg. This section applies
To activate debug messages for core code and built-in modules during
the boot process, even before userspace and debugfs exists, use
dyndbg="QUERY", module.dyndbg="QUERY", or ddebug_query="QUERY"
(ddebug_query is obsoleted by dyndbg, and deprecated). QUERY follows
the syntax described above, but must not exceed 1023 characters. Your
bootloader may impose lower limits.
These dyndbg params are processed just after the ddebug tables are
processed, as part of the arch_initcall. Thus you can enable debug
messages in all code run after this arch_initcall via this boot
parameter.
On an x86 system for example ACPI enablement is a subsys_initcall and
dyndbg="file ec.c +p"
will show early Embedded Controller transactions during ACPI setup if
your machine (typically a laptop) has an Embedded Controller.
PCI (or other devices) initialization also is a hot candidate for using
this boot parameter for debugging purposes.
If foo module is not built-in, foo.dyndbg will still be processed at
boot time, without effect, but will be reprocessed when module is
loaded later. dyndbg_query= and bare dyndbg= are only processed at
boot.

You can do it as a part of post boot script. Say after Android boots(since i see Android tag added here), it runs certain sh scripts as a part of init procedure. Modify the sh files with the dynamic debug commands added and push it in and reboot .
Refer here for post boot script files info.
https://android.stackexchange.com/questions/6558/how-can-i-run-a-script-on-boot

You can have dynamic_debug for some kernel module foo activated automatically during boot using the /etc/modprobe.d/ directory.
### BEGIN prerequisite
# in case module "foo" is not yet automatically loaded during boot,
# then make it load during boot:
# create a file /etc/modules-load.d/foo.conf with contents "foo"
echo "foo" | sudo tee /etc/modules-load.d/foo.conf
### END prerequisite
# create a configuration file to activate dynamic debug for module "foo"
# at its "module initialization time"
echo "options foo dyndbg=+p" | sudo tee /etc/modprobe.d/foo.conf
Reference: Debug Messages at Module Initialization Time from kernel.org documentation.

How to compile Busybox?

(The i9100 and i9100p phones have Exynos 4210 SoC which includes Cortex A9 dual core 1.2Ghz processor which supports NEON.)
I will compile the latest busybox source snapshot available and upload it for everyone for free on internet and maybe even make my own free BusyboxInstaller.apk (I already downloaded today's 14th March snapshot from the official website) because so many busybox installers have very outdated versions and I want to take advantage of possible optimizations for the Cortex A9 cpu.
(NEON technology is a 128-bit SIMD (Single Instruction, Multiple Data) architecture extension.
It can can accelerate multimedia and signal processing algorithms such as video encode/decode, 2D/3D graphics, gaming, audio and speech processing, image processing, telephony, and sound synthesis.)
1 does such as mean it will also benefit my busybox?
2 What other instructions/anythingelse I can use to optimize for i9100 devices?
3 Can I compile on my phone instead of PC with only 1x2Ghz and 2GB RAM? I think it should be much quicker because it has 2x1.5GHz and 1GB RAM (I overclock only a bit) versus 1x2GHz, am I right?
4 How to compile busybox?
What are these options for?
Force NOMMU build
Additional CFLAGS
Additional LDFLAGS
Additional LDLIBS
all in general configuration and Busybox Library Tuning and debug (I googled for each for a few hours but nothing satisfactory)
What kind of applet links to install?
(./_install) BusyBox installation prefix
**I have eeror when following vinayhunachyai instructions.
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- install
/home/euphoria/Sourcery/busybox/scripts/gcc-version.sh: line 11: arm-none-linux-gnueabi-gcc: command not found
CC applets/applets.o
/bin/sh: 1: arm-none-linux-gnueabi-gcc: not found
scripts/Makefile.build:197: recipe for target 'applets/applets.o' failed
make[1]: *** [applets/applets.o] Error 127
Makefile:372: recipe for target 'applets_dir' failed
make: *** [applets_dir] Error 2

How to compile busybox?
1st download toolchain. You can refer to this: Advice regarding installing ARM toolchain on Ubuntu VM (64bit)
untar new source-code
create default config
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- defconfig
A default configuration file is created.
update config to change it to our needs:
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- menuconfig
The option to compile Busybox as a static executable, so that we don’t have to copy the dynamic libraries inside the root filesystem. The setting can be found in “Busybox Settings --> Build Options“.
Also, select what utilities you want embedded in Busybox.
Then, the following command builds Busybox and creates a directory called _install containing the root filesystem tree:
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- install.
Compile statically. If you compile dynamically then you need to copy libraries to target.
For more details see this link: http://balau82.wordpress.com/2010/03/27/busybox-for-arm-on-qemu/

For Android you can use config android_defconfig and build with examples/android-build.
I successfully used busybox patched by sherpya: his config sherpya_android_defconfig and slightly modified build script examples/android-build.

How to build i686-linux-android-gfortran for android-ndk8b (x86 arch Android)?

I tried building i686-linux-android-gfortran using build-gcc.sh following this
(it's for androdindk-7b) but I get error about link.h. I added link.h from here, but it gives further more errors.
Has anyone tried enabling i686-linux-android-gfortran for x86 Android?

From https://groups.google.com/forum/#!msg/android-ndk/QR1qiN0jIpE/g0MHkhTd4YMJ as selalerer suggested. I didn't try this, so I'm posting as a community wiki for reference purposes.
Fortran for x86 Android
=================
The guide is based on this one, many thanks to Phil:
Compiling Android NDK with Objective-C-enabled gcc errors
1) Download and unpack Android NDK 'android-ndk-r8c', (the older -r8b NDK won't work due to missing link.h!):
wget http://dl.google.com/android/ndk/android-ndk-r8c-linux-x86.tar.bz2
2) Create somewhere a folder called 'toolchain-src' (e.g. inside the folder android-ndk-r8c),
'cd' to this new folder
3) Make sure to have git installed ('yum install git' or whatever..) and download
the toolchain sources:
git clone https://android.googlesource.com/toolchain/build.git
git clone https://android.googlesource.com/toolchain/gmp.git
git clone https://android.googlesource.com/toolchain/gdb.git
git clone https://android.googlesource.com/toolchain/mpc.git
git clone https://android.googlesource.com/toolchain/mpfr.git
git clone https://android.googlesource.com/toolchain/expat.git
4) Create the folder 'binutils', 'cd' to this directory, unpack
binutils-2.23 there:
wget ftp.gnu.org/gnu/binutils/binutils-2.23.tar.gz
tar -xvzf binutils-2.23.tar.gz
You should now have a folder toolchain-src/binutils/binutils-2.23
5) Change to folder toolchain-src/build, edit the Makefile.in, changing the line:
--with-gnu-as --with-gnu-ld --enable-languages=c,c++
to
--with-gnu-as --with-gnu-ld --enable-languages=c,c++,fortran
6) In the file android-ndk-r8c/build/tools/build-mingw64-toolchain.sh change the line:
var_append GCC_CONFIGURE_OPTIONS "--enable-languages=c,c++"
to
var_append GCC_CONFIGURE_OPTIONS "--enable-languages=c,c++,fortran"
7) In the file android-ndk-r8c/build/tools/build-gcc.sh, change the line:
EXTRA_CONFIG_FLAGS=$EXTRA_CONFIG_FLAGS" --disable-libquadmath --disable-plugin"
to
EXTRA_CONFIG_FLAGS=$EXTRA_CONFIG_FLAGS" --disable-libquadmath --disable-libquadmath-support --disable-plugin"
8) In the file android-ndk-r8c/build/tools/build-host-gcc.sh, change the line:
ARGS=$ARGS" --enable-languages=c,c++"
to
ARGS=$ARGS" --enable-languages=c,c++,fortran"
And change the line
ARGS=$ARGS" --disable-libquadmath --disable-plugin --disable-libitm --disable-bootstrap"
to
ARGS=$ARGS" --disable-libquadmath --disable-libquadmath-support --disable-plugin --disable-libitm --disable-bootstrap"
9) Build your new toolchain:
/your/path/to/android-ndk-r8c/build/tools/build-gcc.sh -j1 --gmp-version=5.0.5 --mpfr-version=2.4.2 --mpc-version=0.8.1 --binutils-version=2.23 --gdb-version=7.3.x /your/path/to/toolchain-src /your/path/to/android-ndk-r8c x86-4.7
(don't worry about messages like 'expr: warning: unportable BRE:')
10) And go down to your knees in front of the screen, praying to the Lord that somehow these
countless configure scripts doing checks that nobody needs, using an ugly shell language
that cooks your brain with indentation going from right to left, will somehow manage to
compile a zillion of far too small files (so that 10% of the time is spent on compilation
and 90% on starting up GCC), and after an hour of watching progress with
tail -F /tmp/ndk-YourUserName/build/toolchain/config.log
your toolchain will be magically ready. You'll find it in the android-ndk-r8c/toolchains folder.
11) Finally, 'cd' to the folder
'/your/path/to/android-ndk-r8c/toolchains/x86-4.7/prebuilt/linux-x86/i686-linux-android'
and run this command:
ln -s ../libexec libexec
Without this command, it may happen that g++ raises the error message
"g++: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found".
Using strace, I found that g++ looks in the wrong folder, but the link
above lets it find the file liblto_plugin.so nevertheless.
And here are a few lessons learned on the way, so that Google finds this page:
*) To speed up the compilation, you can remove the '-j1'. But only after you got
it to work once, since building in parallel on multiple CPU cores was reported to
cause additional troubles.
*) The error message "Link tests are not allowed after GCC_NO_EXECUTABLES" shows up
when linking fails for x86 (works for ARM). The reason is that GCC does not include
the proper ANDROID_STARTFILE_SPEC and ANDROID_ENDFILE_SPEC from
gcc-4.6.1/gcc/config/linux-android.h. GCC 4.6.1 only specifies them for ARM, but not
for i386, GCC 4.8.0 however does. The GCCs downloaded from Google also do,
so best use Google's GCC.
*) The error message "fatal error: link.h: No such file or directory" also happens
with Google's GCC, and apparently (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=50877)
only when you enable additional languages like objc or fortran.
The bug thread is here: http://gcc.gnu.org/ml/gcc-bugs/2012-08/msg00494.html
MIPS has link.h in android-ndk-r8b/platforms/android-9/arch-mips/usr/include
In android-ndk-r8c, link.h is now also present in android-9/arch-x86/usr/include/link.h,
so this bug was fixed.
*) The error message "fatal error: quadmath_weak.h: No such file or directory":
It also happens with the latest gcc-4.8, so we can just continue using Googles GCC 4.7.
Google itself uses --disable-libquadmath, but we additionally need --disable-libquadmathsupport
(see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47648). So this needs to be added in
android-ndk-r8c/build/tools/build-gcc.sh
and
android-ndk-r8c/build/tools/build-host-gcc.sh
*) The error message "error: Pthreads are required to build libatomic"
Happens when building the ARM version of gcc-4.8 downloaded from gnu.org,
better stay with Google's GCCs.
*) The GCC that came with android-ndk-r8c didn't work for me (error message about
libstdc++.so.6 being too old), while the one in android-ndk-r8b worked
without problems. Since the android-ndk should support as many environments
as possible, I'm not sure why the Googlers decided to depend on a newer libstdc++,
but the good news are that building your own toolchain solves the issue.
*) If you get an error while compiling generic-morestack.c, then replace
#ifdef linux
// On Linux, the first two real time signals are used by the NPTL
with
#if defined(GLIBC) && defined(linux)
// On Linux, the first two real time signals are used by the NPTL

Android device configuration for AOSP

I've downloaded Android source code. Now I want to make it for my own device (LG GT540). I heard that you need to create some 'Device configuration' for that. Although several developers have already created device configurations for my device, but I want to create my own, just for learning.I saw a lot of files like BoardConfig.mk, AndroidProducts.mk, etc. But don't know what they do. Besides they contain a lot of configurations. Over that, there's not a good documentation for that.Can anyone experienced with Android porting and device configurations help me?

Right... So you want to build your own device tree, read on.
Disclaimer: this is by no means complete, and there will be omissions as have explained all this top of my head and copied pasted certain bits that I have here on my own device tree.
The device tree, for example, /device/lg/gt540would consist of the following make files:
Android.mk - this will tell the build system to include and to build sources specifically for your device. See below, for an example. This is dependant on the device and hardware, you could have libsensors, liblights, libcamera subdirectories under the example device tree, i.e. /device/lg/gt540/libsensors, /device/lg/gt540/liblights, /device/lg/gt540/libcamera etc.
AndroidBoard.mk - this is for the kernel, the build system uses that to drop the kernel image in place (more about this in a few minutes)
AndroidProducts.mk - specifies the appropriate device's make file, to use for building. i.e. /device/lg/gt540/device_gt540.mk, this is specific also.
device_xxxxx.mk - specifies the properties and extras to copy over into the final output, in this case, it could be for example, device_gt540.mk
BoardConfig.mk - This is the meat of it all, this is where compiler conditional flags are set, partition layouts, boot addresses, ramdisk size, and so on.
Lets peek into each of those to give a glance as to where it all fits in.
Android.mk:
ifeq ($(TARGET_BOOTLOADER_BOARD_NAME),xxxxx)
include $(call all-named-subdir-makefiles, recovery libsensors liblights libcamera ....)
endif
This is how the build will use that to build recovery, sensors, lights and camera (of course there will be more), its saying 'Yo Builder, go into each of the directories specified, and build the respective sources plskthxbai'
AndroidBoard.mk:
LOCAL_PATH := device/lg/gt540/
#
# Boot files
#
TARGET_PREBUILT_KERNEL := $(LOCAL_PATH)/kernel
file := $(INSTALLED_KERNEL_TARGET)
ALL_PREBUILT += $(file)
$(file): $(TARGET_PREBUILT_KERNEL) | $(ACP)
$(transform-prebuilt-to-target)
Now this, is telling the build system, to be able to drop this kernel into the out/target/product/lg/gt540 (notice the correlation with the device tree directory?)
AndroidProducts.mk:
PRODUCT_MAKEFILES := \
$(LOCAL_DIR)/device_gt540.mk
Its telling the build as in 'Yo Builder, read that device make file please and process it upon completion of build.'
*device_xxxxx.mk: (for this example, device_gt540.mk) *
PRODUCT_NAME := lg_gt540
PRODUCT_DEVICE := gt540
PRODUCT_MODEL := LG GT 540
PRODUCT_COPY_FILES += \
... specific ...
PRODUCT_PROPERTY_OVERRIDES := \
ro.com.android.dateformat=dd-MM-yyyy \
... more stuff ...
This is where all the specifics for the device such as drivers, proprietary libraries, supporting scripts specifically for the device, gets copied over to out/target/product/lg/gt540/system/ in this case. Notice how the overrides for the properties, these end up in the build.prop found in the root of the /system of the Android ROM.
BoardConfig.mk:
LOCAL_PATH:= $(call my-dir)
TARGET_NO_BOOTLOADER := true
TARGET_PREBUILT_KERNEL := device/lg/gt540/kernel
TARGET_PREBUILT_RECOVERY_KERNEL := device/lg/gt540/recovery_kernel
# This will vary from device!
TARGET_BOARD_PLATFORM := msm7k
TARGET_ARCH_VARIANT := armv6-vfp
TARGET_CPU_ABI := armeabi
TARGET_CPU_ABI := armeabi-v6l
TARGET_CPU_ABI2 := armeabi
# OpenGL drivers config file path
BOARD_EGL_CFG := device/lg/gt540/egl.cfg
# Dependant, not to be taken literally!
BOARD_GLOBAL_CFLAGS += -DHAVE_FM_RADIO
# Dependant, not to be taken literally!
BOARD_KERNEL_BASE := 0x02600000
# this will be device specific, and by doing cat /proc/mtd will give you the correct sizes
BOARD_BOOTIMAGE_PARTITION_SIZE := 0x00480000
BOARD_RECOVERYIMAGE_PARTITION_SIZE := 0x00480000
BOARD_SYSTEMIMAGE_PARTITION_SIZE := 0x0cf80000
BOARD_USERDATAIMAGE_PARTITION_SIZE := 0x0d020000
BOARD_FLASH_BLOCK_SIZE := 131072
That is an excerpt, notice how we specify kernel's base address, this is how the boot.img gets generated after compilation is done and yet again, gets dropped into out/target/product/lg/gt540/boot.img. Also, more importantly, we're telling the build system to use the target platform for cross-compiling the sources (*TARGET_BOARD_PLATFORM*/*TARGET_CPU_ABI*) There will be more information in there such as conditional flags to pass to the compiler, for an example. we specified the directive HAVE_FM_RADIO to tell it, when it comes to handling the source for the FM radio system, to conditionally compile parts of the source. Again, this is hardware specific and mileage will vary, also this applies to the address for boot. In a nutshell, this is saying 'Yo Builder, read the damn variables and remember them and apply them when cross-compiling those source files!'
Now that the internals of each of those Android build make-files are shown.
Now, onto the vendor/ part of it, in AOSP, simply, once again, correlation and corresponds with the device/ tree, as in continuing with this example, vendor/lg/gt540/ which gets picked up by the lunch. There's more make files in there but the general consensus is there's a directory called proprietary which contains the proprietary libs (due to close-source etc) that gets copied over. The copying over of the libraries gets specified in the file device-vendor-blobs.mk, in this case, gt540-vendor-blobs.mk.
When the magic happens by doing the following:
. build/envsetup.sh
This is reading in the entire entries found in each of the device/ subdirectories and "remembers them", so the build system knows what type of target is used etc.
When the . lunch gets invoked, a menu appears prompting to pick the device that is required to build. Now the last and final step to do the build...
make -j5 > buildlog.log 2>&1
I run multitail on another terminal and monitor the buildlog.log file to check and make sure its building.
This last step will depend on how many cores you have (n cores + 1 as a rule) and it takes a while to build, GB build takes 40mins on my laptop running Arch Linux 64bit, ICS build takes about 2hrs 30 mins. So mileage will vary on what type of horsepower your machine has.
When the build is done, a little bell goes off and at the bottom of the said log file, I see this:
Combining NOTICE files: out/target/product/xxxxx/obj/NOTICE.html
Target system fs image: out/target/product/xxxxx/obj/PACKAGING/systemimage_intermediates/system.img
Install system fs image: out/target/product/xxxxx/system.img
out/target/product/xxxx/system.img+ total size is 108776448
As matter of interest JBQ (Jean Baptiste Queru - the 'boss' for managing/distributing the source from Google), his build step is this...
make -j32
Yup! 32 cores! That..... is pretty powerful.

There is some information here: http://elinux.org/Android_Device

An excellent resource for anyone building Android for a device is here:
http://com.odroid.com/sigong/nf_file_board/nfile_board_view.php?bid=98
(A Practical Real-World Approach To Android Platform Development In ODROID)
Though some of the stuff in there is particular to the ODROID board, it still offers great insight into the inner workings of Android and the necessary customization for a new board.

If you're looking to get into the hardware side of things probably the single most informative resource I've found has been:
http://source.android.com/compatibility/overview.html
Read through the documentation they wrote for manufacturers looking to build android devices, it's the most thorough/complete reference you will find.