Develop Reference

Missing Kernel DTB after creating signed Android images

I'm porting android to a display device, and have nearly completed this. The device use the Freescale/NXP i.MX6 Dual Lite Soc. The Android version used is Android 8.0.0, and the build is based on the Board Support Packages from NXP/Freescale (link below).
https://www.nxp.com/support/developer-resources/software-development-tools/i.mx-developer-resources/android-os-for-i.mx-applications-processors:IMXANDROID?tab=Design_Tools_Tab
The OS builds fine, and the images (u-boot, boot.img, system.img, vendor.img) resuling from the "make" process works perfercly fine on the device. So my last step is basically to sign the images, and this is where I struggle to get stuff working.
I am following the the guide found here:
https://source.android.com/devices/tech/ota/sign_builds
After completing the steps, I use the now signed images found in the "signed-img.zip" file to flash the device (using the NXP Manufacturing Tool, and not Fast Boot). However, the device now fails to boot the Kernel, giving me an error that the DTB is missing.
Hit any key to stop autoboot: 0
boota mmc0
kernel # 14008000 (8183104)
ramdisk # 15000000 (2036048)
## Booting Android Image at 0x12000000 ...
Kernel load addr 0x14008000 size 7992 KiB
Kernel command line: console=ttymxc0,115200 init=/init video=mxcfb0:dev=ldb video=mxcfb1:off video=mxcfb2:off video=mxcfb3:off vmalloc=128M androidboot.console=ttymxc0 consoleblank=0 ldo_active=on androidboot.hardware=sedevices cma=448M android.selinux=permissive android.dm_verify=disable androidboot.selinux=enforce androidboot.dm_verity=disable androidboot.storage_type=emmc loglevel=8 vt.global_cursor_default=0 buildvariant=userdebug androidboot.serialno=0b2861d4df668b47 androidboot.soc_type=imx6dl androidboot.storage_type=emmc
ERROR: Did not find a cmdline Flattened Device Tree
Could not find a valid device tree
resetting ...
I've narrowed the problem down to the very first step in the guide, where "make dist" is executed in the build directory. This produces a number of ZIP files in the "out/dist" folder, which is processed further in the following steps in the guide. I've tried flashing the device with the images produced in this step (found in the resulting "out/dist/*-img-*.zip" file), and this produces the exact same issue.
So my question is, what does really "make dist" do which cause the DTB to be missing in the "boot.img"? I would've expected it to use the already working "boot.img" found in "out/target/product//". But it instead seems to re-build this image, and in this case not include the DTB. As with so many other aspects of building Android from Source, the workings of "make dist" does not seem to be explained anywhere in the documentation.
I hope anyone with some experience in building Android from source knows something about this, because I seem to be royally stuck.
Just FYI; when I flash the "boot.img" produced after a normal "make", the output after U-boot is as follows:
Hit any key to stop autoboot: 0
boota mmc0
Error: blob decap job completed with errors 0x2000081A
In boota get fastboot lock status error. Set lock status
kernel # 14008000 (8183104)
ramdisk # 15000000 (2036754)
fdt # 14f00000 (40998)
## Booting Android Image at 0x12000000 ...
Kernel load addr 0x14008000 size 7992 KiB
Kernel command line: console=ttymxc0,115200 init=/init video=mxcfb0:dev=ldb video=mxcfb1:off video=mxcfb2:off video=mxcfb3:off vmalloc=128M androidboot.console=ttymxc0 consoleblank=0 ldo_active=on androidboot.hardware=sedevices cma=448M android.selinux=permissive android.dm_verify=disable androidboot.selinux=enforce androidboot.dm_verity=disable androidboot.storage_type=emmc loglevel=8 vt.global_cursor_default=0 buildvariant=userdebug androidboot.serialno=0b2861d4df668b47 androidboot.soc_type=imx6dl androidboot.storage_type=emmc
## Flattened Device Tree blob at 14f00000
Booting using the fdt blob at 0x14f00000
Loading Kernel Image ... OK
Using Device Tree in place at 14f00000, end 14f0d025
switch to ldo_bypass mode!
Starting kernel ...

It seem NXP/Freescale have modified the Build Scripts in their AOSP Board Support Package, and broken the DTB inclusion in the process. It may seem like they where happy when the output from the standard "make" process worked on their Dev Board, and never bothered checking if it was possible to create working signed Release images with it.
The mechanism to include a DTB in the dist package was in place (specifying the location of the DTB in BOARD_KERNEL_DTS), and was also used for their "make otapackage" target. However, that target only generates an unsigned (or signed with dev keys) OTA package, which can not be used with the signing scripts.
In order to make it work as it should, I had to make a small change in the main Makefile which specifies the location of the DTB defined in the BoardConfig.mk. The change was simple enough to make, but the hard part was figuring out where the problem was and how it was intended to work in the first place.
The following patch in "build/make/" fixes the issue, as long as a BoardConfig.mk ONLY specifies one DTB (which at least suits my needs):
diff --git a/core/Makefile b/core/Makefile
index a650565a1..92f3025a9 100644
--- a/core/Makefile
+++ b/core/Makefile
## -621,6 +621,19 ## ifdef INTERNAL_KERNEL_CMDLINE
INTERNAL_BOOTIMAGE_ARGS += --cmdline "$(INTERNAL_KERNEL_CMDLINE)"
endif
+# NOTE! This script has a defect which cause the kernel DTB to be left out when ever 'make dist'
+# is executed. The following addition "fixes" this by adding the first dtb specified in the
+# BoardConfig.mk file. (I would guess in most cases there is never more than one!)
+ifdef dist_goal
+ifndef BOARD_KERNEL_DTS
+ifdef TARGET_BOARD_DTS_CONFIG
+DTS_BOARD=$(word 2, $(subst :, ,$(word 1, $(TARGET_BOARD_DTS_CONFIG))))
+BOARD_KERNEL_DTS="$(KERNEL_OUT)/$(DTS_BOARD)"
+$(info FIXUP: Defining BOARD_KERNEL_DTS:=[$(BOARD_KERNEL_DTS)] for BOOT packaging)
+endif
+endif
+endif
+
INTERNAL_MKBOOTIMG_VERSION_ARGS := \
--os_version $(PLATFORM_VERSION) \
--os_patch_level $(PLATFORM_SECURITY_PATCH)

Linux Kernel Dynamic Debug settings - persist across a reset

With Dynamic Debugging enabled in the Linux Kernel, I am able to control which prints show up by writing to /sys/kernel/debug/dynamic_debug/control. However, It seems that a reboot resets this file to its default settings (all prints disabled).
Is there a way to make the settings persist across a reset? I have a print which is emitted early in the boot process, and I am not able to get it to print, because the reboot disables this print in /sys/kernel/debug/dynamic_debug/control.

You should be able to add dyndbg='your query here' to the kernel command line using the bootloader configuration or manual options entry feature. This will vary with the bootloader although grub is the most common one
If you are using grub as a bootloader you would have to add it to the kernel command line in /boot/grub/grub.conf for the particular kernel you are launching at bootup.
A blog article about this can be found here. In particular it gives this example:
dyndbg='module xhci_hcd +p'
The article also discusses modifying grub as well.
This kernel howto about dynamic debugging has useful information about dyndbg. This section applies
To activate debug messages for core code and built-in modules during
the boot process, even before userspace and debugfs exists, use
dyndbg="QUERY", module.dyndbg="QUERY", or ddebug_query="QUERY"
(ddebug_query is obsoleted by dyndbg, and deprecated). QUERY follows
the syntax described above, but must not exceed 1023 characters. Your
bootloader may impose lower limits.
These dyndbg params are processed just after the ddebug tables are
processed, as part of the arch_initcall. Thus you can enable debug
messages in all code run after this arch_initcall via this boot
parameter.
On an x86 system for example ACPI enablement is a subsys_initcall and
dyndbg="file ec.c +p"
will show early Embedded Controller transactions during ACPI setup if
your machine (typically a laptop) has an Embedded Controller.
PCI (or other devices) initialization also is a hot candidate for using
this boot parameter for debugging purposes.
If foo module is not built-in, foo.dyndbg will still be processed at
boot time, without effect, but will be reprocessed when module is
loaded later. dyndbg_query= and bare dyndbg= are only processed at
boot.

You can do it as a part of post boot script. Say after Android boots(since i see Android tag added here), it runs certain sh scripts as a part of init procedure. Modify the sh files with the dynamic debug commands added and push it in and reboot .
Refer here for post boot script files info.
https://android.stackexchange.com/questions/6558/how-can-i-run-a-script-on-boot

You can have dynamic_debug for some kernel module foo activated automatically during boot using the /etc/modprobe.d/ directory.
### BEGIN prerequisite
# in case module "foo" is not yet automatically loaded during boot,
# then make it load during boot:
# create a file /etc/modules-load.d/foo.conf with contents "foo"
echo "foo" | sudo tee /etc/modules-load.d/foo.conf
### END prerequisite
# create a configuration file to activate dynamic debug for module "foo"
# at its "module initialization time"
echo "options foo dyndbg=+p" | sudo tee /etc/modprobe.d/foo.conf
Reference: Debug Messages at Module Initialization Time from kernel.org documentation.

How to compile Busybox?

(The i9100 and i9100p phones have Exynos 4210 SoC which includes Cortex A9 dual core 1.2Ghz processor which supports NEON.)
I will compile the latest busybox source snapshot available and upload it for everyone for free on internet and maybe even make my own free BusyboxInstaller.apk (I already downloaded today's 14th March snapshot from the official website) because so many busybox installers have very outdated versions and I want to take advantage of possible optimizations for the Cortex A9 cpu.
(NEON technology is a 128-bit SIMD (Single Instruction, Multiple Data) architecture extension.
It can can accelerate multimedia and signal processing algorithms such as video encode/decode, 2D/3D graphics, gaming, audio and speech processing, image processing, telephony, and sound synthesis.)
1 does such as mean it will also benefit my busybox?
2 What other instructions/anythingelse I can use to optimize for i9100 devices?
3 Can I compile on my phone instead of PC with only 1x2Ghz and 2GB RAM? I think it should be much quicker because it has 2x1.5GHz and 1GB RAM (I overclock only a bit) versus 1x2GHz, am I right?
4 How to compile busybox?
What are these options for?
Force NOMMU build
Additional CFLAGS
Additional LDFLAGS
Additional LDLIBS
all in general configuration and Busybox Library Tuning and debug (I googled for each for a few hours but nothing satisfactory)
What kind of applet links to install?
(./_install) BusyBox installation prefix
**I have eeror when following vinayhunachyai instructions.
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- install
/home/euphoria/Sourcery/busybox/scripts/gcc-version.sh: line 11: arm-none-linux-gnueabi-gcc: command not found
CC applets/applets.o
/bin/sh: 1: arm-none-linux-gnueabi-gcc: not found
scripts/Makefile.build:197: recipe for target 'applets/applets.o' failed
make[1]: *** [applets/applets.o] Error 127
Makefile:372: recipe for target 'applets_dir' failed
make: *** [applets_dir] Error 2

How to compile busybox?
1st download toolchain. You can refer to this: Advice regarding installing ARM toolchain on Ubuntu VM (64bit)
untar new source-code
create default config
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- defconfig
A default configuration file is created.
update config to change it to our needs:
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- menuconfig
The option to compile Busybox as a static executable, so that we don’t have to copy the dynamic libraries inside the root filesystem. The setting can be found in “Busybox Settings --> Build Options“.
Also, select what utilities you want embedded in Busybox.
Then, the following command builds Busybox and creates a directory called _install containing the root filesystem tree:
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- install.
Compile statically. If you compile dynamically then you need to copy libraries to target.
For more details see this link: http://balau82.wordpress.com/2010/03/27/busybox-for-arm-on-qemu/

For Android you can use config android_defconfig and build with examples/android-build.
I successfully used busybox patched by sherpya: his config sherpya_android_defconfig and slightly modified build script examples/android-build.

Bluetooth missing under System Settings

I'm porting bluetooth to JB (project code discussed here: https://groups.google.com/forum/#!forum/renesas-emev-osp) and I managed to start it up, both manually (bttest enable) and through the Power Widget button. So code-wise integration seems fine, but a Bluetooth item doesn't appear, even after a full clean rebuild, under the System Settings menu. I have these set of course:
$ grep BLUE *.mk
BoardConfig.mk:BOARD_HAVE_BLUETOOTH := true
BoardConfig.mk:BOARD_HAVE_BLUETOOTH_BCM := true
but looks like it isn't enough, to enable BT settings. What did I miss?

I had the same problem recently when I compiled my own AOSP.
After a lot of trial-and-error it turned out that I was missing one option in one of these files:
/system/etc/permissions/handheld_core_hardware.xml
/system/etc/permissions/tablet_core_hardware.xml
Which file to edit depends on which file you have in the directory /system/etc/permissions.
If you have root access (which I guess) then you can even simply edit the corresponding file on your device without the need of recompiling / flashing (I personally use ES File Explorer to do that).
Just add the following line to either of these files:
<feature name="android.hardware.bluetooth" />
Then start / restart the device and bluetooth option should be enabled in the system settings.
Cheers,
Frank

Android device configuration for AOSP

I've downloaded Android source code. Now I want to make it for my own device (LG GT540). I heard that you need to create some 'Device configuration' for that. Although several developers have already created device configurations for my device, but I want to create my own, just for learning.I saw a lot of files like BoardConfig.mk, AndroidProducts.mk, etc. But don't know what they do. Besides they contain a lot of configurations. Over that, there's not a good documentation for that.Can anyone experienced with Android porting and device configurations help me?

Right... So you want to build your own device tree, read on.
Disclaimer: this is by no means complete, and there will be omissions as have explained all this top of my head and copied pasted certain bits that I have here on my own device tree.
The device tree, for example, /device/lg/gt540would consist of the following make files:
Android.mk - this will tell the build system to include and to build sources specifically for your device. See below, for an example. This is dependant on the device and hardware, you could have libsensors, liblights, libcamera subdirectories under the example device tree, i.e. /device/lg/gt540/libsensors, /device/lg/gt540/liblights, /device/lg/gt540/libcamera etc.
AndroidBoard.mk - this is for the kernel, the build system uses that to drop the kernel image in place (more about this in a few minutes)
AndroidProducts.mk - specifies the appropriate device's make file, to use for building. i.e. /device/lg/gt540/device_gt540.mk, this is specific also.
device_xxxxx.mk - specifies the properties and extras to copy over into the final output, in this case, it could be for example, device_gt540.mk
BoardConfig.mk - This is the meat of it all, this is where compiler conditional flags are set, partition layouts, boot addresses, ramdisk size, and so on.
Lets peek into each of those to give a glance as to where it all fits in.
Android.mk:
ifeq ($(TARGET_BOOTLOADER_BOARD_NAME),xxxxx)
include $(call all-named-subdir-makefiles, recovery libsensors liblights libcamera ....)
endif
This is how the build will use that to build recovery, sensors, lights and camera (of course there will be more), its saying 'Yo Builder, go into each of the directories specified, and build the respective sources plskthxbai'
AndroidBoard.mk:
LOCAL_PATH := device/lg/gt540/
#
# Boot files
#
TARGET_PREBUILT_KERNEL := $(LOCAL_PATH)/kernel
file := $(INSTALLED_KERNEL_TARGET)
ALL_PREBUILT += $(file)
$(file): $(TARGET_PREBUILT_KERNEL) | $(ACP)
$(transform-prebuilt-to-target)
Now this, is telling the build system, to be able to drop this kernel into the out/target/product/lg/gt540 (notice the correlation with the device tree directory?)
AndroidProducts.mk:
PRODUCT_MAKEFILES := \
$(LOCAL_DIR)/device_gt540.mk
Its telling the build as in 'Yo Builder, read that device make file please and process it upon completion of build.'
*device_xxxxx.mk: (for this example, device_gt540.mk) *
PRODUCT_NAME := lg_gt540
PRODUCT_DEVICE := gt540
PRODUCT_MODEL := LG GT 540
PRODUCT_COPY_FILES += \
... specific ...
PRODUCT_PROPERTY_OVERRIDES := \
ro.com.android.dateformat=dd-MM-yyyy \
... more stuff ...
This is where all the specifics for the device such as drivers, proprietary libraries, supporting scripts specifically for the device, gets copied over to out/target/product/lg/gt540/system/ in this case. Notice how the overrides for the properties, these end up in the build.prop found in the root of the /system of the Android ROM.
BoardConfig.mk:
LOCAL_PATH:= $(call my-dir)
TARGET_NO_BOOTLOADER := true
TARGET_PREBUILT_KERNEL := device/lg/gt540/kernel
TARGET_PREBUILT_RECOVERY_KERNEL := device/lg/gt540/recovery_kernel
# This will vary from device!
TARGET_BOARD_PLATFORM := msm7k
TARGET_ARCH_VARIANT := armv6-vfp
TARGET_CPU_ABI := armeabi
TARGET_CPU_ABI := armeabi-v6l
TARGET_CPU_ABI2 := armeabi
# OpenGL drivers config file path
BOARD_EGL_CFG := device/lg/gt540/egl.cfg
# Dependant, not to be taken literally!
BOARD_GLOBAL_CFLAGS += -DHAVE_FM_RADIO
# Dependant, not to be taken literally!
BOARD_KERNEL_BASE := 0x02600000
# this will be device specific, and by doing cat /proc/mtd will give you the correct sizes
BOARD_BOOTIMAGE_PARTITION_SIZE := 0x00480000
BOARD_RECOVERYIMAGE_PARTITION_SIZE := 0x00480000
BOARD_SYSTEMIMAGE_PARTITION_SIZE := 0x0cf80000
BOARD_USERDATAIMAGE_PARTITION_SIZE := 0x0d020000
BOARD_FLASH_BLOCK_SIZE := 131072
That is an excerpt, notice how we specify kernel's base address, this is how the boot.img gets generated after compilation is done and yet again, gets dropped into out/target/product/lg/gt540/boot.img. Also, more importantly, we're telling the build system to use the target platform for cross-compiling the sources (*TARGET_BOARD_PLATFORM*/*TARGET_CPU_ABI*) There will be more information in there such as conditional flags to pass to the compiler, for an example. we specified the directive HAVE_FM_RADIO to tell it, when it comes to handling the source for the FM radio system, to conditionally compile parts of the source. Again, this is hardware specific and mileage will vary, also this applies to the address for boot. In a nutshell, this is saying 'Yo Builder, read the damn variables and remember them and apply them when cross-compiling those source files!'
Now that the internals of each of those Android build make-files are shown.
Now, onto the vendor/ part of it, in AOSP, simply, once again, correlation and corresponds with the device/ tree, as in continuing with this example, vendor/lg/gt540/ which gets picked up by the lunch. There's more make files in there but the general consensus is there's a directory called proprietary which contains the proprietary libs (due to close-source etc) that gets copied over. The copying over of the libraries gets specified in the file device-vendor-blobs.mk, in this case, gt540-vendor-blobs.mk.
When the magic happens by doing the following:
. build/envsetup.sh
This is reading in the entire entries found in each of the device/ subdirectories and "remembers them", so the build system knows what type of target is used etc.
When the . lunch gets invoked, a menu appears prompting to pick the device that is required to build. Now the last and final step to do the build...
make -j5 > buildlog.log 2>&1
I run multitail on another terminal and monitor the buildlog.log file to check and make sure its building.
This last step will depend on how many cores you have (n cores + 1 as a rule) and it takes a while to build, GB build takes 40mins on my laptop running Arch Linux 64bit, ICS build takes about 2hrs 30 mins. So mileage will vary on what type of horsepower your machine has.
When the build is done, a little bell goes off and at the bottom of the said log file, I see this:
Combining NOTICE files: out/target/product/xxxxx/obj/NOTICE.html
Target system fs image: out/target/product/xxxxx/obj/PACKAGING/systemimage_intermediates/system.img
Install system fs image: out/target/product/xxxxx/system.img
out/target/product/xxxx/system.img+ total size is 108776448
As matter of interest JBQ (Jean Baptiste Queru - the 'boss' for managing/distributing the source from Google), his build step is this...
make -j32
Yup! 32 cores! That..... is pretty powerful.

There is some information here: http://elinux.org/Android_Device

An excellent resource for anyone building Android for a device is here:
http://com.odroid.com/sigong/nf_file_board/nfile_board_view.php?bid=98
(A Practical Real-World Approach To Android Platform Development In ODROID)
Though some of the stuff in there is particular to the ODROID board, it still offers great insight into the inner workings of Android and the necessary customization for a new board.

If you're looking to get into the hardware side of things probably the single most informative resource I've found has been:
http://source.android.com/compatibility/overview.html
Read through the documentation they wrote for manufacturers looking to build android devices, it's the most thorough/complete reference you will find.