Develop Reference

How is the AOSP project built?

All the git projects for the AOSP are cloned by the repo tool, which reads this xml: https://android.googlesource.com/platform/manifest/+/refs/heads/master/default.xml.
AOSP guide says the in order to build, we should run source build/envsetup.sh on the folder where repo cloned all repositories. So let's look at the platform/build on the default.xml from the manifest repository. We get
<project path="build/make" name="platform/build" groups="pdk" >
<copyfile src="core/root.mk" dest="Makefile" />
<linkfile src="CleanSpec.mk" dest="build/CleanSpec.mk" />
<linkfile src="buildspec.mk.default" dest="build/buildspec.mk.default" />
<linkfile src="core" dest="build/core" />
<linkfile src="envsetup.sh" dest="build/envsetup.sh" />
<linkfile src="target" dest="build/target" />
<linkfile src="tools" dest="build/tools" />
</project>
We confirm where envsetup.sh is located., it is in platform/build. It defines the function m which according to the AOSP guide, builds the entire AOSP project:
function _trigger_build()
(
local -r bc="$1"; shift
if T="$(gettop)"; then
_wrap_build "$T/build/soong/soong_ui.bash" --build-mode --${bc} --dir="$(pwd)" "$#"
else
echo "Couldn't locate the top of the tree. Try setting TOP."
fi
)
function m()
(
_trigger_build "all-modules" "$#"
)
Ok, so looks like build/soong/soong_ui.bash is the place called when we run the m function, so this script should build everything.
Here's soong_ui.bash. It sources source ${TOP}/build/soong/scripts/microfactory.bash and then calls soong_build_go soong_ui android/soong/cmd/soong_ui
Here's microfactory.bash, where we find function soong_build_go
soong_build_go
{
BUILDDIR=$(getoutdir) \
SRCDIR=${TOP} \
BLUEPRINTDIR=${TOP}/build/blueprint \
EXTRA_ARGS="-pkg-path android/soong=${TOP}/build/soong -pkg-path github.com/golang/protobuf=${TOP}/external/golang-protobuf" \
build_go $#
}
We find build_go in microfactory.bash from build/blueprint:
Looks like all of this is for building the microfactory.go project. I think it has something to do with the soong build system.
I'm now lost. After building microfactory.go, what happens? Where does actual Android code gets built?
microfactory.sh says build_go does this: Bootstrap microfactory from source if necessary and use it to build the requested binary. The requested binary is android/soong/cmd/soong_ui
I'm trying to find android/soong/cmd/soong_ui but I don't know what/where it is, but I'd guess is the soong build system, not the AOSP project yet.
UPDATE:
on soong_ui.bash, I noticed it end with
cd ${TOP}
exec "$(getoutdir)/soong_ui" "$#"
Remember that this is called form envsetup.sh. Well, ${TOP}, I guess, is the place where repo clones everything. Looks like it's trying to execute soong_ui with the arguments from envsetup.sh which are --build-mode --${bc} --dir="$(pwd)" "$#", where this $# is "all-modules" "$#", I guess.
I assume song_ui is the soong executable. It should look for Android.bp on the ${TOP}, but I don't think there is one on the place where repo cloned everything.

You have already found out a lot, and you are right with the link from m to soong_ui.bash and then starting microfactory.
From my reading of the code, the purpose of soong_build_go is to build the package android/soong/cmd/soong_ui, with the binary name soong_ui. Like Yong said in the other answer, this creates the binary soong_ui under the directory $(getoutdir), and the source for that binary is located at build/soong/cmd/soong_ui/main.go.
As for your updated question about an Android.bp file, it is symlinked from build/soong/root.bp when repo sync is run, but as you can see, the file is empty.
Instead, in m it tells Soong to build all_modules, which eventually runs another tool called kati. From the description in https://github.com/google/kati, Kati processes GNU makefiles and turns them into Ninja build files.
At this point we can (mostly) assume regular Make semantics, even though the underlying build system is actually Kati and Ninja and Soong etc. Since the working directory is $TOP, the Makefile at the root directory, which is symlinked from build/make/core/root.mk is used. Which includes main.mk which then includes build/make/core/Makefile. Inside that makefile you can see how the different .img files are built (e.g. system.img).

Let's take make systemimage as an example:
The call sequence is:
prebuilts/build-tools/linux-x86/bin/makeparallel --ninja build/soong/soong_ui.bash --make-mode "systemimage".
And $(getoutdir)/soong_ui is build by "build_go soong_ui android/soong/cmd/soong_ui"
build/soong/cmd/soong_ui/main.go#main()
soong/ui/build/build.go#Build()

How to build boost 1.69.0 for Android with NDK r18b using dec1/Boost-for-Android?

I tried to use https://github.com/dec1/Boost-for-Android to compile boost for Android. It reports 1.69.0 is supported for NDK r18b. I'm under Windows 7.
Even if this is reported as working, I encounter many problems when trying to run the provided script:
Downloaded boost 1.69.0
Downloaded sources
Modified BOOST_SRC_DIR to point to folder containing "1.69.0" boost folder
Modified ANDROID_NDK_ROOT to point to ndk folder
Modified ABIS="armeabi-v7a,x86"
Modified LINKAGE="static"
Added Cygwin to my path (as this is a .sh file I need to run)
Ran doIt.sh
Got non blocking errors:
-nLe fichier sp?cifi? est introuvable.
ERROR: Unsupported HOST_TAG: windows-x86_64
Then I got final error
## COMMAND: mkdir /tmp/ndk-/tmp/build-11600/build-boost/armeabi-v7a/llvm/host-bin
## COMMAND: ./bootstrap.sh --with-toolset=cc Building Boost.Build engine
with toolset cc... Failed to build Boost.Build build engine Consult
'bootstrap.log' for more details ERROR: Could not bootstrap Boost build
'bootstrap.log'reports gcc: not found
Added MinGW to my PATH (to have gcc be found), then retried:
Now I get in 'bootstrap.log':
cc -o bootstrap/jam0 command.c compile.c constants.c debug.c execcmd.c frames.c function.c glob.c hash.c hdrmacro.c headers.c jam.c jambase.c jamgram.c lists.c make.c make1.c object.c option.c output.c parse.c pathsys.c regexp.c rules.c scan.c search.c subst.c timestamp.c variable.c modules.c strings.c filesys.c builtins.c class.c cwd.c native.c md5.c w32_getreg.c modules/set.c modules/path.c modules/regex.c modules/property-set.c modules/sequence.c modules/order.c execunix.c fileunix.c pathunix.c
command.c:1:0: sorry, unimplemented: 64-bit mode not compiled in
There is no 'cc' available in my PATH, dunno where this is supposed to be found
Any idea what I may be doing wrong?

Sorry, only building on linux is officially supported. Mac and Windows should work fine too but details of setting up the relevant environments (eg cgwin or homebrew) is beyond the scope of what this project tries to do.
If you want to use windows as a host you have two easiest options are:
Use virtual machines eg. VirtualBox or and create a linux virtual machine where you do the building.
You can also simply download and use the precompiled binaries I have made available
No matter what os you use to build under, the resulting binaries can then be copied to windows and used from then on as if you had built on windows to start with (theyre cross compiled for android).

Cross compiling c program with openssl for Android

I am facing the following issue when trying to compile a c program using openssl for android x-86. I set up the environment variables as follows using the following script:
setenv-android.sh
After running the script I have the following environment.
./setenv-android_x86.sh
find: /dev/fd/3: Not a directory
find: /dev/fd/4: Not a directory
Error: FIPS_SIG does not specify incore module. Please edit this script.
ANDROID_NDK_ROOT: /opt/android-ndk-r9d-x86
ANDROID_ARCH: arch-x86
ANDROID_EABI: x86-4.6
ANDROID_API: android-18
ANDROID_SYSROOT: /opt/android-ndk-r9d-x86/platforms/android-18/arch-x86
ANDROID_TOOLCHAIN: /opt/android-ndk-r9d-x86/toolchains/x86-4.6/prebuilt/darwin-x86_64/bin
FIPS_SIG:
CROSS_COMPILE: i686-linux-android-
ANDROID_DEV: /opt/android-ndk-r9d-x86/platforms/android-18/arch-x86/usr
However when trying to compile with the following command I get the following error:
pwd
/opt/android-ndk-r9d-x86/bin
./i686-linux-android-gcc Test.c -o test -lcrypto
fatal error: openssl/conf.h: No such file or directory
When I locate for the openssl/conf.h I see the file is available in many places:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/openssl/conf.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/openssl/conf.h
/data/local/arm/ssl/include/openssl/conf.h
/data/local/ssl/include/openssl/conf.h
/openssl/include/openssl/conf.h
/openssl-1.0.1e/include/openssl/conf.h
/openssl-1.0.1g/include/openssl/conf.h
/opt/local/include/openssl/conf.h
/usr/include/openssl/conf.h
/usr/local/include/openssl/conf.h
/usr/local/ssl/android-14/include/openssl/conf.h
/usr/local/ssl/android-18/include/openssl/conf.h
/usr/local/ssl/android-18-x86/include/openssl/conf.h
/usr/local/ssl/include/openssl/conf.h
I think the proper path is this:
/usr/local/ssl/android-18-x86/include/openssl/conf.h
I tried with the -L option but with no luck.
-L/usr/local/ssl/android-18-x86/include/
Can anybody please explain how to setup the path properly to solve this issue. (finally there is no issue with the C code as it compiles properly with gcc)

You are compiling your code using the Android NDK, which creates a "virtual" compiling environment by placing everything you need to compile for android into the ${ANDROID_SYSROOT} directory.
Those directories you listed are outside that sysroot, and therefore are not available to the build system, hence the errors. You must install OpenSSL for Android and putting the resulting headers and binaries there. You might follow this guide to assist you in that.

Using Boost in android ndk with windows

I am trying to use boost library with Android ndk in Eclipse with Windows. I tried to follow this tutorial
I am stuck in the step with "bjam" command in cygwin.
bjam --without-python --without-serialization toolset=gcc-android4.4.3 link=static runtime-link=static target-os=linux --stagedir=android
Error: bjam command not found.
What is bjam? Also I used the boost 1.53 along ndk r8e. Can someone help me with this please?

Android NDK is no longer dependent on Cygwin, so you can build Boost with the NDK from within Windows command prompt (cmd).
In order to make Boost.Build find the NDK, edit boost\tools\build\v2\user-config.jam file and append the following text:
import os ;
androidNDKRoot = C:/android-ndk-r8e ; # put the relevant path
using gcc : android :
$(androidNDKRoot)/toolchains/arm-linux-androideabi-4.7/prebuilt/windows/bin/arm-linux-androideabi-g++ :
<compileflags>--sysroot=$(androidNDKRoot)/platforms/android-9/arch-arm
<compileflags>-mthumb
<compileflags>-Os
<compileflags>-fno-strict-aliasing
<compileflags>-O2
<compileflags>-DNDEBUG
<compileflags>-g
<compileflags>-lstdc++
<compileflags>-I$(androidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.7/include
<compileflags>-I$(androidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.7/libs/armeabi/include
<compileflags>-D__GLIBC__
<compileflags>-D_GLIBCXX__PTHREADS
<compileflags>-D__arm__
<compileflags>-D_REENTRANT
<archiver>$(androidNDKRoot)/toolchains/arm-linux-androideabi-4.7/prebuilt/windows/bin/arm-linux-androideabi-ar
<ranlib>$(androidNDKRoot)/toolchains/arm-linux-androideabi-4.7/prebuilt/windows/bin/arm-linux-androideabi-ranlib
;
Certainly, instead of c:/android-ndk-r8e you have to put the right location of the NDK on your PC.
Besides, you can select more recent platform API, instead of android-9.
Also note that the NDK supplies several tool-chains, and the above settings point to gcc-4.7. If you prefer to build boost with some other tool-chain, change arm-linux-androideabi-4.7 to the relevant path.
After you've put the configuration in user-config.jam, open cmd, cd to the directory where Boost resides, and invoke bootstrap. Then invoke b2 like this (for example):
b2 --without-python --without-serialization threading=multi link=static runtime-link=static toolset=gcc-android target-os=linux threadapi=pthread --stagedir=android stage
UPDATE: As of 11/2015, older NDK toolchains seem to have issues with the newer Boost versions, causing compiler crash, so consider using a more recent compiler. To do this, just change every 4.7 occurrence in the above script to 4.9. Also, it's worth compiling with a more recent Android API (eg. andoroid-9 -> andoroid-16 or so).

How to build i686-linux-android-gfortran for android-ndk8b (x86 arch Android)?

I tried building i686-linux-android-gfortran using build-gcc.sh following this
(it's for androdindk-7b) but I get error about link.h. I added link.h from here, but it gives further more errors.
Has anyone tried enabling i686-linux-android-gfortran for x86 Android?

From https://groups.google.com/forum/#!msg/android-ndk/QR1qiN0jIpE/g0MHkhTd4YMJ as selalerer suggested. I didn't try this, so I'm posting as a community wiki for reference purposes.
Fortran for x86 Android
=================
The guide is based on this one, many thanks to Phil:
Compiling Android NDK with Objective-C-enabled gcc errors
1) Download and unpack Android NDK 'android-ndk-r8c', (the older -r8b NDK won't work due to missing link.h!):
wget http://dl.google.com/android/ndk/android-ndk-r8c-linux-x86.tar.bz2
2) Create somewhere a folder called 'toolchain-src' (e.g. inside the folder android-ndk-r8c),
'cd' to this new folder
3) Make sure to have git installed ('yum install git' or whatever..) and download
the toolchain sources:
git clone https://android.googlesource.com/toolchain/build.git
git clone https://android.googlesource.com/toolchain/gmp.git
git clone https://android.googlesource.com/toolchain/gdb.git
git clone https://android.googlesource.com/toolchain/mpc.git
git clone https://android.googlesource.com/toolchain/mpfr.git
git clone https://android.googlesource.com/toolchain/expat.git
4) Create the folder 'binutils', 'cd' to this directory, unpack
binutils-2.23 there:
wget ftp.gnu.org/gnu/binutils/binutils-2.23.tar.gz
tar -xvzf binutils-2.23.tar.gz
You should now have a folder toolchain-src/binutils/binutils-2.23
5) Change to folder toolchain-src/build, edit the Makefile.in, changing the line:
--with-gnu-as --with-gnu-ld --enable-languages=c,c++
to
--with-gnu-as --with-gnu-ld --enable-languages=c,c++,fortran
6) In the file android-ndk-r8c/build/tools/build-mingw64-toolchain.sh change the line:
var_append GCC_CONFIGURE_OPTIONS "--enable-languages=c,c++"
to
var_append GCC_CONFIGURE_OPTIONS "--enable-languages=c,c++,fortran"
7) In the file android-ndk-r8c/build/tools/build-gcc.sh, change the line:
EXTRA_CONFIG_FLAGS=$EXTRA_CONFIG_FLAGS" --disable-libquadmath --disable-plugin"
to
EXTRA_CONFIG_FLAGS=$EXTRA_CONFIG_FLAGS" --disable-libquadmath --disable-libquadmath-support --disable-plugin"
8) In the file android-ndk-r8c/build/tools/build-host-gcc.sh, change the line:
ARGS=$ARGS" --enable-languages=c,c++"
to
ARGS=$ARGS" --enable-languages=c,c++,fortran"
And change the line
ARGS=$ARGS" --disable-libquadmath --disable-plugin --disable-libitm --disable-bootstrap"
to
ARGS=$ARGS" --disable-libquadmath --disable-libquadmath-support --disable-plugin --disable-libitm --disable-bootstrap"
9) Build your new toolchain:
/your/path/to/android-ndk-r8c/build/tools/build-gcc.sh -j1 --gmp-version=5.0.5 --mpfr-version=2.4.2 --mpc-version=0.8.1 --binutils-version=2.23 --gdb-version=7.3.x /your/path/to/toolchain-src /your/path/to/android-ndk-r8c x86-4.7
(don't worry about messages like 'expr: warning: unportable BRE:')
10) And go down to your knees in front of the screen, praying to the Lord that somehow these
countless configure scripts doing checks that nobody needs, using an ugly shell language
that cooks your brain with indentation going from right to left, will somehow manage to
compile a zillion of far too small files (so that 10% of the time is spent on compilation
and 90% on starting up GCC), and after an hour of watching progress with
tail -F /tmp/ndk-YourUserName/build/toolchain/config.log
your toolchain will be magically ready. You'll find it in the android-ndk-r8c/toolchains folder.
11) Finally, 'cd' to the folder
'/your/path/to/android-ndk-r8c/toolchains/x86-4.7/prebuilt/linux-x86/i686-linux-android'
and run this command:
ln -s ../libexec libexec
Without this command, it may happen that g++ raises the error message
"g++: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found".
Using strace, I found that g++ looks in the wrong folder, but the link
above lets it find the file liblto_plugin.so nevertheless.
And here are a few lessons learned on the way, so that Google finds this page:
*) To speed up the compilation, you can remove the '-j1'. But only after you got
it to work once, since building in parallel on multiple CPU cores was reported to
cause additional troubles.
*) The error message "Link tests are not allowed after GCC_NO_EXECUTABLES" shows up
when linking fails for x86 (works for ARM). The reason is that GCC does not include
the proper ANDROID_STARTFILE_SPEC and ANDROID_ENDFILE_SPEC from
gcc-4.6.1/gcc/config/linux-android.h. GCC 4.6.1 only specifies them for ARM, but not
for i386, GCC 4.8.0 however does. The GCCs downloaded from Google also do,
so best use Google's GCC.
*) The error message "fatal error: link.h: No such file or directory" also happens
with Google's GCC, and apparently (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=50877)
only when you enable additional languages like objc or fortran.
The bug thread is here: http://gcc.gnu.org/ml/gcc-bugs/2012-08/msg00494.html
MIPS has link.h in android-ndk-r8b/platforms/android-9/arch-mips/usr/include
In android-ndk-r8c, link.h is now also present in android-9/arch-x86/usr/include/link.h,
so this bug was fixed.
*) The error message "fatal error: quadmath_weak.h: No such file or directory":
It also happens with the latest gcc-4.8, so we can just continue using Googles GCC 4.7.
Google itself uses --disable-libquadmath, but we additionally need --disable-libquadmathsupport
(see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47648). So this needs to be added in
android-ndk-r8c/build/tools/build-gcc.sh
and
android-ndk-r8c/build/tools/build-host-gcc.sh
*) The error message "error: Pthreads are required to build libatomic"
Happens when building the ARM version of gcc-4.8 downloaded from gnu.org,
better stay with Google's GCCs.
*) The GCC that came with android-ndk-r8c didn't work for me (error message about
libstdc++.so.6 being too old), while the one in android-ndk-r8b worked
without problems. Since the android-ndk should support as many environments
as possible, I'm not sure why the Googlers decided to depend on a newer libstdc++,
but the good news are that building your own toolchain solves the issue.
*) If you get an error while compiling generic-morestack.c, then replace
#ifdef linux
// On Linux, the first two real time signals are used by the NPTL
with
#if defined(GLIBC) && defined(linux)
// On Linux, the first two real time signals are used by the NPTL