Develop Reference

Unable to build Botan for Android on Windows

I cannot understand how to build Botan for android, according on the instruction here:
$ export CXX=/opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android28-clang++
$ ./configure.py --os=android --cc=clang --cpu=arm64
i cannot understand how to use this commands on Windows, also reading previous issues did not help me, can you tell me how did you build this library on windows step-by-step, just your command examples?
I used --cc-bin option of configure.py to specify the path to the compiler, it is considered a solution for windows, but what i have is:
D:\Programming\Libraries\botanAndroid\botan-master>python configure.py --cc-bin=D:\Android\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++ --os=android --cc=clang --cpu=armv7
INFO: configure.py invoked with options "--cc-bin=D:\Android\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++ --os=android --cc=clang --cpu=armv7"
INFO: Configuring to build Botan 2.14.0 (revision unknown)
INFO: Running under 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 22:20:52) [MSC v.1916 32 bit (Intel)]
INFO: Autodetected platform information: OS="Windows" machine="AMD64" proc="Intel64 Family 6 Model 142 Stepping 10, GenuineIntel"
INFO: Canonicalized CPU target armv7 to arm32
WARNING: Could not execute ['D:\Android\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++', '-E', 'src\build-data\detect_version.cpp']: [WinError 193] %1 is not an application of Win32
WARNING: Tried to get clang version, but output '0.0' does not match expected version format
WARNING: Could not execute ['D:\Android\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++', '-E', '-fstack-protector', '-pthread', 'src\build-data\detect_arch.cpp']: [WinError 193] %1 is not an application of Win32
WARNING: Unable to detect target architecture via compiler macro checks
INFO: Target is clang:0.0-android-arm32
INFO: Assuming target arm32 is little endian
INFO: Skipping (dependency failure): asio certstor_sqlite3 rdrand sessions_sqlite3
INFO: Skipping (incompatible CPU): aes_armv8 aes_ni aes_power8 chacha_avx2 clmul_cpu clmul_ssse3 idea_sse2 p9_darn rdrand_rng rdseed serpent_avx2 sha1_armv8 sha1_sse2 sha1_x86 sha2_32_armv8 sha2_32_bmi2 sha2_32_x86 sha2_64_bmi2 sha3_bmi2 shacal2_avx2 shacal2_x86 simd_avx2 sm4_armv8 threefish_512_avx2
INFO: Skipping (incompatible OS): certstor_system_macos certstor_system_windows commoncrypto getentropy proc_walk win32_stats
INFO: Skipping (no enabled compression schemes): compression
INFO: Skipping (requires external dependency): boost bzip2 lzma openssl sqlite3 tpm zlib
INFO: Loading modules: adler32 aead aes aes_vperm aont argon2 aria asn1 auto_rng base base32 base58 base64 bcrypt bcrypt_pbkdf bigint blake2 block blowfish camellia cascade cast128 cast256 cbc cbc_mac ccm cecpq1 certstor_flatfile certstor_sql certstor_system cfb chacha chacha20poly1305 chacha_rng chacha_simd32 checksum cmac comb4p cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa dyn_load eax ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea iso9796 kasumi kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion locking_allocator mac mce mceies md4 md5 mdx_hash mem_pool mgf1 misty1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pgp_s2k pk_pad pkcs11 poly1305 poly_dbl prf_tls prf_x942 psk_db pubkey rc4 rfc3394 rfc6979 rmd160 rng roughtime rsa salsa20 scrypt seed serpent serpent_simd sessions_sql sha1 sha2_32 sha2_64 sha3 shacal2 shacal2_simd shake shake_cipher simd siphash siv skein sm2 sm3 sm4 socket sodium sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng thread_utils threefish_512 tiger tls tls_10 tls_cbc tss twofish utils uuid whirlpool x509 x919_mac xmss xtea xts
INFO: Using hardlink to link files into build dir (use --link-method to change)
INFO: Botan 2.14.0 (revision unknown) (unreleased undated) build setup is complete
Now i'm currently using VisualStudio 2017 native tool command prompt, or calling vcvarsall.bat, to set up the environment.

It seems Botan support for building Android binaries on Windows hosts is limited. You will have to use dark magic to make this work.
The build process consists of two phases, the configuration phase and the make phase.
The Android-specific instructions in the documentation you linked do not cover the whole build process, only the configuration phase. For the make phase, you then have to follow the Windows-specific instructions (link).
Configuration phase:
You will need the following binaries, adjust the paths to your machine:
clang++ (note the .cmd at the end): C:\Development\android-ndk-r19c-windows-x86_64\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++.cmd
ar: C:\Development\android-ndk-r19c-windows-x86_64\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\arm-linux-androideabi-ar.exe
In the Botan folder, run the configure command:
python.exe .\configure.py --cc-bin=C:\Development\android-ndk-r19c-windows-x86_64\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++.cmd --ar-command=C:\Development\android-ndk-r19c-windows-x86_64\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\arm-linux-androideabi-ar.exe --os=android --cpu=armv7 --verbose
Make phase
The configuration phase generates a Makefile in the Botan folder. You will have to make some adjustments to this file:
In the line all: libs cli tests docs remove docs
Reason: Additional tools are needed for building the documentation files. If you really need the documentation, you could also try to install these tools, but I have not tested this.
Replace the occurrences of ln -fs with copy.
Reason: On Linux ln -fs would create a symbolic link from the second file in the parameter list to the first. This command is not available, so changing it to copying the first file to the second seems like a pragmatic work-around to me. You could also change it to the appropriate command for creating a link on Windows, but then you might have to adjust it again when deploying to your Android target.
In the lines starting with LIBOBJS =, CLIOBJS = and TESTOBJS =, replace all occurrences of \ with /. In the whole file, replace occurrences of .\ with ./. Reason: Using the Windows-style path separator \ seems to cause problems in some places.
Find the block with # Executable targets and # Library targets. Insert #<< ... << around the parameter lists (known as the nmake inline file feature, based on this answer), to make it look like this:
# Executable targets
$(CLI): $(LIBRARIES) $(CLIOBJS)
$(EXE_LINK_CMD) #<<
$(ABI_FLAGS) $(CLIOBJS) $(EXE_LINKS_TO) $(LDFLAGS) -o $#
<<
$(TEST): $(LIBRARIES) $(TESTOBJS)
$(EXE_LINK_CMD) #<<
$(ABI_FLAGS) $(TESTOBJS) $(EXE_LINKS_TO) $(LDFLAGS) -o $#
<<
# Library targets
./libbotan-2.a: $(LIBOBJS)
$(AR) #<<
$(AR_OPTIONS) $# $(LIBOBJS)
<<
./libbotan-2.so.13: $(LIBOBJS)
$(CXX) #<<
-shared -fPIC -Wl,-soname,libbotan-2.so.13 $(ABI_FLAGS) $(LDFLAGS) $(LIBOBJS) $(LIB_LINKS_TO) -o $#
<<
Reason: Without this change, I got an error about the parameter list being too long.
You will need nmake (part of Visual Studio). On my machine it is installed in C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\bin\Hostx64\x64\nmake.exe
In the Botan folder, run nmake.exe. Afterwards, your Botan folder should contain the binaries botan, botan-test and libraries libbotan-2....

Cross Compiling ghostscript for android, what host should I use?

Hi Guys I am using this tutorial to build ghostscript-9.19 to be able to use in my android application to convert eps document to pdf. It fails while configure. Here are the logs
checking whether to enable maintainer-specific portions of Makefiles...
no
checking for gcc... arm-linux-androideabi-gcc --
sysroot=/<path>/android-ndk-
r11c/platforms/android-17/arch-arm/
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... configure: error: in
`/<path>/ghostscript-9.19/tiff-config':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
configure: error: libtiff configure script failed
This is the build file I am running
#!/bin/sh
# Compiles ghostscript for Android
# Make sure you have NDK_ROOT defined in .bashrc or .bash_profile
INSTALL_DIR="`pwd`/app/jni/gs"
SRC_DIR="`pwd`/../ghostscript-9.19"
cd $SRC_DIR
export
PATH="/<path>/android-ndk-r11c/toolchains/arm-
linux-androideabi-4.9/prebuilt/darwin-x86_64/bin:$PATH"
export SYS_ROOT="/<path>/Android/android-ndk-
r11c/platforms/android-17/arch-arm/"
export CC="arm-linux-androideabi-gcc --sysroot=$SYS_ROOT"
export LD="arm-linux-androideabi-ld"
export AR="arm-linux-androideabi-ar"
export RANLIB="arm-linux-androideabi-ranlib"
export STRIP="arm-linux-androideabi-strip"
mkdir -p $INSTALL_DIR
./configure --host=arm-linux-androideabi --build=x86_64-apple-darwin
--prefix=$INSTALL_DIR LIBS="-lc -lgcc"
make PREFIX=$INSTALL_DIR
make install DESTDIR=$INSTALL_DIR
exit 0
I am using --host=arm-linux-androideabi. What host should i use? What do I need to change in ghostScript project to make compile in successfully?
Any help is highly appreciated.

Cross compiling Ghostscript is pretty involved, partly because the Ghostscript build relies on building and running interim tools (genarch, genconf, mkromfs and echogs) which, obviously, must be built with the native compiler, rather than the cross compiler.
I think the problem you are seeing is because the call to the libtiff configure doesn't pass on the required options.
You may be better served grabbing, and tweaking the two files (a makefile and a header) from this commit:
Makefile for Android MuPDF libgs.so
and tweaking it to match your requirements.
There is a basic guide of what to do for cross compiling at the bottom of this page:
Ghostscript FAQ
I have a "project" to improve support for cross compiling, but it is slow going at the moment.

Building OpenSSL for Android (ARMv7) on Win32

How can I build OpenSSL for Android ARM v7 (using Android NDK) on Win32?

Until the OpenSSL's wiki and setenv-android.sh are updated accordingly, I'll publish the recipe here. The required fixes to the process are:
Update setenv-android.sh to support Windows.
Update PATH to use Android NDK's (mingw) GNU make (rather than Cygwin's).
Invoke make with a Windows-style path to Cygwin's perl.
This recipe will be a strange hybrid of Cygwin and mingw (since Android NDK gcc toolchains for win32 are mingw). I'm assuming a Windows x86_64 build of the Android NDK unpacked into c:\android-ndk-r9d, and that you wish to use a gcc 4.8 toolchain.
Install Android NDK (duh!).
Install Cygwin -- make sure to include perl
Start Cygwin shell as an administrator to make sure native symlinks will work.
Within the console, run the following script to set the variables:
export \
CYGWIN=winsymlinks:native \
ANDROID_API=android-14 \
ANDROID_DEV=c:/android-ndk-r9d/platforms/android-14/arch-arm/usr \
PATH=/cygdrive/c/android-ndk-r9d/prebuilt/windows-x86_64/bin:/cygdrive/c/android-ndk-r9d/toolchains/arm-linux-androideabi-4.8/prebuilt/windows-x86_64/bin:$PATH \
MACHINE=armv7 \
SYSTEM=android \
ARCH=arm \
CROSS_COMPILE=arm-linux-androideabi-
Now, unpack openssl:
tar xzfv openssl-1.0.1i.tar.gz (or whatever your tarball is)
cd openssl-1.0.1i (or whatever your version is)
Make sure you have actual native Win32 (!) symlinks in include/openssl:
cmd /c "dir include\openssl"
You should see something like:
13-Aug-14 05:59 PM <SYMLINK> aes.h [..\..\crypto\aes\aes.h]
13-Aug-14 05:59 PM <SYMLINK> asn1.h [..\..\crypto\asn1\asn1.h]
(etc.)
Now it's time to configure:
./config shared -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine --openssldir=/foo/bar
Ignore the failure to build (due to failure to find perl). We'll rectify this right away. Do this:
make PERL=$(cygpath -w $(which perl))
Now wait for a few minutes until it builds, and presto, you have your libcrypto.so etc.

Just a couple of comments on my experience with this:
Executing this statement:
PERL=$(cygpath -w $(which perl))
in the cygwin shell allows the shell to interpret the backslashes as escape characters and the build process chokes. To solve this I did the following:
$ echo $(cygpath -w $(which perl))
which produced the windows formatted path to the perl executable:
C:\cygwin64\bin\perl
Then I added this line to the export shown above:
PERL=c:/cygwin64/bin/perl \
There are other ways of doing this, but it worked and headed off the problem with the ./config statement documented above (not finding perl).
Second issue was the -no- statements. After running the configure, the script reports that you'll have to run make depend. I wanted to exclude MD5 (i.e. -no-md5) and when I did the make depend, it errored out with a report that MD5 was disabled. Uhhh, yes, that was kind of the idea, but I just won't use MD5 hashes. I did use the -no-ssl2 and didn't get any complaints after the make depend.
Third issue and this is a mystery. The build broke on compiling crypto because it could not find a symbol that is supposed to be defined in /crypto/objects/obj_xref.h. When I looked at the file, it was empty. Something in the perl script I suppose, but no time to debug right now, since I'm at proof of concept phase. I placed a copy from a patch that I picked up at https://github.com/devpack/openssl-android
After that, my build ran to completion. I've done no testing with this and it is not a trustworthy solution, but it did compile and produce the static libraries that I need for proof of concept for my client.
Just as an update, my shared library built with these libraries loaded fine on my target.

Compiling against C++ standard libraries on the android toolchain

I have a really simple helloworld.cpp program
#include <iostream>
using namespace std;
int main ()
{
cout << "Hello World!";
return 0;
}
And I'm trying to compile it for android x86 with the cross-compiler from the toolchain:
/Users/me/android-ndk-r8/toolchains/x86-4.4.3/prebuilt/darwin-x86/bin/i686-android-linux-g++ helloworld.cpp -L "/Users/me/android-ndk-r8/sources/cxx-stl/stlport/libs/x86/" -lstlport_static
However, I get errors:
helloworld.cpp:2:20: error: iostream: No such file or directory
Any idea why?

Check the documentation.html file included with the NDK, under "Standalone Toolchain". It says that if you invoke the compiler in this way you won't be able to "use any C++ STL". However it is possible, as the documentation explains, if you first create a "customized" toolchain installation, using something like the following command:
$NDK/build/tools/make-standalone-toolchain.sh --platform=android-8 --install-dir=/tmp/my-android-toolchain --arch=x86
where $NDK is the path to your NDK directory. Note the --arch=x86 which means that the toolchain is prepared specifically for the x86 Android. This prepares what you need in one directory, including the STL headers and folders. You should then be able to use -lstdc++ to link against the STL (static version), i.e. something like:
/tmp/my-android-toolchain/bin/i686-android-linux-g++ helloworld.cpp -lstdc++
For a more complete explanation, please see the NDK documentation.

The NDK documentation is not entirely accurate, at least not currently. In fact, it states when using the prebuilt toolchain "You won't be able to use any C++ STL (either STLport or the GNU libstdc++) with it.", but this is out of date. I created a small hello world program using the include with the same error. It can be solved without creating your own toolchain though, which is nice if you don't want to have to add one more step to your configuration process and allows you to always use the latest SDK platform without creating a new toolchain every time.
The NDK ships with the source code for several versions of standard C++ libraries: GAbi++, STLport, and GNU STL. Each flavor comes with prebuilt shared and static libs as well. My example below will use stlport.
To use the stand-alone toolchain at its installed location, you can do something like this:
export CXX='$NDK_ROOT/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-g++ --sysroot="$NDK_ROOT/platforms/android-19/arch-arm"'
This, for example, would set your CXX compiler to compile ARM on the OS X system using SDK platform level 19. This much you probably already knew. Also, you would want to export your CC, CPP, LD, AR, and RANLIB if you use it. I also personally create an envar for READELF.
To add support for C++ libs, you could do something like follows:
$CXX helloworld.cpp -I$NDK_ROOT/sources/cxx-stl/stlport/stlport -L$NDK_ROOT/sources/cxx-stl/stlport/libs/armeabi -lstlport_shared
Note that this will link the libstlport_shared.so which will now be required at runtime, so you may need to add a relative path to the command above to support that, depending on your APK structure. If you want to just test this simple case, you can just run this on the emulator as follows:
adb push a.out /data
adb push $NDK_ROOT/sources/cxx-stl/stlport/libs/armeabi/libstlport_shared.so /data
adb shell
# su
# cd /data
# chmod 777 a.out
# ./a.out
To get rid of the headache of dealing with shared library paths, you can also statically link the C++ library in by changing "-lstlport_shared" to "-lstlport_static". There are some consequences of doing this, as explained in the NDK Dev Guide. The biggest issue is due to having the static library linked in multiple places, causing:
- memory allocated in one library, and freed in the other would leak or even corrupt the heap.
- exceptions raised in libfoo.so cannot be caught in libbar.so (and may simply crash the program).
- the buffering of std::cout not working properly
A useful tool is also included to see what dependencies your program has, the readelf tool.
To see what other shared libraries your program requires, you can run the following:
$NDK_ROOT/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-readelf -a a.out | grep NEEDED
Other cool standard tools:
addr2line - convert a stack trace address to a line of code
nm - Displays the symbol table
objdump - displays info on an object

i call one of the functions from gnustl after it runs fine from prebuilt aosp arm-linux-androideabi-gcc --std=c++11 and after crashing error i cant get a backtrace from logs or reason, my hope is turning to crossbuilt qemu-linux-user to run host compiled i686 binary on the arm, difficulty is interacting with crosshost libs aapt from adt always crashes on any other than platform specific libs, unless kernel module packaged update is possible...

Building FIPS module for android

I am working on an android project that requires the cryptographic libraries that are present in the application to be FIPS certified. To my knowledge there are no FIPS validated JAVA security libraries. Boucy Castle is good but its not validated. After reading some forum posts, I found out that OpenSSL's FIPS module can be used with the help of NDK.
Right now I am trying to build the fips-openssl module for Android, to do that I have created a script for the environment variables for cross compiling.
I am using openssl-fips, and ndk-r8 for this project. I followed the fips guideline I found on google. I hope this gives a clear picture of what I am trying to do.
#! /bin/sh
export ANDROID_NDK="~/Android/android-ndk-r8"
export FIPS_SIG="${ANDROID_NDK}/incore"
export GCC_C1="/usr/lib/gcc/i686-pc-linux-gnu/4.7.0/"
export PATH=$PATH:"${ANDROID_NDK}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin";
export MACHINE=armv71
export ARCH=arm.
export CROSS_COMPILE="arm-linux-androideabi-"
export SYSTEM=android
#export RELEASE=2.6.32.GMU
export ANDROID_DEV="$ANDROID_NDK/platforms/android-14/arch-arm/usr"
export HOSTCC=/usr/bin/gcc
when doing the make this is the error that i get.
arm-linux-androideabi-gcc: error trying to exec 'cc1': execvp: No such file or directory
make[1]: *** [cryptlib.o] Error 1
make[1]: Leaving directory `/home/abhiram/fips/openssl-fips-1.2.3/crypto'
make: *** [build_crypto] Error 1
When i do a "find", the cc1 executable is present in this specific directory.
find . -name cc1
./toolchains/mipsel-linux-android-4.4.3/prebuilt/linux-x86/libexec/gcc/mipsel-linux-android/4.4.3/cc1
./toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/libexec/gcc/arm-linux-androideabi/4.4.3/cc1
./toolchains/x86-4.4.3/prebuilt/linux-x86/libexec/gcc/i686-android-linux/4.4.3/cc1

Looks like the problem is in the export statement, there is a blank space where a dash should be in the PATH line. Change this:
export PATH=$PATH:"${ANDROID_NDK}/toolchains/arm-linux-androideabi 4.4.3/prebuilt/linux-x86/bin";
to this:
export PATH=$PATH:"${ANDROID_NDK}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin";
Also, your find shows that the cc1 executable is not in the path, so add its location to the path export as well:
export PATH=$PATH:"${ANDROID_NDK}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin":"${ANDROID_NDK}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/libexec/gcc/arm-linux-androideabi/4.4.3/";

Right now I am trying to build the fips-openssl module for Android, to do that I have created a script for the environment variables for cross compiling.
OpenSSL provides a script for Android, if you are interested. You can find it on the OpenSSL wiki: FIPS Library and Android.
when doing the make this is the error that i get:
arm-linux-androideabi-gcc: error trying to exec 'cc1'
It looks like your PATH does not include the cross-compile toolchain.
How are you invoking the script? You need to include a leading dot (".") to ensure the changes are applied to the current shell (and not the sub-shell that executes the script (which simply exits)).
Here' the first step of OpenSSL's build procedures for Android located at FIPS Library and Android. Notice the leading dot:
$ . .setenv-android.sh
The results of running the script set a bunch of variables used by the OpenSSL build system:
$ . ./setenv-android.sh
ANDROID_NDK_ROOT: /opt/android-ndk-r9
ANDROID_EABI: arm-linux-androideabi-4.6
ANDROID_API: android-14
ANDROID_SYSROOT: /opt/android-ndk-r9/platforms/android-14/arch-arm
ANDROID_TOOLCHAIN: /opt/android-ndk-r9/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86_64/bin
FIPS_SIG:
CROSS_COMPILE: arm-linux-androideabi-
ANDROID_DEV: /opt/android-ndk-r9/platforms/android-14/arch-arm/usr
"${ANDROID_NDK}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/libexec/gcc/arm-linux-androideabi/4.4.3/"
I don't believe this is needed.
export ANDROID_NDK="~/Android/android-ndk-r8"
According to the folks on the Android NDK user list, you should set both ANDROID_NDK_ROOT and ANDROID_SDK_ROOT. The various NDK and SDK tools use those environmental variables. I suppose the SDK value would be "~/Android/android-sdk" for your installation.
See Recommended NDK Directory? for details.
I also think you should be using ANDROID_SYSROOT. Its not used by the NDK or SDK tools; rather, its used by OpenSSL and passed as sysroot during compile.