Develop Reference

Option -no-engine causes problems with OpenSSL build for Android

I have to build the OpenSSL 1.0.1j libraries for Android, following the instructions at http://wiki.openssl.org/index.php/Android, on a Debian 7 system.
My configuration options are
./Configure dist -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine
The build fails due to the error
make[2]: *** No rule to make target `../../include/openssl/engine.h', needed by `rsa_lib.o'.
(Remark: Using linux-generic64 instead of dist made no difference)
Providing the option -no-rsa leads to complaints from dsa_lib.o. It
also does not make sense to disable RSA and DSA, does it?
I read the NEWS file, http://wiki.openssl.org/ and questions here on SO,
but could not find a solution.
Any suggestions?
Besides that: What is the actual meaning of -no-engine? According to my understanding,
ENGINE is the interface to the crypto algorithms of openssl. Why should it be possible to disable it at all?

Option -no-engine causes problems with OpenSSL build for Android ...
./Configure dist -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine
You can safely omit the no-engine option. The option was used to reduce the size of the binary.
What is the actual meaning of -no-engine? According to my understanding, ENGINE is...
That's a good point, and I can't answer it. But I can say I've used the procedures on the wiki page for a few years, and I know OpenSSL still works (compiles/links/runs) when the no-engine option is used.
Maybe something has changed for 1.0.1j. I did not upgrade (meaning I did not build 1.0.1j for Android and iOS) because I'm not interested in that Downgrade SCSV to accommodate the browsers and their broken-shit, insecure practices of retrying with SSLv3.
Using linux-generic64 instead of dist made no difference...
The cross-compile script (setenv-android.sh) sets the paths to the Android NDK tools AND it sets a few key environmental variables. Of them, CROSS_COMPILE are ANDROID_DEV are critical. From the tail of setenv-android.sh:
# For the Android toolchain
# https://android.googlesource.com/platform/ndk/+/ics-mr0/docs/STANDALONE-TOOLCHAIN.html
export ANDROID_SYSROOT="$ANDROID_NDK_ROOT/platforms/$_ANDROID_API/$_ANDROID_ARCH"
export SYSROOT="$ANDROID_SYSROOT"
export NDK_SYSROOT="$ANDROID_SYSROOT"
export ANDROID_NDK_SYSROOT="$ANDROID_SYSROOT"
export ANDROID_API="$_ANDROID_API"
# CROSS_COMPILE and ANDROID_DEV are DFW (Don't Fiddle With). Its used by OpenSSL build system.
# export CROSS_COMPILE="arm-linux-androideabi-"
export ANDROID_DEV="$ANDROID_NDK_ROOT/platforms/$_ANDROID_API/$_ANDROID_ARCH/usr"
export HOSTCC=gcc
Configuration for Android is picked up through SYSTEM and ARCH. Once Android kicks in, CROSS_COMPILE and ANDROID_DEV are utilized.
Because of the environmental variables, all you need to do is configure no-ssl2 no-ssl3 ....

A symbolic link to engine.h is not created when building OpenSSL with no-engine. I just added
(cd include/openssl ; ln -s ../../crypto/engine/engine.h .)
to my build process.

Building OpenSSL for Android (ARMv7) on Win32

How can I build OpenSSL for Android ARM v7 (using Android NDK) on Win32?

Until the OpenSSL's wiki and setenv-android.sh are updated accordingly, I'll publish the recipe here. The required fixes to the process are:
Update setenv-android.sh to support Windows.
Update PATH to use Android NDK's (mingw) GNU make (rather than Cygwin's).
Invoke make with a Windows-style path to Cygwin's perl.
This recipe will be a strange hybrid of Cygwin and mingw (since Android NDK gcc toolchains for win32 are mingw). I'm assuming a Windows x86_64 build of the Android NDK unpacked into c:\android-ndk-r9d, and that you wish to use a gcc 4.8 toolchain.
Install Android NDK (duh!).
Install Cygwin -- make sure to include perl
Start Cygwin shell as an administrator to make sure native symlinks will work.
Within the console, run the following script to set the variables:
export \
CYGWIN=winsymlinks:native \
ANDROID_API=android-14 \
ANDROID_DEV=c:/android-ndk-r9d/platforms/android-14/arch-arm/usr \
PATH=/cygdrive/c/android-ndk-r9d/prebuilt/windows-x86_64/bin:/cygdrive/c/android-ndk-r9d/toolchains/arm-linux-androideabi-4.8/prebuilt/windows-x86_64/bin:$PATH \
MACHINE=armv7 \
SYSTEM=android \
ARCH=arm \
CROSS_COMPILE=arm-linux-androideabi-
Now, unpack openssl:
tar xzfv openssl-1.0.1i.tar.gz (or whatever your tarball is)
cd openssl-1.0.1i (or whatever your version is)
Make sure you have actual native Win32 (!) symlinks in include/openssl:
cmd /c "dir include\openssl"
You should see something like:
13-Aug-14 05:59 PM <SYMLINK> aes.h [..\..\crypto\aes\aes.h]
13-Aug-14 05:59 PM <SYMLINK> asn1.h [..\..\crypto\asn1\asn1.h]
(etc.)
Now it's time to configure:
./config shared -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine --openssldir=/foo/bar
Ignore the failure to build (due to failure to find perl). We'll rectify this right away. Do this:
make PERL=$(cygpath -w $(which perl))
Now wait for a few minutes until it builds, and presto, you have your libcrypto.so etc.

Just a couple of comments on my experience with this:
Executing this statement:
PERL=$(cygpath -w $(which perl))
in the cygwin shell allows the shell to interpret the backslashes as escape characters and the build process chokes. To solve this I did the following:
$ echo $(cygpath -w $(which perl))
which produced the windows formatted path to the perl executable:
C:\cygwin64\bin\perl
Then I added this line to the export shown above:
PERL=c:/cygwin64/bin/perl \
There are other ways of doing this, but it worked and headed off the problem with the ./config statement documented above (not finding perl).
Second issue was the -no- statements. After running the configure, the script reports that you'll have to run make depend. I wanted to exclude MD5 (i.e. -no-md5) and when I did the make depend, it errored out with a report that MD5 was disabled. Uhhh, yes, that was kind of the idea, but I just won't use MD5 hashes. I did use the -no-ssl2 and didn't get any complaints after the make depend.
Third issue and this is a mystery. The build broke on compiling crypto because it could not find a symbol that is supposed to be defined in /crypto/objects/obj_xref.h. When I looked at the file, it was empty. Something in the perl script I suppose, but no time to debug right now, since I'm at proof of concept phase. I placed a copy from a patch that I picked up at https://github.com/devpack/openssl-android
After that, my build ran to completion. I've done no testing with this and it is not a trustworthy solution, but it did compile and produce the static libraries that I need for proof of concept for my client.
Just as an update, my shared library built with these libraries loaded fine on my target.

Entangled with Google test

I am having trouble to get up and running with Google test. I have read the suggested steps from Google, I also looked a previous post, and read some other examples, but it doesn't clear much things up.
To keep things simple, I am trying the suggested example from Google test that is available from the directory in the Android ndk - sample1:
// main.cpp
#include <QtGui/QGuiApplication>
#include "qtquick2applicationviewer.h"
#include "gtest/gtest.h"
int main(int argc, char *argv[])
{
QGuiApplication app(argc, argv);
testing::InitGoogleTest(&argc, argv);
QtQuick2ApplicationViewer viewer;
viewer.setMainQmlFile(QStringLiteral("qml/factorial/main.qml"));
viewer.showExpanded();
return RUN_ALL_TESTS();
}
// sample1_unittest.cpp
#include <limits.h>
#include "sample1.h"
#include "gtest/gtest.h"
// Tests factorial of 0.
TEST(FactorialTest, Zero) {
EXPECT_EQ(1, Factorial(0));
}
The files sample1.h, sample1.cpp are also in the project, which contain the factorial function. Google test was equally informed to the project file - factorial.pro:
INCLUDEPATH +=
/opt/android-studio/ndk/sources/third_party/googletest/googletest/include
When I press [Build > Build Project "factorial"] it gets the following error:
main.cpp:8: error: undefined reference to 'testing::InitGoogleTest(int*, char**)'
sample1_unittest.cpp:17: error: undefined reference to 'testing::Test::Test()'
I am working with Ubuntu, QtCreator, Android and C++. Indeed I have spent already 3 days mocking around, but getting not much anywhere so far. Thus, I am posting here in hope some guru may give any hint on this. Any help will be mostly appreciated.

It seems you haven't built Google Test from what you describe. You need to compile the project into a library and then link against it. If you have CMake installed, then you have two options:
Use CMake's GUI (it's fairly intuitive) to generate the build system files, and then use those as usual (e.g. if you generate a Visual Studio solution, open the .sln file and build the project).
Use the command line to do the same thing; essentially you just create a new directory and do cmake <path-to-google-test> inside of it. The rest is the same.
You could also build the library by yourself. The distribution contains a folder named fused-src which should contain at least two files: gtest_main.cpp and gtest-all.cpp. Compile those files and you're done. You need to generate two libraries here: gtest out of gtest-all.cpp and gtest_main out of gtest_main.cpp.
Another alternative would be to get already built libraries. I've never searched for them, but they might be out there.

Try something like this:
$ g++ -I $GTEST_HOME/include -L $GTEST_HOME/lib -lgtest -lgtest_main -lpthread test.cpp
For more details:
How to setup googleTest as a shared library on Linux
If it still doesn't work, may find interesting to consider to use Makefile:
# A sample Makefile for building Google Test and using it in user
# tests. Please tweak it to suit your environment and project. You
# may want to move it to your project's root directory.
#
# SYNOPSIS:
#
# make [all] - makes everything.
# make TARGET - makes the given target.
# make clean - removes all files generated by make.
# Please tweak the following variable definitions as needed by your
# project, except GTEST_HEADERS, which you can use in your own targets
# but shouldn't modify.
# Points to the root of Google Test, relative to where this file is.
# Remember to tweak this if you move this file.
GTEST_DIR = ..
# Where to find user code.
USER_DIR = ../samples
# Flags passed to the preprocessor.
# Set Google Test's header directory as a system directory, such that
# the compiler doesn't generate warnings in Google Test headers.
CPPFLAGS += -isystem $(GTEST_DIR)/include
# Flags passed to the C++ compiler.
CXXFLAGS += -g -Wall -Wextra -pthread
# All tests produced by this Makefile. Remember to add new tests you
# created to the list.
TESTS = sample1_unittest
# All Google Test headers. Usually you shouldn't change this
# definition.
GTEST_HEADERS = $(GTEST_DIR)/include/gtest/*.h \
$(GTEST_DIR)/include/gtest/internal/*.h
# House-keeping build targets.
all : $(TESTS)
clean :
rm -f $(TESTS) gtest.a gtest_main.a *.o
# Builds gtest.a and gtest_main.a.
# Usually you shouldn't tweak such internal variables, indicated by a
# trailing _.
GTEST_SRCS_ = $(GTEST_DIR)/src/*.cc $(GTEST_DIR)/src/*.h $(GTEST_HEADERS)
# For simplicity and to avoid depending on Google Test's
# implementation details, the dependencies specified below are
# conservative and not optimized. This is fine as Google Test
# compiles fast and for ordinary users its source rarely changes.
gtest-all.o : $(GTEST_SRCS_)
$(CXX) $(CPPFLAGS) -I$(GTEST_DIR) $(CXXFLAGS) -c \
$(GTEST_DIR)/src/gtest-all.cc
gtest_main.o : $(GTEST_SRCS_)
$(CXX) $(CPPFLAGS) -I$(GTEST_DIR) $(CXXFLAGS) -c \
$(GTEST_DIR)/src/gtest_main.cc
gtest.a : gtest-all.o
$(AR) $(ARFLAGS) $# $^
gtest_main.a : gtest-all.o gtest_main.o
$(AR) $(ARFLAGS) $# $^
# Builds a sample test. A test should link with either gtest.a or
# gtest_main.a, depending on whether it defines its own main()
# function.
sample1.o : $(USER_DIR)/sample1.cc $(USER_DIR)/sample1.h $(GTEST_HEADERS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $(USER_DIR)/sample1.cc
sample1_unittest.o : $(USER_DIR)/sample1_unittest.cc \
$(USER_DIR)/sample1.h $(GTEST_HEADERS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $(USER_DIR)/sample1_unittest.cc
sample1_unittest : sample1.o sample1_unittest.o gtest_main.a
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -lpthread $^ -o $#
If you have to use Makefile to get gtest working, you probably may need to adjust the given template for your case, as you intend to build it to use with Android.

Loading kernel module in Android kernel

I am listing my problem here.
I have a Google Nexus one a.k.a. "passion" phone with me. Fastboot and adb tools are installed in the phone. And the boot loader is unlocked.
My task: I have to add a linux kernel module to the Android kernel.
What I have done:
I followed the steps in http://source.android.com/source/initializing.html and downloaded the kernel for android-2.3.6_r1 (passion) and have built it. I am also able to flash it on the phone and the new android kernel also works fine. Now what I want is to modify the kernel and add my own kernel module and then flash it on the phone, so that the kernel on the phone is my modified kernel.
Now I have come across two approaches to do this.
1)
Cross Compile my kernel module with the android kernel and push it on the device with adb command. The Makefile I use in the kernel is as follows.
VERSION = 2
PATCHLEVEL = 3
SUBLEVEL = 6
EXTRAVERSION = -00054-g5f01537
obj-m += hello-1.o
KDIR=/home/apurva/android_dir
PWD := $(shell pwd)
all:
make -C $(KDIR) ARCH=arm CROSS_COMPILE=/home/apurva/android_dir/prebuilt/linux- x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi- SUBDIRS=$(PWD) modules
clean:
make -C $(KDIR) ARCH=arm CROSS_COMPILE=/home/apurva/android_dir/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi- SUBDIRS=$(PWD) clean
Now this is not able to generate new hello-1.ko. I do not know why, I guess there is some problem with the VERSION, PATCHLEVEL, SUBLEVEL and EXTRAVERSION values. Are these necessary? I tried these value from android-2.3.6_r1 also but still it does not work. I am not sure what is this EXTRAVERSION value?
I even tried with the hello-1.ko generated from the compiler in my ubuntu. I pushed this hello-1.ko into the emulator with the following adb command.
/root/bin/src/out/host/linux-x86/bin/adb shell mount
/root/bin/src/out/host/linux-x86/bin/adb push hello-1.ko /data
/root/bin/src/out/host/linux-x86/bin/adb insmod /data/hello-1.ko
But that hello-1.ko is not able to insmod and I get the following error.
insmod : Error in init_module() hello-1.ko function not implemented
Whereas the hello-1.c is quite simple:
#include <linux/module.h> /* Needed by all modules */
#include <linux/kernel.h> /* Needed for KERN_INFO */
int init_module(void)
{
printk(KERN_INFO "Hello world 1.\n");
return 0;
}
void cleanup_module(void)
{
printk(KERN_INFO "Goodbye world 1.\n");
}
2)
The second approach of doing this can be placing my source files of the kernel module in the kernel directory of android. May be in the system directory or somewhere else and ask the make to build these source files also along with the other source. But I am not sure where to ask the make process to do so. I tried to do so in main.mk and created a Android.mk file in the source directory of my source files but it did not work. May be this is a better solution but I could not find any help on this.
After doing this, my kernel modules should be able to control the wnic (Wireless Network Interface device) of the android phone. It should be able to put the wnic in sleep mode and then wake it up after receiving command from my kernel module. If you have some pointers on how to do this, that will be a help. I have found that on Android it is controlled through wpa_supplicant private driver. Commands, like:
wpa_cli driver powermode 0 - auto
wpa_cli driver powermode 1 - active
can do my task, but I am not sure since I have not tried. I have not reached that stage.
Please look into this and provide some help/guidance.
Thanks,
Apurva

Kernel modules (KO's) are much easier to work with than a static kernel - as long as the kernel has enabled them. The easiest way to tell is do an "adb shell lsmod". Second is to see if the kernel .config has enabled CONFIG_MODULES=y and CONFIG_MODULE_UNLOAD=y. Lots of info on the web about linux KO development.
Hummm, you're close but it looks like the makefile is screwy. First try to build the hello KO on your host for unit test, then build on your target. Here's a sample makefile I use on an OMAP36xx running gingerbread:
# Makefile for trivial android kernel module
obj-m += mod_hello.o
CROSS_COMPILE=/opt/distros/ARM/bin/arm-none-linux-gnueabi-
TARG_KDIR ?= /opt/android/dal/nook_kernel
HOST_KDIR=/lib/modules/$(shell uname -r)/build
# target creates:
# .<obj>.o: CC command line for the .o, including dependencies
# .<obj>.mod.o.cmd: CC command line for the mod.o, including dependencies
# .<obj>.ko.cmd: LD command line which links the .o and .mod.o to create the .ko
target:
#echo "Make module for target arm"
make -C $(TARG_KDIR) M=$(PWD) ARCH=arm CROSS_COMPILE=$(CROSS_COMPILE) modules
host:
#echo "Make module for host"
make -C $(HOST_KDIR) M=$(PWD) modules
clean:
#echo "cleaning target"
make -C $(TARG_KDIR) M=$(PWD) clean
#echo "cleaning host"
make -C $(HOST_KDIR) M=$(PWD) clean

First check in the .config if the module support is enabled. (CONFIG_MODULES=y and CONFIG_MODULE_UNLOAD=y) if not enable them using menuconfig.
Then place your module on the root of the kernel source and add this to the main makefile you find at the root
core-y := usr/ yourModule/
and this to the yourModule folders makefile
obj-m := yourModule.o

Cross compile LKM module for Android platform

For work reasons, I need to develop a LKM for Android platform. I'm not very sure how to cross compile my AndroidModule.c and what tools to use for that. I guess that I'll need the source code of Android in order to tell the compiler to link the libraries from there right?
I will also need to download the ARM compiler for Android.
I think with those three things is enough (LKM Code in C, Android Source Code, ARM compiler for android).
The problem is that I can't find any tutorial that explains how to compile LKM for Android.
I'll be very pleased to have more info about it.

Here's a makefile that I use to build modules for Android.
I'm assuming you have a copy of the linux source somewhere and that you have built the kernel for your phone.
In your module directory I put a Makefile like this:
ifneq ($(KERNELRELEASE),)
obj-m := mymod.o
else
COMPILER ?=/pathtoandroid/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-
CROSS_COMPILE ?=$(COMPILER)
ARCH ?=arm
KERNELDIR ?= /home/kernel/androidkerneldir/
PWD := $(shell pwd)
EXTRACFLAGS += -I$(PWD)/somedirectory/shomewhere
default:
$(MAKE) -C $(KERNEL_DIR) M=`pwd` ARCH=$(ARCH) CROSS_COMPILE=$(COMPILER) EXTRA_CFLAGS=$(EXTRACFLAGS) modules
clean:
rm *.o *.ko
install:
adb push mymod.ko /system/lib/modules
This should do it for you. Make sure you have write permissions to /system directory.

Try the Android URL at the bottom it has detailed instructions on how to build the source.
Then follow this URL for final building (this is for dream release), I am assuming the procedure should hold good for other releases as well.

This should help.
To cross compile a module you'll need the kernel source code and the ARM compiler which is in the Android tool chain. You'll need a Makefile something along the lines of
obj-m:= AndroidModule.o
all: module
module:
$(MAKE) -C $(KERNELSRC) SUBDIRS=$(PWD) modules
clean:
$(MAKE) -C $(KERNELSRC) SUBDIRS=$(PWD) clean
#rm -f Module.symvers Module.markers modules.order
Then compile by configuring CROSS_COMPILE as the ARM compiler and KERNELSRC as the kernel source location, and calling make. Here's the command I use on 0xdriod.
CROSS_COMPILE=~/beagle-eclair/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi- ARCH=arm KERNELSRC=~/kernel make