I'm trying to build the wireguard-android sources on my windows machine, fixing the problems step by step with trial and error.
I have CMake and Make installed, the first problem I encountered was the 'bad' value reported by uname (mingw64), not matching with the golang filenames hosted by Google.
I've edited (hacked) the make file to point to the real filename of the windows version (amd64). The problem is the windows version has a .zip extension instead of .tar.gzand the result from zip doesn't seem compatible with the piped command.
Original command:
curl "https://dl.google.com/go/go$(DESIRED_GO_VERSION).$(shell uname -s | tr '[:upper:]' '[:lower:]')-$(NDK_GO_ARCH_MAP_$(shell uname -m)).tar.gz" | tar -C "$(dir $#)" --strip-components=1 -xzf -
My attempted changes:
(uses an if-else because it still needs to run on mac too)
ifeq "msys" "$(shell uname -o | tr '[:upper:]' '[:lower:]')"
# Note: when enclosed in the ifeq, the ARCH_MAP part no longer worked
# Note: using tar with the zip fails, because we cant untar a zip (signal 13)
# Note: using unzip with the zip fails with confusing paths
curl "https://dl.google.com/go/go$(DESIRED_GO_VERSION).windows-amd64.zip" | unzip -C "$(dir $#)" --strip-components=1 -xzf -
else
curl "https://dl.google.com/go/go$(DESIRED_GO_VERSION).$(shell uname -s | tr '[:upper:]' '[:lower:]')-$(NDK_GO_ARCH_MAP_$(shell uname -m)).tar.gz" | tar -C "$(dir $#)" --strip-components=1 -xzf -
endif
Resulting error if using unzip:
unzip: cannot find or open C:\wireguard-android\app\build\intermediates\cmake\debug\obj\armeabi-v7a/../generated-src/go-1.13.7/, C:\wireguard-android\app\build\intermediates\cmake\debug\obj\armeabi-v7a/../generated-src/go-1.13.7/.zip or C:\wireguard-android\app\build\intermediates\cmake\debug\obj\armeabi-v7a/../generated-src/go-1.13.7/.ZIP.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 127M 0 5419 0 0 26694 0 1:23:47 --:--:-- 1:23:47 26694
curl: (23) Failed writing body (34 != 1357)
make: *** [C:\wireguard-android\app\build\intermediates\cmake\debug\obj\armeabi-v7a/../generated-src/go-1.13.7/.prepared] Error 9
Resulting error if using tar:
curl "https://dl.google.com/go/go1.13.7.windows-amd64.zip" | tar -C "C:\wireguard-android\app\build\intermediates\cmake\debug\obj\armeabi-v7a/../generated-src/go-1.13.7/" --strip-components=1 -xzf -
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0gzip: stdin has more than one entry--rest ignored
tar: Child died with signal 13
tar: Error is not recoverable: exiting now
0 127M 0 108k 0 0 248k 0 0:08:47 --:--:-- 0:08:47 248k
curl: (23) Failed writing body (738 != 1357)
make: *** [C:\wireguard-android\app\build\intermediates\cmake\debug\obj\armeabi-v7a/../generated-src/go-1.13.7/.prepared] Error 2
How should I get this command working? (With either tar or zip/unzip)
Somebody pointed out to me, that maybe I need the Android distro, not the Windows one. In that case I would assume I need armeabi-v7a, but I can only see ARMv6 and ARMv8 packages. What distribution should I use?
I've managed to solve this, I'm answering my own question rather than deleting because I think this will eventually help another developer.
First some clarifications:
It may seem like you should be including the target-platform Go libs (i.e. ARM for Android), but if you try this, you will get an error that the file is not executable when building. (So the build platform is the version to match, i.e. Windows here)
Because Google provides the Windows libs as a Zip (vs tar.gz for other platforms), you cannot use the tar command.
Because zip's cannot be read from a curl stream (the index is at the end of the file), you need to download the file and then unzip. Some people have suggested to use the jar command, however this did not work successfully with cmake/make
The zip file will be extracted into a subdirectory 'go' inside of whichever path you choose. For this reason you need to modify the paths
Some of the commands on Windows would not accept .. as the middle of a path, so I had to modify the CMakeLists file to use an absolute path
The changed end of the Makefile: (from after the .prepared/mkdir statements)
# Warning the (lack-of) indentation here is critical https://stackoverflow.com/a/4483467/984830
# Note: I've hardcoded the windows filename below, so you'll need to do an `if` (like in my question) if you want to support other platforms as well
curl -o "$(BUILDDIR)/go-$(DESIRED_GO_VERSION)/gofile.zip" "https://dl.google.com/go/go$(DESIRED_GO_VERSION).windows-amd64.zip"
unzip -o "$(BUILDDIR)/go-$(DESIRED_GO_VERSION)/gofile.zip" -d "$(dir $#)"
rm -f "$(BUILDDIR)/go-$(DESIRED_GO_VERSION)/gofile.zip"
patch -p1 -f -N -r- -d "$(dir $#)go/" < goruntime-boottime-over-monotonic.diff
touch "$#"
$(DESTDIR)/libwg-go.so: export PATH := $(BUILDDIR)/go-$(DESIRED_GO_VERSION)/go/bin/:$(PATH)
$(DESTDIR)/libwg-go.so: $(BUILDDIR)/go-$(DESIRED_GO_VERSION)/.prepared go.mod
go build -tags linux -ldflags="-X golang.zx2c4.com/wireguard/ipc.socketDirectory=/data/data/$(ANDROID_PACKAGE_NAME)/cache/wireguard" -v -trimpath -o "$#" -buildmode c-shared
The changed end of the CMakeLists file: (replacing the add_custom_target section)
# added a new variable to get the parent dir without using .. in the path
get_filename_component(destdirparent "${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/.." ABSOLUTE)
# referred to the variable on the last line of this statment
add_custom_target(libwg-go.so WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/libwg-go" COMMENT "Building wireguard-go" VERBATIM COMMAND make
ANDROID_ARCH_NAME=${ANDROID_ARCH_NAME}
ANDROID_C_COMPILER=${ANDROID_C_COMPILER}
ANDROID_TOOLCHAIN_ROOT=${ANDROID_TOOLCHAIN_ROOT}
ANDROID_LLVM_TRIPLE=${ANDROID_LLVM_TRIPLE}
ANDROID_SYSROOT=${ANDROID_SYSROOT}
ANDROID_PACKAGE_NAME=${ANDROID_PACKAGE_NAME}
CFLAGS=${CMAKE_C_FLAGS}\ -Wno-unused-command-line-argument
LDFLAGS=${CMAKE_SHARED_LINKER_FLAGS}\ -fuse-ld=gold
DESTDIR=${CMAKE_LIBRARY_OUTPUT_DIRECTORY}
BUILDDIR=${destdirparent}/generated-src
)
For reference the original files are here: Makefile & CMakeLists and are mirrored on Github
Thanks for #Nick Cardoso's answer. After repeated modification and debugging, I successfully worked out the makefile and CMakeLists.txt available on the Windows platform.
When using, you only need to modify these three variables: GIT_BIN_PATH, DESIRED_GO_VERSION, MAKE_PATH.
Notes:
No spaces are allowed in the project path! Otherwise the path of CURDIR will be truncated
Don't install Cygwin! All you need is Git and the NDK, because Git contains the required commands: unzip.exe, patch.exe and touch.exe
tunnel/tools/libwg-go/Makefile
# SPDX-License-Identifier: Apache-2.0
#
# Copyright © 2017-2019 WireGuard LLC. All Rights Reserved.
BUILDDIR ?= $(CURDIR)/build
DESTDIR ?= $(CURDIR)/out
NDK_GO_ARCH_MAP_x86 := 386
NDK_GO_ARCH_MAP_x86_64 := amd64
NDK_GO_ARCH_MAP_arm := arm
NDK_GO_ARCH_MAP_arm64 := arm64
NDK_GO_ARCH_MAP_mips := mipsx
NDK_GO_ARCH_MAP_mips64 := mips64x
CLANG_FLAGS := --target=$(ANDROID_LLVM_TRIPLE) --gcc-toolchain=$(ANDROID_TOOLCHAIN_ROOT) --sysroot=$(ANDROID_SYSROOT)
export CGO_CFLAGS := $(CLANG_FLAGS) $(CFLAGS)
export CGO_LDFLAGS := $(CLANG_FLAGS) $(LDFLAGS) -Wl,-soname=libwg-go.so
export CC := $(ANDROID_C_COMPILER)
export GOARCH := $(NDK_GO_ARCH_MAP_$(ANDROID_ARCH_NAME))
export GOOS := android
export CGO_ENABLED := 1
default: $(DESTDIR)/libwg-go.so
GIT_BIN_PATH := "D:\Program Files\Git\usr\bin"
UNZIP_PATH := $(GIT_BIN_PATH)\unzip.exe
PATCH_PATH := $(GIT_BIN_PATH)\patch.exe
TOUCH_PATH := $(GIT_BIN_PATH)\touch.exe
DESIRED_GO_VERSION := 1.16.3
$(BUILDDIR)/go-$(DESIRED_GO_VERSION)/.prepared:
if not exist "$(dir $#)" mkdir "$(dir $#)"
curl -o "$(BUILDDIR)\go-$(DESIRED_GO_VERSION)\gofile.zip" "https://dl.google.com/go/go$(DESIRED_GO_VERSION).windows-amd64.zip"
$(UNZIP_PATH) -o "$(BUILDDIR)\go-$(DESIRED_GO_VERSION)\gofile.zip" -d "$(dir $#)"
del /f "$(BUILDDIR)\go-$(DESIRED_GO_VERSION)\gofile.zip"
$(PATCH_PATH) -p1 -f -N -r- -d "$(dir $#)go/" < goruntime-boottime-over-monotonic.diff
$(TOUCH_PATH) "$#"
$(DESTDIR)/libwg-go.so: export PATH := $(BUILDDIR)/go-$(DESIRED_GO_VERSION)/go/bin/:$(PATH)
$(DESTDIR)/libwg-go.so: $(BUILDDIR)/go-$(DESIRED_GO_VERSION)/.prepared go.mod
go build -tags linux -ldflags="-X golang.zx2c4.com/wireguard/ipc.socketDirectory=/data/data/$(ANDROID_PACKAGE_NAME)/cache/wireguard" -v -trimpath -o "$#" -buildmode c-shared
.DELETE_ON_ERROR:
/tunnel/tools/CMakeLists.txt
# SPDX-License-Identifier: Apache-2.0
#
# Copyright © 2018-2019 WireGuard LLC. All Rights Reserved.
cmake_minimum_required(VERSION 3.4.1)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_LIBRARY_OUTPUT_DIRECTORY}")
# Work around https://github.com/android-ndk/ndk/issues/602
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=gold")
file(GLOB WG_SOURCES wireguard-tools/src/*.c ndk-compat/compat.c)
add_executable(libwg.so ${WG_SOURCES})
target_include_directories(libwg.so PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/wireguard-tools/src/uapi/linux/" "${CMAKE_CURRENT_SOURCE_DIR}/wireguard-tools/src/")
target_compile_options(libwg.so PUBLIC -O3 -std=gnu11 -D_GNU_SOURCE -include ${CMAKE_CURRENT_SOURCE_DIR}/ndk-compat/compat.h -DHAVE_VISIBILITY_HIDDEN -DRUNSTATEDIR=\"/data/data/${ANDROID_PACKAGE_NAME}/cache\")
get_filename_component(destdirparent "${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/.." ABSOLUTE)
set(MAKE_PATH "D:/AndroidSDK/ndk/21.1.6352462/prebuilt/windows-x86_64/bin/make.exe")
# referred to the variable on the last line of this statment
add_custom_target(libwg-go.so WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/libwg-go" COMMENT "Building wireguard-go" VERBATIM COMMAND ${MAKE_PATH}
ANDROID_ARCH_NAME=${ANDROID_ARCH_NAME}
ANDROID_C_COMPILER=${ANDROID_C_COMPILER}
ANDROID_TOOLCHAIN_ROOT=${ANDROID_TOOLCHAIN_ROOT}
ANDROID_LLVM_TRIPLE=${ANDROID_LLVM_TRIPLE}
ANDROID_SYSROOT=${ANDROID_SYSROOT}
ANDROID_PACKAGE_NAME=${ANDROID_PACKAGE_NAME}
CFLAGS=${CMAKE_C_FLAGS}\ -Wno-unused-command-line-argument
LDFLAGS=${CMAKE_SHARED_LINKER_FLAGS}\ -fuse-ld=gold
DESTDIR=${CMAKE_LIBRARY_OUTPUT_DIRECTORY}
BUILDDIR=${destdirparent}/generated-src
)
# Hack to make it actually build as part of the default target
add_dependencies(libwg.so libwg-go.so)
Related
I have an app which locate in /system/priv-app/MyTestApp.
The android source code environment is Android P (API 28).
At first, the MyTestApp.apk was build with gradle build tools 3.6.1. Then I upgrade it to 4.1.0 and build a new MyTestApp.apk and prebuilt it in the rom. So the crash happens.
E AndroidRuntime: java.lang.UnsatisfiedLinkError: dlopen failed: library "/system/priv-app/MyTestApp/MyTestApp.apk!/lib/armeabi-v7a/libmytest.so" not found
E AndroidRuntime: at java.lang.Runtime.loadLibrary0(Runtime.java:1016)
E AndroidRuntime: at java.lang.System.loadLibrary(System.java:1669)
I pull the file /system/priv-app/MyTestApp/MyTestApp.apk, extract the file and found that the libmytest.so exists. It definitely caused by gradle build tools upgrade. But I can't find the reason. Could anyone give some help?
After looking to the source code, I found that in bionic/linker/linker.cpp
if (entry.method != kCompressStored || (entry.offset % PAGE_SIZE) != 0) {
close(fd);
return -1;
}
entry.offset % PAGE_SIZE != 0 this condition fails.
So I guess there's something wrong with zipalign of AGP 4.1+.
Still looking into it.
Android build system will uncompress dex files embedded in apk for privileged apps as long as the property DONT_UNCOMPRESS_PRIV_APPS_DEXS not defined. The uncompress definition is as below
# Uncompress dex files embedded in an apk.
#
define uncompress-dexs
$(hide) if (zipinfo $# '*.dex' 2>/dev/null | grep -v ' stor ' >/dev/null) ; then \
tmpdir=$#.tmpdir; \
rm -rf $$tmpdir && mkdir $$tmpdir; \
unzip -q $# '*.dex' -d $$tmpdir && \
zip -qd $# '*.dex' && \
( cd $$tmpdir && find . -type f | sort | zip -qD -X -0 ../$(notdir $#) -# ) && \
rm -rf $$tmpdir; \
fi
endef
When apk build with AGP 4.0, after uncompress-dexs, it can be success verified by zipalign tool.
zipalign -v -c -p 4 MyTestApp.apk
But when build with AGP 4.1+ (also include 7.0-alpha), it can't be verified by zipalign tool.
But I haven't find the reason yet. Just guess that there is something new change about apk archive in AGP 4.1+.
android.useNewApkCreator=false
Add this line in gradle.properties can solve this issue.
But since AGP 3.6+, the default value of this property is true. So it should not be the difference between AGP 4.0 and AGP 4.1.
I'm so confused. There maybe some else conditions to make effect.
refer to lastest code, the uncompress-dexs method is like below
https://android.googlesource.com/platform/build/+/master/core/definitions.mk#2385
# Uncompress dex files embedded in an apk.
#
define uncompress-dexs
if (zipinfo $# '*.dex' 2>/dev/null | grep -v ' stor ' >/dev/null) ; then \
$(ZIP2ZIP) -i $# -o $#.tmp -0 "classes*.dex" && \
mv -f $#.tmp $# ; \
fi
endef
I am trying to compile a driver as a loadable module but whenever I adb shell into my phone and do an insmod test.ko I get the error message insmod: failed to load /data/local/tmp/test.ko: Exec format error. Grep'ing dmesg I find the following log: test: no symbol version for module_layout.
I've done quite a bit of googling [Ref] [Ref] [Ref] [Ref] [Ref] and reading the linux kbuild documentation txt files [Ref] [Ref] to no avail, so if someone knows the answer it'd be fab :) (I've only referenced the most helpful links I found).
What I've done so far is this:
I have an Qualcomm Aurora checkout and in the kernel directory I type the following
cp arch/arm64/configs/gemini_user_defconfig .config [B]# Has CONFIG_MODVERSIONS enabled and MODULE_SIG*=n[/B]
yes "" | make oldconfig ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- V=1
make prepare ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- V=1
make scripts ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- V=1
make modules ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- V=1
I have my PATH set to point to /MyAOSBDir/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin so I pick up the correct toolchain.
At this point appears to be well. In my kernel directory I have Module.symvers, so I appear to be set.
Now to the directory I have set up outside of this build tree with a dummy test... The Makefile looks like this:
KERNEL_DIR:=/solomon-build/MiNote2AOSB/kernel/
obj-m += test.o
PWD := $(shell pwd)
.PHONY: all
all:
$(MAKE) M=$(PWD) ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- -C $(KERNEL_DIR) modules V=1
clean:
$(MAKE) M=$(PWD) -C $(KERNEL_DIR) clean
The driver is just a dummy:
#include <linux/module.h> /* Needed by all modules */
#include <linux/kernel.h> /* Needed for KERN_INFO */
#include <linux/init.h> /* Needed for the macros */
static int __init hello_start(void)
{
printk(KERN_INFO "Hello world\n");
return 0;
}
static void __exit hello_end(void)
{
printk(KERN_INFO "Goodbye world\n");
}
module_init(hello_start);
module_exit(hello_end);
I do make make all at the comment line and get this output:
make M=/solomon-build/build_mxt_kmod ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- -C /solomon-build/MiNote2AOSB/kernel/ modules V=1
make[1]: Entering directory `/solomon-build/MiNote2AOSB/kernel'
test -e include/generated/autoconf.h -a -e include/config/auto.conf || ( \
echo >&2; \
echo >&2 " ERROR: Kernel configuration is invalid."; \
echo >&2 " include/generated/autoconf.h or include/config/auto.conf are missing.";\
echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix it."; \
echo >&2 ; \
/bin/false)
mkdir -p /solomon-build/build_mxt_kmod/.tmp_versions ; rm -f /solomon-build/build_mxt_kmod/.tmp_versions/*
make -f ./scripts/Makefile.build obj=/solomon-build/build_mxt_kmod
./scripts/gcc-wrapper.py aarch64-linux-android-gcc -Wp,-MD,/solomon-build/build_mxt_kmod/.test.o.d -nostdinc -isystem /solomon-build/MiNote2AOSB/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/../lib/gcc/aarch64-linux-android/4.9.x-google/include -I./arch/arm64/include -Iarch/arm64/include/generated -Iinclude -I./arch/arm64/include/uapi -Iarch/arm64/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__ -mlittle-endian -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -mgeneral-regs-only -fno-pic -fno-delete-null-pointer-checks -Os -Wno-maybe-uninitialized --param=allow-store-data-races=0 -Wframe-larger-than=2048 -fstack-protector -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-var-tracking-assignments -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(test)" -D"KBUILD_MODNAME=KBUILD_STR(test)" -c -o /solomon-build/build_mxt_kmod/.tmp_test.o /solomon-build/build_mxt_kmod/test.c
(cat /dev/null; echo kernel//solomon-build/build_mxt_kmod/test.ko;) > /solomon-build/build_mxt_kmod/modules.order
make -f ./scripts/Makefile.modpost
find /solomon-build/build_mxt_kmod/.tmp_versions -name '*.mod' | xargs -r grep -h '\.ko$' | sort -u | sed 's/\.ko$/.o/' | scripts/mod/modpost -m -i ./Module.symvers -I /solomon-build/build_mxt_kmod/Module.symvers -o /solomon-build/build_mxt_kmod/Module.symvers -S -E -w -s -T -
./scripts/gcc-wrapper.py aarch64-linux-android-gcc -Wp,-MD,/solomon-build/build_mxt_kmod/.test.mod.o.d -nostdinc -isystem /solomon-build/MiNote2AOSB/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/../lib/gcc/aarch64-linux-android/4.9.x-google/include -I./arch/arm64/include -Iarch/arm64/include/generated -Iinclude -I./arch/arm64/include/uapi -Iarch/arm64/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__ -mlittle-endian -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -mgeneral-regs-only -fno-pic -fno-delete-null-pointer-checks -Os -Wno-maybe-uninitialized --param=allow-store-data-races=0 -Wframe-larger-than=2048 -fstack-protector -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-var-tracking-assignments -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(test.mod)" -D"KBUILD_MODNAME=KBUILD_STR(test)" -DMODULE -c -o /solomon-build/build_mxt_kmod/test.mod.o /solomon-build/build_mxt_kmod/test.mod.c
aarch64-linux-android-ld -EL -r -T ./scripts/module-common.lds --build-id --fix-cortex-a53-843419 -o /solomon-build/build_mxt_kmod/test.ko /solomon-build/build_mxt_kmod/test.o /solomon-build/build_mxt_kmod/test.mod.o
make[1]: Leaving directory `/solomon-build/MiNote2AOSB/kernel'
So, there are no messages saying there is a missing Module.symvers, so that's good, and I do get a built .ko file.
What I don't understand is the message ERROR: Kernel configuration is invalid. The files it is talking about exist, so I know they have been created in my kernel build.
Anyway, trying to insmod this onto the phone gives the message I mentioned previously. If I cat Module.symvers I see:
0xff924338 backlight_force_update drivers/video/backlight/backlight EXPORT_SYMBOL
0x30254daf test_iosched_register block/test-iosched EXPORT_SYMBOL
<snip>
0xdd502540 backlight_device_unregister drivers/video/backlight/backlight EXPORT_SYMBOL
0x9939eba0 backlight_unregister_notifier drivers/video/backlight/backlight EXPORT_SYMBOL
So no symbol module_layout in the file, and thus no CRC, which is what I presume is creating the error message?
Any clues would be much appreciated, thank you.
Oh I should also mention that the module and kernel vermagic match:
# modinfo /data/local/tmp/test.ko
filename: /data/local/tmp/test.ko
depends:
vermagic: 3.18.20-mytestkernel-perf-g671c431-dirty SMP preempt mod_unload modversions aarch64
# uname -a
Linux localhost 3.18.20-mytestkernel-perf-g671c431-dirty #1 SMP PREEMPT Mon Jun 5 15:46:13 BST 2017 aarch64
*EDIT - FOUND SOLUTION - NEW QUESTON *:
I've gotten much further since posting. What I realised was, was that the compilation of the kernel using the Android build system is producing a radically different Module.symvers one from my method above (for some reason it took a while for the penny to drop and for me to check the Andoird build output - duh!). In fact, I've realised I always had the Module.symvers file, it was just located in ../out/target/product/msm8996/obj/KERNEL_OBJ/Module.symvers relative to the kernel dir. Copying this file into kernel and re-running my module build now produces a module that loads correctly :)
So, I guess the question becomes, why do the two files differ? What is Android doing differently? I assume it is setting up some more detailed config, but how/what/why?
So, the answer was relatively simple in the end...
I just use the normal Android build process of sourcing envsetup.sh, running lunch and then building the kernel:
. build/envsetup.sh
lunch msm8996-userdebug
make kernel -j16
(Remember to make sure in your .config file to set CONFIG_MODULES=y and set any kernel drivers to =m if you want them as modules).
In the separate directory, not in the Android build tree I now have my new module and the following Makefile, which now looks like this:
MY_ANDROID_ROOT_DIR :=/path/to/my/android/checkout
KERNEL_DIR:=$(MY_ANDROID_ROOT_DIR)/kernel
obj-m += my_driver.o
PWD := $(shell pwd)
.PHONY: all
all:
$(MAKE) \
M=$(PWD) \
ARCH=arm64 \
CROSS_COMPILE=aarch64-linux-android- \
-C $(KERNEL_DIR) \
modules \
V=1 \
O=$(MY_ANDROID_ROOT_DIR)/out/target/product/msm8996/obj/KERNEL_OBJ
clean:
$(MAKE) M=$(PWD) -C $(KERNEL_DIR) clean
The make all output will still be in your modules directory (not the Android output dir).
This is basically identical to all of the examples relating to pure Linux builds. The only extra bit of information I needed to provide was where the Android build system output its build files, which is O=$(MY_ANDROID_ROOT_DIR)/out/target/product/<your-product>/obj/KERNEL_OBJ.
The O= define tells Make where to find the output of the kernel build.
The 'V=1` define tells Make to output verbose build information.
modules is a kernel target that will build you module defined in the Makefile variable obj-m. This variable "specifies the object files which are built as loadable kernel modules".
Now my modules build fine and I don't need to worry about independently compiling the kernel or anything like that to fix missing file messages (although it will still output about autoconf.h, which I don't understand because it exists in my tree and doesn't seem to stop my modules working).
PS If you module includes more than one file define obj-m += my_driver.o and then my_driver-m:= <extra objects>.
How can I fix this problem？When I was building the Android source of 5.0 with
'make -j4', it was causing a problem.
This is the error log:
Install: out/host/linux-x86/bin/acp
Yacc: aidl <= frameworks/base/tools/aidl/aidl_language_y.y
prebuilts/misc/linux-x86/bison/bison -d -o out/host/linux-x86/obj32/EXECUTABLES/aidl_intermediates/aidl_language_y.cpp frameworks/base/tools/aidl/aidl_language_y.y
Lex: aidl <= frameworks/base/tools/aidl/aidl_language_l.l
logtags: out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/src/core/java/android/content/EventLogTags.java <= frameworks/base/core/java/android/content/EventLogTags.logtags
prebuilts/misc/linux-x86/bison/bison: m4 子进程失败
build/core/binary.mk:539: recipe for target 'out/host/linux-x86/obj32/EXECUTABLES/aidl_intermediates/aidl_language_y.cpp' failed
make: *** [out/host/linux-x86/obj32/EXECUTABLES/aidl_intermediates/aidl_language_y.cpp] Error 1
make: *** 正在等待未完成的任务....
frameworks/base/tools/aidl/aidl_language_l.l:55: warning, 无法匹配规则
flex-2.5.39: fatal internal error, exec of /usr/bin/m4 failed
build/core/binary.mk:576: recipe for target 'out/host/linux-x86/obj32/EXECUTABLES/aidl_intermediates/aidl_language_l.cpp' failed
make: *** [out/host/linux-x86/obj32/EXECUTABLES/aidl_intermediates/aidl_language_l.cpp] Error 1
make: *** Deleting file 'out/host/linux-x86/obj32/EXECUTABLES/aidl_intermediates/aidl_language_l.cpp'
logtags: out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/src/core/java/android/speech/tts/EventLogTags.java <= frameworks/base/core/java/android/speech/tts/EventLogTags.logtags
In my case, I get similar error when build Marshmallow 6.0.1_r1 in Ubuntu 18.04.
If you get similar errors like this:
build/core/binary.mk:646: recipe for target 'out/host/linux-x86/obj/EXECUTABLES/aidl_intermediates/aidl_language_l.cpp' failed
make: *** [out/host/linux-x86/obj/EXECUTABLES/aidl_intermediates/aidl_language_l.cpp] Aborted (core dumped)
Look at the line carefully, it's a core dumped, so I use coredumpctl list, navigate to the end of list with G, then figure out the core dump executed path (right column) is prebuilts/misc/linux-x86/flex/flex-2.5.39
And the PREBUILT file shows:
xb#dnxb:~/my_android_src/prebuilts/misc/linux-x86/flex$ cat PREBUILT
The objects in this prebuilt directory can be rebuilt
using the source archive hosted at
http://sourceforge.net/projects/flex/files/
Commands:
./configure
make CFLAGS="-static" LDFLAGS="-static"
# (Disregard the errors at the end of the build)
rm flex
make CFLAGS="-static" LDFLAGS="-static" flex
# Copy the linker command line, replace it with static libm.a
gcc -static -static -o flex ccl.o dfa.o ecs.o scanflags.o gen.o main.o misc.o nfa.o parse.o scan.o skel.o sym.o tblcmp.o yylex.o options.o scanopt.o buf.o tables.o tables_shared.o filter.o regex.o /usr/lib/x86_64-linux-gnu/libm.a
xb#dnxb:~/my_android_src/prebuilts/misc/linux-x86/flex$
That's it, the first 2 lines said that you probably need to rebuild yourself if something went wrong since this flex-2.5.39 binary is prebuilt in another platform.
So simply untar the flex-2.5.39.tar.gz of this directory, go to the extracted flex-2.5.39 directory, then follows the PREBUILT instruction above.
In short, you need to do the following steps:
cd <ANDROID_TREE_DIR>/prebuilts/misc/linux-x86/flex
tar -zxvf flex-2.5.39.tar.gz
cd flex-2.5.39
make CFLAGS="-static" LDFLAGS="-static"
rm flex
make CFLAGS="-static" LDFLAGS="-static" flex
/bin/bash ./libtool --tag=CC --mode=link gcc -static -static -o flex ccl.o dfa.o ecs.o scanflags.o gen.o main.o misc.o nfa.o parse.o scan.o skel.o sym.o tblcmp.o yylex.o options.o scanopt.o buf.o tables.o tables_shared.o filter.o regex.o lib/libcompat.la /usr/lib/x86_64-linux-gnu/libm.a
cd ..
cp flex-2.5.39/flex .
mv flex flex-2.5.39
ls -la
flex-2.5.39 #### <<- YOUR new generated file
flex-2.5.39.tar.gz
MODULE_LICENSE_BSD_LIKE
NOTICE
PREBUILT
Now try to do make <module> -j4 again, no longer abort on such error.
p/s: Alternative way will be LANG=C make pm -j4 if you see something like LC_* error message.
p/s2: I also need symlink ld as below to fixed 'out/host/linux-x86/obj/lib/libart.so' failed and error: unsupported reloc 42:
xb#dnxb:~/my_android_src$ cp prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8//x86_64-linux/bin/ld ../ld_bk #backup android source's ld first
xb#dnxb:~/my_android_src$ rm prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8//x86_64-linux/bin/ld #remove android source's ld
xb#dnxb:~/my_android_src$ ln -s /usr/bin/ld.gold prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8//x86_64-linux/bin/ld #create symlink for android source's ld to system's ld.gold
Check carefully you have all the required packages installed https://source.android.com/source/initializing.html
I am trying to load two shared libraries in my Android application through the loadLibrary call:
System.loadLibrary("mywrapper");
System.loadLibrary("crypto");
I keep running catching the `UnsatisfiedLinkError. Here is a more detailed version of the error.
Caused by: java.lang.UnsatisfiedLinkError: Cannot load library: link_image[1969]:
130 could not load needed library 'libcrypto.so.1.0.0' for
'libmywrapper.so' (load_library[1111]: Library 'libcrypto.so.1.0.0' not found)
Any ideas?
After spending some time I found out that Android doesn't support versioned libraries. Has any one faced the same issue?
I had the same problem on building libwebsockets for Android, which needs to link with OpenSSL. I use libssl.so as example. You should do the same for related .so files.
Before:
huiying#huiying-PORTEGE-R835:~$ objdump -p libssl.so | grep so
libssl.so: file format elf32-little
NEEDED libcrypto.so.1.0.0
NEEDED libdl.so
NEEDED libc.so
SONAME libssl.so.1.0.0
After
huiying#huiying-PORTEGE-R835:~$ rpl -R -e .so.1.0.0 "_1_0_0.so" libssl.so
Replacing ".so.1.0.0" with "_1_0_0.so" (case sensitive) (partial words matched)
.
A Total of 2 matches replaced in 1 file searched.
huiying#huiying-PORTEGE-R835:~$ objdump -p libssl.so | grep so
libssl.so: file format elf32-little
NEEDED libcrypto_1_0_0.so
NEEDED libdl.so
NEEDED libc.so
SONAME libssl_1_0_0.so
And don't forget to change file name "libssl.so" to "libssl_1_0_0.so".
The hack works. I have running Android app to prove it. See my rant at http://computervisionandjava.blogspot.com/2015/05/trouble-with-versioned-shared-libraries.html.
It seems android has an issue with loading versioned libraries.The issue at hand was because of library so-name in my case libcrypto.so.1.0.0. Even if you rename the library and try to load it as a prebuilt shared library in an android make file it fails.( It has to be because the library name is somehow embedded in the file. And any library that links with it expects to be linked with a library of with the same name )
I hope there are other ways out there when it comes to handling libraries with version names in android.
For now I am evading the problem all together by using static libraries of openssl and linking them with my own shared library.
Year 2014 and still no support for versioned shared libs. So I made a script to patch SONAME. Just point the script to input dir where all versioned libs placed. Then check output dir "unver".
#!/bin/bash
DIR="$1"
if [ "$DIR" == "" ]; then
echo "Usage: fix-soname.sh <target dir>"
exit
fi
if [ ! -d $DIR ]; then
echo "Not found: $DIR"
exit
fi
OUT="$DIR/unver"
echo "Input=$DIR"
echo "Output=$OUT"
CWD=$(pwd)
cd $DIR
# prep dirs
mkdir -p $OUT
rm -f -R $OUT/*
# rename libs and copy to out dir
find "$DIR" -type f -name '*.so*' | while read FILE; do
NAME=$(basename "$FILE")
SONAME=$NAME
while read SYMLINK; do
X=$(basename "$SYMLINK")
#echo "$X (${#X}) -> $NAME (${#NAME})"
if [ "${#X}" -lt "${#SONAME}" ]; then
SONAME=$X
fi
done<<EOT
`find -L $DIR -samefile $FILE`
EOT
#echo $SONAME
cp -f $SONAME $OUT/
done
# patch libs in out dir
find "$OUT" -type f -name '*.so*' | while read FILE; do
# get file name without path
NAME=$(basename "$FILE")
# extract SONAME from shared lib
SONAME=`readelf -d $FILE | grep '(SONAME)' | grep -P '(?<=\[)(lib.*?)(?=\])' -o`
#echo "$NAME [$SONAME]"
# patch SONAME if required
if [ "$NAME" != "$SONAME" ]; then
L1=${#NAME}
L2=${#SONAME}
LDIFF=$((L2-L1))
#echo "$NAME [$SONAME] ($LDIFF)"
if [ "$LDIFF" -gt "0" ]; then
SONEW=$NAME
for (( c=1; c<=$LDIFF; c++ )); do
SONEW+="\x00"
done
echo "$NAME [$SONAME] -> $SONEW ($LDIFF)"
rpl -R -e "$SONAME" "$SONEW" $OUT
fi
fi
done
cd $CWD
I would like to cross-compile ICU static libs for Android using Cygwin. So far, I have been able to configure and make the Cygwin/MSVC and Cygwin versions. I have installed the android-ndk-r7 and can see a version of gcc in the toolchains directory. Several examples suggest using host:arm-eabi - but this is not present on my machine.
I have copied mh-linux to mh-unknown in /icu/source/config and run the following:
export HOST_ICU=/cygdrive/d/__/External/SQLite/icu
export ICU_CROSS_BUILD=/cygdrive/d/__/External/SQLite/icu-cygwin
export NDK_ROOT=/cygdrive/d/__/android-ndk-r7
export CPPFLAGS="-I$NDK_ROOT/platforms/android-8/arch-arm/usr/include/ -O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=0 -DU_GNUC_UTF16_STRING=0 -fno-short-enums -nostdlib"
export CXXFLAGS="-I$NDK_ROOT/platforms/android-8/arch-arm/usr/include/ -O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=0 -DU_GNUC_UTF16_STRING=0 -fno-short-enums -nostdlib"
export CFLAGS="-I$NDK_ROOT/platforms/android-8/arch-arm/usr/include/ -O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=0 -DU_GNUC_UTF16_STRING=0 -fno-short-enums -nostdlib"
export LDFLAGS="-lc -Wl,-rpath-link=$NDK_ROOT/platforms/android-8/arch-arm/usr/lib/ -L $NDK_ROOT/platforms/android-8/arch-arm/usr/lib/"
$HOST_ICU/source/configure --with-cross-build=$ICU_CROSS_BUILD --enable-extras=no --enable-strict=no --enable-static --enable-shared=no --enable-tests=no --enable-samples=no --enable-dyload=no --enable-tools=no --host=arm-eabi --with-data-packaging=archive
I get the following error:
checking for ICU version numbers... release 4.8.1.1, library 48.1.1, unicode version 6.0
checking build system type... i686-pc-cygwin
checking host system type... arm-unknown-eabi
checking target system type... arm-unknown-eabi
checking whether to build debug libraries... no
checking whether to build release libraries... yes
checking for arm-eabi-gcc... no
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/cygdrive/d/Projects/__/External/SQLite/icu-android':
configure: error: C compiler cannot create executables
See `config.log' for more details
I'm sure this is a "stupid" question, but how do I get the ICU configure script to point to the gcc under $NDK_ROOT\toolchains\arm-linux-androideabi-4.4.3\prebuilt\windows\arm-linux-androideabi\bin? Am I missing some setup or install? Should I be setting my PATH so that the first gcc found is the one in arm-linux-androiedeabi?
Update 1. I just noticed that while windows\arm-linux=androideabi\bin contains gcc, windows\bin contains arm-linux-androideabi-gcc. How do I get ICU to call this?
Update 2. On the suggestion of Steven R. Loomis, I picked up updates for config.sub and config.guess from
http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD
http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
placed android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin into my path, and re-ran configure with --host=arm-linux-androideabi... this time:
checking for arm-linux-androideabi-gcc... arm-linux-androideabi-gcc
checking whether the C compiler works... no
Definitely closer. Detailed error from config.log
gcc version 4.4.3 (GCC)
configure:3125: $? = 0
configure:3114: arm-linux-androideabi-gcc -V >&5
arm-linux-androideabi-gcc.exe: '-V' option must have argument
configure:3125: $? = 1
configure:3114: arm-linux-androideabi-gcc -qversion >&5
arm-linux-androideabi-gcc.exe: unrecognized option '-qversion'
arm-linux-androideabi-gcc.exe: no input files
configure:3125: $? = 1
configure:3145: checking whether the C compiler works
configure:3167: arm-linux-androideabi-gcc -I/cygdrive/d/Projects/android-ndk-r7/platforms/android-8/arch-arm/usr/include
/ -O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=0 -DU_GNUC_UTF16_STRING=0 -fno-short-enums -nostdlib -I/cygdrive/d/Projec
ts/android-ndk-r7/platforms/android-8/arch-arm/usr/include/ -O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=0 -DU_GNUC_UTF1
6_STRING=0 -fno-short-enums -nostdlib -lc -Wl,-rpath-link=/cygdrive/d/Projects/android-ndk-r7/platforms/android-8/arch-a
rm/usr/lib/ -L /cygdrive/d/Projects/android-ndk-r7/platforms/android-8/arch-arm/usr/lib/ conftest.c >&5
D:/Projects/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/
4.4.3/../../../../arm-linux-androideabi/bin/ld.exe: cannot find -lc
collect2: ld returned 1 exit status
configure:3171: $? = 1
configure:3209: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME ""
| #define PACKAGE_TARNAME ""
| #define PACKAGE_VERSION ""
| #define PACKAGE_STRING ""
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| /* end confdefs.h. */
|
| int
| main ()
| {
|
| ;
| return 0;
| }
configure:3214: error: in `/cygdrive/d/Projects/__/External/SQLITE/icu-android':
configure:3216: error: C compiler cannot create executables
See `config.log' for more details
Update 3. The changes to config.sub and config.guess worked in that we are now using the right gcc compiler. The -lc failure comes from not being able to find libc.so (which is in android-ndk-r7/platforms/android-8/arch-arm/usr/lib" even though this is in the LDFLAGS. I did have an extra space after -L in the original LDFLAGS, but removing this did not help.
Update 4. According to an older post in http://groups.google.com/group/android-ndk/browse_thread/thread/46295616a889bc12
"The windows ndk toolchain is (thankfully) native to windows, so it
doesnt go through the cygwin translation layer which would translate
/cygdrive paths."
Update 5. Swapped all instances of /cygdrive/d/ with D:/. Now C compiler works though it still doesn't make. Suspect that ICU_CROSS_BUILD has to be in the icu/source directory.
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... yes
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether arm-linux-androideabi-gcc accepts -g... yes
checking for arm-linux-androideabi-gcc option to accept ISO C89... none needed
checking for arm-linux-androideabi-g++... arm-linux-androideabi-g++
checking whether we are using the GNU C++ compiler... yes
checking whether arm-linux-androideabi-g++ accepts -g... yes
checking how to run the C preprocessor... arm-linux-androideabi-gcc -E
checking for a BSD-compatible install... /usr/bin/install -c
checking for gmake... /usr/bin/gmake
configure: error: D:/Projects/__/External/SQLite/icu-cygwin/config/icucross.mk not found. Please build ICU in
D:/Projects/__/External/SQLite/icu-cygwin first.
Update 6. I re-configured and rbuilt my Cygwin folder in icu-cygwin. Go figure, this time icucross.mk was there. Successful configuration! But...
Update 7 Make did not end up so well.
$ make
D:/Projects/__/External/SQLite/icu/source/config/mh-linux:41: *** target pattern contains no `%'. Stop.
What?!?! It seems that now we want Cygwin paths again.. :(
Update 8. Changed my paths so that HOST_ICU and ICU_CROSS_BUILD use Cygwin paths, but NDK_ROOT is Windows path (since Android NDK ld can't handle cygwin paths).
THis time further but
arm-linux-androideabi-gcc.exe: /cygdrive/d/Projects/__/External/SQLit e/icu/source/stubdata/stubdata.c: No such file or directory
arm-linux-androideabi-gcc.exe: no input files
Seems that what has to happen is that arm-linux-androideabi-gcc needs to be made for cygwin, or the cross-build will not work.
Update 9. It seems that arm-linux-androideabi-gcc does not support cygwin paths - though ndk_build does. However, ICU is set to call arm-linux-androideabi-gcc while "make" requires cygwin paths. Maybe time to switch to OSX or Linux to do this.
Update 10 - Still no success.
Cygwin - Apparently the arm-linux-androideabi Crystax build also doesn't support cygwin paths in -L. Attempting to cross-compile under Cygwin will give the -lc error, since it cannot parse the -L/cygdrive/d/... path to the library. Changing to D:/ helps, but later causes make to fail since it is Cygwin make.
Linux - Using the normal NDK r7 build, configuration will fail with a wchar_t = 0 error. The Crystax NDK build will fix this, and make will fail complaining about uint64_t in Android's sys/type.h. See ICU library in Android NDK. You can force it to be defined and it will lead to yet another error about size mismatch.
OSX - Probably the most successful, compiling using the official build or the Crystax build, it leads directly to the uint64_t bug. If you hack around it, it will lead you to
icu/source/common/ustrenum.cpp:118: error: must #include <typeinfo> before using typeid
Help!
00. Download android ndk http://developer.android.com/tools/sdk/ndk/index.html
example install d:\android-r9b
01. Download install example d:\msys MSys http://www.mingw.org/wiki/MSYS 1.01
02. Download gcc 3.8.1 w64 http://mingw-w64.sourceforge.net/ example install d:\mingw32_64
03. Download icu versin 52.1 http://site.icu-project.org/download/52#TOC-ICU4C-Download
04. Extract icu source code to d:\icu
05. Enter msys use export PATH=/d/msys/1.0/bin:/d/ming32_64/mingw32/bin:$PATH
06. cd /d/icu;mkdir mingw;mkdir android;cd mingw
this is follow icu readme.html cross compile steps
07. cd /d/icu/ming ;../source/runConfigureICU MinGW ;make
no problem,all is working and under /d/icu/mingw/bin generate tools for cross compile later.
08. cd /d/icu/android
09. /d/android-ndk-r9b/build/tools/make-standalone-toolchain.sh --platform=android-9 --install-dir=/d/androidgcc/ -- toolchain=arm-linux-androideabi-4.8
if use window 7/8 64bit add additional options: --system=windowx86_64
10. export PATH again; export PATH=/d/msys/1.0/bin/:/d/androidgcc/bin/:/d/androidgcc/arm-linux-androideabi/bin/
the path must include the ar.exe execute for create library.
11. create shared library.
sh ../source/configure --host=armv6-google-linux --enable-shared=yes --disable-static -with-cross-build=/d/icu/mingw CC=arm-linux-androideabi-gcc CXX=arm-linux-androideabi-g++ AR=arm-linux-androideabi-ar --with-data-packaging=archive
12. make
all is successful