This question is exploratory in nature, not sure if this fits stack overlflow Q&A.
Context:
I have a library written in golang that I need to compile for multiple services to use.
These services are in Kotlin android,Rust,Golang.
The only option I am aware of is using something like SWIG to compile the go library for different languages.
Problem:
I don't think SWIG works for Kotlin.
I am trying to fish for the best methods to do this and different approaches this can be done.
For any language that can generate a C shared library and header file you can use SWIG to wrap it. Equally for any language that runs within a JVM and can call Java classes you can make use of SWIG's auto generated Java bindings.
With that we can therefore do a sequence of things that looks like this:
Go -> C -> JNI -> Java -> Kotlin
It's actually fairly sane. I've put together an example below for this to show how it works since I was curious having never written Go nor Kotlin before. (Take this with a pinch of salt therefore, I've probably not hit "best practice" for either!)
This example assumes you have a working JDK/JRE, C compiler, Go installation and kotlinc.
My demo.go looks like this:
package main
import (
"C"
"fmt"
)
//export TestGoFunc
func TestGoFunc(str *C.char) *C.char {
fmt.Printf("Got string: %s\n", C.GoString(str))
return nil
}
func main() {}
And hello.kt looks like this:
fun main() {
println("Hello, World!")
test.TestGoFunc("Another string")
}
To wrap this I wrote the following SWIG interface:
%module test
%{
#include "golib.h"
%}
%include <typemaps.i>
%pragma(java) jniclasscode=%{
static {
System.loadLibrary("test");
}
%}
// Assuming you don't care about these in your library silence/neaten stuff
#define _Complex
%ignore _GoString_;
%ignore GoComplex64;
%ignore GoComplex128;
%ignore GoSlice;
%ignore GoInterface;
%include "golib.h"
This is a fairly standard SWIG interface for targeting Java - it hides some of the stuff in the generated header file we don't care about and autoloads the .so file inside Java using a pragma for us.
Then I put together a small Makefile to build everything since there's a bunch of steps to this build:
all: libtest.so hello.jar
golib.so: demo.go
go build -o golib.so -buildmode=c-shared demo.go
test_wrap.c: golib.so test.i
swig3.0 -java -Wall test.i
libtest.so: test_wrap.c
gcc -shared -Wall -Wextra test_wrap.c -o libtest.so ./golib.so -I/usr/lib/jvm/default-java/include/ -I/usr/lib/jvm/default-java/include/linux
hello.jar: hello.kt
javac *.java
kotlinc hello.kt -include-runtime -d hello.jar -cp .
jar uvf hello.jar *.class
If we build and run this then it all works nicely:
$ make
go build -o golib.so -buildmode=c-shared demo.go
swig3.0 -java -Wall test.i
gcc -shared -Wall -Wextra test_wrap.c -o libtest.so ./golib.so -I/usr/lib/jvm/default-java/include/ -I/usr/lib/jvm/default-java/include/linux
javac *.java
kotlinc hello.kt -include-runtime -d hello.jar -cp .
jar uvf hello.jar *.class
adding: test.class(in = 302) (out= 216)(deflated 28%)
adding: testJNI.class(in = 389) (out= 268)(deflated 31%)
$ LD_LIBRARY_PATH=. java -jar hello.jar
Hello, World!
Got string: Another string
I'd be tempted to use -buildmode=c-archive for Go to build a static library and then link that into the SWIG shared object instead just to keep things simpler in that regards though.
Related
I'm having an Error in my Native C++ Code in Android NDK Application
My main.cpp
#include <stdio.h>
int main()
{
printf("Hello, world\n");
return 0;
}
The main.c is exactly the same.
If i run
/home/rip/Music/android-ndk-r19b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android26-clang -pie main.c
then
adb push a.out /data/local/tmp
and
adb shell /data/local/tmp/a.out
all works fine. But if i run
/home/rip/Music/android-ndk-r19b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android26-clang++ -pie main.cpp
then
adb push a.out /data/local/tmp
and
adb shell /data/local/tmp/a.out
The error message is:
CANNOT LINK EXECUTABLE "/data/local/tmp/a.out": library "libc++_shared.so" not found
Then i tried to run
/home/rip/Music/android-ndk-r19b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android26-clang++ -pie hello1.cpp /home/rip/Music/android-ndk-r19b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so
to link the library, but it doesn't work anyway.
The error message is:
CANNOT LINK EXECUTABLE "/data/local/tmp/a.out": library "libc++_shared.so" not found
That's the expected behavior. Unlike the standard C library (to which your program is linking when building with simple *-clang), C++ is not a system library. You have to make it available on the device just like any other third party library.
Quoted from official documentation:
Note: libc++ is not a system library. If you use libc++_shared.so, it must be included in your APK. If you're building your application with Gradle this is handled automatically.
And:
If you're using clang directly in your own build system, clang++ will use c++_shared by default. To use the static variant, add -static-libstdc++ to your linker flags.
So either link with C++ statically by passing -static-libstdc++ to compiler. Or copy the libc++_shared.so (from <NDK>/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/ in your case) and run like:
adb push a.out libc++_shared.so /data/local/tmp/
adb shell
cd /data/local/tmp/
LD_LIBRARY_PATH=. ./a.out
Other than the LLVM's Standard C++ library discussed above, there's also a limited system C++ runtime (/system/lib(64)/libstdc++.so) which "provides support for the basic C++ Runtime ABI". But "The system STL will be removed in a future NDK release."
I compiled the same "hello world" code in a .c and .cpp file and didn't have the same issue when I pushed the application to my device. I assume you're having the same issue as in this topic:
Application can't find libc++_shared.so
The issue may come from your toolchain or toolchain parameters as you are calling clang manually.
I created a simple project that you can run and test:
android-ndk-example
add_executable( # Sets the name of the library.
ndk_example_c
# Provides a relative path to your source file(s).
main.c
)
add_executable( # Sets the name of the library.
ndk_example_cpp
# Provides a relative path to your source file(s).
main2.cpp
)
In generated cmake script, I can see the following definition for cpp compiler:
rule CXX_COMPILER__ndk_example_cpp
depfile = $DEP_FILE
deps = gcc
command = D:\Users\$USER\AppData\Local\Android\Sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe --target=armv7-none-linux-androideabi19 --gcc-toolchain=D:/Users/$USER/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64 --sysroot=D:/Users/$USER/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/sysroot $DEFINES $INCLUDES $FLAGS -MD -MT $out -MF $DEP_FILE -o $out -c $IN_ABS
description = Building CXX object $out
I write a new Answer for the Solution because i cannot edit my question.
The Solution is the following command for android devices with armv7:
/home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ --target=armv7-none-linux-androideabi19 --gcc-toolchain=/home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64 --sysroot=/home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/sysroot main.cpp
For aarch64 armv8 the command is:
/home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ --target=aarch64-none-linux-android21 --gcc-toolchain=/home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64 --sysroot=/home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/sysroot main.cpp
A CMakeLists.txt file should look as:
cmake_minimum_required(VERSION 3.1)
set(CMAKE_CXX_COMPILER /home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++)
project(Test CXX)
set(CMAKE_CXX_FLAGS "--target=aarch64-none-linux-android21 --gcc-toolchain=/home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64 --sysroot=/home/tony/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/sysroot")
set(SOURCES
main.cpp
)
add_executable(Test ${SOURCES})
Then can the app build with
cmake
make
adb push Test /data/local/tmp
adb shell /data/local/tmp/Test
I've two files:
lib.c
#include<stdio.h>
void hi() {
printf("Hi i'm a library function in lib.so\n");
}
and main.c
#include<stdio.h>
#include<dlfcn.h>
/* based on Jeff Scudder's code */
int main() {
void *SharedObjectFile;
void (*hi)();
// Load the shared libary;
SharedObjectFile = dlopen("./lib.so", RTLD_LAZY);
// Obtain the address of a function in the shared library.
ciao = dlsym(SharedObjectFile, "hi");
// Use the dynamically loaded function.
(*hi)();
dlclose(SharedObjectFile);
}
And I've tried to build an executables using the following commands:
export LD_LIBRARY_PATH=pwd
gcc -c -fpic lib.c
gcc -shared -lc -o lib.so lib.o
gcc main.c -ldl
And it works pretty well.
Then I've tried to export my program on Android (Nexus One, with ARM-v7-0a arch) using the following commands:
export LD_LIBRARY_PATH=pwd
arm-none-linux-gnueabi-gcc -c -fpic lib.c
arm-none-linux-gnueabi-gcc -shared -lc -o lib.so lib.o
arm-none-linux-gnueabi-gcc main.c -ldl -o main
adb push main /system/app
The result of executing ./main on the correct folder on my smartphone is just:
./main: not found
even if my file is right there!
Am I missing anything during the cross-compile process? Any help?
I'm using the cross-compiler from CodeSourcery and it works well for static programs without .so libraries.
Thanks
EDIT: as Igor states below, that was a linker issue. This command fixes it:
arm-none-linux-gnueabi-gcc -o test main.c -Wl,--dynamic-linker=/system/bin/linker -ldl
in my very case I need other libraries because in /system/lib/ there are no many .so files.
The "not found" message refers not to the shared object but to the dynamic linker. Linux uses /lib/ld-linux.so.2 (or /lib64/ld-linux-x86-64.so.2 for x64) while Android uses /bin/linker. You can check which dynamic loader your program uses with readelf -l, e.g.:
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000034 0x08048034 0x08048034 0x00100 0x00100 R E 0x4
INTERP 0x000134 0x08048134 0x08048134 0x00013 0x00013 R 0x1
[Requesting program interpreter: /lib/ld-linux.so.2]
You can specify a linker to use with ld's --dynamic-linker switch, but there are likely to be other differences. For example, Android uses a stripped-down libc implementation called bionic, and it may be missing functionality that your program relies on, or have different behavior.
You should use NDK or another Android-targeted toolchain when compiling programs for Android. Even though it's based on Linux kernel, the differences are large enough that Linux-targeted toolchains are not sufficient.
All discussions are for x86.
If I wrote a simple hello program such as the one below:
#include <stdio.h>
int main(){
printf("Hello\n");
return 0;
}
And compile it on my PC with ubuntu
$gcc -shared -mPIC -o hello_new hello.c
Then it will give me segmentation fault when I try to execute hello_new. Same error when I move this binary to an android phone. (But I can compile it as a binary with statically linked libc and run it on the android phone)
Yes, I want to execute a shared object directly.
The reason is below:
I recently get a linux file compiled by someone else. When I use linux command file and readelf to analyze the file. It says it is a shared object (32-bit, compiled with -m32). But I can execute the shared object like an executable in android on a phone:
$./hello
This really confuses me. This shared object file contains printf function calls, not sure if it is statically link or dynamically linked. But since it can run on Android through ADB, I assume it is statically linked against libc.
What kind of compilation technique can allow one to execute shared object directly?
It happens that I am currently working on this type of thing.
One of the main differences between executables and shared object under linux, is that an executable has an interpreter and a (valid) entry point.
For example, on a minimal program :
$ echo 'int main;' | gcc -xc -
If you look at it's elf program headers:
$ readelf --program-headers a.out
...
INTERP 0x0000000000000200 0x0000000000400200 0x0000000000400200
0x000000000000001c 0x000000000000001c R 1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
...
The interpreter program is responsible of the execution of the program, to achieve this, it will perform some initializations like loading the needed shared objects. In fact, it is quite analogous to a script shebang, but for elf files.
In this case, /lib64/ld-linux-x86-64.so.2 is the loader for amd64. You can have multiples loaders: e.g., one for 32bits, one for 64.
Now the entry point :
$ readelf --file-header a.out
ELF Header:
...
Entry point address: 0x4003c0
...
$ readelf -a a.out | grep -w _start
57: 00000000004003c0 0 FUNC GLOBAL DEFAULT 13 _start
By default, you can see that _start is defined as the entry point.
So if you consider the following minimal example :
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#ifdef INTERPRETER
const char interp[] __attribute__((section(".interp"))) = INTERPRETER;
#endif /* INTERPRETER */
void entry_point(void) {
fprintf(stderr, "hello executable shared object world !\n");
_exit(EXIT_SUCCESS);
}
If you compile it as a "normal" shared object and execute it :
$ gcc libexecutable.c -Wall -Wextra -fPIC -shared -o libexecutable.so
$ ./libexecutable.so
Erreur de segmentation
You can see it segfaults. But now if you define an interpreter (adapt it's path to what readelf --program-headers gave you before) and tell to the linker what is your entry point :
$ gcc libexecutable.c -Wall -Wextra -fPIC -shared -o libexecutable.so -DINTERPRETER=\"/lib64/ld-linux-x86-64.so.2\" -Wl,-e,entry_point
$ ./libexecutable.so hello executable shared object world !
now it works. Please note that the _exit() call is necessary to avoid a segfault at the end of the execution.
But in the end, remember that because you specify a custom entry point, you will bypass libc initialization steps which could be needed or not, depending on your needs.
I think your android and pc boths are x86 or arm at the same time, else executable should not run in both platform. Now to make a shared library executable at the same time you can use -pie command line option of gcc. Details can be found in this answer.
I am having trouble to get up and running with Google test. I have read the suggested steps from Google, I also looked a previous post, and read some other examples, but it doesn't clear much things up.
To keep things simple, I am trying the suggested example from Google test that is available from the directory in the Android ndk - sample1:
// main.cpp
#include <QtGui/QGuiApplication>
#include "qtquick2applicationviewer.h"
#include "gtest/gtest.h"
int main(int argc, char *argv[])
{
QGuiApplication app(argc, argv);
testing::InitGoogleTest(&argc, argv);
QtQuick2ApplicationViewer viewer;
viewer.setMainQmlFile(QStringLiteral("qml/factorial/main.qml"));
viewer.showExpanded();
return RUN_ALL_TESTS();
}
// sample1_unittest.cpp
#include <limits.h>
#include "sample1.h"
#include "gtest/gtest.h"
// Tests factorial of 0.
TEST(FactorialTest, Zero) {
EXPECT_EQ(1, Factorial(0));
}
The files sample1.h, sample1.cpp are also in the project, which contain the factorial function. Google test was equally informed to the project file - factorial.pro:
INCLUDEPATH +=
/opt/android-studio/ndk/sources/third_party/googletest/googletest/include
When I press [Build > Build Project "factorial"] it gets the following error:
main.cpp:8: error: undefined reference to 'testing::InitGoogleTest(int*, char**)'
sample1_unittest.cpp:17: error: undefined reference to 'testing::Test::Test()'
I am working with Ubuntu, QtCreator, Android and C++. Indeed I have spent already 3 days mocking around, but getting not much anywhere so far. Thus, I am posting here in hope some guru may give any hint on this. Any help will be mostly appreciated.
It seems you haven't built Google Test from what you describe. You need to compile the project into a library and then link against it. If you have CMake installed, then you have two options:
Use CMake's GUI (it's fairly intuitive) to generate the build system files, and then use those as usual (e.g. if you generate a Visual Studio solution, open the .sln file and build the project).
Use the command line to do the same thing; essentially you just create a new directory and do cmake <path-to-google-test> inside of it. The rest is the same.
You could also build the library by yourself. The distribution contains a folder named fused-src which should contain at least two files: gtest_main.cpp and gtest-all.cpp. Compile those files and you're done. You need to generate two libraries here: gtest out of gtest-all.cpp and gtest_main out of gtest_main.cpp.
Another alternative would be to get already built libraries. I've never searched for them, but they might be out there.
Try something like this:
$ g++ -I $GTEST_HOME/include -L $GTEST_HOME/lib -lgtest -lgtest_main -lpthread test.cpp
For more details:
How to setup googleTest as a shared library on Linux
If it still doesn't work, may find interesting to consider to use Makefile:
# A sample Makefile for building Google Test and using it in user
# tests. Please tweak it to suit your environment and project. You
# may want to move it to your project's root directory.
#
# SYNOPSIS:
#
# make [all] - makes everything.
# make TARGET - makes the given target.
# make clean - removes all files generated by make.
# Please tweak the following variable definitions as needed by your
# project, except GTEST_HEADERS, which you can use in your own targets
# but shouldn't modify.
# Points to the root of Google Test, relative to where this file is.
# Remember to tweak this if you move this file.
GTEST_DIR = ..
# Where to find user code.
USER_DIR = ../samples
# Flags passed to the preprocessor.
# Set Google Test's header directory as a system directory, such that
# the compiler doesn't generate warnings in Google Test headers.
CPPFLAGS += -isystem $(GTEST_DIR)/include
# Flags passed to the C++ compiler.
CXXFLAGS += -g -Wall -Wextra -pthread
# All tests produced by this Makefile. Remember to add new tests you
# created to the list.
TESTS = sample1_unittest
# All Google Test headers. Usually you shouldn't change this
# definition.
GTEST_HEADERS = $(GTEST_DIR)/include/gtest/*.h \
$(GTEST_DIR)/include/gtest/internal/*.h
# House-keeping build targets.
all : $(TESTS)
clean :
rm -f $(TESTS) gtest.a gtest_main.a *.o
# Builds gtest.a and gtest_main.a.
# Usually you shouldn't tweak such internal variables, indicated by a
# trailing _.
GTEST_SRCS_ = $(GTEST_DIR)/src/*.cc $(GTEST_DIR)/src/*.h $(GTEST_HEADERS)
# For simplicity and to avoid depending on Google Test's
# implementation details, the dependencies specified below are
# conservative and not optimized. This is fine as Google Test
# compiles fast and for ordinary users its source rarely changes.
gtest-all.o : $(GTEST_SRCS_)
$(CXX) $(CPPFLAGS) -I$(GTEST_DIR) $(CXXFLAGS) -c \
$(GTEST_DIR)/src/gtest-all.cc
gtest_main.o : $(GTEST_SRCS_)
$(CXX) $(CPPFLAGS) -I$(GTEST_DIR) $(CXXFLAGS) -c \
$(GTEST_DIR)/src/gtest_main.cc
gtest.a : gtest-all.o
$(AR) $(ARFLAGS) $# $^
gtest_main.a : gtest-all.o gtest_main.o
$(AR) $(ARFLAGS) $# $^
# Builds a sample test. A test should link with either gtest.a or
# gtest_main.a, depending on whether it defines its own main()
# function.
sample1.o : $(USER_DIR)/sample1.cc $(USER_DIR)/sample1.h $(GTEST_HEADERS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $(USER_DIR)/sample1.cc
sample1_unittest.o : $(USER_DIR)/sample1_unittest.cc \
$(USER_DIR)/sample1.h $(GTEST_HEADERS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $(USER_DIR)/sample1_unittest.cc
sample1_unittest : sample1.o sample1_unittest.o gtest_main.a
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -lpthread $^ -o $#
If you have to use Makefile to get gtest working, you probably may need to adjust the given template for your case, as you intend to build it to use with Android.
I've two files:
lib.c
#include<stdio.h>
void hi() {
printf("Hi i'm a library function in lib.so\n");
}
and main.c
#include<stdio.h>
#include<dlfcn.h>
/* based on Jeff Scudder's code */
int main() {
void *SharedObjectFile;
void (*hi)();
// Load the shared libary;
SharedObjectFile = dlopen("./lib.so", RTLD_LAZY);
// Obtain the address of a function in the shared library.
ciao = dlsym(SharedObjectFile, "hi");
// Use the dynamically loaded function.
(*hi)();
dlclose(SharedObjectFile);
}
And I've tried to build an executables using the following commands:
export LD_LIBRARY_PATH=pwd
gcc -c -fpic lib.c
gcc -shared -lc -o lib.so lib.o
gcc main.c -ldl
And it works pretty well.
Then I've tried to export my program on Android (Nexus One, with ARM-v7-0a arch) using the following commands:
export LD_LIBRARY_PATH=pwd
arm-none-linux-gnueabi-gcc -c -fpic lib.c
arm-none-linux-gnueabi-gcc -shared -lc -o lib.so lib.o
arm-none-linux-gnueabi-gcc main.c -ldl -o main
adb push main /system/app
The result of executing ./main on the correct folder on my smartphone is just:
./main: not found
even if my file is right there!
Am I missing anything during the cross-compile process? Any help?
I'm using the cross-compiler from CodeSourcery and it works well for static programs without .so libraries.
Thanks
EDIT: as Igor states below, that was a linker issue. This command fixes it:
arm-none-linux-gnueabi-gcc -o test main.c -Wl,--dynamic-linker=/system/bin/linker -ldl
in my very case I need other libraries because in /system/lib/ there are no many .so files.
The "not found" message refers not to the shared object but to the dynamic linker. Linux uses /lib/ld-linux.so.2 (or /lib64/ld-linux-x86-64.so.2 for x64) while Android uses /bin/linker. You can check which dynamic loader your program uses with readelf -l, e.g.:
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000034 0x08048034 0x08048034 0x00100 0x00100 R E 0x4
INTERP 0x000134 0x08048134 0x08048134 0x00013 0x00013 R 0x1
[Requesting program interpreter: /lib/ld-linux.so.2]
You can specify a linker to use with ld's --dynamic-linker switch, but there are likely to be other differences. For example, Android uses a stripped-down libc implementation called bionic, and it may be missing functionality that your program relies on, or have different behavior.
You should use NDK or another Android-targeted toolchain when compiling programs for Android. Even though it's based on Linux kernel, the differences are large enough that Linux-targeted toolchains are not sufficient.