Building an Android executable gRPC server that uses protocol buffers (without APK) - android

I compiled the gRPC Android example from here.
I want to run the program as executable from adb shell.
Added these lines to grpc-helloworld.cc:
#include <iostream>
int main() {
std::cout << "qwerty" << std::endl;
return 0;
}
And these lines to its CMakeLists.txt:
add_executable(avocado
src/main/cpp/grpc-helloworld.cc)
target_include_directories(avocado
PRIVATE ${HELLOWORLD_PROTO_HEADERS})
target_link_libraries(avocado
helloworld_proto_lib
android
${log-lib})
Then I pushed the generated executable and libs file and tried to run it:
LD_LIBRARY_PATH=. ./avocado
I got the following error:
[libprotobuf FATAL
/home/buga/grpc/third_party/protobuf/src/google/protobuf/stubs/common.cc:79]
This program was compiled against version 3.0.0 of the Protocol Buffer
runtime library, which is not compatible with the installed version
(3.5.1). Contact the program author for an update. If you compiled
the program yourself, make sure that your headers are from the same
version of Protocol Buffers as your link-time library. (Version
verification failed in
"out/soong/.intermediates/frameworks/av/drm/libmediadrm/libmediadrm/android_arm64_armv8-a_kryo300_shared_core/gen/proto/frameworks/av/drm/libmediadrm/protos/plugin_metrics.pb.cc".)terminating
with uncaught exception of type google::protobuf::FatalException: This
program was compiled against version 3.0.0 of the Protocol Buffer
runtime library, which is not compatible with the installed version
(3.5.1). Contact the program author for an update. If you compiled
the program yourself, make sure that your headers are from the same
version of Protocol Buffers as your link-time library. (Version
verification failed in
"out/soong/.intermediates/frameworks/av/drm/libmediadrm/libmediadrm/android_arm64_armv8-a_kryo300_shared_core/gen/proto/frameworks/av/drm/libmediadrm/protos/plugin_metrics.pb.cc".)
Aborted
What am I doing wrong?
We realized that there is a version of the protobuf library called libprotobuf-cpp-full.so and libprotobuf-cpp-lite.so, and it seems that their version is 3.0.0. This is conflicting with our version (3.5.1) which is compiled into either a static lib, or as a shared lib.

I'm not quite sure why this happens. Something about once the linker loads helloworld_proto_lib, it overrides all loaded protobuf symbols, and for some reason another library that you had nothing to do with crashes your program. But that's not telling you anything new.
Here's one way to solve this problem:
1. Changes to grpc-helloworld.cc
Make the main extern "C", and change its name maybe. For example:
extern "C" int my_main() {
std::cout << "qwerty" << std::endl;
return 0;
}
2. Add file grpc-avocado.cc
This will contain the actual main of the executable, which will dynamically load the libraries helloworld_proto_lib and grpc-helloworld. Here's how to do it:
#include <iostream>
#include <android/dlext.h>
#include <dlfcn.h>
int main() {
android_dlextinfo extinfo;
extinfo.flags = ANDROID_DLEXT_FORCE_LOAD;
void* proto_lib = android_dlopen_ext("/path/to/libhelloworld_proto_lib.so", RTLD_LAZY, &extinfo);
void* helloworld = dlopen("/path/to/libgrpc-helloworld.so", RTLD_LAZY);
int (*my_main)() = (int (*)())dlsym(helloworld, "my_main");
return my_main();
}
The function android_dlopen_ext from #include <android/dlext.h>, and its flag argument, are described here: https://developer.android.com/ndk/reference/group/libdl . In the above code we pass the flag ANDROID_DLEXT_FORCE_LOAD, which is documented as:
When set, do not use stat(2) to check if the library has already been loaded.
This flag allows forced loading of the library in the case when for some reason multiple ELF files share the same filename (because the already-loaded library has been removed and overwritten, for example).
Note that if the library has the same DT_SONAME as an old one and some other library has the soname in its DT_NEEDED list, the first one will be used to resolve any dependencies.
I think the text in bold is what explains why this solution works.
3. Change CMakeLists.txt
Since you'll be loading helloworld_proto_lib dynamically, you can now remove it from the executable definition, and there's no need for any proto headers:
add_executable(avocado
src/main/cpp/grpc-avocado.cc)
target_link_libraries(avocado
android
${log-lib})
Build, push, and run
You can now build, push the executable avocado and the two libraries libgrpc-helloworld.so, libhelloworld_proto_lib.so, and run. You don't need LD_LIBRARY_PATH. Good luck with the rest of your project!

Related

cross compile for android on linux with __ANDROID_API__ predefinition fails my program

I setup conan for cross building android app on my linux, i have my_profile below for conan for this cross building, which that I run conan create . user/testing -pr=my_profile
include(default)
target_host=aarch64-linux-android
android_ndk=$HOME/android-ndk-r21
api_level=21
[settings]
arch=armv8
build_type=Release
compiler=clang
compiler.libcxx=libc++
compiler.version=9
os=Android
os.api_level=$api_level
[build_requires]
[options]
[env]
PATH=[$android_ndk/toolchains/llvm/prebuilt/linux-x86_64/bin]
CHOST=$target_host
AR=$target_host-ar
AS=$target_host-as
LD=$target_host-ld
STRIP=$target_host-strip
RANLIB=$target_host-ranlib
CC=$target_host$api_level-clang
CXX=$target_host$api_level-clang++
CONAN_MAKE_PROGRAM=$android_ndk/prebuilt/linux-x86_64/bin/make
CONAN_CMAKE_TOOLCHAIN_FILE=$android_ndk/build/cmake/android.toolchain.cmake
and I made a very simple file:
#include <fcntl.h>
int raw_fallocate(int fd, off_t length) {
if (fallocate(fd, 0, 0, length) == 0) return 0;
return -1;
}
I found that in the fcntl.h, it only defines fallocate when __ANDROID_API >=21 with #ifdef
so in my CMakeLists.txt, I need to put
target_compile_definitions(hello PRIVATE __ANDROID_API__=21) to make it compile, otherwise, the compiler will complain it cannot find definition of fallocate.
That all make sense. However, when I put this preprocessor definition, I still got a warning message saying:
In file included from <built-in>:413:
<command line>:1:9: warning: '__ANDROID_API__' macro redefined [-Wmacro-redefined]
#define __ANDROID_API__ 21
^
<built-in>:405:9: note: previous definition is here
#define __ANDROID_API__ 16
^
1 warning generated.
What I don't understand is I could not find this built-in thing..., i searched my whole android_ndk folder, and could not find where is this #define __ANDROID_API__ 16
Also I only have android_ndk v21 installed, I have no idea where this version of 16 came from.
Any idea?
This #define __ANDROID_API__ 16 comes from the NDK itself (this is the lowest supported API for android-ndk-r21. To set it to 21, you must pass ANDROID_PLATFORM parameter to CMake. Update: this is actually wrong. For ABI arm64-v8a the minimal API is 21. So, the problem is that arch=armv8 didn't work.
According to the conan instructions, set os.api_level=21 should have worked. But with this approach, you should not supply the CONAN_CMAKE_TOOLCHAIN_FILE. Update: this does not work because conan is not compatible with NDK r21.
I assume that when you do supply CONAN_CMAKE_TOOLCHAIN_FILE, all the settings, like CC= and AR= become irrelevant. Same for os.api_level and arch. But if you replace cmake with a script that calls the original cmake binary and sets the necessarycommand-line parameters, including -DANDROID_PLATFORM=android-21, you should be set. Simply add to my_profile:
CONAN_CMAKE_PROGRAM=cmake-wrapper
This approach is used in https://github.com/bincrafters/conan-android_ndk_installer package.
So I've been digging into this a little bit, and I think I found where the problem is. As #Alex Cohn said, the android ndk sets the min api level to 16 in the file
$android_ndk/build/cmake/platforms.cmake, it says on the first line set(NDK_MIN_PLATFORM_LEVEL "16"). After I change it to 21, everything works.
Now the question becomes: How can I override this value externally without touching this platforms.cmake file (I just don't want to touch files come with ndk package)? I tried to put set(NDK_MIN_PLATFORM_LEVEL "16") in my CMakelists.txt file for my project, but that does not work as I believe it is overridden later on by platforms.cmake.
Of course the second method of using conan's ndk-installer also works, and I checked the same platforms.cmake file in the installed ndk folder, it has the same value of 16. So there must be somewhere in conan settings that updated the value to 21, and I would like to learn where / how this is changed when using ndk-installer, so I can do the same for my manually installled android ndk.

Android NDK CMake linking issues

I'm quite new with NDK + Gradle + CMake integration and I'm trying to understand why linking doesn't export symbols as intended.
I have a static library built by a CMakeLists.txt which is not the main CMakeLists.txt.
The scripts does something like:
# main CMakeLists.txt
add_subdirectory(${LIBS}/foo libs}
add_library(native SHARED native.cpp)
# omitting standard android libraries
target_link_libraries(native foo ${android-lib} ${log-lib})
while CMakeLists.txt inside ${libs}/foo is the following:
# misc configuration of ${SRC}
add_library(foo STATIC ${SRC})
The script works fine, it's able to link libnative.so and I'm able to find the generated libfoo.a. Everything seems fine.
I then try to define a native method in foo.cpp contained in foo library:
extern "C" JNIEXPORT void JNICALL Java_com_mypackage_Controls_onTap(JNIEnv*, jobject, int x, int y) {
// log something
}
But I'm not able to call the native method defined in foo library. I get an UnsatisfiedLinkError at runtime. If, instead, I move (directly by copying and pasting) the method to native.cpp then everything goes fine.
So basically:
Java -> method in native.cpp works
Java -> method in native.cpp -> method defined in foo library works
Java -> method in foo library doesn't work (UnsatisfiedLinkError)
I tried to inspect the exported functions with nm and it looks like that foo.a correctly exports the native function as I can see
00011060 T Java_com_mypackage_Controls_onTap
But this entry disappears from libnative.so. If, instead, I define the method directly in native.cpp then I can see it correctly with nm also on libnative.so.
In addition calling any method in foo library from native.cpp works as intended so the library is effectively statically linked.
I am not able to understand the reason behind this, the approach should be fine, visibility should be correct as specified by JNIEXPORT macro so I'm really groping in the dark (and Gradle doesn't provide any output of compilation phase so I can't understand what's happening, but the build.ninja file seems correct)
This behavior, even if unpleasant, is correct. The linker drops any object "files" (in your case, foo.o) from used static libraries, unless they are "pinned" by one of the objects in the shared lib (in your case, native.o). There are three ways to solve the problem:
compile foo.cpp as part of libnative.so instead of a
static lib.
reference Java_com_mypackage_Controls_onTap or any other
external symbol from foo.cpp in native.cpp
use SET(native -Wl,--whole-archive foo -Wl,--no-whole-archive) (see https://stackoverflow.com/a/17477559/192373)

Linker can't find function

Im trying to build an Ubuntu Touch tree, based on an Aosp tree I already correctly built.
It fails with this error
CAPEWrapper.cpp:16: error: undefined reference to '__xlog_buf_printf'
this is the header that file includes
#include "CAPEWrapper.h"
which on cascade includes
#include <cutils/xlog.h>
which in turn defines
#if defined(__cplusplus)
extern "C" {
#endif
int __xlog_buf_printf(int bufid, const struct xlog_record *rec, ...);
#if defined(__cplusplus)
}
#endif
I suspect that my g++ doesn't set __cplusplus macro. Could it be a realistic scenario with this kind of error? If this could be the problem, should I need to specify a standard implementation with "stdc=something" to solve it?
Any other idea is welcome.
Make sure that your project is linking libcutils, and that it's linking it in the correct order (i.e. that -lcutils appears in the linker command line after any module that depends on it).
In the end, I found that the modules was listed inside a macro called LOCAL_WHOLE_STATIC_LIBRARIES, that in Android environment passes its content to the --whole-archive flag of GCC linker.

How does Android source building #ifdefs work in compilation

I'm familiar with Android kernel programming but I'm a newbie at building the Android source. I'm wondering how to enable the #ifdefs in android source building. Are there any defconfig file in android source like in android kernel to choose what we want to compile in the compilation?.. How can i enable the codings defined with #ifdef to get compile during the Android source compilation?
Ex:
#ifdef USE_ION
int alloc_map_ion_memory(OMX_U32 buffer_size,
OMX_U32 alignment, struct ion_allocation_data *alloc_data,
struct ion_fd_data *fd_data,int flag);
void free_ion_memory(struct vdec_ion *buf_ion_info);
#else
bool align_pmem_buffers(int pmem_fd, OMX_U32 buffer_size,
OMX_U32 alignment);
#endif
I want to make sure the ion part is being compiled not the pmem part.
Try add the line:
#error "USE_ION"
after #ifdef USE_ION
Build again, if the build failed, USE_ION is defined.

cocos2d android compile time linking not working

I am trying to get cocos2d-android (cocos2d-2.0-rc2-x-2.0.1) "Helloworld" sample to run under windows. I am using latest version of cygwin along side with android ndk r6, android sdk API 8. And I tried the manual here
after a lot of challenges I am down to this problem which I think is in linking the classes at compile time. when I try to run the *build_native.sh* script I get an error stating that in CCGL.h, PFNGLDELETEVERTEXARRAYSOESPROC which is defined as extern, does not name a type.
//declare here while define in CCEGLView_android.cpp
extern PFNGLGENVERTEXARRAYSOESPROC glGenVertexArraysOESEXT;
extern PFNGLBINDVERTEXARRAYOESPROC glBindVertexArrayOESEXT;
extern PFNGLDELETEVERTEXARRAYSOESPROC glDeleteVertexArraysOESEXT;
'PFNGLDELETEVERTEXARRAYSOESPROC' is declared in CCEGLView.h.
#if CC_TEXTURE_ATLAS_USE_VAO
#include <EGL/egl.h>
PFNGLGENVERTEXARRAYSOESPROC glGenVertexArraysOESEXT = 0;
PFNGLBINDVERTEXARRAYOESPROC glBindVertexArrayOESEXT = 0;
PFNGLDELETEVERTEXARRAYSOESPROC glDeleteVertexArraysOESEXT = 0;
#endif
I tried to add the header address to android.mk under jni directory, to LOCAL_C_INCLUDES, as suggested here,but there were no rules to make them. so can anybody help me with how to compile/link this, it would be much appreciated.
I had this problem, and using a higher version of NDK worked. Try to get the most recent version, reset the environment variables, and run build_native.sh again.
http://developer.android.com/tools/sdk/ndk/index.html

Categories

Resources