go with networking on Android - android

I've been trying to get a Go application running on Android 10 and hit a snag. I'm pretty sure I now know why I'm hitting this snag, but not sure how to fix it. First, simple things work with cross compilation. For example,
package main
import "fmt"
func main() {
fmt.Println("Hello, Android!")
}
will run just fine compiled with GOARM=arm64 GOOS=linux go build
But now if I add some networking code:
package main
import (
"fmt"
"net/http"
"io/ioutil"
)
func main() {
fmt.Println("Hello, Android!")
client := &http.Client{}
resp, err := client.Get("https://www.google.com")
if err != nil {
fmt.Println(err)
return
}
bodyBytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%s", bodyBytes)
}
And compile the same way it will run, but I'll get an error like:
Get "https://www.google.com": dial tcp: lookup www.google.com on [::1]:53: read udp [::1]:47618->[::1]:53: read: connection refused
Initially I thought this was some networking config on my Android device, but after lots of messing around I'm pretty sure it has to do with the fact that Android networking is "different". Maybe this is a bionic vs libc thing (?) but at least one other person seems to have hit the same snag and solved by building using the toolchain that comes with the NDK.
Does anyone know how to actually get this to work though?
Initially I was hoping it was as easy as GOOS=android GOARCH=arm64 go build but that fails with:
# command-line-arguments
/usr/lib/golang/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-384615804/go.o: Relocations in generic ELF (EM: 183)
<bunch of these>
/usr/bin/ld: /tmp/go-link-384615804/go.o: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
Some Googling turned up https://github.com/golang/go/issues/30216 - which says you need to explicitly enable CGO. But GOARCH=arm64 GOOS=android CGO_ENABLED=1 go build still fails, this time with:
# runtime/cgo
gcc_android.c:6:10: fatal error: android/log.h: No such file or directory
6 | #include <android/log.h>
| ^~~~~~~~~~~~~~~
compilation terminated.
I messed around with a lot of other combinations and the closest I got was pointing Go to compilers in my NDK installation. Basically something like GOARCH=arm64 GOOS=android CC=$NDKROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc CGO_ENABLED=1 CGO_CFLAGS="--sysroot $NDKROOT/sysroot -I$NDKROOT/sysroot/usr/include/arm-linux-androideabi/" go build. But this failed with:
# runtime/cgo
In file included from _cgo_export.c:4:0:
cgo-gcc-export-header-prolog:25:14: error: size of array '_check_for_64_bit_pointer_matching_GoInt' is negative
There's something broken in the way I'm pointing to the compiler - for example I won't find any header files (stdlib.h) without specifying sysroot, and even then it won't find the architecture specific includes (asm/types.h) without specifying the additional include path. So the error is probably a mismatch between architecture and compiler I'm just not sure where or how to fix it.
If you've read all this, thanks! Some additional info:
go version: 1.14.13 linux/amd64
NDK version: r16b
I've also looked at things like gomobile but that seems more geared towards building Android applications (like apks) although maybe I'm missing something here.

Figured it out! Well, got my specific example working but I'm not really sure why this works. The answer came basically from https://github.com/golang/go/issues/20755 and you have to:
Build a standalone version of the NDK (the comment in the Python script that does this says it packages it in a more convenient way for other tools to use)
Rebuild Go's standard library using the cross compiler in your NDK
Build your application using the NDK cross compiler.
So for my example:
$ $(NDKROOT)/build/tools/make_standalone_toolchain.py --arch arm64 --api 24 --install-dir $(HOME)/ndk_standalone
$ CC=$(HOME)/ndk_standalone/bin/clang CXX=$(HOME)/ndk_standalone/bin/clang GOOS=android GOARCH=arm64 go install std
$ CC=$(HOME)/ndk_standalone/bin/clang CXX=$(HOME)/ndk_standalone/bin/clang GOOS=android GOARCH=arm64 CGO_ENABLED=1 go build
And networking works!
I'm not really sure why (or if) clang makes a difference as opposed to invoking the compiler directly. Maybe I'll play around with this more. Also, I had to still enabled CGO explicitly when compiling my application (but not for the go std library) otherwise linking failed.

Related

Unable to get qmake to cross compile correctly

I am new to this build system, but I've spent dozens of hours trying to get qmake + clang to produce a working android binary. I would appreciate any help or advice. Here is as simple an example as I can come up with:
First, here is a working build command. It gives me a shared object that I can call from android (with Unity, using C# native interops):
C:\Users\deltav\AppData\Local\Android\Sdk2\ndk\21.3.6528147\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe ^
-target aarch64-none-linux-android21 -shared -v ^
-DANDROID_ABI=arm64-v8a -DANDROID_ARM_MODE=arm -DANDROID ^
M:\simpleExample\simpleExample.c ^
-o M:\simpleExample\libsimpleExample.so
Here is the verbose output of running that command: https://pastebin.com/18HcnDYh
On the other hand, here is the simpleExample.pro file that is NOT giving me a good binary: https://pastebin.com/wxrx6Myc
Here is the qmake output for building in release mode: https://pastebin.com/KWavwrb7
As you will know, qmake uses a kit to execute the project. My kit: https://imgur.com/a/Ehtmgin
The specific issue that has pushed me down this debugging path is from C# trying to load my shared object like this:
[DllImport("simpleExample")]
public static extern float getSpeedY();
The error that I get is this:
"DllNotFoundException: Unable to load DLL 'simpleExample': The
specified module could not be found."
Again, this error is not an issue if I just build from the CLI without qmake. I do not beleive this is a unity or C# error, as I worked the problem from that side first.
What I have tried so far:
I have used multiple ELF & DWARF tools to compare the ABIs of the generated binaries, which all told me the binaries were identical.
I tweaked and shaved down the clang calls that qmake generated until I got a vaild build command. Unfortunately I can't just replace qmake with the complile command because there are other projects that I need to apply this fix to.
QMake version 3.1
Ironically, CMake basically works out of the box.
Answer: Looking at the clang linking call, you can see QMake 3.1 was adding "libc++shared" as a dependency. I'm not sure why, as this was a C project.
The equivalent CMake project did not do this.
Packaging libc++shared with my binary fixed the issue.

Cross Compiling ghostscript for android, what host should I use?

Hi Guys I am using this tutorial to build ghostscript-9.19 to be able to use in my android application to convert eps document to pdf. It fails while configure. Here are the logs
checking whether to enable maintainer-specific portions of Makefiles...
no
checking for gcc... arm-linux-androideabi-gcc --
sysroot=/<path>/android-ndk-
r11c/platforms/android-17/arch-arm/
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... configure: error: in
`/<path>/ghostscript-9.19/tiff-config':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
configure: error: libtiff configure script failed
This is the build file I am running
#!/bin/sh
# Compiles ghostscript for Android
# Make sure you have NDK_ROOT defined in .bashrc or .bash_profile
INSTALL_DIR="`pwd`/app/jni/gs"
SRC_DIR="`pwd`/../ghostscript-9.19"
cd $SRC_DIR
export
PATH="/<path>/android-ndk-r11c/toolchains/arm-
linux-androideabi-4.9/prebuilt/darwin-x86_64/bin:$PATH"
export SYS_ROOT="/<path>/Android/android-ndk-
r11c/platforms/android-17/arch-arm/"
export CC="arm-linux-androideabi-gcc --sysroot=$SYS_ROOT"
export LD="arm-linux-androideabi-ld"
export AR="arm-linux-androideabi-ar"
export RANLIB="arm-linux-androideabi-ranlib"
export STRIP="arm-linux-androideabi-strip"
mkdir -p $INSTALL_DIR
./configure --host=arm-linux-androideabi --build=x86_64-apple-darwin
--prefix=$INSTALL_DIR LIBS="-lc -lgcc"
make PREFIX=$INSTALL_DIR
make install DESTDIR=$INSTALL_DIR
exit 0
I am using --host=arm-linux-androideabi. What host should i use? What do I need to change in ghostScript project to make compile in successfully?
Any help is highly appreciated.
Cross compiling Ghostscript is pretty involved, partly because the Ghostscript build relies on building and running interim tools (genarch, genconf, mkromfs and echogs) which, obviously, must be built with the native compiler, rather than the cross compiler.
I think the problem you are seeing is because the call to the libtiff configure doesn't pass on the required options.
You may be better served grabbing, and tweaking the two files (a makefile and a header) from this commit:
Makefile for Android MuPDF libgs.so
and tweaking it to match your requirements.
There is a basic guide of what to do for cross compiling at the bottom of this page:
Ghostscript FAQ
I have a "project" to improve support for cross compiling, but it is slow going at the moment.

How to get libreadline for Android?

I tried to download the gnu readline-6.2 source code but could not manage to build it on Android platform. Could anyone point out a way for me to tackle this problem? Thanks a lot!
I try to get the default Android build flags on my PC and use it to run ./configure and got problems saying that:
checking build system type... x86_64-unknown-linux-gnu
checking host system type... arm-linux-androideabi
configure: cross-compiling for arm-linux-androideabi is not supported
Beginning configuration for readline-6.2 for arm-linux-androideabi
checking whether make sets $(MAKE)... yes
checking for arm-linux-gcc... /home/huangwei/ICS/prebuilt/linux-x86/toolchain/arm-linux-androideabi-4.4.x/bin/arm-linux-androideabi-gcc
checking for C compiler default output file name...
configure: error: in `/home/huangwei/ICS/external/readline':
configure: error: C compiler cannot create executables
See `config.log' for more details.
However, I opened the config.log and use the same build command to compile the conftest.c, it works, the .o file is there. I don't know why the configure still fails. Is there anyone could help?
I found myself also needing to compile libreadline for arm-linux-androideabi. What I did was the following:
download source code for libreadline gnu.
set CC to the android-ndk cross compiler (CC=~/android/lib/bin/arm-linux-androideabi-gcc)
update the config.sub and config.guess files (the ones in the libreadline source are very old and don't have arm-linux-androideabi)
$ cp /usr/share/misc/config.{sub,guess} support/.
run ./configure with --host and --prefix set to the target architecture and the locations for the compiled objects
$ CC=~/android/lib/bin/arm-linux-androideabi-gcc ./configure --build=arm-linux-androideabi --prefix=~/android/lib/
make && make install and it worked for me! I have libreadline.a and libreadline.so files in ~/android/lib/lib
Note, before I had updated the support/config.sub and support/config.guess files I was getting the same error described in the question.

Android OpenCV shared library build error

When I build as static library, the source code builds well in Android JellyBean 4.1.2 using the mm command. But when I try to build the opencv folder under android/external/opencv as static library, with the following changes in android/external/opencv/Android.mk
+ LOCAL_SHARED_LIBRARIES+= libdl
and
- include $(BUILD_STATIC_LIBRARY)
+ include $(BUILD_SHARED_LIBRARY)
- LOCAL_STATIC_LIBRARIES := libcxcore libcv libcvaux libcvml libcvhighgui
+ LOCAL_SHARED_LIBRARIES := libcxcore libcv libcvaux libcvml libcvhighgui
I get the following error:
android/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.6/bin/../lib/gcc/arm-linux-androideabi/4.6.x-google/../../../../arm-linux-androideabi/bin/ld: internal error in thumb_branch_common, at /tmp/android-8532/src/build/../binutils/binutils-2.21/gold/arm.cc:4148
collect2: ld returned 1 exit status
make: *** [out/target/product/mydevice/obj/SHARED_LIBRARIES/libcxcore_intermediates/LINKED/libcxcore.so] Error 1
Any idea on how to fix this error?
Thanks in advance.
Regards,
Jai
,,, arm-linux-androideabi/bin/ld: internal error in thumb_branch_common,
at /tmp/android-8532/src/build/../binutils/binutils-2.21/gold/arm.cc:4148
collect2: ld returned 1 exit status
Any idea on how to fix this error?
In Binutils, file arm.cc, Line 4148 there was an Internal Error (in Binutils).
Try newer Binutils (or older) or fix the Bug and submit the Patch.
You can also switch to a different Toolchain (using a different Binutils) and
try that. Sometimes Compiler switches will alter the operation enough to avoid
the Bug.
There is a CHANCE that simply altering the order of the Function in the Source File
of the Code you are compiling (NOT gold/arm.cc, that is where the Compiler Bug is,
change YOUR Source) and that Binutils will read the newly ordered Code differently
and thus avoid the Binutils Bug.
It is also possible that the Code you are linking (the .o Files) was corrupted
either on Disk or by a Bug in the Compiler, or that it is for the wrong endian
(etc.) and that is not being detected.
PS: Thumb-2 Code (in the Compiler's Toolchain) is not Bug free (as you noticed).
See here for some People working on a fix:
http://git.openembedded.org/openembedded-core-contrib/commit/?h=kraj/gold&id=b72f3238ad32ab420306a0226d8c2e57c52ddf45
... and this is what they came up with:
http://sourceware.org/bugzilla/attachment.cgi?id=6284&action=diff

Compiling against C++ standard libraries on the android toolchain

I have a really simple helloworld.cpp program
#include <iostream>
using namespace std;
int main ()
{
cout << "Hello World!";
return 0;
}
And I'm trying to compile it for android x86 with the cross-compiler from the toolchain:
/Users/me/android-ndk-r8/toolchains/x86-4.4.3/prebuilt/darwin-x86/bin/i686-android-linux-g++ helloworld.cpp -L "/Users/me/android-ndk-r8/sources/cxx-stl/stlport/libs/x86/" -lstlport_static
However, I get errors:
helloworld.cpp:2:20: error: iostream: No such file or directory
Any idea why?
Check the documentation.html file included with the NDK, under "Standalone Toolchain". It says that if you invoke the compiler in this way you won't be able to "use any C++ STL". However it is possible, as the documentation explains, if you first create a "customized" toolchain installation, using something like the following command:
$NDK/build/tools/make-standalone-toolchain.sh --platform=android-8 --install-dir=/tmp/my-android-toolchain --arch=x86
where $NDK is the path to your NDK directory. Note the --arch=x86 which means that the toolchain is prepared specifically for the x86 Android. This prepares what you need in one directory, including the STL headers and folders. You should then be able to use -lstdc++ to link against the STL (static version), i.e. something like:
/tmp/my-android-toolchain/bin/i686-android-linux-g++ helloworld.cpp -lstdc++
For a more complete explanation, please see the NDK documentation.
The NDK documentation is not entirely accurate, at least not currently. In fact, it states when using the prebuilt toolchain "You won't be able to use any C++ STL (either STLport or the GNU libstdc++) with it.", but this is out of date. I created a small hello world program using the include with the same error. It can be solved without creating your own toolchain though, which is nice if you don't want to have to add one more step to your configuration process and allows you to always use the latest SDK platform without creating a new toolchain every time.
The NDK ships with the source code for several versions of standard C++ libraries: GAbi++, STLport, and GNU STL. Each flavor comes with prebuilt shared and static libs as well. My example below will use stlport.
To use the stand-alone toolchain at its installed location, you can do something like this:
export CXX='$NDK_ROOT/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-g++ --sysroot="$NDK_ROOT/platforms/android-19/arch-arm"'
This, for example, would set your CXX compiler to compile ARM on the OS X system using SDK platform level 19. This much you probably already knew. Also, you would want to export your CC, CPP, LD, AR, and RANLIB if you use it. I also personally create an envar for READELF.
To add support for C++ libs, you could do something like follows:
$CXX helloworld.cpp -I$NDK_ROOT/sources/cxx-stl/stlport/stlport -L$NDK_ROOT/sources/cxx-stl/stlport/libs/armeabi -lstlport_shared
Note that this will link the libstlport_shared.so which will now be required at runtime, so you may need to add a relative path to the command above to support that, depending on your APK structure. If you want to just test this simple case, you can just run this on the emulator as follows:
adb push a.out /data
adb push $NDK_ROOT/sources/cxx-stl/stlport/libs/armeabi/libstlport_shared.so /data
adb shell
# su
# cd /data
# chmod 777 a.out
# ./a.out
To get rid of the headache of dealing with shared library paths, you can also statically link the C++ library in by changing "-lstlport_shared" to "-lstlport_static". There are some consequences of doing this, as explained in the NDK Dev Guide. The biggest issue is due to having the static library linked in multiple places, causing:
- memory allocated in one library, and freed in the other would leak or even corrupt the heap.
- exceptions raised in libfoo.so cannot be caught in libbar.so (and may simply crash the program).
- the buffering of std::cout not working properly
A useful tool is also included to see what dependencies your program has, the readelf tool.
To see what other shared libraries your program requires, you can run the following:
$NDK_ROOT/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-readelf -a a.out | grep NEEDED
Other cool standard tools:
addr2line - convert a stack trace address to a line of code
nm - Displays the symbol table
objdump - displays info on an object
i call one of the functions from gnustl after it runs fine from prebuilt aosp arm-linux-androideabi-gcc --std=c++11 and after crashing error i cant get a backtrace from logs or reason, my hope is turning to crossbuilt qemu-linux-user to run host compiled i686 binary on the arm, difficulty is interacting with crosshost libs aapt from adt always crashes on any other than platform specific libs, unless kernel module packaged update is possible...

Categories

Resources