Develop Reference

go with networking on Android

I've been trying to get a Go application running on Android 10 and hit a snag. I'm pretty sure I now know why I'm hitting this snag, but not sure how to fix it. First, simple things work with cross compilation. For example,
package main
import "fmt"
func main() {
fmt.Println("Hello, Android!")
}
will run just fine compiled with GOARM=arm64 GOOS=linux go build
But now if I add some networking code:
package main
import (
"fmt"
"net/http"
"io/ioutil"
)
func main() {
fmt.Println("Hello, Android!")
client := &http.Client{}
resp, err := client.Get("https://www.google.com")
if err != nil {
fmt.Println(err)
return
}
bodyBytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%s", bodyBytes)
}
And compile the same way it will run, but I'll get an error like:
Get "https://www.google.com": dial tcp: lookup www.google.com on [::1]:53: read udp [::1]:47618->[::1]:53: read: connection refused
Initially I thought this was some networking config on my Android device, but after lots of messing around I'm pretty sure it has to do with the fact that Android networking is "different". Maybe this is a bionic vs libc thing (?) but at least one other person seems to have hit the same snag and solved by building using the toolchain that comes with the NDK.
Does anyone know how to actually get this to work though?
Initially I was hoping it was as easy as GOOS=android GOARCH=arm64 go build but that fails with:
# command-line-arguments
/usr/lib/golang/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-384615804/go.o: Relocations in generic ELF (EM: 183)
<bunch of these>
/usr/bin/ld: /tmp/go-link-384615804/go.o: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
Some Googling turned up https://github.com/golang/go/issues/30216 - which says you need to explicitly enable CGO. But GOARCH=arm64 GOOS=android CGO_ENABLED=1 go build still fails, this time with:
# runtime/cgo
gcc_android.c:6:10: fatal error: android/log.h: No such file or directory
6 | #include <android/log.h>
| ^~~~~~~~~~~~~~~
compilation terminated.
I messed around with a lot of other combinations and the closest I got was pointing Go to compilers in my NDK installation. Basically something like GOARCH=arm64 GOOS=android CC=$NDKROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc CGO_ENABLED=1 CGO_CFLAGS="--sysroot $NDKROOT/sysroot -I$NDKROOT/sysroot/usr/include/arm-linux-androideabi/" go build. But this failed with:
# runtime/cgo
In file included from _cgo_export.c:4:0:
cgo-gcc-export-header-prolog:25:14: error: size of array '_check_for_64_bit_pointer_matching_GoInt' is negative
There's something broken in the way I'm pointing to the compiler - for example I won't find any header files (stdlib.h) without specifying sysroot, and even then it won't find the architecture specific includes (asm/types.h) without specifying the additional include path. So the error is probably a mismatch between architecture and compiler I'm just not sure where or how to fix it.
If you've read all this, thanks! Some additional info:
go version: 1.14.13 linux/amd64
NDK version: r16b
I've also looked at things like gomobile but that seems more geared towards building Android applications (like apks) although maybe I'm missing something here.

Figured it out! Well, got my specific example working but I'm not really sure why this works. The answer came basically from https://github.com/golang/go/issues/20755 and you have to:
Build a standalone version of the NDK (the comment in the Python script that does this says it packages it in a more convenient way for other tools to use)
Rebuild Go's standard library using the cross compiler in your NDK
Build your application using the NDK cross compiler.
So for my example:
$ $(NDKROOT)/build/tools/make_standalone_toolchain.py --arch arm64 --api 24 --install-dir $(HOME)/ndk_standalone
$ CC=$(HOME)/ndk_standalone/bin/clang CXX=$(HOME)/ndk_standalone/bin/clang GOOS=android GOARCH=arm64 go install std
$ CC=$(HOME)/ndk_standalone/bin/clang CXX=$(HOME)/ndk_standalone/bin/clang GOOS=android GOARCH=arm64 CGO_ENABLED=1 go build
And networking works!
I'm not really sure why (or if) clang makes a difference as opposed to invoking the compiler directly. Maybe I'll play around with this more. Also, I had to still enabled CGO explicitly when compiling my application (but not for the go std library) otherwise linking failed.

Replacement for BUILD_BROKEN_PHONY and phony-rebuild in android R

i am trying to build android R but getting the following error in the kernel makefile
kernel/exynos/AndroidKernel.mk:155: error: writing to readonly directory: "/data/home/sumkumar/work/workspace/R/out/../out/target/product/**/obj/KERNEL_OBJ/arch/arm64/boot/Image"
this was handled in Android Q by using the flag "BUILD_BROKEN_PHONY_TARGETS=true" but in R this is depreciated hence cannot use the same.
after looking at following link given for phony rules changes from google for android R
https://android.googlesource.com/platform/build/+/master/Changes.md#phony_targets
i have updated my Makefile as follows by removing the "../" in the erronous path and past the previous error ,
#KERNEL_OUT ?= $(if $(filter /% ~%,$(TARGET_OUT_INTERMEDIATES)),,$(realpath $(OUT_DIR))/../)$(TARGET_OUT_INTERMEDIATES)/KERNEL_OBJ
KERNEL_OUT ?= $(TARGET_OUT_INTERMEDIATES)/KERNEL_OBJ
but now i am getting errors with phony-rebuild as mentioned below
kernel/exynos/AndroidKernel.mk:139: error: real file "out/target/product/**/obj/KERNEL_OBJ/.config" depends on PHONY target "phony-rebuild"
14:21:04 ckati failed with: exit status 1
i understood as phony targets are always dirty hence every file will be rebuild which depends on that phony, but in my case this rebuilding is required .
Attaching a snap of the file depicting the same
KERNEL_CONFIG := $(KERNEL_OUT)/.config
KERNEL_CONFIG := $(KERNEL_OUT)/.config
.PHONY: phony-rebuild
$(KERNEL_CONFIG): phony-rebuild
$(show) echo "make $(KERNEL_DEFCONFIG)"
$(MAKE_CONFIG_CMD)
$(info MAKE_CONFIG_CMD is $(MAKE_CONFIG_CMD))
Please let me know if there is any way to avoid using phony-rebuild and replace it with an alternate mechanism.
Regards,

Building AOSP master branch fails in Mac OSX

I am trying to build the master branch of AOSP, but it fails with following error:
host C++: validatekeymaps <= frameworks/base/tools/validatekeymaps/Main.cpp
In file included from frameworks/base/tools/validatekeymaps/Main.cpp:17:
In file included from frameworks/native/include/input/KeyCharacterMap.h:26:
In file included from frameworks/native/include/input/Input.h:25:
system/core/include/utils/Vector.h:20:10: fatal error: 'new' file not found
#include <new>
^
1 error generated.
make: *** [out/host/darwin-x86/obj/EXECUTABLES/validatekeymaps_intermediates/Main.o] Error 1
Does anyone has any idea on this? Where the "new" file should present? How would I solve this? Any help would be greatly appreciated.

I believe is a clang error. clang does not need .h header extensions. I wouldn't bet my life on it but I will suggest updating your host clang to the newest or newer version. It is definitely a host problem. I've seen it before, but don't remember all, plus I'm on Ubuntu and don't build AOSP. I do see a recent change added -Werror now into system/core so try doing a Host clang update... even if you already have that file, an update may fix HOST include paths.
One more thought. Try: echo $(HOST_TOOLCHAIN_ROOT)
If you don't get a location you may have a clang which is too new... my reason/reference:
https://android.googlesource.com/platform/build/+/master/core/combo/HOST_darwin-x86_64.mk:
# gcc location for clang; to be updated when clang is updated
# HOST_TOOLCHAIN_ROOT is a Darwin-specific define
HOST_TOOLCHAIN_FOR_CLANG := $(HOST_TOOLCHAIN_ROOT)

Problems compiling avahi into Android NDK project

Over the last few days I've been having a difficult time trying to build avahi into a static or shared library for use with an existing Android NDK project.
We have a few games in the App and Play stores and my task is to get multiplayer working in the Android versions.
Specifically the task involves replacing the Bonjour component so that these games can connect to each other via zeroconf.
Research seemed to indicate that avahi is the lib that we're looking for, but at this point I'm open to anything that will work!
I'm hoping that someone here can either help me get avahi compiling or suggest another more appropriate (and easier to install) lib.
Project uses android-ndk-r8b and is being built on OSX 10.7.4 using command line (not eclipse)
Got the latest Avahi source from here:
http://www.linuxfromscratch.org/blfs/view/svn/basicnet/avahi.html
homebrewed all the necessary libs to get ./configure to run without errors.
./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --disable-static --disable-mono --disable-monodoc --disable-gdbm --disable-libdaemon --disable-nls --disable-gtk --disable-gtk3 --disable-python --disable-qt3 --disable-qt4 --enable-core-docs --with-distro=none
./configure runs with no apparent red flags.
make results in this compile error:
socket.c: In function 'ipv6_pktinfo':
socket.c:271: warning: unused variable 'yes' [-Wunused-variable]
socket.c:270: warning: unused parameter 'fd' [-Wunused-parameter]
socket.c: In function 'avahi_send_dns_packet_ipv6':
socket.c:609: error: 'IPV6_PKTINFO' undeclared (first use in this function)
socket.c:609: error: (Each undeclared identifier is reported only once
socket.c:609: error: for each function it appears in.)
socket.c: In function 'avahi_recv_dns_packet_ipv6':
socket.c:869: error: 'IPV6_HOPLIMIT' undeclared (first use in this function)
socket.c:878: error: 'IPV6_PKTINFO' undeclared (first use in this function)
make[2]: *** [libavahi_core_la-socket.lo] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
Figured that it's not building against the android-ndk-r8b libs or finding ipv6.h or something.
Checked my .bash_profile.sh file:
export PATH=/Users/Muy01/Projects/Development/Android/android-sdks/tools/:$PATH
export PATH=/Users/Muy01/Projects/Development/Android/android-sdks/platform-tools/:$PATH
export PATH=/Users/Muy01/Projects/Development/Android/android-ndk-r8b/:$PATH
added --host=arm-linux-androideabi to the ./configure arguments list
resulting in this error:
checking host system type... Invalid configuration `arm-linux-androideabi': system `androideabi' not recognized
Couldn't figure out how to get a list of available host system types so changed direction and decided to try and build the static lib via Android.mk file.
Found this post on creating an appropriate Android.mk file:
can't compile avahi on android
Realized that I don't have Android.mk files within all the subdirectories.
Researched, downloaded, built, Androgenizer to try and convert all the Makefile.am files into Android.mk files.
http://cgit.collabora.com/git/user/derek/androgenizer.git/
Couldn't figure out or find info on how to do that though =/
Decided to try and create my own Android.mk file:
LOCAL_PATH := $(call my-dir)
ROOT_LOCAL_PATH :=$(call my-dir)
#Build avahi into a static lib
include $(CLEAR_VARS)
AVAHI_TOP := $(ROOT_LOCAL_PATH)/../avahi-0.6.31
MY_SOURCES := $(wildcard $(AVAHI_TOP)/avahi-core/*.c*)
MY_SOURCES += $(wildcard $(AVAHI_TOP)/avahi-common/*.c*)
LOCAL_C_INCLUDES := $(AVAHI_TOP)
LOCAL_SRC_FILES := $(MY_SOURCES:$(LOCAL_PATH)%=%)
LOCAL_MODULE := avahi
include $(BUILD_STATIC_LIBRARY)
Results in compile time errors for avahi-core/iface-linux.c:33:0:
/avahi-0.6.31/avahi-core/iface-linux.h:27:8: Redefinition of 'struct AvahiInterfaceMonitorOSDep'
/avahi-0.6.31/avahi-core/iface.h:46:16: Originally defined here
/avahi-0.6.31/avahi-core/iface-linux.h:33:9: Redeclaration of enumerator 'LIST_IFACE'
/avahi-0.6.31/avahi-core/iface.h:52:9: Previous definition of 'LIST_IFACE' was here
/avahi-0.6.31/avahi-core/iface-linux.h:34:9: Redeclaration of enumerator 'LIST_ADDR'
/avahi-0.6.31/avahi-core/iface.h:53:9: Previous definition of 'LIST_ADDR' was here
/avahi-0.6.31/avahi-core/iface-linux.h:35:9: Redeclaration of enumerator 'LIST_DONE'
/avahi-0.6.31/avahi-core/iface.h:54:9: Previous definition of 'LIST_DONE' was here
/jni//../avahi-0.6.31/avahi-core/iface-linux.c: In function 'netlink_callback':
And now I'm pretty much stuck.
I tried #if 0'ing out the iface-linux.c and h files resulting in a cascading slew of other errors, so prob a bad idea.
Thinking that it may be something I've done wrong with the ./configure command?
Perhaps an issue with my Android.mk file?
I figure this must be something that quite a few developers are dealing with so I'm probably missing something because I can't seem to find any good information via google.
Any help would be much appreciated!
I've sent this out to the avhi mailing list as well, if I get a response there I will post here for posterity.
Thanks,
Chris

I'll follow up here with the solution that worked for me.
My solution was to use JMDNS instead of Avahi.
There's not much traffic on the Avahi mailing list.
JMDNS has working examples available.
JMDNS took me about 4 hours to set up within my NDK environment and about a day to work out some "kinks."

tar avahi-0.6.31
patch -p1 < 0001-Add-Android-support.patch
patch -p1 < 0002-Add-uninstalled.pc.in-files.patch
cd avahi-0.6.31
./configure --sysconfdir=/etc --localstatedir=/var
make
cd the subdirectories: make Android.mk
then, you will see Android.mk in all the subdirectories.
ndk-build V=1 NDK_LOG=2 APP_ABI="armeabi armeabi-v7a"

cannot find -lgnustl_static: Compiling Android SDL port with NDK r7

I'm trying to build this port of libSDL for Android, using the new Android NDK r7 release:
https://github.com/pelya/commandergenius
I know I am doing some things to some degree "unsupported": I am using cygwin&windows despite that being mentioned in the README as no longer being supported. That being said, I can hack out shell scripts and solve my problems as necessary and have it compiling but not linking.
So anyways I have compiling working until the linking (sorry for large block, relevant part trimmed below):
/cygdrive/c/and/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin/arm-linux-androideabi-ar crs obj/local/armeabi/libflac.a ./obj/local/armeabi/objs-debug/flac/src/ogg_decoder_aspect.o ./obj/local/armeabi/objs-debug/flac/src/md5.o ./obj/local/armeabi/objs-debug/flac/src/stream_decoder.o ./obj/local/armeabi/objs-debug/flac/src/fixed.o ./obj/local/armeabi/objs-debug/flac/src/memory.o ./obj/local/armeabi/objs-debug/flac/src/stream_encoder.o ./obj/local/armeabi/objs-debug/flac/src/window.o ./obj/local/armeabi/objs-debug/flac/src/cpu.o ./obj/local/armeabi/objs-debug/flac/src/ogg_encoder_aspect.o ./obj/local/armeabi/objs-debug/flac/src/lpc.o ./obj/local/armeabi/objs-debug/flac/src/float.o ./obj/local/armeabi/objs-debug/flac/src/bitmath.o ./obj/local/armeabi/objs-debug/flac/src/metadata_object.o ./obj/local/armeabi/objs-debug/flac/src/ogg_helper.o ./obj/local/armeabi/objs-debug/flac/src/metadata_iterators.o ./obj/local/armeabi/objs-debug/flac/src/bitreader.o ./obj/local/armeabi/objs-debug/flac/src/bitwriter.o ./obj/local/armeabi/objs-debug/flac/src/stream_encoder_framing.o ./obj/local/armeabi/objs-debug/flac/src/crc.o ./obj/local/armeabi/objs-debug/flac/src/format.o ./obj/local/armeabi/objs-debug/flac/src/ogg_mapping.o
C:/and/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.4.3/../../../../arm-linux-androideabi/bin/ld.exe: cannot find -lgnustl_static
collect2: ld returned 1 exit status
/cygdrive/c/and/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin/arm-linux-androideabi-ar crs obj/local/armeabi-v7a/libflac.a ./obj/local/armeabi-v7a/objs-debug/flac/src/ogg_decoder_aspect.o ./obj/local/armeabi-v7a/objs-debug/flac/src/md5.o ./obj/local/armeabi-v7a/objs-debug/flac/src/stream_decoder.o ./obj/local/armeabi-v7a/objs-debug/flac/src/fixed.o ./obj/local/armeabi-v7a/objs-debug/flac/src/memory.o ./obj/local/armeabi-v7a/objs-debug/flac/src/stream_encoder.o ./obj/local/armeabi-v7a/objs-debug/flac/src/window.o ./obj/local/armeabi-v7a/objs-debug/flac/src/cpu.o ./obj/local/armeabi-v7a/objs-debug/flac/src/ogg_encoder_aspect.o ./obj/local/armeabi-v7a/objs-debug/flac/src/lpc.o ./obj/local/armeabi-v7a/objs-debug/flac/src/float.o ./obj/local/armeabi-v7a/objs-debug/flac/src/bitmath.o ./obj/local/armeabi-v7a/objs-debug/flac/src/metadata_object.o ./obj/local/armeabi-v7a/objs-debug/flac/src/ogg_helper.o ./obj/local/armeabi-v7a/objs-debug/flac/src/metadata_iterators.o ./obj/local/armeabi-v7a/objs-debug/flac/src/bitreader.o ./obj/local/armeabi-v7a/objs-debug/flac/src/bitwriter.o ./obj/local/armeabi-v7a/objs-debug/flac/src/stream_encoder_framing.o ./obj/local/armeabi-v7a/objs-debug/flac/src/crc.o ./obj/local/armeabi-v7a/objs-debug/flac/src/format.o ./obj/local/armeabi-v7a/objs-debug/flac/src/ogg_mapping.o
C:/and/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin/../lib/gcc/arm-linux-androideabi/4.4.3/../../../../arm-linux-androideabi/bin/ld.exe: cannot find -lgnustl_static
collect2: ld returned 1 exit status
Relevant failure, if you can't parse that wall of text:
ld.exe: cannot find -lgnustl_static
Now, if I look at the Windows NDK release I find gnustl_* files here:
under ./android-ndk-r7/sources:
./cxx-stl/gnu-libstdc++/libs/armeabi/libgnustl_shared.so
./cxx-stl/gnu-libstdc++/libs/armeabi/libgnustl_static.a
./cxx-stl/gnu-libstdc++/libs/armeabi-v7a/libgnustl_shared.so
./cxx-stl/gnu-libstdc++/libs/armeabi-v7a/libgnustl_static.a
./cxx-stl/gnu-libstdc++/libs/x86/libgnustl_shared.so
./cxx-stl/gnu-libstdc++/libs/x86/libgnustl_static.a
I looked also at the linux android NDK distribution and they are packaged the same (only have libgnustl files, built, under the sources directory).
So I have 3 questions:
why would we be using -l to link libgnustl_static when it's a .a
file? Shouldn't that be just including the .a file in the object
list, or alternately be -lgnustl_shared?
why are these files only under "sources"? I would have expected them to be in the "toolchains" library NDK directory. How does anyone ever link against libgnustl?
Can I copy these files somewhere where they'll be seen by the linker? Just adding their directory with -L hasn't seemed to work, and plus I need to get the right versions linked with the right build target (armeabi vs armeabi-v7 vs x86)

This is a bug in gcc/NDK that manifests itself slightly different on Linux, Cygwin, and Windows (non Cygwin).
Here's what I did to fix it (works for Linux and for one of my developers who is blind and has to use Cygwin)...
Remove the line that says, APP_STL := gnustl_static from your Application.mk (it's broken in the NDK, and it's broken slightly differently on Linux Cygwin, Windows)
add a line to $(LOCAL_LDLIBS) in your Android.mk file that says:
$(NDK_ROOT)/sources/cxx-stl/gnu-libstdc++/4.6/libs/armeabi-v7a/libgnustl_static.a
also add two lines to Android.mk that say:
LOCAL_C_INCLUDES := $(NDK_ROOT)/sources/cxx-stl/gnu-libstdc++/4.6/include
LOCAL_C_INCLUDES += $(NDK_ROOT)/sources/cxx-stl/gnu-libstdc++/4.6/libs/armeabi-v7a/include
The foregoing will accomplish what having, APP_STL := gnustl_static in your Application.mk was supposed to.

In case this plagues anyone else, the solution is to remove -lgnustl_static and instead list
c:\path\to\armeabi\libs\libgnustl_static.a in the linker command.
Do not use /cygdrive style path, as the linker will fail to find the file. Use actual c:\...