Stockfish 13, released February 19, 2021, is the strongest chess engine at the moment I am writing these lines. You can download it for windows, linux, mac os, and also android. Yet at the moment I am writing these lines, the android binaries download link points to Stockfish 12 and not 13. (This happens often with Stockfish, as the android binaries are not the ones people need the most.) As Stockfish is open source (github here), how could we cross-compile Stockfish 13 on a windows 10 computer for an Android phone ? (In my case, a Samsung Galaxy S21 Ultra running Android 11.)
We are going to need the source code : git clone https://github.com/official-stockfish/Stockfish.git (see the right address at Stockfish's github as this address could change) somewhere on your PC. (You need the command line utilities of Tortois Git for instance.) Let's call StockFishDIr the full path to the StockFish folder (that contains src as subfolder) on the PC.
We also need a cross-compiler to ... cross-compile (!) the source-code on the PC into an executable targeting the Android OS, meaning : being executable on the ANdroid OS, i.e. on the phone. We are going to use Android-NDK, which contains toolchains and cross-compilers. We download the "Latest Stable Version" r22 (as of the moment I am writing these lines) and unzip it to a folder on the PC ; I call "AndroidNdkDir" the full path to this directory (containing build, meta, prebuilt, toolchains etc folders) on the PC.
Finally, as in Stockfish's source code there is only a Makefile, we are going to need make under windows. Yes. Just get it from GnuWin32 and note MakeDir the installation directory on you PC, the one containg the bin folder containing make.exe. Analyzing the MakeFile linearly show that the cross-compilation boils down to only one command line provided some file is downloaded, and to another "optimizing" command line.
Now we look at the MakeFile from StockFishDIr\src ; we will modify it slightly.
First we change the executable name by changine the line
EXE = stockfish
into
EXE = stockfish_13_armv8_xcomp_win10x64
to give it a meaningful name.
Second, we locate the occurence of armv8 (it is the ARCH of the Samsung Galaxy S21 Ultra ; if you have another Android phone, maybe you should look for 'armv7' and adapt consequently what follows) concerning the compiler : namely, we replace the bit
ifeq ($(arch),armv8)
CXX=aarch64-linux-android21-clang++
STRIP=aarch64-linux-android-strip
endif
with
ifeq ($(arch),armv8)
CXX=AndroidNdkDir\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android30-clang++.cmd
STRIP=AndroidNdkDir\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-strip.exe
endif
endif
(Note that there was the initial android21 in the CXX line that I replaced with android30. It is because the Samung Galaxy s21 Ultra runs Android 11 whose real version number is android30. If you have an android phone with another android version, adapt the lines consequently.)
Now, before compiling, we need one last piece : the net file (technical, net = networks = neural network = the weights file of the neural network Stockfish operates with). "Normally" (ah ah), under a linux setup (all what I do here works almost mutatis mutandis on Linux), we would just do a "MakeDir\bin\make.exe" profile-build ARCH=x86-64 which would download the net file where needed, and we would not even bother we it. Sadly, running here "MakeDir\bin\make.exe" profile-build ARCH=x86-64 doesn't work for some reason, and we need to download the net file ourselves manually. Hence we have to look in the Makefile to find how the net file is downloaded. The bit that matters in the Makefile is :
# evaluation network (nnue)
net:
$(eval nnuenet := $(shell grep EvalFileDefaultName evaluate.h | grep define | sed 's/.*\(nn-[a-z0-9]\{12\}.nnue\).*/\1/'))
#echo "Default net: $(nnuenet)"
$(eval nnuedownloadurl := https://tests.stockfishchess.org/api/nn/$(nnuenet))
$(eval curl_or_wget := $(shell if hash curl 2>/dev/null; then echo "curl -skL"; elif hash wget 2>/dev/null; then echo "wget -qO-"; fi))
#if test -f "$(nnuenet)"; then \
echo "Already available."; \
else \
if [ "x$(curl_or_wget)" = "x" ]; then \
echo "Automatic download failed: neither curl nor wget is installed. Install one of these tools or download the net manually"; exit 1; \
else \
echo "Downloading $(nnuedownloadurl)"; $(curl_or_wget) $(nnuedownloadurl) > $(nnuenet);\
fi; \
fi;
$(eval shasum_command := $(shell if hash shasum 2>/dev/null; then echo "shasum -a 256 "; elif hash sha256sum 2>/dev/null; then echo "sha256sum "; fi))
#if [ "x$(shasum_command)" != "x" ]; then \
if [ "$(nnuenet)" != "nn-"`$(shasum_command) $(nnuenet) | cut -c1-12`".nnue" ]; then \
echo "Failed download or $(nnuenet) corrupted, please delete!"; exit 1; \
fi \
else \
echo "shasum / sha256sum not found, skipping net validation"; \
fi
We see that the address of the file is https://tests.stockfishchess.org/api/nn/$(nnuenet)) and that the variable nnuenet is defined by greping the string EvalFileDefaultName inside the header file evaluate.h (which is inside StockFishDIr\src), the line of interest being :
#define EvalFileDefaultName "nn-62ef826d1a6d.nnue"
Hence the variable nnuenet is equal to the string "nn-62ef826d1a6d.nnue" and the address we are going to download the nets from manually is https://tests.stockfishchess.org/api/nn/nn-62ef826d1a6d.nnue.
Of course, this address may change in the future, but you see the idea. Now we place the net file inside StockFishDIr\src
Now we are ready to cross-compile, with the following command to execute inside StockFishDIr\src :
"MakeDir\bin\make.exe" stockfish_13_armv8_xcomp_win10x64 COMP=ndk COMPCC="AndroidNdkDir\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android30-clang --target=aarch64-linux-android30" ARCH=armv8
(Adapt here also the aarch64-linux-android30 bits according to your Android phone.)
When the preceeding command executes properly it produces the stockfish_13_armv8_xcomp_win10x64 binary inside the StockFishDIr\src folder. This binary is already usuable on you phone (by putting it inside the DroidFish\uci folder of your DroidFish app, for instance) but is not optimal as the size is big and the execution is a little bit slow. Hence we have to "strip" the binary with the following command line, still inside the StockFishDIr\src folder :
"AndroidNdkDir\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-strip" stockfish_13_armv8_xcomp_win10x64
(There's no android30 anymore but there is still a aarch64 : adapt here also the aarch64-linux-android30 bits according to your Android phone.)
Now, your binary file stockfish_13_armv8_xcomp_win10x64 is perfectly usable. You can play chess against StockFish 13 on your phone, and suffer ! ;)
Related
I'm experimenting with native libs cross compiled from ubuntu. What I really want is to be able to compile my c++ libraries and use them in a Xamarin.Android app.
First, I have the arm gcc compiler: arm-linux-gnueabi-gcc. I have a simple lib (libmr.so) that has one function void Print(); that prints something to the console. I'm compiling with:
arm-linux-gnueabi-gcc -Wall -shared -o libmr.so mr.c
When inspecting it using file libmr.so everything seems to be good. However when I'm including it with my android app and try to load it, it is as if it doesn't exist. I'm certain it is there, the path is absolutely correct as I tried to load another lib (libmonodroid.so) from the same folder and it worked.
I tried inspecting both libs to find some kind of a difference:
$ objdump -x libmr.so | grep NEEDED
NEEDED libc.so.6
$ objdump -x libmonodroid.so | grep NEEDED
NEEDED libc.so
... (in addition to other libs)
This is the only difference I'm finding between the two. libmonodroid.so loads properly but libmr.so acts as if it doesn't exist. (I'm using dlopen to load a lib)
EDIT:
I built an executable using the same toolchain, gave me a clue:
Static linking with libc: arm-linux-gnueabi-gcc -Wall -o hi source.c -static. Pushed hi to my android devices and executed it with adb. Result: SUCCESS!
Dynamic linking with libc: arm-linux-gnueabi-gcc -Wall -o hi source.c. Result: it's not even there! Meaning ./hi gives /system/bin/sh: ./hi: not found although it's absolutely there.
So, looks like libc is really the culprit? Maybe I need to link dynamically with not libc.so.6 but with libc.so just like libmonodroid.so is doing?
Check this out for anyone having the same problem. Download the ndk, there's a standalone toolchain for building native libs that run on android that you can extract (you won't need the whole ndk).
I was able to run a basic app on ubuntu 15.04 with this Makefile in the same dir as my hi.c:
$ cat hi.c # create hi.c with favorite editor
#include <stdio.h>
int main(int argc, char** argv) {
int uid = getuid();
int eid = geteuid();
printf("Hello world\n");
printf("You are uid: %d and eid; %d", uid, eid);
return 0;
}
$ cat Makefile # create two line Makefile
CC=arm-linux-gnueabi-gcc
LDFLAGS=-static
$ make hi # build arm based hi executable file
arm-linux-gnueabi-gcc -static hi.c -o hi
$ file hi # check file type
hi: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=17b65e60cdd32449ac237bfd1b8238bfa1d416a0, not stripped
$ adb push hi /data/local/tmp # copy to droid fon
4403 KB/s (593252 bytes in 0.131s)
$ adb shell /data/local/tmp/hi # run hi executable
adb shell /data/local/tmp/hi
Hello world
You are uid: 2000 and eid; 2000
$ uname -a
Linux lenny 3.19.0-28-generic #30-Ubuntu SMP Mon Aug 31 15:52:51 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Note that I do not have any NDK installed.
How can I build OpenSSL for Android ARM v7 (using Android NDK) on Win32?
Until the OpenSSL's wiki and setenv-android.sh are updated accordingly, I'll publish the recipe here. The required fixes to the process are:
Update setenv-android.sh to support Windows.
Update PATH to use Android NDK's (mingw) GNU make (rather than Cygwin's).
Invoke make with a Windows-style path to Cygwin's perl.
This recipe will be a strange hybrid of Cygwin and mingw (since Android NDK gcc toolchains for win32 are mingw). I'm assuming a Windows x86_64 build of the Android NDK unpacked into c:\android-ndk-r9d, and that you wish to use a gcc 4.8 toolchain.
Install Android NDK (duh!).
Install Cygwin -- make sure to include perl
Start Cygwin shell as an administrator to make sure native symlinks will work.
Within the console, run the following script to set the variables:
export \
CYGWIN=winsymlinks:native \
ANDROID_API=android-14 \
ANDROID_DEV=c:/android-ndk-r9d/platforms/android-14/arch-arm/usr \
PATH=/cygdrive/c/android-ndk-r9d/prebuilt/windows-x86_64/bin:/cygdrive/c/android-ndk-r9d/toolchains/arm-linux-androideabi-4.8/prebuilt/windows-x86_64/bin:$PATH \
MACHINE=armv7 \
SYSTEM=android \
ARCH=arm \
CROSS_COMPILE=arm-linux-androideabi-
Now, unpack openssl:
tar xzfv openssl-1.0.1i.tar.gz (or whatever your tarball is)
cd openssl-1.0.1i (or whatever your version is)
Make sure you have actual native Win32 (!) symlinks in include/openssl:
cmd /c "dir include\openssl"
You should see something like:
13-Aug-14 05:59 PM <SYMLINK> aes.h [..\..\crypto\aes\aes.h]
13-Aug-14 05:59 PM <SYMLINK> asn1.h [..\..\crypto\asn1\asn1.h]
(etc.)
Now it's time to configure:
./config shared -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine --openssldir=/foo/bar
Ignore the failure to build (due to failure to find perl). We'll rectify this right away. Do this:
make PERL=$(cygpath -w $(which perl))
Now wait for a few minutes until it builds, and presto, you have your libcrypto.so etc.
Just a couple of comments on my experience with this:
Executing this statement:
PERL=$(cygpath -w $(which perl))
in the cygwin shell allows the shell to interpret the backslashes as escape characters and the build process chokes. To solve this I did the following:
$ echo $(cygpath -w $(which perl))
which produced the windows formatted path to the perl executable:
C:\cygwin64\bin\perl
Then I added this line to the export shown above:
PERL=c:/cygwin64/bin/perl \
There are other ways of doing this, but it worked and headed off the problem with the ./config statement documented above (not finding perl).
Second issue was the -no- statements. After running the configure, the script reports that you'll have to run make depend. I wanted to exclude MD5 (i.e. -no-md5) and when I did the make depend, it errored out with a report that MD5 was disabled. Uhhh, yes, that was kind of the idea, but I just won't use MD5 hashes. I did use the -no-ssl2 and didn't get any complaints after the make depend.
Third issue and this is a mystery. The build broke on compiling crypto because it could not find a symbol that is supposed to be defined in /crypto/objects/obj_xref.h. When I looked at the file, it was empty. Something in the perl script I suppose, but no time to debug right now, since I'm at proof of concept phase. I placed a copy from a patch that I picked up at https://github.com/devpack/openssl-android
After that, my build ran to completion. I've done no testing with this and it is not a trustworthy solution, but it did compile and produce the static libraries that I need for proof of concept for my client.
Just as an update, my shared library built with these libraries loaded fine on my target.
I need to add the timetest directory's executable to the system.img file so that I can access the command from the shell. The location is at : https://android.googlesource.com/platform/system/extras/+/android-4.4.4_r1/tests/timetest/. So what I did was to go into the timetest directory and issue the following command:
mm snod
The output was as follows:
PRODUCT_COPY_FILES device/generic/goldfish/data/etc/apns-conf.xml:system/etc/apns-conf.xml ignored.
make: Entering directory `/home/username/android_built/android'
target thumb C: timetest <= frameworks/base/cmds/timetest/timetest.c
target StaticExecutable: timetest (out/target/product/hammerhead/obj/EXECUTABLES/timetest_intermediates/LINKED/timetest)
target Symbolic: timetest (out/target/product/hammerhead/symbols/sbin/timetest)
target Strip: timetest (out/target/product/hammerhead/obj/EXECUTABLES/timetest_intermediates/timetest)
Install: out/target/product/hammerhead/root/sbin/timetest
make: Leaving directory `/home/username/android_built/android'
The file installation part has been left out for clarity.
The following output is after I tried copying the timetest into frameworks/base/cmds as detailed in this question : android AOSP, adding new executable [.c] code. I also tried it directly.
I then looked at the entire phone including the xbin,bin directories . I also used find / -name "timetest", but the executable is nowhere to be found. What am I doing wrong?
I'm not actually sure how you are installing timetest to your device, but if you are just adb pushing the result to your device then you will likely have a hard time running that.
If you look at the Android.mk file for timetest you will see that it is labeled as "optional"
LOCAL_MODULE_TAGS := optional
However after grepping through all the make files, there is no build variant which includes this in a build. If you have your own device make file you can add it there, but if you just want to make this work quickly you can add it to the list of PRODUCT_PACKAGES in the build/target/product/core_minimal.mk file. There will be a list that looks something like:
PRODUCT_PACKAGES += \
BackupRestoreConfirmation \
DownloadProvider \
HTMLViewer \
MediaProvider \
Just add timetest into that list. After that do a make installclean and rebuild your device and you should have timetest included in sbin.
I have following AOSP project setup
1) I have test.cpp in (AOSP root directory)/vendor/myProject/test.cpp
2) In Android.mk i have used
LOCAL_CFLAGS += -g -fprofile-arcs -ftest-coverage
LOCAL_LDFLAGS += -lgcov
3) When I compile the code i get test.gcno in:
(AOSP root directory)/out/target/product/generic/obj/EXECUTABLES/myProject_intermediates
4) Then i put the test on device (adb sync)
5) Then on device i have used following:
export GCOV_PREFIX=/vendor
export GCOV_PREFIX_STRIP=13 (to strip down the un-necessary path)
6) I run the test ./system/bin/test and i get test.gcda file on
device (shell) /vendor/test.gcda
7) I copy the test.gcda (from device) to my build directory (/out/target/product/generic/obj/EXECUTABLES/myProject_intermediates) where i already have test.gcno
8) Now i am in /out/target/product/generic/obj/EXECUTABLES/myProject_intermediates
Then run gcov as follows:
(AOSP root
directory)/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-gcov
test
For this i get output as follows:
File 'vendor/myProject/test.cpp' Lines executed:23.00% of 223
vendor/myProject/test.cpp:creating 'test.cpp.gcov'
vendor/myProject/test.cpp:cannot open source file
Can anybody help me on how to solve this. It says test.cpp:cannot open source file
gcov is not generating complete report. I have also checked -b option by specifying the path to source as follows:
(AOSP root
directory)/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-gcov
-b (AOSP root directory)/vendor/myProject test
It didn't worked.
I guess the problem is because of distributed files (gcno,gcda,test.cpp) in different directories.
Thanks
Solution is while building the test in which directory you were, you should be in the same directory while running gcov later, then all the .gcov files will be created successfully.
For example:
Project directory : android/frameworks/Myproj
In this you have test.cpp and Android.mk
Then say suppose from android directory you'll build the app or test.cpp, i.e. android$
Then the *.gcno files will be present in android/out/Myproj_Intermediates/
You will get the *.gcda files by running app in device, you will bring that *.gcda files from device to folder android/out/Myproj_intermediates/
Now to run gcov on these:
You should be in the android directory (since you built app from there)
android$ ./prebuilts/gcc/linux-x86/arm/arm-eabi-4.6/bin/arm-eabi-gcov \
-o out/Myproj_intermediates/ -a frameworks/Myproj/test.cpp -b -c
There has been a better approach in AOSP for several months now.
Add LOCAL_NATIVE_COVERAGE := true to your makefile.
$ export NATIVE_COVERAGE := true
Build, sync, run.
$ acov
acov will pull all the coverage files off the device, put them in the right place, and lcov (iirc this will prompt you to install if you don't already have it), and generate and HTML report for you.
I have a Device on which I installed Android Gingerbread 2.3.4
Here i want to run C executable file on android device
I am able to run android NDK application on this Device and it runs perfect.
But I want to run only one hello.c executable file on the Device.
/* #includes #defines ... */
int main(){
// Do something when this is executed
return 0;
}
Is there any way to compile this file with Android NDK tool chain so I can run this file's executable?
I found one thing here but this is not working for me. I am using Android NDK, Revision 7b for Linux.
There is no directory structure like this.
First, let me say that my answer is dependent on your using NDK r7b (it'll work for r7c as well) on Linux (change paths appropriately for other systems).
Edit: Last tested with NDK r8e on Linux and Nexus 4 with adb from SDK Platform-Tools Rev 18 on Windows 7 (latest as of 2013-07-25) without root access.
Yet Another Edit: Please read this question for altering my instruction for native binaries that need to run on Android 5.0(Lollypop) and later.
Go to $NDK_ROOT (The topmost folder of NDK zip when unzipped).
Copy $NDK_ROOT/samples/hello-jni directory as $NDK_ROOT/sources/hello-world.
Go to $NDK_ROOT/sources/hello-world.
Edit AndroidManifest.xml to give the application an appropriate name (This is optional).
Go to $NDK_ROOT/sources/hello-world/jni. This is where the source code is.
Edit hello-jni.c, remove all the code, and put in your hello world code. Mine is:#include
int main( int argc, char* argv[])
{
printf("Hello, World!");
return 0;
}
Edit Android.mk and change the line include $(BUILD_SHARED_LIBRARY) to include $(BUILD_EXECUTABLE). You can also change the LOCAL_MODULE line to the name you want for your executable(default is hello-jni)
Go back to $NDK_ROOT/sources/hello-world
Run ../../ndk-build to create the executable.
Copy it from $NDK_ROOT/sources/hello-jni/libs/armeabi/hello-jni to /data/local/tmp on the Android device and change it's permissions to 755 (rwxr-xr-x). If you changed the LOCAL_MODULE line in $NDK_ROOT/sources/hello-world/jni/Android.mk, the executable name will be the new value of LOCAL_MODULE instead of hello-jni. (All this is done via adb from the Android SDK.)
Execute the binary with full path as /data/local/tmp/hello-jni, or whatever you named it to.
And you're done( and free to start on the documentation in $NDK_ROOT/docs to get a better idea of what to do).
The best/easiest place to put a executable is /data/local. You'll also need to chmod the binary as executable. Often you'll also need to do this in two steps to get the binary from /sdcard/ to /data/local:
$ adb push mybin /sdcard/
$ adb shell
$ cp /sdcard/mybin /data/local/mybin
$ cd /data/local
$ chmod 751 mybin
Caveats:
Not all systems have cp. You can use cat if this is the case:
$ cat /sdcard/mybin > /data/local/mybin
Some systems don't allow write in /data/local for the "shell" user. Try /data/local/tmp
the "/sdcard" location is not executable, meaning that any file there is not executable at all.
the only way to "adb push" executable would be to put them in "/data/local", which should be writable for adb, and allow execution for anyone.
I recently had the same problem on a new nexus-5. I'd like to add that /data/local was not writable by the user ("shell", uid 2000) I got with adb shell. But putting the executable in the subdirectory /data/local/tmp/ worked fine.
In a nutshell,
First, to cross-compile your C code from your host machine, use NDK toolchain with sysroot option and position independent option -fPIE -pie.
$NDKROOT/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/bin/aarch64-linux-android-gcc \
--sysroot=$NDKROOT/platforms/android-22/arch-arm64 -fPIE -pie main.c -o main
the arch part arrch64 or arch-arm64, the toolchain version part 4.9, platform version part android-22, and the binary format for your host machine darwin-x86_64 may vary by your environment.
Second, push your binary under /data/local/tmp and execute it from adb shell.