Develop Reference

Unable to build Botan for Android on Windows

I cannot understand how to build Botan for android, according on the instruction here:
$ export CXX=/opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android28-clang++
$ ./configure.py --os=android --cc=clang --cpu=arm64
i cannot understand how to use this commands on Windows, also reading previous issues did not help me, can you tell me how did you build this library on windows step-by-step, just your command examples?
I used --cc-bin option of configure.py to specify the path to the compiler, it is considered a solution for windows, but what i have is:
D:\Programming\Libraries\botanAndroid\botan-master>python configure.py --cc-bin=D:\Android\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++ --os=android --cc=clang --cpu=armv7
INFO: configure.py invoked with options "--cc-bin=D:\Android\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++ --os=android --cc=clang --cpu=armv7"
INFO: Configuring to build Botan 2.14.0 (revision unknown)
INFO: Running under 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 22:20:52) [MSC v.1916 32 bit (Intel)]
INFO: Autodetected platform information: OS="Windows" machine="AMD64" proc="Intel64 Family 6 Model 142 Stepping 10, GenuineIntel"
INFO: Canonicalized CPU target armv7 to arm32
WARNING: Could not execute ['D:\Android\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++', '-E', 'src\build-data\detect_version.cpp']: [WinError 193] %1 is not an application of Win32
WARNING: Tried to get clang version, but output '0.0' does not match expected version format
WARNING: Could not execute ['D:\Android\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++', '-E', '-fstack-protector', '-pthread', 'src\build-data\detect_arch.cpp']: [WinError 193] %1 is not an application of Win32
WARNING: Unable to detect target architecture via compiler macro checks
INFO: Target is clang:0.0-android-arm32
INFO: Assuming target arm32 is little endian
INFO: Skipping (dependency failure): asio certstor_sqlite3 rdrand sessions_sqlite3
INFO: Skipping (incompatible CPU): aes_armv8 aes_ni aes_power8 chacha_avx2 clmul_cpu clmul_ssse3 idea_sse2 p9_darn rdrand_rng rdseed serpent_avx2 sha1_armv8 sha1_sse2 sha1_x86 sha2_32_armv8 sha2_32_bmi2 sha2_32_x86 sha2_64_bmi2 sha3_bmi2 shacal2_avx2 shacal2_x86 simd_avx2 sm4_armv8 threefish_512_avx2
INFO: Skipping (incompatible OS): certstor_system_macos certstor_system_windows commoncrypto getentropy proc_walk win32_stats
INFO: Skipping (no enabled compression schemes): compression
INFO: Skipping (requires external dependency): boost bzip2 lzma openssl sqlite3 tpm zlib
INFO: Loading modules: adler32 aead aes aes_vperm aont argon2 aria asn1 auto_rng base base32 base58 base64 bcrypt bcrypt_pbkdf bigint blake2 block blowfish camellia cascade cast128 cast256 cbc cbc_mac ccm cecpq1 certstor_flatfile certstor_sql certstor_system cfb chacha chacha20poly1305 chacha_rng chacha_simd32 checksum cmac comb4p cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa dyn_load eax ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea iso9796 kasumi kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion locking_allocator mac mce mceies md4 md5 mdx_hash mem_pool mgf1 misty1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pgp_s2k pk_pad pkcs11 poly1305 poly_dbl prf_tls prf_x942 psk_db pubkey rc4 rfc3394 rfc6979 rmd160 rng roughtime rsa salsa20 scrypt seed serpent serpent_simd sessions_sql sha1 sha2_32 sha2_64 sha3 shacal2 shacal2_simd shake shake_cipher simd siphash siv skein sm2 sm3 sm4 socket sodium sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng thread_utils threefish_512 tiger tls tls_10 tls_cbc tss twofish utils uuid whirlpool x509 x919_mac xmss xtea xts
INFO: Using hardlink to link files into build dir (use --link-method to change)
INFO: Botan 2.14.0 (revision unknown) (unreleased undated) build setup is complete
Now i'm currently using VisualStudio 2017 native tool command prompt, or calling vcvarsall.bat, to set up the environment.

It seems Botan support for building Android binaries on Windows hosts is limited. You will have to use dark magic to make this work.
The build process consists of two phases, the configuration phase and the make phase.
The Android-specific instructions in the documentation you linked do not cover the whole build process, only the configuration phase. For the make phase, you then have to follow the Windows-specific instructions (link).
Configuration phase:
You will need the following binaries, adjust the paths to your machine:
clang++ (note the .cmd at the end): C:\Development\android-ndk-r19c-windows-x86_64\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++.cmd
ar: C:\Development\android-ndk-r19c-windows-x86_64\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\arm-linux-androideabi-ar.exe
In the Botan folder, run the configure command:
python.exe .\configure.py --cc-bin=C:\Development\android-ndk-r19c-windows-x86_64\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\armv7a-linux-androideabi28-clang++.cmd --ar-command=C:\Development\android-ndk-r19c-windows-x86_64\android-ndk-r19c\toolchains\llvm\prebuilt\windows-x86_64\bin\arm-linux-androideabi-ar.exe --os=android --cpu=armv7 --verbose
Make phase
The configuration phase generates a Makefile in the Botan folder. You will have to make some adjustments to this file:
In the line all: libs cli tests docs remove docs
Reason: Additional tools are needed for building the documentation files. If you really need the documentation, you could also try to install these tools, but I have not tested this.
Replace the occurrences of ln -fs with copy.
Reason: On Linux ln -fs would create a symbolic link from the second file in the parameter list to the first. This command is not available, so changing it to copying the first file to the second seems like a pragmatic work-around to me. You could also change it to the appropriate command for creating a link on Windows, but then you might have to adjust it again when deploying to your Android target.
In the lines starting with LIBOBJS =, CLIOBJS = and TESTOBJS =, replace all occurrences of \ with /. In the whole file, replace occurrences of .\ with ./. Reason: Using the Windows-style path separator \ seems to cause problems in some places.
Find the block with # Executable targets and # Library targets. Insert #<< ... << around the parameter lists (known as the nmake inline file feature, based on this answer), to make it look like this:
# Executable targets
$(CLI): $(LIBRARIES) $(CLIOBJS)
$(EXE_LINK_CMD) #<<
$(ABI_FLAGS) $(CLIOBJS) $(EXE_LINKS_TO) $(LDFLAGS) -o $#
<<
$(TEST): $(LIBRARIES) $(TESTOBJS)
$(EXE_LINK_CMD) #<<
$(ABI_FLAGS) $(TESTOBJS) $(EXE_LINKS_TO) $(LDFLAGS) -o $#
<<
# Library targets
./libbotan-2.a: $(LIBOBJS)
$(AR) #<<
$(AR_OPTIONS) $# $(LIBOBJS)
<<
./libbotan-2.so.13: $(LIBOBJS)
$(CXX) #<<
-shared -fPIC -Wl,-soname,libbotan-2.so.13 $(ABI_FLAGS) $(LDFLAGS) $(LIBOBJS) $(LIB_LINKS_TO) -o $#
<<
Reason: Without this change, I got an error about the parameter list being too long.
You will need nmake (part of Visual Studio). On my machine it is installed in C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\bin\Hostx64\x64\nmake.exe
In the Botan folder, run nmake.exe. Afterwards, your Botan folder should contain the binaries botan, botan-test and libraries libbotan-2....

Android chromium full browser apk build issues

I trying to build chromium android apk with this instruction
But on GN configuration step I have following problem after arguments file editing and saving
ERROR at //build/config/android/internal_rules.gni:1966:13: Script returned non-zero exit code.
exec_script("//build/android/gyp/find.py",
^----------
Current dir: /home/artem/chromium/src/out/Default/
Command: python -- /home/artem/chromium/src/build/android/gyp/find.py ../../third_party/android_tools/sdk/extras/google/google_play_services/libproject/google-play-services_lib/res
Returned 1.
stderr:
../../third_party/android_tools/sdk/extras/google/google_play_services/libproject/google-play-services_lib/res does not exist
See //build/config/android/rules.gni:634:3: whence it was called.
process_resources(process_resources_target_name) {
^-------------------------------------------------
See //third_party/android_tools/BUILD.gn:120:1: whence it was called.
android_resources("google_play_services_default_resources") {
^------------------------------------------------------------
See //base/BUILD.gn:1074:7: which caused the file to be included.
"//third_party/android_tools:cpu_features",
^-----------------------------------------
Then I try manually download google play services with command from this issue thread:
sudo ./build/android/play_services/update.py download
And get this warning:
Your version of the Google Play services library is not up to date. You might run into issues building or running the app. Please run `./build/android/play_services/update.py download` to retry downloading it.
After that I try to build full browser apk
ninja -C out/Default chrome_public_apk
and get error:
ninja: Entering directory `out/Default'
ninja: fatal: chdir to 'out/Default' - No such file or directory
Is there the way to fix this issue and build working apk file?

This works for me:
Run this command at chromium folder:
git rebase-update
gclient sync
Download google play services:
sudo ./build/android/play_services/update.py download
Configure GN: gn args opt/Full
Add this line to file: target_os="android"
Build apk with command: ninja -C opt/Full chrome_public_apk

How to compile aFreeRDP on Linux

aFreeRDP is the Android client of the FreeRDP project.
I'm using 64 bit Ubuntu 15.04 Vivid.
My goal is to compile aFreeRDP. I'm at the finish line, but cannot cross it. I successfully installed the required packages, Android NDK, and then compiled the external openssl (the android shell script does that automatically). Then I generated make files. Interestingly neither
cmake -DCMAKE_TOOLCHAIN_FILE=cmake/AndroidToolchain.cmake -DANDROID_NDK=/home/csaba/android-ndk -DFREERDP_EXTERNAL_SSL_PATH=/home/csaba/GitRepos/FreeRDP/external/openssl -DANDROID_SDK=/home/csaba/android-sdk-linux
nor
cmake -DCMAKE_TOOLCHAIN_FILE=cmake/AndroidToolchain.cmake -DANDROID_NDK=/home/csaba/android-ndk -DFREERDP_EXTERNAL_SSL_PATH=/home/csaba/GitRepos/FreeRDP/external/openssl -DCMAKE_BUILD_TYPE=Debug -DANDROID_BUILD_JAVA=OFF -DANDROID_SDK=/home/csaba/android-sdk-linux -DFREERDP_EXTERNAL_JPEG_PATH=/home/csaba/GitRepos/FreeRDP/external/jpeg8d
produced any apk, but they happily compiled libfreerdp-android.so.
I need an apk, so I went to the client/Android/aFreeRDP and since I had a build.xml there, I said ant build install. The build process started, but then died at the resource compilation phase. I have the current level 23 android build tools and companion stuff with the Android SDK. I see in the source, that the source was originally targeted to level 21. I added level 21 SDK too, and tried to force maxSdkVersion to 21 in the AndroidManifest.xml, but that didn't help. The compilation log:
http://pastebin.com/Nc0mqGrQ
As you can see after resource errors
[aapt] Generating resource IDs...
[aapt] /home/csaba/GitRepos/FreeRDP/client/Android/appcompat_v7/res/values-v23/styles_base.xml:20: error: Error retrieving parent for item: No resource found that matches the given name 'android:Widget.Material.Button.Colored'.
[aapt]
[aapt] /home/csaba/GitRepos/FreeRDP/client/Android/appcompat_v7/res/values-v23/styles_base_text.xml:19: error: Error retrieving parent for item: No resource found that matches the given name 'android:TextAppearance.Material.Widget.Button.Inverse'.
[aapt]
it dies
BUILD FAILED
/home/csaba/android-sdk-linux/tools/ant/build.xml:597: The following error occurred while executing this line:
/home/csaba/android-sdk-linux/tools/ant/build.xml:649: The following error occurred while executing this line:
/home/csaba/android-sdk-linux/tools/ant/build.xml:694: null returned: 1
Comparing the level 21 and the level 23 resources, the level 21 has way more stuff, the level 23 has single entries, very vague. How can I fix this? Let me know what files should I post.
Update 1:
Get rid of an annoying issue with ADT, which could have affected this bug too: Eclipse Error: Error parsing ...\android-22\android-wear\armeabi-v7a\devices.xml
Unfortunately uninstalling the Wear images made the ADT error go away, but the aFreeRDP compilation error still remained.
Update 2:
I tried to go back in time and my goal was to have an Android SDK which provides 21, but doesn't have level 23.
I backed up my current Android SDK. SDK download tgz 22.6.2 (https://dl.google.com/android/android-sdk_r22.6.2-linux.tgz) only provides target 21. But SDK download tgz 23.0.0 (https://dl.google.com/android/android-sdk_r23-linux.tgz) already wants to upgrade to 24. So I cannot get the system to a state, when I don't have the target 23 (which indirectly causes errors somehow - more correctly, the level 24's compatibility facility for target 23 (which satisfies target 21)), but have the target 21.
(Bonus question: why Google makes it so hard to download older SDKs? I had to fiddle and guess URLs).
Update 3:
I managed to debug aFreeRDP from Android Studio (v 1.5) using the directions github.com/FreeRDP/FreeRDP/blob/master/docs/README.android

I want to post the same topic but you already made one so I will post here.
it should be comment but I dont have 50 reputation to comment so i make it answer here: Notes: (xxx = https://github.com in this post)
Reference Links:
xxx/FreeRDP/FreeRDP/wiki/Compilation
xxx/awakecoding/FreeRDP-Manuals/blob/master/Developer/FreeRDP-Developer-Manual.markdown
and other posts from internet
Goal: Build aFreeRDP for Android devices.
2 years ago I used to build it for Ubuntu 12 / 14 but now the build is broken.
A new environment needs to setup again:
-Windows 7 Enterprise 64 bit (MAIN OS)
-WMware WorkStation 10
-VM Ubuntu 15.10 Wily 64bit
Prepare Ubuntu with NDK10e, JDK7, SDK23...
shaun#ubuntu:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 15.10
Release: 15.10
Codename: wily
Install JDK
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer
Install ANDROID NDK
chmod a+x android-ndk-r10e-linux-x86_64.bin
./android-ndk-r10e-linux-x86_64.bin
sudo chown -R root:root /home/shaun/Main/android-ndk-r10e
sudo mv /home/shaun/Main/android-ndk-r10e /opt
export NDK=/opt/android-ndk-r10e
Install ANDROID SDK
Download and extract android-sdk_r23.0.2-linux.tgz
sudo mv /home/shaun/Main/android-sdk-linux /opt/android-sdk
shaun#ubuntu:/opt/android-sdk/tools$ ./android
Install dependences GIT / CMAKE / GCC
sudo apt-get install \
build-essential git-core cmake \
libssl-dev \
libx11-dev libxext-dev libxinerama-dev libxcursor-dev libxkbfile-dev \
libxv-dev libxi-dev libxdamage-dev libxrender-dev libxrandr-dev \
libasound2-dev libcups2-dev libpulse-dev \
libavutil-dev libavcodec-dev \
libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libcunit1-dev libdirectfb-dev xmlto doxygen libxtst-dev
sudo apt-get install lib32stdc++6
sudo apt-get install lib32z1
GET FreeRDP
git clone git://github.com/FreeRDP/FreeRDP.git
cd FreeRDP
Add PATH
shaun#ubuntu:~$ gedit ~/.bashrc
export ANDROID_NDK=/opt/android-ndk-r10e
export ANDROID_SDK=/opt/android-sdk
export FREERDP_EXTERNAL_SSL_PATH=/home/shaun/Main/FreeRDP-master/external/Android-external-openssl-ndk-static
export PATH=$PATH:$FREERDP_EXTERNAL_SSL_PATH
export PATH="$PATH:$ANDROID_SDK/tools:$ANDROID_SDK/platform-tools:$ANDROID_NDK"
from FreeRDP-->run ./scripts/android_setup_build_env.sh
--> see long output .... then at the end you see:
Prepared external libraries, you can now build the application.
Finally run:
cmake -DCMAKE_TOOLCHAIN_FILE=cmake/AndroidToolchain.cmake -DANDROID_NDK=$ANDROID_NDK -DANDROID_SDK=$ANDROID_SDK -DFREERDP_EXTERNAL_SSL_PATH=$FREERDP_EXTERNAL_SSL_PATH -DFREERDP_EXTERNAL_JPEG_PATH="/home/shaun/Main/FreeRDP/external/jpeg8d" -DANDROID_NATIVE_API_LEVEL:STRING=L -DCMAKE_MAKE_PROGRAM=$ANDROID_NDK/prebuilt/linux-x86/bin/make -DCMAKE_BUILD_TYPE=Debug -DANDROID_BUILD_JAVA=OFF -L .
make
After this make, I have all libs i need but still fail to have APK:...
...long output....
Total time: 2 seconds
client/Android/aFreeRDP/CMakeFiles/android-package.dir/build.make:50: recipe for target 'client/Android/aFreeRDP/bin/aFreeRDP-debug.apk' failed
make[2]: * [client/Android/aFreeRDP/bin/aFreeRDP-debug.apk] Error 1
CMakeFiles/Makefile2:2494: recipe for target 'client/Android/aFreeRDP/CMakeFiles/android-package.dir/all' failed
make[1]: * [client/Android/aFreeRDP/CMakeFiles/android-package.dir/all] Error 2
Makefile:136: recipe for target 'all' failed
make: *** [all] Error 2
==========================
Import to ECLIPSE and build from Eclipse sucessfully -- APK and application run on all my devices again with new features most advanced feature is microphone redirection works as well :-)
But Still not work in ANDROID STUDIO , Still trying :(, please let me know if you have it work in AS. Thanks

I started with the 23.0 SDK tgz, upgraded one round (not upgrading up to Android 6.0). Then the troublesome xml files are copied over from the Android SDK. So I deleted that values-v23 folder from the SDK. I also modified the AndroidManifest.xml of the aFreeRDP and set minSdkVersion to 17 from 14. The directory deletion certainly helped, I'm not sure if the other two steps contributed, but I have an apk now. I consider this an ugly hack, only a workaround though. Because So I won't mark this as an answer for now.
Update:
I started from a totally updated Android SDK, and the minSdkVersion bump from 14 to 17 also doesn't matter. The only thing which made the error go away is the invasive removal of that values-v23 directory from inside the Android SDK.
Note:
The compilation is simpler than it was a year/couple of years ago. The android_setup_build_env.sh takes care of downloading and compiling ssl library and other stuff.

`gomobile build` and `gomobile install` throws "gomobile: EOF"

I'm trying to build an android application using gomobile but the commands gomobile install and gomobile build fail with gomobile: EOF . Before this error I was getting this error:
# golang.org/x/mobile/gl
In file included from /root/go/src/golang.org/x/mobile/gl/gl.go:17:0:
work.h:6:23: fatal error: GLES2/gl2.h: No such file or directory
compilation terminated. but it was fixed with yum install make gcc mesa-libGLES-devel mesa-libEGL-devel.
The application is mostly a networking application. Here are the imports. Do only certain packages work with gomobile or all of them?
"golang.org/x/mobile/app"
"golang.org/x/net/websocket"
"os/exec"
"flag"
"fmt"
log "github.com/golang/glog"
"io/ioutil"
"net"
"net/http"
"net/url"
"strconv"
OS: Fedora 22.
More info: I'm able to build https://github.com/golang/mobile/blob/master/example/network/main.go so I can only assume the issue is specific to my application.

It failed because AndroidManifest.xml was empty.

Invalid syntax zipfile(apkAbsPath) in line 202 ninjadroid.py

I cloned ninjadroid from https://github.com/rovellipaolo/NinjaDroid recently and followed the instructions for decompiling apk files. As instructed I have installed android sdk and python, but when I tried running the command in my command prompt (python ninjadroid.py -t myPackage.apk), it throws the below error
File "ninjadroid.py", line 202
with ZipFile(apkAbsPath) as z:

Mmm... that's strange.
Is 'myPackage.apk' an existing file (in your computer)? Is it a real APK package? From the error, I would say you are trying to reverse a non-APK file.
Unfortunately the "is-APK" check in the current, available version is a lazy check - done pretty badly. I completely reimplemented it in the new version I am working on.
UPDATE
With the newest NinjaDroid version (i.e. 2.0):
$ python ninjadroid.py this_is_a_non_existing_file
>> NinjaDroid: [ERROR] The target file (i.e. 'this_is_a_non_existing_file') must be an existing, readable file!
$ touch this_is_an_existing_file_but_not_an_apk_package.txt
$ python ninjadroid.py this_is_an_existing_file_but_not_an_apk_package.txt
>> NinjaDroid: [ERROR] The target file (i.e. 'this_is_an_existing_file_but_not_an_apk_package.txt') must be an APK package!
$ python ninjadroid.py test/data/Example.apk
{
"app_name": "Example",
...
}