Develop Reference

cargo ndk fails due to pkg-config issue

I think I've read every suggestion on the internet at this point to no avail, so I'm hoping I can get some help here.
Repro
git clone https://github.com/banool/aclip.git
cd aclip/frontend/native
cargo ndk -t armeabi-v7a -o ../android/app/src/main/jniLibs build
The code in question: https://github.com/banool/aclip.
Relevant env:
PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig/
LDFLAGS=-L/opt/homebrew/opt/openssl#1.1/lib
CPPFLAGS=-I/usr/local/opt/openssl#1.1/include
CFLAGS=-I/opt/homebrew/opt/openssl#1.1/include
ANDROID_NDK_HOME=/Users/dport/Library/Android/sdk/ndk
Output
Running the above gives me this: https://gist.github.com/d2564965bbd5e4d8bf72c3132c87255f.
Some thoughts on what's going on here:
I see it says NDK API level: 21. Is this level not the same as the NDK level, which as you can see in the previous line is 24?
I have tried setting PKG_CONFIG_SYSROOT_DIR to the NDK directory. That gives me instead this rather cryptic error: https://gist.github.com/b28eae5f666fccaffe80278a5a8a80e1. I wonder if there is a better directory to use.
I did everything here relevant to the second part, where I'm trying to use a newer NDK: http://cjycode.com/flutter_rust_bridge/template/setup_android.html.
I have set the libunwind stuff suggested here: http://cjycode.com/flutter_rust_bridge/template/setup_android.html. That helped with the gcc error in the second gist here, but I've still got the ssl and crypto errors.
Any tips on how to proceed would be very much appreciated.
Thanks!

For me I was able to skirt around the issue by doing this.
First, figure out what openssl crate version we're relying on:
$ cargo tree --target=x86_64-unknown-linux-musl -i openssl-sys | grep openssel
openssl-sys v0.9.74
└── openssl v0.10.40
Then add something like this to your Cargo.toml:
openssl = { version = "0.10.40", features = ["vendored"] }
This avoids the need for the system openssl.

add_qt_android_apk strips debug symbols of shared library

I use add_qt_android_apk to build an APK using Qt 5:
add_qt_android_apk(my_app_apk gustavsfairyland NAME "#string/ApplicationName"
DEPENDS
${QTAV_LIBRARIES}
)
QTAV_LIBRARIES is defined before:
set(QTAV_LIBRARIES
"${QTAV_FFMPEG_LIB_DIR}/libavcodec.so"
"${QTAV_FFMPEG_LIB_DIR}/libavfilter.so"
"${QTAV_FFMPEG_LIB_DIR}/libavformat.so"
"${QTAV_FFMPEG_LIB_DIR}/libavresample.so"
"${QTAV_FFMPEG_LIB_DIR}/libavutil.so"
"${QTAV_FFMPEG_LIB_DIR}/libswresample.so"
"${QTAV_FFMPEG_LIB_DIR}/libswscale.so"
"${CMAKE_BINARY_DIR}/buildqtav/${QTAV_LIB_DIR}/libQtAV.so"
"${CMAKE_BINARY_DIR}/buildqtav/${QTAV_LIB_DIR}/libQtAVWidgets.so"
)
I build libQtAV.so using the debug mode (user.conf):
CONFIG += no_config_tests
CONFIG += config_avutil config_avformat config_avcodec config_swscale config_swresample
CONFIG -= release
CONFIG += debug
When I use nm to check for symbols I get many symbols:
nm ../buildqtav/lib_android_arm/libQtAV.so
00062884 t $a
00061d88 t $a
0005f9d0 t $a
...
But when I use nm on the copied library in the libs directory I get nothing:
bash-4.3$ nm armeabi-v7a/libQtAV.so
nm: armeabi-v7a/libQtAV.so: no symbols
Does add_qt_android_apk remove the debugging symbols?
In the CMake module for Qt APK I found this:
if(EXTRA_LIBS)
set(EXTRA_LIBS "${EXTRA_LIBS},${LIB}")
else()
set(EXTRA_LIBS "${LIB}")
endif()
endforeach()
set(QT_ANDROID_APP_EXTRA_LIBS "\"android-extra-libs\": \"${EXTRA_LIBS}\",")
so it uses the specified external .so path. It is then added to qtdeploy.json in the CMake variable QT_ANDROID_APP_EXTRA_LIBS. Which has the entry
buildandroidarmeabi-v7a/buildqtav/lib_android_arm/libQtAV.so
in "android-extra-libs":
So it actually has the correct entry but somehow strips the debug symbols. The library in the "libs" folder has a size of 1.1 MiBytes while the original library in "lib_android_arm" has a size of 1.6 MiBytes.
I'd like to see the routines using ndk-stack which prints at the moment:
Stack frame #05 pc 000b714f /data/app/org.qtproject.gustavsfairyland-1/lib/arm/libQtAV.so: Routine ??
edit:
I use the following CMake module: https://github.com/LaurentGomila/qt-android-cmake
edit2:
It looks like androiddeployqt does always strip symbols of the libraries:
http://code.qt.io/cgit/qt/qttools.git/tree/src/androiddeployqt/main.cpp
stripLibraries() is always called when building the application.

As already mentioned in the edited question and in the answer, androiddeployqt is what strips the debuginfo from the .so file.
I belive the important point here is that androiddeployqt does not modify the file, it creates a copy in the process of generating the apk, and only this copy has the debug infos stripped. In the build output there is still the unmodified .so file which contains the debug infos and which can be used to debug the application (Qt creator does that AFAIK) and which can be used to grab symbol information - I use this for creating readable stacktraces with breakpad crashdump files.
I believe that's also the reason why QTBUG-57771 was rejected as incomplete: The Qt maintainers see no good reason to have the symbols available on the device, as long as debugging the application with Qt creator works with the current build process (and demonstrates that debugging is possible with a properly configured toolchain).

Well, androiddeployqt does ALWAYS strip the libraries to reduce their size. I have opened a bug report for Qt: https://bugreports.qt.io/browse/QTBUG-57771

How to call OpenSSL SRP APIs from Android app?

I am trying to implement the SRP protocol for a Server and Client app in Android. I checked that OpenSSL supports SRP protocol but how can I access OpenSSL APIs from the Android app? Is JNI the only way to access the OpenSSL C APIs? Are there any samples that I can refer how to build the OpenSSL for Android and to call the OpenSSL SRP APIs through JNI.

I am trying to implement the SRP protocol
I like SRP, too. Be sure to use Thomas Wu's specification, or version 6 of the IETF's specification.
OpenSSL supports SRP protocol but how can I access OpenSSL APIs from the Android app? Is JNI the only way to access the OpenSSL C APIs?
Yes.
You might also look to Bouncy Castle for a Java implementation.
Are there any samples that I can refer how to build the OpenSSL for Android...
See OpenSSL and Android on the OpenSSL wiki.
Android carries around a copy of OpenSSL, but I'm not sure of it includes SRP.
Are there any samples that I can refer ... to call the OpenSSL SRP APIs through JNI.
Not that I am aware. The closest you will find is the source code to s_client, options like -srpuser <user>, and data structures like srp_arg_st, and functions like ssl_srp_verify_param_cb and ssl_give_srp_client_pwd_cb.
You can find the source code for s_client at <openssl src dir>/apps/s_client.c. Line 1365 looks interesting (from 1.0.2h):
# ifndef OPENSSL_NO_SRP
if (srp_arg.srplogin) {
if (!srp_lateuser && !SSL_CTX_set_srp_username(ctx, srp_arg.srplogin)) {
BIO_printf(bio_err, "Unable to set SRP username\n");
goto end;
}
srp_arg.msg = c_msg;
srp_arg.debug = c_debug;
SSL_CTX_set_srp_cb_arg(ctx, &srp_arg);
SSL_CTX_set_srp_client_pwd_callback(ctx, ssl_give_srp_client_pwd_cb);
SSL_CTX_set_srp_strength(ctx, srp_arg.strength);
if (c_msg || c_debug || srp_arg.amp == 0)
SSL_CTX_set_srp_verify_param_callback(ctx,
ssl_srp_verify_param_cb);
}
# endif
And the srp_arg_st from around line 475:
/* This is a context that we pass to all callbacks */
typedef struct srp_arg_st {
char *srppassin;
char *srplogin;
int msg; /* copy from c_msg */
int debug; /* copy from c_debug */
int amp; /* allow more groups */
int strength /* minimal size for N */ ;
} SRP_ARG;
Obviously, OpenSSL is native C and does not use JNI.
How to call OpenSSL SRP APIs ...?
At the highest levels, you need to do two or three things in C. Think of them as a supplement to a the standard TLS Client from the OpenSSL wiki. (I'm side stepping Android/JNI part).
First, you need to set the SSL_CTX_set_srp_*_callback. The callbacks are how the library prompts your TLS client for information like username and password.
Second, you remove all non-SRP cipher suites. That means you do not use a cipher list like "HIGH:!aNULL:!MD5:!RC4".
Third, you use only SRP cipher suites. I'm not sure how the cipher list would look when using "HIGH:!aNULL:...". But you can hand pick the list of ciphers with:
$ openssl ciphers -v | grep SRP | grep -v DSS | cut -f 1 -d ' '
SRP-RSA-AES-256-CBC-SHA
SRP-AES-256-CBC-SHA
SRP-RSA-AES-128-CBC-SHA
SRP-AES-128-CBC-SHA
SRP-RSA-3DES-EDE-CBC-SHA
SRP-3DES-EDE-CBC-SHA
If you go to the openssl ciphers man page, then you should be able to cross reference SRP-RSA-AES-256-CBC-SHA with the name needed in the cipher list. Unfortunately, the SRP cipher suites are missing.
However, you can go to ICANN's TLS paramter registry and get the names:
SRP-RSA-AES-256-CBC-SHA → TLS_SRP_SHA_RSA_WITH_AES_256_CBC_SHA
SRP-AES-256-CBC-SHA → TLS_SRP_SHA_WITH_AES_256_CBC_SHA
SRP-RSA-AES-128-CBC-SHA → TLS_SRP_SHA_RSA_WITH_AES_128_CBC_SHA
SRP-AES-128-CBC-SHA → TLS_SRP_SHA_WITH_AES_128_CBC_SHA
SRP-RSA-3DES-EDE-CBC-SHA → TLS_SRP_SHA_RSA_WITH_3DES_EDE_CBC_SHA
SRP-3DES-EDE-CBC-SHA → TLS_SRP_SHA_WITH_3DES_EDE_CBC_SHA
So the string you would use for SSL_CTX_set_cipher_list or SSL_set_cipher_list:
static const char const PREFERRED_CIPHERS[] =
"TLS_SRP_SHA_RSA_WITH_AES_256_CBC_SHA:TLS_SRP_SHA_WITH_AES_256_CBC_SHA:
TLS_SRP_SHA_RSA_WITH_AES_128_CBC_SHA:TLS_SRP_SHA_WITH_AES_128_CBC_SHA:
TLS_SRP_SHA_RSA_WITH_3DES_EDE_CBC_SHA:TLS_SRP_SHA_WITH_3DES_EDE_CBC_SHA";

OpenSSL for android

I am using following OpenSSL library for android
My application need to support TLS 1.2 version(TLSv2).I want to know Does above library support TLS 1.2 and above library is build upon which Openssl version?

According to the code this is only OpenSSL 1.0.0a. TLS1.2 support was added with OpenSS L1.0.1, so this library does not support TLS1.2. BTW, TLS1.2 is TLSv1_2 and not TLSv2.

I am using following OpenSSL library for android
According to Steffen, its a dwonlevel version. You might consider building it yourself. You can find information on OpenSSL's wiki: OpenSSL and Android.
Or, you could fetch 1.0.1h from this Github: Noloader GitHub. The GitHub has OpenSSL 1.0.1h built for both API 14 (GCC 4.6 toolchain) and API 18 (GCC 4.8 toolchain).
Does OpenSSL library support TLS 1.2 and above library is build upon which Openssl version?
If you are working with a modern version of OpenSSL, then TLS 1.2 will be available by default (unless on a distro like Debian and Ubuntu, which disables TLS 1.1 and TLS 1.2 prior to about 2014). If the distro has not disabled the protocol, then you should perform the following to tighten up the protocols:
/* Useless return value ??? */
SSL_library_init();
const SSL_METHOD* method = SSLv23_method();
if(!(NULL != method)) handleFailure();
SSL_CTX* ctx = SSL_CTX_new(method);
if(!(ctx != NULL)) handleFailure();
/* Cannot fail ??? */
const long flags = SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_COMPRESSION;
SSL_CTX_set_options(ctx, flags);
The code above provides you with TLS 1.0 and above. You will be fine with TLS since its ubiquitous (hence, no need for SSLv3). The code will also ensure TLS 1.3 is available once its standardized (the IETF is standardizing it now). And it disables compression because of attacks like CRIME.
There's also an opportunity to tighten up the cipher suites. You should provide 16 or 20 or so approved ciphers and no more. There's no reason to be using export grade ciphers, RC4 or MD5 in 2014. Plus, if you advertise all 80+ ciphers, it causes some appliances to break like older F5's and IronPorts. The applicances used a fixed-size buffer that was too small, and they choke/hang on the ClientHello.

Cross-Compiling Swiften library for ARM-embedded with SCons makery

Swiften is a XMPP client library and my objective was to build it for a ARM-embedded target running Linux.
I hacked my way to a successful cross-compile with little knowledge of SCons. I'll lay out my hack here with the hope that someone can point me to a maintainable solution using the two makery files, SConscript.boot and SConstruct.
I had two tasks (neither accomplished satisfactorily):
Successfully switching the tool-chain from native-compile to cross-compile
Ensuring that OpenSSL libraries were successfully linked (not supplied by the swiftim project; they has to be installed and built in the 3rdParty folder).
Switching the tool-chain from native-compile to cross-compile for ARM
My ARM cross tool-chain components, gcc, g++, ld, etc are located here.
/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/
I couldn't find a way to tell scons to use the cross tool-chain (from the above location) instead of the native tool (in the usual place, /usr/bin). Prefacing the invocation (./scons Swiften) with the fully-qualified values for the environment variables, CC and CXX didn't work (while not recommended, its alluded to in one place).
Scons would only pick up the native tool-chain even after many ad hoc changes to the makery.
So, as a hack, I had to change the native tool-chain to point to the cross tool-chain.
/usr/bin/gcc -> /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/bin/arm-linux-gnueabihf-gcc-4.7.3*
/usr/bin/g++ -> /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/bin/arm-linux-gnueabihf-g++*
The first compile-break for ARM was fixed by adding the line below to the default portion of the build script, SConscript.boot.
env.Append(CPPDEFINES = ["_LITTLE_ENDIAN"])
The next compile-break has to do with the OpenSSL header files not being found. To fix the location issue, I had to introduce the line below into SConscript.boot
vars.Add(PackageVariable("openssl", "OpenSSL location", "/home/auro-tripathy/swiftim/swift/3rdParty/OpenSSL/openssl-1.0.1c/"))
Linking with OpenSSL
For the sample Switften programs to link with the OpenSSL libraries, I had to move libssl.a and libcrypto.a (built separately) from the location they were built to the toolchain library-location like so.
mv ~/swiftim/swift/3rdParty/OpenSSL/openssl-1.0.1c/libcrypto.a /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/lib/gcc/arm-linux-gnueabihf/4.7.3/.
Help
Not understanding of the working of scons, I've made some hacks to get it to work.
I’d like some help to:
Introduce a new target called ARM-embedded, just like other targets; iPhone, android, etc
Clean way to integrate OpenSSL into the build .
Update
Per dirkbaechle, retried the script below and it works
export CC=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/gcc
export CXX=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/g++
./scons Swiften

Brady's answer is correct, regarding how you'd do it in plain SCons. I'd just like to mention that the top-level SConstruct of Swiften already provides arguments like "cc=" and "cxx=" for using local toolchains.
You might want to inspect the ouput of scons -h for a complete list of available options.
In addition, the SConscript for the OpenSSL build expects the sources to be located in the relative folder named "openssl", not "openssl-1.0.1c" as in your case. Maybe that's where your build problems are mainly coming from.

I left a comment above regarding the cross-compilation. Its already been answered in the link provided, but basically you just need to set the appropriate construction variables: CC, CXX, LINK, etc.
As for a "Clean way to integrate OpenSSL into the build" this can be performed simply by adding library and include paths appropriately as follows replacing the quoted values appropriately:
(without having to copy/move the original files)
# This sets the location of the OpenSSL Include paths
env.Append(CPPPATH="path/to/openssl/includes")
# This sets the location of the OpenSSL Libraries
env.Append(LIBPATH="path/to/openssl/libraries")
# These are the OpenSSL libraries to be linked into the binary
env.Append(LIBS=["OpenSSL_lib", "OpenSSL_lib2"])

The choice of compiler, and additional flags, can all be set in Swift's config.py file. A snippet from config.py using a custom compiler and flags is below (the one I use on one of my dev boxes):
cc = link = "/usr/local/llvm-git/bin/clang"
cxx = "/usr/local/llvm-git/bin/clang++"
bothflags = " -std=c++11 -stdlib=libc++ -nostdinc++"
cxxflags = bothflags + " -I/usr/local/libcxx/include -Wno-deprecated"
linkflags = bothflags + " -L/usr/local/libcxx/lib"
This should work for cross-compiling in the same manner.
To use a bundled openssl, you should just be able to extract into 3rdParty/OpenSSL, and add openssl_force_bundled = True to your config.py. You should not need to fiddle with setting include paths to this yourself. It's conceivable that this is tied to a particular openssl release as I've not compiled a bundled openssl since 1.0.0a, but if it doesn't work with the current version it's probably a bug that ought to be fixed. You could also cross-compile openssl yourself and use openssl='/path/to/openssl', but that's a little more of a nuisance for you.