I'm trying to port Android SDK Tools to unsupported system (OpenBSD for that matter).
I've already fixed few things in go scripts for 'lunch' to work.
Now I'm at the point where I need to provide binaries to actually build SDK for current machine.
Some of the tools are provided inside AOSP.
There are bunch of prebuilt executables and libraries inside prebuilts/build-tools/ in linux-x86 and darwin-x86 subdirs for specific platforms.
Question is, how did they get there?
There are aidl, m4, ninja, soong_zip, zipalign, etc.
I could link some of them from base system but for some of them (soong_zip, for example) I didn't find any information how to obtain them.
Is there some kind of google repo where one can build them from?
Any information that could be relevant to the topic are also will be welcomed.
Related
Note - I am relatively new to Android and AOSP...
Where can I find the Android SDK in the AOSP? As we know, we build apk files in the android studio using the separately installed SDK. But for some reason, I wish to use the SDK in the AOSP if it is available. I see the source code of SDK in the AOSP; do we need to build it?
Why/How I have arrived at the above question:
I had created an .apk in the AOSP which had JNI files and dependency on some existing Broadcom libs (so). It is a system app. What I noticed is the .apk package does not contain the JNI libs but rather is copied to /system/lib folder separately. Hence I had a doubt how the .apk upgrade will work? Is it possible?
I assumed the .apk upgrade won't work that way and the .apk should be packaged including the JNI libs. Hence I planned to build the .apk in the studio and use the .apk as prebuilt and just sign it in the AOSP (we have the keys). Then I also wanted to allow the developers to build the .apk in AOSP itself by running the Gradle in the command line. I did so by adding the command to the Android.mk. But the point here is, it is still using the SDK installed in /home//Android/Sdk. But there may be build machines which may not have the SDK installed. So I am putting this question - Do we (where?) have the SDK in AOSP? Can I use that instead?
I tried to explain the problem.. in case it's not clear please let me know... Will try to give more details...
OTA update will work. Update with Package Manager - won't. This is normal for system apps with native libraries.
In AOSP applications are built differently depending on their location in build tree. Apps placed in ~/packages/apps and ~/device/some_vendor are system apps and they are handled differently by the system. One of differences is that during build process they are stripped of their native libraries and those libraries are simply copied to the /system/ partition.
Including pre-built apk is a good solution.
Yes you can build sdk yourself from sources. Yes, it's there. But I don't understand why you need that. Are there any changes to the API?
There are other options. For example, you can mangle your build scripts. Say, you can add a global FLAG that would disable lib stripping for system apps.
Is there a reference like this (http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html) that explains the folders inside the main Android SDK folder?
Based on the response I've got so far, I think I need to clarify further what I'm looking for.
I wanted to know the basis for the SDK to be divided into these folders (screenshot below). For e.g. something similar to the following excerpt in the 'SDK Readme.txt'
"Platform-tools contains build tools that are periodically updated to support new features in the Android platform (which is why they are separate from basic SDK tools), including adb, dexdump, and others."
I did put copy the android-sdk folder at a location separate from IDEs so that it can be shared.
It's not a guaranteed standard, but the folders have been consistent for many years now. Here are the core basics, see the links for more info:
# platform-tools: Updated each AOSP version; always backwards compatible
# tools/bin: Platform-independent tools to help building/using android
# eg lint, sdkmanager, monkeyrunner
# build-tools/<version>: Compilation tools, required to build.
# AOSP version specific, typically used by AS directly
See https://developer.android.com/studio/command-line/
https://www.programering.com/a/MDMyMzNwATk.html
No, this is proprietary project from Google, so they don't need to follow any standard.
After searching a lot and reading a lot of information, I cannot decide which tool I should use for compiling my code. My codebase is mainly c++. I use primarily linux as my development machine.
Based on opinions I read before, my final candidates are waf and cmake, but I cannot decide myself which one should be more appropiate.
My primary requirements are:
Must be able to compile software in windows/linux and android.
Must be ready to run tests.
Must be able to play nicely with other libraries that must be compiled with another build system but most likely will have to be compiled from source.
Must be able to add custom steps, like for example, generating some data from some files (mainly graphics) before compiling, all integrated in the build system.
Some strong preferences are:
Being ready to support MAC compilation.
Being able to cross-compile from linux as many platforms as I can (maybe windows/linux/android but cannot MAC?)
Being able to add support for iOS compilation if the need arises.
Would be nice if the invocation interface was similar to that of autotools, since it is the one many people know and it is well documented.
Some questions:
If I have some rare requirement, which build system would be more ready to be extended?
Are both currently well maintained? (I wonder about waf mainly).
Community: if I find a problem, both communities are big enough to support me, in your experience?
For now my feeling is that I favour waf a bit as a tool, but cmake seems to have been quite successful for whatever reason.
Don't know much about waf, but CMake fits your requirements pretty well. I do know waf is written in Python, my personal favourite programming language ATM.
My primary requirements are:
Must be able to compile software in windows/linux and android.
CMake does Windows and Linux very well but so does any other build system worth its salt.
Someone wrote some Android scripts for CMake. Can't find anything similar for waf (my Google-fu turns up nothing.)
Must be ready to run tests.
CMake has a sibling testing framework.
Must be able to play nicely with other libraries that must be compiled with another build system but most likely will have to be compiled from source.
CMake has good integration with pkg-config, and can link against arbitrary shared libraries.
Must be able to add custom steps, like for example, generating some data from some files (mainly graphics) before compiling, all integrated in the build system.
CMake can generate custom rules.
Some strong preferences are:
Being ready to support MAC compilation.
CMake supports Mac quite well. It will even make you an Xcode project if you want, but it can also do command line builds.
Being able to cross-compile from linux as many platforms as I can (maybe windows/linux/android but cannot MAC?)
Cross-compiling is supported in CMake. CMake will not be the primary source of pain with cross-compiling - literally everything else will.
Especially with regards to cross-compiling for Mac. It's possible, but not worth it to cross-compile for that platform, considering you need access to a Mac anyways to get the libraries and header files, you need to patch GCC and clang and LLVM, etc. The only sound reason I've heard for going through this much pain is running an automated build server. Anyways, if you get a working Linux -> Mac toolchain, you should be able to cross-compile with CMake as if it were any other Unix platform.
Being able to add support for iOS compilation if the need arises.
iOS cross-compilation can be done, but you need a Mac.
Would be nice if the invocation interface was similar to that of autotools, since it is the one many people know and it is well documented.
Write a configure script that just calls CMake (cmake .). Then your users can do a ./configure && make && make install on platforms where that makes sense. There's also CPack which lets you generate DEB, RPM, NSIS (Windows) and DMG (Mac) installers/packages.
Some questions:
If I have some rare requirement, which build system would be more ready to be extended?
CMake is very extensible. It can be extended to support new languages and target platforms. (Given that waf is written in Python, it's going to be pretty hackable too.)
Are both currently well maintained? (I wonder about waf mainly).
CMake is mature and well-maintained.
Community: if I find a problem, both communities are big enough to support me, in your experience?
The community and extensions available are what keeps me coming back to CMake, from things like bakefile, honestly.
WAF
is pure Python
becomes part of your project, i.e. no external dependency
supports many build tools
can be used to do all kind of automations, not just building
It works perfectly for Linux, Mac or Windows.
On Android, gradle is the chosen build tool of Google. To use that is
wise, because it is set up to work by Google. You can call waf from
gradle and vice-versa, though.
If you want to learn all the low level Android
SDK tools, you could also use
WAF directly.
The SDK has
javac for Android Runtime (formerly Dalvik), Android\'s JVM, and produces a .class file
jar can also be used for Android
d8 (formerly dx) produces .dex files, with Dalvik executable code
aapt2 can then produce the .apk
javac and jar are known to WAF. For dx and aapt2 you would need
to create your own tasks, which is very
easy.
You would best make a WAF tool and
share it. Tools are either part of WAF or there is
waftools.
There are also these Steinwurf
tools.
If you make Android native code using
NDK:
you use CLANG, which is known to WAF
Further on you mentioned requirements:
WAF has waf_unit_test
WAF can do gnu_cross compilation. The Gnu toolchain knowns many
targets. But for Android you would need to set things up yourself
using the SDK or NDK. For NDK you could use the Gnu toolchain.
You would do waf configure, waf build instead of configure,
make, but you could wrap a Configure or Makefile around waf to
have the same commands.
WAF is very easily extendible with Python
WAF is now on gitlab and
constantly worked on.
The community is surely smaller than for CMake. But it is Python.
You can look into it and find out for yourself. You can also
contribute and become part of the community.
Is there some sort of alternative toolchain or language for Android, which can generate standalone APK files?
Ideally it should not depend on the huge and ever-changing, ever-upgraded official Android SDK.
As a parable, I am looking for a rough equivalent to how PowerBASIC and Mingw targets plain Windows just fine, despite Microsoft releasing new Visual Studios all the time.
Bonus points if this language or toolchain itself is an Android program...
As you may or may not be aware, the Android toolchain is based on a few simple ideas:
Your code is compiled using the plain old java compiler, and linked against the Android stubs (android.jar) for linkage against the system library.
After being compiled, the code is converted to dex format. You can actually run this yourself, just do a dx --help. The job of the dx tool is to take Java class files and convert them to dex code, a pretty straightforward compilation which involves going from a stack based to register based vm, and a few other changes.
After having this in place, an apk is built using a set of apk tools. It used to be apkbuilder, but this has since been deprecated. You can actually run this yourself as well. All an APK is is simply a collection of the manifest, resources, and a single file for all the code in dex form. (I.e., many .class files compile to a single .dex which is quite a bit smaller because of a wrapped web of pointers).
So the Android toolchain isn't really all that complex. The custom build process is handled by ant build rules, which are defined in an SDK wide build.xml, which you can find in the platform-tools/ directory (iirc). However, to generate new baseline projects using this custom build environment you simply use the android update project command.
While I'm not sure if this is the response you'd hoped for, I hope it will disambiguate the build process. It's not really all that complex of a toolchain, the majority of it is off the shelf Java, and not Android specific (all that makes it Android specific is library specific stubs for dynamically linked system code). Beyond this, once you have a set of classes, you need only run a few commands to make an executable APK which Android unpacks. I would suspect that any tool targeting the JVM (and capable of linking with the Android specific dynamically linked API) could perform a similar process of producing class files and using this toolchain to compile the rest of the way, though obviously the automated ant build process makes it much simpler.
EDIT:
After some more digging, I found this relevant android-developers thread. An unsettling quote:
At this time we simply don't have the resources to support people who
want to use their own build system, but we really wish we could. In
many ways we try to make it easy on other tools vendor by clearly
separating logic to eclipse or ant specific code (hence the multitude
of jar files everywhere in the tools and in ADT), but this is not one
of them.
However, you may also find this link helpful.
Terminal-IDE and AIDE are pretty much what I was looking for. Both runs on Android.
I want to use some unix tools on my rooted android arm6 based phone. I will be using cross compiler tools provided here. If I want to compile gnu netcat, how can I set the cross compiler prefix to arm-none-linux-gnueabi- and how to enable static linking (no shared library).
I managed to cross-compile rsync for Android using Ubuntu's arm-linux-gnueabi toolchain. See this related question.
Unless you particularly need to build against a more standard libc than bionic, you can just use the ndk's toolchain, either by copying the hello-jni example and changing BUILD_SHARED_LIBRARY to BUILD_EXECUTABLE in the jni/Android.mk or using the script to generate a stand alone toolchain. You may want to use the V=1 option to the ndk-build script to see the commands it's issuing to its gcc.
Otherwise you may need to pass the prefix to the configure script or manually edit it into the Makefile for the project. This often has not gone well as many projects have make systems not really set up for cross compiling, I've had to resort to editing the configure script to set prefixes and skip tests where it tries to execute a test program.
An option that sometimes works when the build system is more complicated than the project requires is to do a configure for your host (let's hope that's linux). Then manually edit the generated Makefile to change anything needed to build for android instead. Might not be a bad idea to do a clean just in case (especially if you did a test host build). And then do the build which will pick up the arm compiler from your Makefile modifications.
Lastly, if you can be content with the original netcat by Hobbit rather than the gnu version, you hardly need to port it to android yourself as that's already been done. There's already an android version in the google tree at https://android.googlesource.com/platform/external/netcat
which may be on your device already (as 'nc'), and is definitely included in alternate ROMs such as Cyanogenmod.