I recently switched to Apple M1 and am having a problem creating a docker image that eventually runs on a Buildkite linux CI. The same code runs just fine on my MacBook with an Intel chip and successfully creates a docker image.
The problem happens when sdkmanager tries to pull in build-tools;${ANDROID_BUILD_TOOLS_VERSION} with the latest commandlinetools, it fails with the following errors:
Warning: Dependant package with key emulator not found!
Warning: Unable to compute a complete list of dependencies.ates...
The closest issues I can find are Install build-tools: emulator not found and Error with android sdk, both without any resolutions. Also note that I have run sdkmanager --list, and emulator is not present as an available package there (just on M1).
Here is my Dockerfile (I don't work with docker too often so please excuse if the code is not the cleanest):
FROM gradle:7.4-jdk11
ENV SDK_URL="https://dl.google.com/android/repository/commandlinetools-linux-8092744_latest.zip" \
ANDROID_HOME="/usr/local/android-sdk" \
ANDROID_VERSION=32 \
ANDROID_BUILD_TOOLS_VERSION=32.0.0
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_HOME --licenses \
&& $ANDROID_HOME/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_HOME --update \
&& $ANDROID_HOME/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_HOME "platform-tools" "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
&& $ANDROID_HOME/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_HOME "platforms;android-${ANDROID_VERSION}" \
&& apt-get update \
&& apt-get install -y build-essential file apt-utils curl gnupg \
&& curl -sL https://deb.nodesource.com/setup_14.x | bash - \
&& apt-get -y install nodejs \
&& npm install -g firebase-tools -y \
&& rm -rf /var/lib/apt/lists/*
Side note; I had to upgrade usage of jdk8 to jdk11 for the Android build agent, the previous implementation was pulling in sdk-tools-linux-3859397.zip instead of commandlinetools-linux-8092744_latest.zip, and that was able to pull in build-tools via the sdkmanager just fine on the M1 as well and created a docker image.
Given that it builds on Intel, my task is technically complete, but it's going to be much easier in the long run that it runs on M1. Any ideas? Or can anyone suggest what the right place would be to raise this? Do you reckon it is a google command line tool issue or docker issue?
I just had the same problem on M1 Pro and this solved the issue for me.
In short: emulator is not available on Arm (M1) yet so you need to install it manually. You can follow the official guide but in short, this worked for me
Download emulator for Apple Silicon (example or you find more links in the [official guide](https://developer.android.com/studio/emulator_archive
Unzip the folder in Android SDK folder (it will create a SKD/emulator folder with files inside)
Create SDK/emulator/package.xml file with content from here and update the last part of the xml <revision><major>31</major><minor>1</minor><micro>4</micro></revision> with the version you downloaded
You should not see the same error again on M1 🎉
Spoiler Alert: I'm still not able to fully build on M1 because, during the actual build, I get the error
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
I have same issue. I think there is no android 'emulator' package for linux/armv8 . So you won't be able to use docker with native images.
I tried to configure docker-compose file in order to use linux/x86_64 instead of linux/armv8 but there are others issues.
I add this line in docker-compose.yaml :
platform: linux/x86_64
like this :
version: '3.7'
services:
android:
platform: linux/x86_64
build:
dockerfile: Dockerfile
context: ./Docker
working_dir: /app/Android/Scripts
volumes:
- ../:/app:rw,cached
command: ./build_distribution.sh
# command: sh -c "while true; do sleep 10; done"
But I get a lot of issue with network. It seems that network layer on this architecture is buggy and sometimes it is stuck downloading an image or a component randomly...
Is there a way to make it work with one or other architecture ?
Related
We are building an app in React Native using wix/Detox for writing e2e tests.
We are using Docker for doing builds and running tests. Specifically, we're using community react-native-android Docker image.
I have added fastlane to the image for building / publishing our app and now trying to add an emulator so that we can run tests as well.
This is our Dockerfile:
# reactnativecommunity/react-native-android:4.0
FROM reactnativecommunity/react-native-android#sha256:c3ad9b8ed5caac0718b1c1b8f10469eb75b0ac77c86143c94f9616ee46b80b4b
RUN gem install fastlane -NV
RUN sdkmanager --install "system-images;android-31;google_apis;arm64-v8a"
RUN echo no | avdmanager create avd --name default -k "system-images;android-31;google_apis;arm64-v8a"
However, when I run the container and attempt to start the emulator, I get the following:
root#7b862aa149c8:/# emulator -no-audio -no-boot-anim -no-window -use-system-libs #default
emulator: Android emulator version 30.7.5.0 (build_id 7491168) (CL:N/A)
emulator: INFO: ignore sdcard for arm at api level >= 30
emulator: feeding guest with passive gps data, in headless mode
emulator: ERROR: AdbHostServer.cpp:102: Unable to connect to adb daemon on port: 5037
cannot add library /opt/android/emulator/qemu/linux-x86_64/lib64/vulkan/libvulkan.so: failed
added library /opt/android/emulator/lib64/vulkan/libvulkan.so
emulator: INFO: ignore sdcard for arm at api level >= 30
emulator: INFO: userspace-boot-properties.cpp:242: Sending adb public key [QAAAACG/f/Yfe1MON5LKVoRU+iuzZRCKQTvsqLWcWh8i5ii03oHLY7y7U+0uKSE5x84OrGS0g6G9HTU7Mazh18yhbmPAQociSCfrSRIWg3IAdduFDgYPnY2G3Lj11ZLQoyRt4+xXO7zVcUPtwERpgpYQXEN5KkkEZBUxxWvrejENPAffPp4DHFqJf63sAUPh9qo6cwfSMylnl7RTBicxZ+tuDwrxA1JgcvklAs38E9gzxjNDed+1VibNUjzCtRl4DnPPmIk1OqhvjU0xZV//YXDrdhb++jQOpcKAzkCvVks4RUjJ1okytnmDfh+YUH4thQYuqNdEBnGM4wz8cPQH0YjO1SsK2sOrD0VJaGjSDAwsjne2QCFy5ET+HOUcMNKCaC60MAcCgWXLm8MdoaVLgjoG0jbUEkr/BJ2hpN7/p+qi8qMSg3Vv2M/4kdHmIEzDpJTd8TFD1bbrRjdUIheDzE1b00SKhQzDJ39sQim31IqWGuRIJ4Cb39VaHCFK6ZwZPpG9KKLSV1ww5KCKVgSsJ5zaAvhagCjXXXVrtl40v72zYq/c0BtUG4LhLixtz/zuT1WFKq8HoGFkJBvw5fBVqcvOn6yN7xI3DAuzAlQcHPyRQdHm72+//KimCVHnkxlL3hpRMir06QhMk2DsKTntXjbYuhCCn0aetEUUpai6gl1NuCbv92QlDgEAAQA= #unknown]
emulator: INFO: GrpcServices.cpp:315: Started GRPC server at 127.0.0.1:8554, security: Local
WARNING. Using fallback path for the emulator registration directory.
emulator: INFO: EmulatorAdvertisement.cpp:93: Advertising in: /root/.android/avd/running/pid_9.ini
qemu-system-aarch64-headless: PCI bus not available for hda
saving arm snapshot.... !!!
saving done.... !!!
root#7b862aa149c8:/#
This is what worked in the end BUT required bare metal machine
# reactnativecommunity/react-native-android:4.0
FROM reactnativecommunity/react-native-android#sha256:c3ad9b8ed5caac0718b1c1b8f10469eb75b0ac77c86143c94f9616ee46b80b4b
ARG IMAGE="system-images;android-31;google_apis;x86_64"
RUN sdkmanager --install "${IMAGE}"
RUN echo no | avdmanager create avd --name emulator_for_e2e_testing -k "${IMAGE}"
ENTRYPOINT adb start-server && emulator -avd emulator_for_e2e_testing -no-audio -no-window -no-boot-anim
So,
Is it possible to run the (Android) emulator through Docker? yes.
I don't know much about Fastlane|wix/Detox, but I made this in a e2e/Appium project
If so, how?
Do you need a hardware that support virtualization
In Linux find out if CPU Support Intel VT/AMD-V Virtualization For KVM
lscpu | grep Virtualization
Virtualization: VT-x (if 'Intel VT-x' or 'VT-x' feature supported)
or
Virtualization: full (if 'full' not supported)
On AWS, you need a bare metal instance (supported x86) like c5.metal
Android Emulator Container Scripts
https://github.com/google/android-emulator-container-scripts
Follow this git, create an image with google API (I my case, 27-google-x86) and put in docker repo, like AWS ECR.
And, I created an image too to Appium (I believe with is possible with wix/detox)
To SDK
Dockerfile
FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
# Installs i386 architecture required for running 32 bit Android tools
RUN dpkg --add-architecture i386 && \
apt-get update && \
apt-get dist-upgrade -y && \
apt-get install -y --no-install-recommends openjdk-8-jdk && \
apt-get install -y --no-install-recommends git wget unzip curl make && \
rm -rf /var/lib/apt/lists/* && \
apt-get autoremove -y && \
apt-get clean
ENV JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64/jre" \
PATH=$PATH:$JAVA_HOME/bin
# Installs Android SDK
ENV ANDROID_HOME=/android-sdk
ENV ANDROID_SDK_HOME $ANDROID_HOME
ARG ANDROID_SDK_VERSION=6514223
ARG ANDROID_BUILD_TOOLS_VERSION=27.0.0
ARG ANDROID_PLATFORM_VERSION="android-27"
RUN mkdir -p ${ANDROID_SDK_HOME}/cmdline-tools && \
wget -q https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_VERSION}_latest.zip && \
unzip *tools*linux*.zip -d ${ANDROID_SDK_HOME}/cmdline-tools && \
rm *tools*linux*.zip
ENV PATH ${PATH}:${ANDROID_SDK_HOME}/tools:${ANDROID_SDK_HOME}/platform-tools
RUN mkdir -p ~/.android && \
touch ~/.android/repositories.cfg && \
echo y | ${ANDROID_SDK_HOME}/cmdline-tools/tools/bin/sdkmanager --licenses && \
echo y | ${ANDROID_SDK_HOME}/cmdline-tools/tools/bin/sdkmanager "platform-tools" && \
echo y | ${ANDROID_SDK_HOME}/cmdline-tools/tools/bin/sdkmanager "build-tools;$ANDROID_BUILD_TOOLS_VERSION" && \
echo y | ${ANDROID_SDK_HOME}/cmdline-tools/tools/bin/sdkmanager "platforms;$ANDROID_PLATFORM_VERSION"
# Installs Node
ARG NODE_VERSION=12.x
RUN curl -sL https://deb.nodesource.com/setup_${NODE_VERSION} | bash
RUN apt-get install --yes nodejs
Finally, I orchestrated everything a Jenkinsfile
I have a simple dockerfile that configured to build android apps
It's all working nicely, but the problem is that each time i run the ./gradlew command, it downloads and installs all the Gradle artifacts and dependencies. How can i just install once?
this is the dockerfile:
FROM openjdk:8
ENV SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip" \
ANDROID_HOME="/usr/local/android-sdk" \
ANDROID_VERSION=26 \
ANDROID_BUILD_TOOLS_VERSION=26.0.2
# Download Android SDK
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
# Install Android Build Tool and Libraries
RUN $ANDROID_HOME/tools/bin/sdkmanager --update
RUN $ANDROID_HOME/tools/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
"platforms;android-${ANDROID_VERSION}" \
"platform-tools"
RUN mkdir /application
WORKDIR /application
and this is the command :
docker run -it --rm -v "$PWD":/application packsdkandroiddocker.image bash ./gradlew assembleRelease --debug
Based on your Dockerfile, your gradle's cache directory is /root/.gradle. So, you need to mount a volume to the cache directory to persist cache.
Use host volumes
docker run -it --rm -v $PWD/.gradle:/root/.gradle -v $PWD:/application packsdkandroiddocker.image bash ./gradlew assembleDebug
Use named volumes
docker volume create gradle-cache
docker run -it --rm -v gradle-cache:/root/.gradle -v $PWD:/application packsdkandroiddocker.image bash ./gradlew assembleDebug
The problem, as you've no doubt discovered, is that enumerating all possible dependencies of all Gradle tasks and downloading them is...non-trivial.
It's not the most elegant solution, but I've solved this problem previously (in the context of GitLab CI Docker runners) by mounting a directory on the host to be the GRADLE_USER_HOME directory. This way, the image itself is fresh every run, but the caches are shared across runs. It does mean that you'll still have an initial slow run per host, but it dramatically reduces the time taken for subsequent runs.
Depending on exactly what you're trying to do, this may or may not be a satisfactory solution.
I want to use Make install command and I have installed all requirements like yasm, nasm, curl, ant, rsync and the autotools: autoconf, automake, aclocal, pkgconfig, libtool. (Exactly, I want to compile Linphone Android NDK from Here : https://github.com/BelledonneCommunications/linphone-android. I have follow all steps from there)
I have try to install libtoolize using this command:
brew install libtoolize
But terminal always show :
Error: No available formula for libtoolize
If i try to make install, terminal will show :
Could not find libtoolize. Please install libtool.
Anybody can help ?
You should install the package libtool via
brew install libtool
This package contains the tool libtoolize as you can check via
brew list libtool
Note the warning
In order to prevent conflicts with Apple's own libtool we have prepended a "g"
so, you have instead: glibtool and glibtoolize.
You may try again installing the tools you want to. If the come with a ./configure script, re-execute it to let it find glibtoolize. If this does not work, you may need to set the environment variable LIBTOOL to the version Homebrew installed:
export LIBTOOL=`which glibtool`
export LIBTOOLIZE=`which glibtoolize`
As a last resort, you may need to set a symbolic link from glibtoolize to libtoolize. You can do so via
ln -s `which glibtoolize` libtoolize
Then, add the directory with the link to the path by
export PATH=$(pwd):$PATH
Then, libtoolize should be found.
Maybe you should refer to this Linphone for android is not working/missing libraries. Autotools installation for mac as suggested a part of the step.
# Assume we want to install them below $HOME/local.
myprefix=$HOME/local
# Ensure the tools are accessible from PATH.
# It is advisable to set this also in ~/.profile, for development.
PATH=$myprefix/bin:$PATH
export PATH
# Do the following in a scratch directory.
wget http://ftp.gnu.org/gnu/m4/m4-1.4.14.tar.gz
wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.64.tar.gz
wget http://ftp.gnu.org/gnu/automake/automake-1.11.1.tar.gz
wget http://ftp.gnu.org/gnu/libtool/libtool-2.4.tar.gz
gzip -dc m4-1.4.14.tar.gz | tar xvf -
gzip -dc autoconf-2.64.tar.gz | tar xvf -
gzip -dc automake-1.11.1.tar.gz | tar xvf -
gzip -dc libtool-2.4.tar.gz | tar xvf -
cd m4-1.4.14
./configure -C --prefix=$myprefix && make && make install
cd ../autoconf-2.64
./configure -C --prefix=$myprefix && make && make install
cd ../automake-1.11.1
./configure -C --prefix=$myprefix && make && make install
cd ../libtool-2.4
./configure -C --prefix=$myprefix && make && make install
I am trying to make a docker image that I can use to build Android projects, using Shippable.
The problem is the android update sdk command, which gives the following error:
Installing Android SDK Tools, revision 24.2
Failed to rename directory /opt/android-sdk-linux/tools to /opt/android-sdk-linux/temp/ToolPackage.old01.
Failed to create directory /opt/android-sdk-linux/tools
I found somewhat of a solution here: https://stackoverflow.com/a/8839359/867099 but it's for Windows, and does not seem to fix the problem on linux. It appears that during the update command, the current directory is in use and therefore cannot be renamed.
My workaround sofar, using that workaroundsuggestion, is this:
RUN cp -r /opt/android-sdk-linux/tools /opt/android-sdk-linux/tools_copy
RUN cd /opt/android-sdk-linux/tools && echo 'y' | /opt/android-sdk-linux/tools_copy/android update sdk --no-ui -a --filter tools,platform-tools,build-tools-22.0.1,android-21,extra-android-support,extra-google-google_play_services --force
In order to automatically accept the license, I echo 'y' to the android command.
But I think the android command should also run in the correct directory, which is why I cd into it first.
But, it still fails. I'm rather stumped on how to fix this issue, so any help is appreciated.
------ UPDATE --------
I run the android sdk update command without the tools filter, and in the end, my gradle builds are successful. So I don't know for sure whether it's a problem to not update them...
This can be solved by combining all Android SDK commands in a single Dockerfile's RUN command. It has something to do with Docker's file system.
For a detailed explanation from a post on Issue Tracker for the Android Open Source Project:
The problem is caused when you run the SDK update in a Docker
container. Docker uses a special filesystem which works like a version
control system (e.g. git) and records all changes made to the
filesystem. The problem is that the SDK update uses a hardlink move
operation to move the 'tools' directory, which is not supported by the
underlying Docker filesystem.
The solution for this is to simply run all Android SDK commands in a
single 'RUN' command in Docker. That way, the hardlink move works as
normal, as long as you don't use the 'hardlink move' operation between
multiple RUN command (snapshots). The advantage of this is also that
your Docker layers will be smaller (compared to running multiple
seperate RUN commands).
Here is the line from the Dockerfile:
RUN wget https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz && \
tar xzf android-sdk_r24.4.1-linux.tgz && \
rm android-sdk_r24.4.1-linux.tgz && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter platform-tools,tools -a ) && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter extra-android-m2repository,extra-android-support,extra-google-google_play_services,extra-google-m2repository -a) && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter build-tools-23.0.2,android-24 -a)
This is what's currently working for me, you can see the update command successfully running below. In my environment these are 3 different docker images in one FROM hierarchy so you can likely combine a lot of the apt-gets if that's not your case.
FROM ubuntu:14.04
# Set debconf to run non-interactively
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
# Install base dependencies
RUN apt-get update && apt-get install -y -q --no-install-recommends \
apt-transport-https \
build-essential \
ca-certificates \
curl \
git \
libssl-dev \
python \
rsync \
software-properties-common \
wget \
&& rm -rf /var/lib/apt/lists/*
# Install the JDK
RUN echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections && \
add-apt-repository -y ppa:webupd8team/java && \
apt-get update -qq && \
DEBIAN_FRONTEND=noninteractive apt-get install -qqy --force-yes oracle-java7-installer && \
rm -rf /var/lib/apt/lists/* /var/cache/oracle-jdk7-installer
# Install Android Dev Tools
RUN apt-get update && apt-get install -y -q --no-install-recommends \
lib32ncurses5 \
lib32stdc++6 \
lib32z1 \
libswt-gtk-3-java \
unzip \
&& rm -rf /var/lib/apt/lists/*
RUN wget -qO- "http://dl.google.com/android/android-sdk_r23.0.2-linux.tgz" | tar -zxv -C /opt/
RUN cd /opt/android-sdk-linux/tools/ && \
echo y | ./android update sdk --all --filter platform-tools,build-tools-20.0.0,android-17,sysimg-17,system-image,extra-android-support --no-ui --force
ENV PATH /opt/android-sdk-linux/build-tools/20.0.0:$PATH
I feel as though I'm trying to learn android programming in the middle of a fireworks display during a rodeo. All the fancy IDE stuff recommended by all the books I seem to find is just monumentally distracting from discovering what I really and truly need just to develop an android app.
Can anyone point me at documentation for the minimal set of the tools needed to actually build an app? I feel like if I could understand what the heck was actually going on, I'd be better able to use the fancy IDE.
Primitive? So, not Eclipse, and also not ant. You can use aapt, javac, dx, apkbuilder, and signer directly. Or more-or-less directly; you're still a programmer, you have ways of dealing with repetition.
I do some on-device app development with Terminal IDE. This is one my build scripts (named 'make'):
P=me/rapacity/stitch
rm src/$P/R.java
mkdir -m 770 -p dist || exit
mkdir -m 770 -p build/classes || exit
mkdir -m 770 -p tmp || exit
./index.perl
aapt p -f -M AndroidManifest.xml -F build/resources.res \
-I ~/sdk/3.2/android.jar -S res/ -J src/$P || exit
cd src
for F in \
SelectActivity.java Tilesets.java \
StitchActivity.java \
TilesetView.java \
; do
if test $P/$F -nt ../build/classes/$P/$(dirname $F)/$(basename $F .java).class; then
echo Building $P/$F
REBUILD=1
javac -d ../build/classes $P/$F 2> ../tmp/javac.out
../redcat ../tmp/javac.out
grep error ../tmp/javac.out && exit
fi
done
cd ..
if [ ! -z $REBUILD ]; then
set -x
( cd src; javac -d ../build/classes $P/R.java )
( cd build/classes; dx --dex --verbose --no-strict --output=../core.dex me ) || # 'me' as in me.rapacity.
apkbuilder dist/core.apk -u -z build/resources.res -f build/core.dex || exit
signer dist/core.apk core-debug.apk
else
echo +++ No need to rebuild .apk
fi
in which some lengths are gone to to avoid recompilation and to promptly exit after an error. Very little of that needs to be edited per-project.
Java SE 6 (NOT Java7!)
Recommended IDE is Eclipse (recommended as there are guides for it on the official documentation)
Android SDK (you need to download the API you want to develop for) and ADT - the android development tools - Guide
Optional:
Device and a connection for it, recommended, not necessary as an Emulator is bundled in the Android SDK
Give AIDE a try for Android on-device App development. It is very easy to get started and the project format is fully compatible with Eclipse, so if you want to continue development on your PC you can.
Getting started video: http://www.youtube.com/watch?feature=player_embedded&v=NGT9MqT3W2w