Develop Reference

Unable to get gradle build-tools in docker - Apple M1

I recently switched to Apple M1 and am having a problem creating a docker image that eventually runs on a Buildkite linux CI. The same code runs just fine on my MacBook with an Intel chip and successfully creates a docker image.
The problem happens when sdkmanager tries to pull in build-tools;${ANDROID_BUILD_TOOLS_VERSION} with the latest commandlinetools, it fails with the following errors:
Warning: Dependant package with key emulator not found!
Warning: Unable to compute a complete list of dependencies.ates...
The closest issues I can find are Install build-tools: emulator not found and Error with android sdk, both without any resolutions. Also note that I have run sdkmanager --list, and emulator is not present as an available package there (just on M1).
Here is my Dockerfile (I don't work with docker too often so please excuse if the code is not the cleanest):
FROM gradle:7.4-jdk11
ENV SDK_URL="https://dl.google.com/android/repository/commandlinetools-linux-8092744_latest.zip" \
ANDROID_HOME="/usr/local/android-sdk" \
ANDROID_VERSION=32 \
ANDROID_BUILD_TOOLS_VERSION=32.0.0
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_HOME --licenses \
&& $ANDROID_HOME/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_HOME --update \
&& $ANDROID_HOME/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_HOME "platform-tools" "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
&& $ANDROID_HOME/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_HOME "platforms;android-${ANDROID_VERSION}" \
&& apt-get update \
&& apt-get install -y build-essential file apt-utils curl gnupg \
&& curl -sL https://deb.nodesource.com/setup_14.x | bash - \
&& apt-get -y install nodejs \
&& npm install -g firebase-tools -y \
&& rm -rf /var/lib/apt/lists/*
Side note; I had to upgrade usage of jdk8 to jdk11 for the Android build agent, the previous implementation was pulling in sdk-tools-linux-3859397.zip instead of commandlinetools-linux-8092744_latest.zip, and that was able to pull in build-tools via the sdkmanager just fine on the M1 as well and created a docker image.
Given that it builds on Intel, my task is technically complete, but it's going to be much easier in the long run that it runs on M1. Any ideas? Or can anyone suggest what the right place would be to raise this? Do you reckon it is a google command line tool issue or docker issue?

I just had the same problem on M1 Pro and this solved the issue for me.
In short: emulator is not available on Arm (M1) yet so you need to install it manually. You can follow the official guide but in short, this worked for me
Download emulator for Apple Silicon (example or you find more links in the [official guide](https://developer.android.com/studio/emulator_archive
Unzip the folder in Android SDK folder (it will create a SKD/emulator folder with files inside)
Create SDK/emulator/package.xml file with content from here and update the last part of the xml <revision><major>31</major><minor>1</minor><micro>4</micro></revision> with the version you downloaded
You should not see the same error again on M1 🎉
Spoiler Alert: I'm still not able to fully build on M1 because, during the actual build, I get the error
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

I have same issue. I think there is no android 'emulator' package for linux/armv8 . So you won't be able to use docker with native images.
I tried to configure docker-compose file in order to use linux/x86_64 instead of linux/armv8 but there are others issues.
I add this line in docker-compose.yaml :
platform: linux/x86_64
like this :
version: '3.7'
services:
android:
platform: linux/x86_64
build:
dockerfile: Dockerfile
context: ./Docker
working_dir: /app/Android/Scripts
volumes:
- ../:/app:rw,cached
command: ./build_distribution.sh
# command: sh -c "while true; do sleep 10; done"
But I get a lot of issue with network. It seems that network layer on this architecture is buggy and sometimes it is stuck downloading an image or a component randomly...
Is there a way to make it work with one or other architecture ?

Each time I execute docker run ./gradlew command, it downloads all the gradle files and dependencies. How to download and install only once?

I have a simple dockerfile that configured to build android apps
It's all working nicely, but the problem is that each time i run the ./gradlew command, it downloads and installs all the Gradle artifacts and dependencies. How can i just install once?
this is the dockerfile:
FROM openjdk:8
ENV SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip" \
ANDROID_HOME="/usr/local/android-sdk" \
ANDROID_VERSION=26 \
ANDROID_BUILD_TOOLS_VERSION=26.0.2
# Download Android SDK
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
# Install Android Build Tool and Libraries
RUN $ANDROID_HOME/tools/bin/sdkmanager --update
RUN $ANDROID_HOME/tools/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
"platforms;android-${ANDROID_VERSION}" \
"platform-tools"
RUN mkdir /application
WORKDIR /application
and this is the command :
docker run -it --rm -v "$PWD":/application packsdkandroiddocker.image bash ./gradlew assembleRelease --debug

Based on your Dockerfile, your gradle's cache directory is /root/.gradle. So, you need to mount a volume to the cache directory to persist cache.
Use host volumes
docker run -it --rm -v $PWD/.gradle:/root/.gradle -v $PWD:/application packsdkandroiddocker.image bash ./gradlew assembleDebug
Use named volumes
docker volume create gradle-cache
docker run -it --rm -v gradle-cache:/root/.gradle -v $PWD:/application packsdkandroiddocker.image bash ./gradlew assembleDebug

The problem, as you've no doubt discovered, is that enumerating all possible dependencies of all Gradle tasks and downloading them is...non-trivial.
It's not the most elegant solution, but I've solved this problem previously (in the context of GitLab CI Docker runners) by mounting a directory on the host to be the GRADLE_USER_HOME directory. This way, the image itself is fresh every run, but the caches are shared across runs. It does mean that you'll still have an initial slow run per host, but it dramatically reduces the time taken for subsequent runs.
Depending on exactly what you're trying to do, this may or may not be a satisfactory solution.

Android SDK on docker

Im trying to run android emulator on docker but I get the following error, I have done some research but none of the answers helped me in achieving my goal, here is the error that I get and my docker file, I have tried armeabi and system-images;android-29;google_apis;x86 and both did not work.
# emulator -avd device
emulator: WARNING: encryption is off
emulator: INFO: QtLogger.cpp:68: Warning: could not connect to display ((null):0, (null))
emulator: INFO: QtLogger.cpp:68: Info: Could not load the Qt platform plugin "xcb" in "/opt/android-sdk/emulator/lib64/qt/plugins" even though it was found. ((null):0, (null))
Fatal: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: xcb.
((null):0, (null))
emulator: INFO: QtLogger.cpp:68: Fatal: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: xcb.
((null):0, (null))
Aborted
Im doing the same steps on my MAC and it's working fine, here is my docker file:
FROM ubuntu:18.04
RUN dpkg --add-architecture i386 && \
apt-get update -y && \
apt-get install -y --no-install-recommends libncurses5:i386 libc6:i386 libstdc++6:i386 lib32gcc1 lib32ncurses5 lib32z1 zlib1g:i386 && \
apt-get install -y --no-install-recommends openjdk-8-jdk && \
apt-get install -y --no-install-recommends git wget unzip && \
apt-get install -y --no-install-recommends qt5-default
ARG GRADLE_VERSION=5.2.1
ARG GRADLE_DIST=bin
RUN cd /opt && \
wget -q https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-${GRADLE_DIST}.zip && \
unzip gradle*.zip && \
ls -d */ | sed 's/\/*$//g' | xargs -I{} mv {} gradle && \
rm gradle*.zip
# download and install Android SDK
# https://developer.android.com/studio/#downloads
ARG ANDROID_SDK_VERSION=4333796
ENV ANDROID_HOME /opt/android-sdk
RUN mkdir -p ${ANDROID_HOME} && cd ${ANDROID_HOME} && \
wget -q https://dl.google.com/android/repository/sdk-tools-linux-${ANDROID_SDK_VERSION}.zip && \
unzip *tools*linux*.zip && \
rm *tools*linux*.zip
# set the environment variables
ENV APK_SIGNER=/opt/android-sdk/build-tools/29.0.2/
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV ANDROID_SDK_ROOT /opt/android-sdk
ENV GRADLE_HOME /opt/gradle
ENV PATH ${PATH}:${GRADLE_HOME}/bin:${ANDROID_HOME}/emulator:${ANDROID_HOME}/tools/bin:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools:${ANDROID_HOME}/tools/bin/apksigner:${APK_SIGNER}
ENV _JAVA_OPTIONS -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
# WORKAROUND: for issue https://issuetracker.google.com/issues/37137213
ENV LD_LIBRARY_PATH ${ANDROID_HOME}/emulator/lib64:${ANDROID_HOME}/emulator/lib64/qt/lib
# accept the license agreements of the SDK components
ADD docker/license_accepter.sh /opt/
RUN chmod +x /opt/license_accepter.sh && /opt/license_accepter.sh $ANDROID_HOME
# setup adb server
EXPOSE 5037
# install and configure SSH server
EXPOSE 22
ADD docker/sshd-banner /etc/ssh/
ADD docker/authorized_keys /tmp/
RUN apt-get update -y && \
apt-get install -y --no-install-recommends openssh-server supervisor locales && \
mkdir -p /var/run/sshd /var/log/supervisord && \
locale-gen en en_US en_US.UTF-8 && \
apt-get remove -y locales && apt-get autoremove -y && \
FILE_SSHD_CONFIG="/etc/ssh/sshd_config" && \
echo "\nBanner /etc/ssh/sshd-banner" >> $FILE_SSHD_CONFIG && \
echo "\nPermitUserEnvironment=yes" >> $FILE_SSHD_CONFIG && \
ssh-keygen -q -N "" -f /root/.ssh/id_rsa && \
FILE_SSH_ENV="/root/.ssh/environment" && \
touch $FILE_SSH_ENV && chmod 600 $FILE_SSH_ENV && \
printenv | grep "JAVA_HOME\|GRADLE_HOME\|ANDROID_HOME\|LD_LIBRARY_PATH\|PATH" >> $FILE_SSH_ENV && \
FILE_AUTH_KEYS="/root/.ssh/authorized_keys" && \
touch $FILE_AUTH_KEYS && chmod 600 $FILE_AUTH_KEYS && \
for file in /tmp/*.pub; \
do if [ -f "$file" ]; then echo "\n" >> $FILE_AUTH_KEYS && cat $file >> $FILE_AUTH_KEYS && echo "\n" >> $FILE_AUTH_KEYS; fi; \
done && \
(rm /tmp/*.pub 2> /dev/null || true)
ADD docker/supervisord.conf /etc/supervisor/conf.d/
ADD docker/supervisord.conf /etc/supervisor/conf.d/
ADD docker/android.keystore /Android-APK/
ADD app/build/outputs/apk/release/ /Android-APK/
RUN sdkmanager "platform-tools" "platforms;android-29" "build-tools;29.0.2" "add-ons;addon-google_apis-google-24"
RUN apksigner sign --ks /Android-APK/android.keystore --ks-pass pass:android --key-pass pass:android --in /Android-APK/app-release-unsigned.apk --out /Android-APK/signed-app.apk
RUN zipalign -f 4 /Android-APK/signed-app.apk /Android-APK/aligned-app.apk
#RUN sdkmanager "system-images;android-23;google_apis;x86"
#RUN adb install /Android-APK/aligned-app.apk
#RUN sdkmanager "system-images;android-25;google_apis;arm64-v8a"
#RUN avdmanager create avd --force --name "device" --abi "arm64-v8a" --package 'system-images;android-25;google_apis;arm64-v8a' --device "Nexus 10"
CMD ["/usr/bin/supervisord"]

I had a similar problem, and it ended up being that our script was running the incorrect version of the emulator (26, which doesn't have good support for headless yet)
It should be running
$ANDROID_HOME/emulator/emulator -avd avd-name -no-skin -no-audio -no-window
and not the emulator in the tools folder.
Check the version of the emulator by just running the executable. My output (that works) is
emulator: Android emulator version 30.4.5.0 (build_id 7140946) (CL:N/A)
emulator: ERROR: No AVD specified. Use '#foo' or '-avd foo' to launch a virtual device named 'foo'

first run
xhost +
then run your docker container like this
docker run -i -t --net=host --privileged --env="DISPLAY" \
-v $HOME/.Xauthority:/root/.Xauthority:rw \
-v ~/disk1/android_docker:/home/android \
android-image /bin/bash
then you can run
emulator

How to create and start the emulator using Jenkins

I am new to the docker and Jenkins. First i create the docker image file. In that image install sdk and start the emulator. when i am start the emulator i am getting a following error.
Here is my code
FROM openjdk:8
# Install Git and dependencies
RUN dpkg --add-architecture i386 \
&& apt-get update \
&& apt-get install -y file git curl zip libncurses5:i386 libstdc++6:i386 zlib1g:i386 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists /var/cache/apt
# Set up environment variables
ENV ANDROID_HOME="/opt/android-sdk" \
SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip" \
GRADLE_URL="https://services.gradle.org/distributions/gradle-4.5.1-all.zip"
# Create a non-root user
RUN chmod 777 -Rf /opt
# Download Android SDK
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
RUN chmod 777 -Rf /opt/*
# Install Gradle
RUN wget $GRADLE_URL -O gradle.zip \
&& unzip gradle.zip \
&& mv gradle-4.5.1 gradle \
&& rm gradle.zip \
&& mkdir .gradle
ENV PATH="/home/user/gradle/bin:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools:${PATH}"
RUN echo y | android update sdk --no-ui
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-19.0.3
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-25.0.3
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-26.0.2
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-27.0.3
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-28.0.1
RUN cd /opt/android-sdk/ && mkdir platforms
RUN cd /opt/android-sdk/tools/bin && ./sdkmanager "system-images;android-23;google_apis;x86" && echo y | ./avdmanager create avd -n test -k "system-images;android-23;google_apis;x86" -b x86 -c 100M -d 7 -f
Here is my error.

Fail to execute command using Docker

I have a docker container, which purpose is to build android NDK project.
This container is build by this tutorial http://ainoya.io/docker-android-walter. Here is the Docker file:
# based on https://registry.hub.docker.com/u/samtstern/android-sdk/dockerfile/ with openjdk-8
FROM java:8
MAINTAINER Lebedevsd <lebedevsd#gmail.com>
ENV DEBIAN_FRONTEND noninteractive
# Install dependencies
RUN dpkg --add-architecture i386 && \
apt-get update && \
apt-get install -yq libstdc++6:i386 zlib1g:i386 libncurses5:i386 --no-install-recommends && \
apt-get install zip && \
apt-get -y install --reinstall locales && \
apt-get clean
# Download and setup android NDK
ENV ANDROID_NDK_URL http://dl.google.com/android/repository/android-ndk-r12b-linux-x86_64.zip
RUN cd /usr/local && \
curl -L "${ANDROID_NDK_URL}" > file.zip && \
unzip -q file.zip && \
rm file.zip
ENV ANDROID_NDK_HOME /usr/local/android-ndk-r12b
ENV PATH ${ANDROID_NDK_HOME}/:$PATH
# Download and untar SDK
ENV ANDROID_SDK_URL http://dl.google.com/android/android-sdk_r24.4.1-linux.tgz
RUN curl -L "${ANDROID_SDK_URL}" | tar --no-same-owner -xz -C /usr/local
ENV ANDROID_HOME /usr/local/android-sdk-linux
ENV ANDROID_SDK /usr/local/android-sdk-linux
ENV PATH ${ANDROID_HOME}/tools:$ANDROID_HOME/platform-tools:$PATH
# Install Android SDK components
ONBUILD COPY android_sdk_components.env /android_sdk_components.env
ONBUILD RUN (while :; do echo 'y'; sleep 3; done) | android update sdk --no-ui --all --filter "$(cat /android_sdk_components.env)"
# Support Gradle
ENV TERM dumb
ENV JAVA_OPTS -Xms512m -Xmx768m
The output is
docker run -t -v $(pwd)/app:/project/app lebedevsd/android_sdk_build:latest ./gradlew clean assemblePRelease -PversionCode=24 -PversionName=0.0.24
docker: Error response from daemon: oci runtime error: exec: "./gradlew": stat ./gradlew: no such file or directory.

The problem was that I was mounting wrong directory: $(pwd)/app instead of $(pwd).
the right way is:
docker run -i -v $(pwd):/project -w /project lebedevsd/android_sdk_build:latest ./gradlew clean