Develop Reference

Each time I execute docker run ./gradlew command, it downloads all the gradle files and dependencies. How to download and install only once?

I have a simple dockerfile that configured to build android apps
It's all working nicely, but the problem is that each time i run the ./gradlew command, it downloads and installs all the Gradle artifacts and dependencies. How can i just install once?
this is the dockerfile:
FROM openjdk:8
ENV SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip" \
ANDROID_HOME="/usr/local/android-sdk" \
ANDROID_VERSION=26 \
ANDROID_BUILD_TOOLS_VERSION=26.0.2
# Download Android SDK
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
# Install Android Build Tool and Libraries
RUN $ANDROID_HOME/tools/bin/sdkmanager --update
RUN $ANDROID_HOME/tools/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
"platforms;android-${ANDROID_VERSION}" \
"platform-tools"
RUN mkdir /application
WORKDIR /application
and this is the command :
docker run -it --rm -v "$PWD":/application packsdkandroiddocker.image bash ./gradlew assembleRelease --debug

Based on your Dockerfile, your gradle's cache directory is /root/.gradle. So, you need to mount a volume to the cache directory to persist cache.
Use host volumes
docker run -it --rm -v $PWD/.gradle:/root/.gradle -v $PWD:/application packsdkandroiddocker.image bash ./gradlew assembleDebug
Use named volumes
docker volume create gradle-cache
docker run -it --rm -v gradle-cache:/root/.gradle -v $PWD:/application packsdkandroiddocker.image bash ./gradlew assembleDebug

The problem, as you've no doubt discovered, is that enumerating all possible dependencies of all Gradle tasks and downloading them is...non-trivial.
It's not the most elegant solution, but I've solved this problem previously (in the context of GitLab CI Docker runners) by mounting a directory on the host to be the GRADLE_USER_HOME directory. This way, the image itself is fresh every run, but the caches are shared across runs. It does mean that you'll still have an initial slow run per host, but it dramatically reduces the time taken for subsequent runs.
Depending on exactly what you're trying to do, this may or may not be a satisfactory solution.

Android SDK on docker

Im trying to run android emulator on docker but I get the following error, I have done some research but none of the answers helped me in achieving my goal, here is the error that I get and my docker file, I have tried armeabi and system-images;android-29;google_apis;x86 and both did not work.
# emulator -avd device
emulator: WARNING: encryption is off
emulator: INFO: QtLogger.cpp:68: Warning: could not connect to display ((null):0, (null))
emulator: INFO: QtLogger.cpp:68: Info: Could not load the Qt platform plugin "xcb" in "/opt/android-sdk/emulator/lib64/qt/plugins" even though it was found. ((null):0, (null))
Fatal: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: xcb.
((null):0, (null))
emulator: INFO: QtLogger.cpp:68: Fatal: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: xcb.
((null):0, (null))
Aborted
Im doing the same steps on my MAC and it's working fine, here is my docker file:
FROM ubuntu:18.04
RUN dpkg --add-architecture i386 && \
apt-get update -y && \
apt-get install -y --no-install-recommends libncurses5:i386 libc6:i386 libstdc++6:i386 lib32gcc1 lib32ncurses5 lib32z1 zlib1g:i386 && \
apt-get install -y --no-install-recommends openjdk-8-jdk && \
apt-get install -y --no-install-recommends git wget unzip && \
apt-get install -y --no-install-recommends qt5-default
ARG GRADLE_VERSION=5.2.1
ARG GRADLE_DIST=bin
RUN cd /opt && \
wget -q https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-${GRADLE_DIST}.zip && \
unzip gradle*.zip && \
ls -d */ | sed 's/\/*$//g' | xargs -I{} mv {} gradle && \
rm gradle*.zip
# download and install Android SDK
# https://developer.android.com/studio/#downloads
ARG ANDROID_SDK_VERSION=4333796
ENV ANDROID_HOME /opt/android-sdk
RUN mkdir -p ${ANDROID_HOME} && cd ${ANDROID_HOME} && \
wget -q https://dl.google.com/android/repository/sdk-tools-linux-${ANDROID_SDK_VERSION}.zip && \
unzip *tools*linux*.zip && \
rm *tools*linux*.zip
# set the environment variables
ENV APK_SIGNER=/opt/android-sdk/build-tools/29.0.2/
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV ANDROID_SDK_ROOT /opt/android-sdk
ENV GRADLE_HOME /opt/gradle
ENV PATH ${PATH}:${GRADLE_HOME}/bin:${ANDROID_HOME}/emulator:${ANDROID_HOME}/tools/bin:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools:${ANDROID_HOME}/tools/bin/apksigner:${APK_SIGNER}
ENV _JAVA_OPTIONS -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
# WORKAROUND: for issue https://issuetracker.google.com/issues/37137213
ENV LD_LIBRARY_PATH ${ANDROID_HOME}/emulator/lib64:${ANDROID_HOME}/emulator/lib64/qt/lib
# accept the license agreements of the SDK components
ADD docker/license_accepter.sh /opt/
RUN chmod +x /opt/license_accepter.sh && /opt/license_accepter.sh $ANDROID_HOME
# setup adb server
EXPOSE 5037
# install and configure SSH server
EXPOSE 22
ADD docker/sshd-banner /etc/ssh/
ADD docker/authorized_keys /tmp/
RUN apt-get update -y && \
apt-get install -y --no-install-recommends openssh-server supervisor locales && \
mkdir -p /var/run/sshd /var/log/supervisord && \
locale-gen en en_US en_US.UTF-8 && \
apt-get remove -y locales && apt-get autoremove -y && \
FILE_SSHD_CONFIG="/etc/ssh/sshd_config" && \
echo "\nBanner /etc/ssh/sshd-banner" >> $FILE_SSHD_CONFIG && \
echo "\nPermitUserEnvironment=yes" >> $FILE_SSHD_CONFIG && \
ssh-keygen -q -N "" -f /root/.ssh/id_rsa && \
FILE_SSH_ENV="/root/.ssh/environment" && \
touch $FILE_SSH_ENV && chmod 600 $FILE_SSH_ENV && \
printenv | grep "JAVA_HOME\|GRADLE_HOME\|ANDROID_HOME\|LD_LIBRARY_PATH\|PATH" >> $FILE_SSH_ENV && \
FILE_AUTH_KEYS="/root/.ssh/authorized_keys" && \
touch $FILE_AUTH_KEYS && chmod 600 $FILE_AUTH_KEYS && \
for file in /tmp/*.pub; \
do if [ -f "$file" ]; then echo "\n" >> $FILE_AUTH_KEYS && cat $file >> $FILE_AUTH_KEYS && echo "\n" >> $FILE_AUTH_KEYS; fi; \
done && \
(rm /tmp/*.pub 2> /dev/null || true)
ADD docker/supervisord.conf /etc/supervisor/conf.d/
ADD docker/supervisord.conf /etc/supervisor/conf.d/
ADD docker/android.keystore /Android-APK/
ADD app/build/outputs/apk/release/ /Android-APK/
RUN sdkmanager "platform-tools" "platforms;android-29" "build-tools;29.0.2" "add-ons;addon-google_apis-google-24"
RUN apksigner sign --ks /Android-APK/android.keystore --ks-pass pass:android --key-pass pass:android --in /Android-APK/app-release-unsigned.apk --out /Android-APK/signed-app.apk
RUN zipalign -f 4 /Android-APK/signed-app.apk /Android-APK/aligned-app.apk
#RUN sdkmanager "system-images;android-23;google_apis;x86"
#RUN adb install /Android-APK/aligned-app.apk
#RUN sdkmanager "system-images;android-25;google_apis;arm64-v8a"
#RUN avdmanager create avd --force --name "device" --abi "arm64-v8a" --package 'system-images;android-25;google_apis;arm64-v8a' --device "Nexus 10"
CMD ["/usr/bin/supervisord"]

I had a similar problem, and it ended up being that our script was running the incorrect version of the emulator (26, which doesn't have good support for headless yet)
It should be running
$ANDROID_HOME/emulator/emulator -avd avd-name -no-skin -no-audio -no-window
and not the emulator in the tools folder.
Check the version of the emulator by just running the executable. My output (that works) is
emulator: Android emulator version 30.4.5.0 (build_id 7140946) (CL:N/A)
emulator: ERROR: No AVD specified. Use '#foo' or '-avd foo' to launch a virtual device named 'foo'

first run
xhost +
then run your docker container like this
docker run -i -t --net=host --privileged --env="DISPLAY" \
-v $HOME/.Xauthority:/root/.Xauthority:rw \
-v ~/disk1/android_docker:/home/android \
android-image /bin/bash
then you can run
emulator

How to create and start the emulator using Jenkins

I am new to the docker and Jenkins. First i create the docker image file. In that image install sdk and start the emulator. when i am start the emulator i am getting a following error.
Here is my code
FROM openjdk:8
# Install Git and dependencies
RUN dpkg --add-architecture i386 \
&& apt-get update \
&& apt-get install -y file git curl zip libncurses5:i386 libstdc++6:i386 zlib1g:i386 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists /var/cache/apt
# Set up environment variables
ENV ANDROID_HOME="/opt/android-sdk" \
SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip" \
GRADLE_URL="https://services.gradle.org/distributions/gradle-4.5.1-all.zip"
# Create a non-root user
RUN chmod 777 -Rf /opt
# Download Android SDK
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
RUN chmod 777 -Rf /opt/*
# Install Gradle
RUN wget $GRADLE_URL -O gradle.zip \
&& unzip gradle.zip \
&& mv gradle-4.5.1 gradle \
&& rm gradle.zip \
&& mkdir .gradle
ENV PATH="/home/user/gradle/bin:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools:${PATH}"
RUN echo y | android update sdk --no-ui
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-19.0.3
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-25.0.3
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-26.0.2
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-27.0.3
RUN echo y | android update sdk --filter tools,platform-tools,build-tools-28.0.1
RUN cd /opt/android-sdk/ && mkdir platforms
RUN cd /opt/android-sdk/tools/bin && ./sdkmanager "system-images;android-23;google_apis;x86" && echo y | ./avdmanager create avd -n test -k "system-images;android-23;google_apis;x86" -b x86 -c 100M -d 7 -f
Here is my error.

Fail to execute command using Docker

I have a docker container, which purpose is to build android NDK project.
This container is build by this tutorial http://ainoya.io/docker-android-walter. Here is the Docker file:
# based on https://registry.hub.docker.com/u/samtstern/android-sdk/dockerfile/ with openjdk-8
FROM java:8
MAINTAINER Lebedevsd <lebedevsd#gmail.com>
ENV DEBIAN_FRONTEND noninteractive
# Install dependencies
RUN dpkg --add-architecture i386 && \
apt-get update && \
apt-get install -yq libstdc++6:i386 zlib1g:i386 libncurses5:i386 --no-install-recommends && \
apt-get install zip && \
apt-get -y install --reinstall locales && \
apt-get clean
# Download and setup android NDK
ENV ANDROID_NDK_URL http://dl.google.com/android/repository/android-ndk-r12b-linux-x86_64.zip
RUN cd /usr/local && \
curl -L "${ANDROID_NDK_URL}" > file.zip && \
unzip -q file.zip && \
rm file.zip
ENV ANDROID_NDK_HOME /usr/local/android-ndk-r12b
ENV PATH ${ANDROID_NDK_HOME}/:$PATH
# Download and untar SDK
ENV ANDROID_SDK_URL http://dl.google.com/android/android-sdk_r24.4.1-linux.tgz
RUN curl -L "${ANDROID_SDK_URL}" | tar --no-same-owner -xz -C /usr/local
ENV ANDROID_HOME /usr/local/android-sdk-linux
ENV ANDROID_SDK /usr/local/android-sdk-linux
ENV PATH ${ANDROID_HOME}/tools:$ANDROID_HOME/platform-tools:$PATH
# Install Android SDK components
ONBUILD COPY android_sdk_components.env /android_sdk_components.env
ONBUILD RUN (while :; do echo 'y'; sleep 3; done) | android update sdk --no-ui --all --filter "$(cat /android_sdk_components.env)"
# Support Gradle
ENV TERM dumb
ENV JAVA_OPTS -Xms512m -Xmx768m
The output is
docker run -t -v $(pwd)/app:/project/app lebedevsd/android_sdk_build:latest ./gradlew clean assemblePRelease -PversionCode=24 -PversionName=0.0.24
docker: Error response from daemon: oci runtime error: exec: "./gradlew": stat ./gradlew: no such file or directory.

The problem was that I was mounting wrong directory: $(pwd)/app instead of $(pwd).
the right way is:
docker run -i -v $(pwd):/project -w /project lebedevsd/android_sdk_build:latest ./gradlew clean

Android SDK tools install in Docker fails

I am trying to make a docker image that I can use to build Android projects, using Shippable.
The problem is the android update sdk command, which gives the following error:
Installing Android SDK Tools, revision 24.2
Failed to rename directory /opt/android-sdk-linux/tools to /opt/android-sdk-linux/temp/ToolPackage.old01.
Failed to create directory /opt/android-sdk-linux/tools
I found somewhat of a solution here: https://stackoverflow.com/a/8839359/867099 but it's for Windows, and does not seem to fix the problem on linux. It appears that during the update command, the current directory is in use and therefore cannot be renamed.
My workaround sofar, using that workaroundsuggestion, is this:
RUN cp -r /opt/android-sdk-linux/tools /opt/android-sdk-linux/tools_copy
RUN cd /opt/android-sdk-linux/tools && echo 'y' | /opt/android-sdk-linux/tools_copy/android update sdk --no-ui -a --filter tools,platform-tools,build-tools-22.0.1,android-21,extra-android-support,extra-google-google_play_services --force
In order to automatically accept the license, I echo 'y' to the android command.
But I think the android command should also run in the correct directory, which is why I cd into it first.
But, it still fails. I'm rather stumped on how to fix this issue, so any help is appreciated.
------ UPDATE --------
I run the android sdk update command without the tools filter, and in the end, my gradle builds are successful. So I don't know for sure whether it's a problem to not update them...

This can be solved by combining all Android SDK commands in a single Dockerfile's RUN command. It has something to do with Docker's file system.
For a detailed explanation from a post on Issue Tracker for the Android Open Source Project:
The problem is caused when you run the SDK update in a Docker
container. Docker uses a special filesystem which works like a version
control system (e.g. git) and records all changes made to the
filesystem. The problem is that the SDK update uses a hardlink move
operation to move the 'tools' directory, which is not supported by the
underlying Docker filesystem.
The solution for this is to simply run all Android SDK commands in a
single 'RUN' command in Docker. That way, the hardlink move works as
normal, as long as you don't use the 'hardlink move' operation between
multiple RUN command (snapshots). The advantage of this is also that
your Docker layers will be smaller (compared to running multiple
seperate RUN commands).
Here is the line from the Dockerfile:
RUN wget https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz && \
tar xzf android-sdk_r24.4.1-linux.tgz && \
rm android-sdk_r24.4.1-linux.tgz && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter platform-tools,tools -a ) && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter extra-android-m2repository,extra-android-support,extra-google-google_play_services,extra-google-m2repository -a) && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter build-tools-23.0.2,android-24 -a)

This is what's currently working for me, you can see the update command successfully running below. In my environment these are 3 different docker images in one FROM hierarchy so you can likely combine a lot of the apt-gets if that's not your case.
FROM ubuntu:14.04
# Set debconf to run non-interactively
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
# Install base dependencies
RUN apt-get update && apt-get install -y -q --no-install-recommends \
apt-transport-https \
build-essential \
ca-certificates \
curl \
git \
libssl-dev \
python \
rsync \
software-properties-common \
wget \
&& rm -rf /var/lib/apt/lists/*
# Install the JDK
RUN echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections && \
add-apt-repository -y ppa:webupd8team/java && \
apt-get update -qq && \
DEBIAN_FRONTEND=noninteractive apt-get install -qqy --force-yes oracle-java7-installer && \
rm -rf /var/lib/apt/lists/* /var/cache/oracle-jdk7-installer
# Install Android Dev Tools
RUN apt-get update && apt-get install -y -q --no-install-recommends \
lib32ncurses5 \
lib32stdc++6 \
lib32z1 \
libswt-gtk-3-java \
unzip \
&& rm -rf /var/lib/apt/lists/*
RUN wget -qO- "http://dl.google.com/android/android-sdk_r23.0.2-linux.tgz" | tar -zxv -C /opt/
RUN cd /opt/android-sdk-linux/tools/ && \
echo y | ./android update sdk --all --filter platform-tools,build-tools-20.0.0,android-17,sysimg-17,system-image,extra-android-support --no-ui --force
ENV PATH /opt/android-sdk-linux/build-tools/20.0.0:$PATH