Develop Reference

Docker: Operation not permitted on workspace directory android

I am trying to set up a CI/CD pipeline for my android project using Jenkins and docker(I'm using docker file) and here's a sample of the Jenkinsfile
pipeline {
agent { dockerfile true }
environment {
appName = 'Yoruba Proverbs'
KEY_PASSWORD = credentials('keyPassword')
KEY_ALIAS = credentials('keyAlias')
KEYSTORE = credentials('keystore')
STORE_PASSWORD = credentials('storePassword')
}
stages {
....
}
}
The build fails at the instance when docker is trying to make a modification under the workspace directory with an operation not permitted error(FileSystemException). Here's a screenshot: build
I have tried a few different tweaks like restarting Jenkins, restarting docker, Granting docker full disk access on my laptop but no positive result
Edit:
This is the content of the Dockerfile
FROM openjdk:8
WORKDIR project/
# Install Build Essentials
RUN apt-get update \
&& apt-get install build-essential -y
# Set Environment Variables
ENV SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip" \
ANDROID_HOME="/usr/local/android-sdk" \
ANDROID_VERSION=30
# Download Android SDK
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& mkdir "$ANDROID_HOME/licenses" || true \
&& echo "24333f8a63b6825ea9c5514f83c2829b004d1fee" > "$ANDROID_HOME/licenses/android-sdk-license" \
&& yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
# Install Android Build Tool and Libraries
RUN $ANDROID_HOME/tools/bin/sdkmanager --update
RUN $ANDROID_HOME/tools/bin/sdkmanager "build-tools;30.0.3" \
"platforms;android-${ANDROID_VERSION}" \
"platform-tools"
CMD ["/bin/bash"]

Sending build context to docker daemon becomes greater than 250 gb for android

I am trying to build a docker image for android, below is my docker file
FROM ubuntu:18.04
LABEL maintainer="Javier Santos"
ENV VERSION_SDK_TOOLS "4333796"
ENV ANDROID_HOME "/sdk"
ENV PATH "$PATH:${ANDROID_HOME}/tools"
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get -qq update
RUN apt-get install -y locales
RUN locale-gen en_US.UTF-8
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
RUN apt-get install -qqy --no-install-recommends \
bzip2 \
curl \
git-core \
html2text \
openjdk-8-jdk \
libc6-i386 \
lib32stdc++6 \
lib32gcc1 \
lib32ncurses5 \
lib32z1 \
unzip \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN rm -f /etc/ssl/certs/java/cacerts; \
/var/lib/dpkg/info/ca-certificates-java.postinst configure
RUN curl -s https://dl.google.com/android/repository/sdk-tools-linux-${VERSION_SDK_TOOLS}.zip > /sdk.zip && \
unzip /sdk.zip -d /sdk && \
rm -v /sdk.zip
RUN mkdir -p $ANDROID_HOME/licenses/ \
&& echo "8933bad161af4178b1185d1a37fbf41ea5269c55\nd56f5187479451eabf01fb78af6dfcb131a6481e" > $ANDROID_HOME/licenses/android-sdk-license \
&& echo "84831b9409646a918e30573bab4c9c91346d8abd" > $ANDROID_HOME/licenses/android-sdk-preview-license
RUN yes | $ANDROID_HOME/tools/bin/sdkmanager "platforms;android-28"
ADD packages.txt /sdk
RUN mkdir -p /root/.android && \
touch /root/.android/repositories.cfg && \
${ANDROID_HOME}/tools/bin/sdkmanager --update
RUN while read -r package; do PACKAGES="${PACKAGES}${package} "; done < /sdk/packages.txt && \
${ANDROID_HOME}/tools/bin/sdkmanager ${PACKAGES}
But when I am building this, then Sending build context to Docker daemon is getting larger from 250GB and still increasing, Is this normal, or I am doing something wrong, please suggest, thanks in advance
I have taken reference from this link https://hub.docker.com/r/javiersantos/android-ci

The build context is not inside your Dockerfile, it's the path passed at the end of the build command, often a . to indicate the current directory. When building with compose it defaults to the current directory. So if this is growing, you have files in that context directory that are growing between builds.
With buildkit, this behavior changes, only sending the requested files rather than the full context. You can enable that by running export DOCKER_BUILDKIT=1 in the shell that runs the docker build command. You can also default this feature to enabled by setting the following in /etc/docker/daemon.json and then reloading the docker engine (often systemctl reload docker):
{
"features": {"buildkit": true }
}
Considering your build only adds a single file, probably the best option is to configure a .dockerignore in the root of your context (the same folder with the packages.txt file) with the following:
*
!packages.txt
The * ignores everything, and the !packages.txt reincludes that file back into the context.

How to resolve: Error: spawnSync ./gradlew ENOBUFS

so im trying to build my app on release mode with Docker.
When I run it on my machine it works with no problem but when I run it on docker I get this error:
Error: spawnSync ./gradlew ENOBUFS
[11:15:54] : [Step 1/2] at Object.spawnSync (internal/child_process.js:1041:20)
[11:15:54] : [Step 1/2] at spawnSync (child_process.js:607:24)
[11:15:54] : [Step 1/2] at execFileSync (child_process.js:634:15)
[11:15:54] : [Step 1/2] at runOnAllDevices (/usr/src/app/node_modules/#react-native-community/cli-platform-android/build/commands/runAndroid/runOnAllDevices.js:74:39)
[11:15:54] : [Step 1/2] at buildAndRun (/usr/src/app/node_modules/#react-native-community/cli-platform-android/build/commands/runAndroid/index.js:158:41)
[11:15:54] : [Step 1/2] at /usr/src/app/node_modules/#react-native-community/cli-platform-android/build/commands/runAndroid/index.js:125:12
[11:15:54] : [Step 1/2] at processTicksAndRejections (internal/process/task_queues.js:93:5)
[11:15:54] : [Step 1/2] at async Command.handleAction (/usr/src/app/node_modules/react-native/node_modules/#react-native-community/cli/build/cliEntry.js:160:7)
when I try to build with develop im getting no problems...
another thing I put attention about is when I build my app on docker with ./gradlew installRelease I get no issue but this command never use Jetifier which I really need for my build.
Im attaching my docker file, let me know if you need more files:
ENV SDK_URL "https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip"
ENV ANDROID_HOME /usr/src/app
ENV ANDROID_BUILD_TOOLS_VERSION 28.0.3
ENV ANDROID_VERSION 28
ENV PATH $PATH:$ANDROID_HOME/bin
ARG NODE_VERSION=12.x
WORKDIR /usr/src/app
COPY package*.json ./
RUN cat package.json
# installing android and java
RUN apt update -qq && apt install -qq -y --no-install-recommends \
apt-transport-https \
curl \
build-essential \
file \
git \
openjdk-8-jre \
gnupg2 \
python \
ruby-full \
openssh-client \
zip \
unzip \
sudo \
&& rm -rf /var/lib/apt/lists/*;
RUN curl -sL -o android.zip ${SDK_URL} && unzip android.zip && rm android.zip
RUN yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
# install nodejs and yarn packages from nodesource and yarn apt sources
RUN echo "deb https://deb.nodesource.com/node_${NODE_VERSION} stretch main" > /etc/apt/sources.list.d/nodesource.list \
&& echo "deb https://dl.yarnpkg.com/debian/ stable main" > /etc/apt/sources.list.d/yarn.list \
&& curl -sS https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add - \
&& curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& apt-get update -qq \
&& apt-get install -qq -y --no-install-recommends nodejs yarn \
&& rm -rf /var/lib/apt/lists/*
RUN npm i -g react-native-cli
#installing libraries
RUN npm install
COPY . .
#Create apk and zip it with the current date (UTC)
RUN react-native run-android --variant=release; exit 0
RUN cd android && cd app && cd build && cd outputs && cd apk && cd release && zip -r `(date +%m_%d_%Y__%H:%M)`.zip app-release.apk
#Create a new container from a linux base image that has the aws-cli installed
FROM mesosphere/aws-cli
#Using the alias defined for the first container, copy the contents of the build folder to this container
COPY --from=builder /usr/src/app/android/app/build/outputs/apk/release/ .
COPY --from=builder /usr/src/app/deploy-android-prod.sh .
ENTRYPOINT chmod +x deploy-android-prod.sh && sh ./deploy-android-prod.sh
Thanks for the help

Docker can not find Android SDK and NDK directory

I've created a docker container in order to run my gradle tasks on it.
I'm downloading the sdk inside it, but when I run a task from outside it says that the sdk folder can not be found because it's getting the path I have in the local.properties file of the project. Which is pointing to my machine sdk folder. How can I specify the sdk folder inside the docker image? Thanks.
Docker image build file:
FROM openjdk:8
ENV SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip" \
ANDROID_HOME="/usr/local/android-sdk" \
ANDROID_NDK_HOME="/usr/local/android-sdk/ndk-bundle" \
ANDROID_VERSION=26 \
ANDROID_BUILD_TOOLS_VERSION=26.0.2
# Download Android SDK
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
# add to PATH
ENV PATH ${PATH}:${ANDROID_HOME}
ENV ANDROID_NDK_HOME /usr/local/android-ndk
ENV ANDROID_NDK_VERSION r19
ENV NDK_URL="https://dl.google.com/android/repository/android-ndk-${ANDROID_NDK_VERSION}-linux-x86_64.zip"
# Download Android NDK
RUN mkdir "$ANDROID_NDK_HOME" \
&& cd "$ANDROID_NDK_HOME" \
&& curl -o ndk.zip $NDK_URL \
&& unzip ndk.zip \
&& rm ndk.zip
# add to PATH
ENV PATH ${PATH}:${ANDROID_NDK_HOME}
# Install Android Build Tool and Libraries
RUN $ANDROID_HOME/tools/bin/sdkmanager --update
RUN $ANDROID_HOME/tools/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
"platforms;android-${ANDROID_VERSION}" \
"platform-tools"
RUN mkdir /application
WORKDIR /application
This is how I run the task:
docker run -it --rm -v "$PWD":/application packsdkandroiddocker.image sh -c "$#" ./gradlew clean
And this is the error I get:
NDK is missing a "platforms" directory. If you are using NDK, verify
the ndk.dir is set to a valid NDK directory. It is currently set to
/Users/adalpari/Library/Android/sdk/ndk-bundle. If you are not using
NDK, unset the NDK variable from ANDROID_NDK_HOME or local.properties
to remove this warning.
FAILURE: Build failed with an exception.
What went wrong: A problem occurred configuring project ':app'.
The SDK directory '/Users/adalpari/Library/Android/sdk' does not exist.

I think you need to remove local.properties file from your root folder as SDK is then searched locally which container cannot access.

Android NDK CI using Bitbucket Pipelines

I am trying to build an NDK Android project using Bitbucket Pipelines continuous integration using ndk-build (not the newer CMAKE). But I am getting the following error when calling ./gradlew assembleDebug:
/opt/android-sdk-linux/ndk-bundle/build/ndk-build: 144: /opt/android-sdk-linux/ndk-bundle/build/ndk-build: file: not found
:sensorylib:ndkBuild
make: Entering directory '/opt/atlassian/pipelines/agent/build/sensorylib/src/main'
/bin/sh: 1: file: not found
make: execvp: /opt/android-sdk-linux/ndk-bundle/build/extract_manifest.py: Permission denied
make: execvp: /opt/android-sdk-linux/ndk-bundle/build/extract_manifest.py: Permission denied
[armeabi] Install : libSensoryVoiceJNI.so => libs/armeabi/libSensoryVoiceJNI.so
/opt/android-sdk-linux/ndk-bundle/build/core/build-binary.mk:797: recipe for target 'libs/armeabi/libSensoryVoiceJNI.so' failed
make: Leaving directory '/opt/atlassian/pipelines/agent/build/sensorylib/src/main'
make: /opt/android-sdk-linux/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86/bin/arm-linux-androideabi-strip: Command not found
make: *** [libs/armeabi/libSensoryVoiceJNI.so] Error 127
make: *** Deleting file 'libs/armeabi/libSensoryVoiceJNI.so'
:sensorylib:ndkBuild FAILED
20 actionable tasks: 20 executed
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sensorylib:ndkBuild'.
> Process 'command '/opt/android-sdk-linux/ndk-bundle/ndk-build'' finished with non-zero exit value 2
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED in 45s
bitbucket-pipelines.yml
image: java:8
pipelines:
default:
- step:
caches:
- gradle
script:
# dependencies
- apt-get update
- apt-get -y install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev ccache libgl1-mesa-dev libxml2-utils xsltproc unzip
# environment vars
- export SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip"
- export ANDROID_HOME="/opt/android-sdk-linux"
- export ANDROID_NDK_HOME="$ANDROID_HOME/ndk-bundle"
- export PATH="$ANDROID_HOME/tools:$ANDROID_HOME/tools/bin:$ANDROID_NDK_HOME:$PATH"
# download and unzip sdk
- wget -q $SDK_URL -O android-sdk.zip
- unzip android-sdk.zip -d $ANDROID_HOME && rm -f android-sdk.zip
# accept all licences http://stackoverflow.com/questions/38096225/automatically-accept-all-sdk-licences
- mkdir -p "$ANDROID_HOME/licenses"
- echo -e "8933bad161af4178b1185d1a37fbf41ea5269c55" > "$ANDROID_HOME/licenses/android-sdk-license"
- echo -e "84831b9409646a918e30573bab4c9c91346d8abd" > "$ANDROID_HOME/licenses/android-sdk-preview-license"
# download ndk
- sdkmanager "ndk-bundle"
# build
- chmod +x gradlew
- ./gradlew assembleDebug
The ANDROID_NDK_HOME path seems correct as I can ls the folder and it looks correct. The permission on ndk-build also looks correctly set...

The error was that I was missing file in the apt-get install dependencies.
Here is the working bitbucket-pipelines.yml.
(I also managed to reduce the overall apt-get dependencies)
image: java:8
pipelines:
default:
- step:
caches:
- gradle
script:
# dependencies
- apt-get update && apt-get -y install file build-essential
# environment vars
- export SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip"
- export ANDROID_HOME="/opt/android-sdk-linux"
- export ANDROID_NDK_HOME="$ANDROID_HOME/ndk-bundle"
- export PATH="$ANDROID_HOME/tools:$ANDROID_HOME/tools/bin:$ANDROID_NDK_HOME:$PATH"
# download and unzip sdk
- wget -q $SDK_URL -O android-sdk.zip && unzip android-sdk.zip -d $ANDROID_HOME && rm -f android-sdk.zip
# accept all licences http://stackoverflow.com/questions/38096225/automatically-accept-all-sdk-licences
- mkdir -p "$ANDROID_HOME/licenses"
- echo -e "8933bad161af4178b1185d1a37fbf41ea5269c55" > "$ANDROID_HOME/licenses/android-sdk-license"
- echo -e "84831b9409646a918e30573bab4c9c91346d8abd" > "$ANDROID_HOME/licenses/android-sdk-preview-license"
# download ndk
- sdkmanager "ndk-bundle"
# build
- chmod +x gradlew
- ./gradlew assembleDebug

Here is a simplified version
image: umbrela/android-ndk-builder:latest
pipelines:
branches:
master:
- step:
caches:
- gradle
script:
# build
- chmod +x gradlew && ./gradlew assembleDebug
Where umbrela/android-ndk-builder:latest is
FROM java:8
RUN apt-get update \
&& apt-get -y install file build-essential
ENV SDK_URL https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip
ENV ANDROID_HOME /opt/android-sdk-linux
ENV ANDROID_NDK_HOME $ANDROID_HOME/ndk-bundle
ENV PATH $ANDROID_HOME/tools:$ANDROID_HOME/tools/bin:$ANDROID_NDK_HOME:$PATH
# accept all licences http://stackoverflow.com/questions/38096225/automatically-accept-all-sdk-licences
RUN mkdir -p "$ANDROID_HOME/licenses"
RUN echo 8933bad161af4178b1185d1a37fbf41ea5269c55 > $ANDROID_HOME/licenses/android-sdk-license
RUN echo 84831b9409646a918e30573bab4c9c91346d8abd > $ANDROID_HOME/licenses/android-sdk-preview-license
RUN ls $ANDROID_HOME/licenses
# download and unzip sdk
RUN mkdir -p $ANDROID_HOME \
&& wget -q $SDK_URL -O android-sdk.zip \
&& unzip android-sdk.zip -d $ANDROID_HOME \
&& rm -f android-sdk.zip
# download ndk
RUN sdkmanager "ndk-bundle"