Develop Reference

MWE of GitLab CI that uses a (QEMU) simulated android phone over adb?

Context
While trying to simulate a phone and unit test adb and uiautomator interactions with the phone, in Python, I am experiencing some difficulties in finding a MWE that:
Simulates an Android phone
Simulates that Android phone over adb.
In a .gitlab-ci.yml file.
Most of the examples 1, 2, 3, 4, 5 I that I found, also compile an Android app and build it, however, I am not interested in that. I merely would like to simulate and test the interaction with the phone. Hence, I would like to ask:
Question
How can one simulate an Android phone with adb in GitLab CI and run (two files, one Bash, one Python) that contain:
adb install some_app.apk
And:
from uiautomator import device
device.screenshot("/some_path/some_file.png")
?
Approach I
I found this script to be automatically installing Android Emulator, installing an Android phone, and running it with GUI:
# Source: https://github.com/Uirado/linux-bash-install/blob/a150a462d2f9d095734f10a7b8c6c39871e5df71/batch-install.sh
# Worked.
yes | sudo apt install openjdk-8-jdk
echo 'export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")' >> ~/.profile
sudo update-java-alternatives --set java-1.8.0-openjdk-amd64
. ~/.profile
sudo apt update
sudo apt upgrade
#installLog "Android SDK"
mkdir -p ~/Android/Sdk
wget -O ~/Android/Sdk/android-sdk.zip "https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip"
unzip -o -qq ~/Android/Sdk/android-sdk.zip -d ~/Android/Sdk
rm ~/Android/Sdk/android-sdk.zip
mv ~/Android/Sdk/tools/emulator ~/Android/Sdk/tools/emulator2
wget -O ~/gradle-4.10.3.zip https://downloads.gradle-dn.com/distributions/gradle-4.10.3-all.zip
unzip -o -qq ~/gradle-4.10.3.zip -d ~/
echo 'export GRADLE_HOME=$HOME/gradle-4.10.3' >> ~/.profile
echo 'export ANDROID_HOME=$HOME/Android/Sdk' >> ~/.profile
echo 'export ANDROID_SDK_ROOT=$ANDROID_HOME' >> ~/.profile
echo 'export PATH=$PATH:$ANDROID_HOME/tools' >> ~/.profile
echo 'export PATH=$PATH:$ANDROID_HOME/tools/bin' >> ~/.profile
echo 'export PATH=$PATH:$GRADLE_HOME/bin' >> ~/.profile
. ~/.profile
mkdir -p ~/.android
touch ~/.android/repositories.cfg
installLog "Android SDK packages"
sdkmanager --install "platform-tools"
sdkmanager --install "build-tools;29.0.2"
sdkmanager --install "extras;google;google_play_services"
sdkmanager --install "system-images;android-28;google_apis_playstore;x86_64"
sdkmanager --install emulator
echo 'export PATH=$PATH:$ANDROID_HOME/emulator' >> ~/.profile
. ~/.profile
# create android VMs android-small and #android-large
avdmanager create avd -n "android-small" -k "system-images;android-28;google_apis_playstore;x86_64"
# avdmanager create avd -n "android-small" -k "system-images;android-28;google_apis_playstore;x86_64" -d "Nexus S" -c 512
# Show installed android devices:
emulator -list-avds
# Launch emulated phone:
emulator -avd android-small -netdelay none -netspeed full
However, I was not yet able to interact with it from Python nor bash.

Android SDK on docker

Im trying to run android emulator on docker but I get the following error, I have done some research but none of the answers helped me in achieving my goal, here is the error that I get and my docker file, I have tried armeabi and system-images;android-29;google_apis;x86 and both did not work.
# emulator -avd device
emulator: WARNING: encryption is off
emulator: INFO: QtLogger.cpp:68: Warning: could not connect to display ((null):0, (null))
emulator: INFO: QtLogger.cpp:68: Info: Could not load the Qt platform plugin "xcb" in "/opt/android-sdk/emulator/lib64/qt/plugins" even though it was found. ((null):0, (null))
Fatal: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: xcb.
((null):0, (null))
emulator: INFO: QtLogger.cpp:68: Fatal: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: xcb.
((null):0, (null))
Aborted
Im doing the same steps on my MAC and it's working fine, here is my docker file:
FROM ubuntu:18.04
RUN dpkg --add-architecture i386 && \
apt-get update -y && \
apt-get install -y --no-install-recommends libncurses5:i386 libc6:i386 libstdc++6:i386 lib32gcc1 lib32ncurses5 lib32z1 zlib1g:i386 && \
apt-get install -y --no-install-recommends openjdk-8-jdk && \
apt-get install -y --no-install-recommends git wget unzip && \
apt-get install -y --no-install-recommends qt5-default
ARG GRADLE_VERSION=5.2.1
ARG GRADLE_DIST=bin
RUN cd /opt && \
wget -q https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-${GRADLE_DIST}.zip && \
unzip gradle*.zip && \
ls -d */ | sed 's/\/*$//g' | xargs -I{} mv {} gradle && \
rm gradle*.zip
# download and install Android SDK
# https://developer.android.com/studio/#downloads
ARG ANDROID_SDK_VERSION=4333796
ENV ANDROID_HOME /opt/android-sdk
RUN mkdir -p ${ANDROID_HOME} && cd ${ANDROID_HOME} && \
wget -q https://dl.google.com/android/repository/sdk-tools-linux-${ANDROID_SDK_VERSION}.zip && \
unzip *tools*linux*.zip && \
rm *tools*linux*.zip
# set the environment variables
ENV APK_SIGNER=/opt/android-sdk/build-tools/29.0.2/
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV ANDROID_SDK_ROOT /opt/android-sdk
ENV GRADLE_HOME /opt/gradle
ENV PATH ${PATH}:${GRADLE_HOME}/bin:${ANDROID_HOME}/emulator:${ANDROID_HOME}/tools/bin:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools:${ANDROID_HOME}/tools/bin/apksigner:${APK_SIGNER}
ENV _JAVA_OPTIONS -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
# WORKAROUND: for issue https://issuetracker.google.com/issues/37137213
ENV LD_LIBRARY_PATH ${ANDROID_HOME}/emulator/lib64:${ANDROID_HOME}/emulator/lib64/qt/lib
# accept the license agreements of the SDK components
ADD docker/license_accepter.sh /opt/
RUN chmod +x /opt/license_accepter.sh && /opt/license_accepter.sh $ANDROID_HOME
# setup adb server
EXPOSE 5037
# install and configure SSH server
EXPOSE 22
ADD docker/sshd-banner /etc/ssh/
ADD docker/authorized_keys /tmp/
RUN apt-get update -y && \
apt-get install -y --no-install-recommends openssh-server supervisor locales && \
mkdir -p /var/run/sshd /var/log/supervisord && \
locale-gen en en_US en_US.UTF-8 && \
apt-get remove -y locales && apt-get autoremove -y && \
FILE_SSHD_CONFIG="/etc/ssh/sshd_config" && \
echo "\nBanner /etc/ssh/sshd-banner" >> $FILE_SSHD_CONFIG && \
echo "\nPermitUserEnvironment=yes" >> $FILE_SSHD_CONFIG && \
ssh-keygen -q -N "" -f /root/.ssh/id_rsa && \
FILE_SSH_ENV="/root/.ssh/environment" && \
touch $FILE_SSH_ENV && chmod 600 $FILE_SSH_ENV && \
printenv | grep "JAVA_HOME\|GRADLE_HOME\|ANDROID_HOME\|LD_LIBRARY_PATH\|PATH" >> $FILE_SSH_ENV && \
FILE_AUTH_KEYS="/root/.ssh/authorized_keys" && \
touch $FILE_AUTH_KEYS && chmod 600 $FILE_AUTH_KEYS && \
for file in /tmp/*.pub; \
do if [ -f "$file" ]; then echo "\n" >> $FILE_AUTH_KEYS && cat $file >> $FILE_AUTH_KEYS && echo "\n" >> $FILE_AUTH_KEYS; fi; \
done && \
(rm /tmp/*.pub 2> /dev/null || true)
ADD docker/supervisord.conf /etc/supervisor/conf.d/
ADD docker/supervisord.conf /etc/supervisor/conf.d/
ADD docker/android.keystore /Android-APK/
ADD app/build/outputs/apk/release/ /Android-APK/
RUN sdkmanager "platform-tools" "platforms;android-29" "build-tools;29.0.2" "add-ons;addon-google_apis-google-24"
RUN apksigner sign --ks /Android-APK/android.keystore --ks-pass pass:android --key-pass pass:android --in /Android-APK/app-release-unsigned.apk --out /Android-APK/signed-app.apk
RUN zipalign -f 4 /Android-APK/signed-app.apk /Android-APK/aligned-app.apk
#RUN sdkmanager "system-images;android-23;google_apis;x86"
#RUN adb install /Android-APK/aligned-app.apk
#RUN sdkmanager "system-images;android-25;google_apis;arm64-v8a"
#RUN avdmanager create avd --force --name "device" --abi "arm64-v8a" --package 'system-images;android-25;google_apis;arm64-v8a' --device "Nexus 10"
CMD ["/usr/bin/supervisord"]

I had a similar problem, and it ended up being that our script was running the incorrect version of the emulator (26, which doesn't have good support for headless yet)
It should be running
$ANDROID_HOME/emulator/emulator -avd avd-name -no-skin -no-audio -no-window
and not the emulator in the tools folder.
Check the version of the emulator by just running the executable. My output (that works) is
emulator: Android emulator version 30.4.5.0 (build_id 7140946) (CL:N/A)
emulator: ERROR: No AVD specified. Use '#foo' or '-avd foo' to launch a virtual device named 'foo'

first run
xhost +
then run your docker container like this
docker run -i -t --net=host --privileged --env="DISPLAY" \
-v $HOME/.Xauthority:/root/.Xauthority:rw \
-v ~/disk1/android_docker:/home/android \
android-image /bin/bash
then you can run
emulator

Android SDK tools install in Docker fails

I am trying to make a docker image that I can use to build Android projects, using Shippable.
The problem is the android update sdk command, which gives the following error:
Installing Android SDK Tools, revision 24.2
Failed to rename directory /opt/android-sdk-linux/tools to /opt/android-sdk-linux/temp/ToolPackage.old01.
Failed to create directory /opt/android-sdk-linux/tools
I found somewhat of a solution here: https://stackoverflow.com/a/8839359/867099 but it's for Windows, and does not seem to fix the problem on linux. It appears that during the update command, the current directory is in use and therefore cannot be renamed.
My workaround sofar, using that workaroundsuggestion, is this:
RUN cp -r /opt/android-sdk-linux/tools /opt/android-sdk-linux/tools_copy
RUN cd /opt/android-sdk-linux/tools && echo 'y' | /opt/android-sdk-linux/tools_copy/android update sdk --no-ui -a --filter tools,platform-tools,build-tools-22.0.1,android-21,extra-android-support,extra-google-google_play_services --force
In order to automatically accept the license, I echo 'y' to the android command.
But I think the android command should also run in the correct directory, which is why I cd into it first.
But, it still fails. I'm rather stumped on how to fix this issue, so any help is appreciated.
------ UPDATE --------
I run the android sdk update command without the tools filter, and in the end, my gradle builds are successful. So I don't know for sure whether it's a problem to not update them...

This can be solved by combining all Android SDK commands in a single Dockerfile's RUN command. It has something to do with Docker's file system.
For a detailed explanation from a post on Issue Tracker for the Android Open Source Project:
The problem is caused when you run the SDK update in a Docker
container. Docker uses a special filesystem which works like a version
control system (e.g. git) and records all changes made to the
filesystem. The problem is that the SDK update uses a hardlink move
operation to move the 'tools' directory, which is not supported by the
underlying Docker filesystem.
The solution for this is to simply run all Android SDK commands in a
single 'RUN' command in Docker. That way, the hardlink move works as
normal, as long as you don't use the 'hardlink move' operation between
multiple RUN command (snapshots). The advantage of this is also that
your Docker layers will be smaller (compared to running multiple
seperate RUN commands).
Here is the line from the Dockerfile:
RUN wget https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz && \
tar xzf android-sdk_r24.4.1-linux.tgz && \
rm android-sdk_r24.4.1-linux.tgz && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter platform-tools,tools -a ) && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter extra-android-m2repository,extra-android-support,extra-google-google_play_services,extra-google-m2repository -a) && \
(echo y | android-sdk-linux/tools/android -s update sdk --no-ui --filter build-tools-23.0.2,android-24 -a)

This is what's currently working for me, you can see the update command successfully running below. In my environment these are 3 different docker images in one FROM hierarchy so you can likely combine a lot of the apt-gets if that's not your case.
FROM ubuntu:14.04
# Set debconf to run non-interactively
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
# Install base dependencies
RUN apt-get update && apt-get install -y -q --no-install-recommends \
apt-transport-https \
build-essential \
ca-certificates \
curl \
git \
libssl-dev \
python \
rsync \
software-properties-common \
wget \
&& rm -rf /var/lib/apt/lists/*
# Install the JDK
RUN echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections && \
add-apt-repository -y ppa:webupd8team/java && \
apt-get update -qq && \
DEBIAN_FRONTEND=noninteractive apt-get install -qqy --force-yes oracle-java7-installer && \
rm -rf /var/lib/apt/lists/* /var/cache/oracle-jdk7-installer
# Install Android Dev Tools
RUN apt-get update && apt-get install -y -q --no-install-recommends \
lib32ncurses5 \
lib32stdc++6 \
lib32z1 \
libswt-gtk-3-java \
unzip \
&& rm -rf /var/lib/apt/lists/*
RUN wget -qO- "http://dl.google.com/android/android-sdk_r23.0.2-linux.tgz" | tar -zxv -C /opt/
RUN cd /opt/android-sdk-linux/tools/ && \
echo y | ./android update sdk --all --filter platform-tools,build-tools-20.0.0,android-17,sysimg-17,system-image,extra-android-support --no-ui --force
ENV PATH /opt/android-sdk-linux/build-tools/20.0.0:$PATH

"ImportError: No module named readline" running "repo init"

I downloaded the source of Android, but when I used repo init according to the website, I got the following error:
Traceback (most recent call last):
File "/root/bin/repo", line 91, in <module>
import readline
ImportError: No module named readline
I am using Ubuntu 10.04 and Python 2.7.2

AFAIK, the default Python environment in Ubuntu 10.04 is Python 2.6.5, so you must be using a self-compiled python.
You should be missing some readline header files when building your python 2.7.2, so you have two choices now:
Re-compile your python, with libreadline?-dev installed.
Install the standalone version of readline, using pip install readline or easy_install readline

I have the same issue, and I fixed it in this way, hope it will be helpful to you.
$ source Install_Python2.7.5.sh
Shell script "Install_Python2.7.5.sh" is like this:
#!/bin/bash
sudo apt-get install libreadline-dev
sudo aptitude install build-essential libreadline5-dev
sudo apt-get install libbz2-dev
if [ ! -d ~/software/Python/Python-2.7.5 ]; then
mkdir ~/software
mkdir ~/software/Python
mkdir ~/software/Python/Python-2.7.5
fi
if [ ! -d ~/bin ]; then
mkdir ~/bin
fi
user_root=~
echo $user_root
echo "unpackage Python-2.7.5.tar.bz2..."
tar -jxvf Python-2.7.5.tar.bz2
echo "prepear to install Python-2.7.5"
cd Python-2.7.5
sed -i "s/#readline/readline/g" Modules/Setup.dist
echo "Start to install Python-2.7.5"
./configure --prefix=$user_root/software/Python/Python-2.7.5/ && make && make install
echo "Make symbol link"
ln -s -f $user_root/software/Python/Python-2.7.5/bin/* $user_root/bin
echo "Set environment variable"
PATH=$user_root/bin:$PATH

Repo import readline error

I have tried to work with Android as described in http://source.android.com/source/downloading.html, but when I used repo, I have faced with this error:
import readline
ImportError: No module named readline
I am using Ubuntu 11.04 and Python 2.7.
I will be appreciated if anybody can help me.

Try installing libreadline5-dev

The readline is not avaliable becase the python installed with out readline module. You need to recompile your Python with readline included. You can do it like this, good luck!
$ source Install_Python2.7.5.sh
Shell script "Install_Python2.7.5.sh" is like this:
#!/bin/bash
sudo apt-get install libreadline-dev
sudo aptitude install build-essential libreadline5-dev
sudo apt-get install libbz2-dev
if [ ! -d ~/software/Python/Python-2.7.5 ]; then
mkdir ~/software
mkdir ~/software/Python
mkdir ~/software/Python/Python-2.7.5
fi
if [ ! -d ~/bin ]; then
mkdir ~/bin
fi
user_root=~
echo $user_root
echo "unpackage Python-2.7.5.tar.bz2..."
tar -jxvf Python-2.7.5.tar.bz2
echo "prepear to install Python-2.7.5"
cd Python-2.7.5
sed -i "s/#readline/readline/g" Modules/Setup.dist
echo "Start to install Python-2.7.5"
./configure --prefix=$user_root/software/Python/Python-2.7.5/ && make && make install
echo "Make symbol link"
ln -s -f $user_root/software/Python/Python-2.7.5/bin/* $user_root/bin
echo "Set environment variable"
PATH=$user_root/bin:$PATH