Develop Reference

How to cache Android NDK in Azure devops pipeline?

While creating a task in Azure devops pipeline, I am requred to cache the NDK to reduce the build time. Hence, I am trying to write a task for the same.
For example here is what I am using for caching the gradle and konan repos
variables:
GRADLE_USER_HOME: $(Pipeline.Workspace)/.gradle
KONAN_USER_HOME: /Users/runner/.konan
steps:
- bash: env
displayName: env vars
- task: Cache#2
inputs:
key: 'gradle | "$(Agent.OS)"'
restoreKeys: gradle
path: $(GRADLE_USER_HOME)
displayName: Gradle build cache
- task: Cache#2
inputs:
key: 'konan | "$(Agent.OS)" | cache'
restoreKeys: konan
path: $(KONAN_USER_HOME)
displayName: Konan build cache
they work just fine!
But for NDK, how do I know the relative repository path for my NDK and cache it, assuming the task will be something like this
- task: Cache#2
inputs:
key: 'ndk | "$(Agent.OS)"'
restoreKeys: ndk
path: $(NDK_PATH)
displayName: NDK build cache
Any help in this is highly helpful.

How do I know the relative repository path for my NDK?
You can use the environment variable ANDROID_NDK_PATH to get the path for the NDK.
If you are using the ndk that pre-installed in Microsoft hosted agent windows-latest, the value of ANDROID_NDK_PATH is C:\Program Files (x86)\Android\android-sdk\ndk-bundle.

So, I figured this out, forgot to post here and this is what I found.
1 - to get the relative path of android-ndk from devops setup (I dont have those permissions).
2 - Then run the caching task to cache into that path
1 - get relative env vars with this bash output. CREDITS So In steps I write:
steps:
- bash: env
displayName: env vars
prints all the env variables available example gist with some vars intentionally removed
from that I picked up the path relevant to my use
2021-01-19T16:17:44.4273060Z ANDROID_HOME=/Users/runner/Library/Android/sdk
2021-01-19T16:17:44.4276800Z ANDROID_SDK_ROOT=/Users/runner/Library/Android/sdk
2021-01-19T16:17:44.4320010Z ANDROID_NDK_18R_PATH=/Users/runner/Library/Android/sdk/ndk/18.1.5063045
2021-01-19T16:17:44.4328690Z ANDROID_NDK_HOME=/Users/runner/Library/Android/sdk/ndk-bundle
now the most relevant path was to ANDROID_SDK_ROOT as this makes sense to then append /ndk to it.
then comes my task
2 the caching task:
create variable for path NDK_HOME: /Users/runner/Library/Android/sdk/ndk
and add it in the cache task which completely looks like this
variables:
GRADLE_USER_HOME: $(Pipeline.Workspace)/.gradle
KONAN_USER_HOME: /Users/runner/.konan
NDK_HOME: /Users/runner/Library/Android/sdk/ndk
pool:
vmImage: 'macos-latest'
name: $(date:yyyy).$(Month)$(rev:.r)
steps:
- bash: env
displayName: env vars
- task: Cache#2
inputs:
key: 'gradle | "$(Agent.OS)"'
restoreKeys: gradle
path: $(GRADLE_USER_HOME)
displayName: Gradle build cache
- task: Cache#2
inputs:
key: 'konan | "$(Agent.OS)" | cache'
restoreKeys: konan
path: $(KONAN_USER_HOME)
displayName: Konan build cache
- task: Cache#2
inputs:
key: 'ndk | "$(Agent.OS)"'
restoreKeys: ndk
path: $(NDK_HOME)/ndk
displayName: NDK build cache
Works like a charm 🤞
One other way to dynamically create your variable from this answer. Thank you
then my task looks like
steps:
- bash: env
displayName: env vars
- bash: echo "##vso[task.setvariable variable=ANDROID_SDK_ROOT;]$ANDROID_SDK_ROOT"
- task: Cache#2
inputs:
key: 'ndk | "$(Agent.OS)"'
restoreKeys: ndk
path: $(ANDROID_SDK_ROOT)/ndk
displayName: NDK build cache
works in same manner

BubbleWrap CLI shows Error The given android-sdk isn't correct when bubblewrap build

I want to build TWA using BubbleWrap CLI
When I try to build my app using bubblewrap build
| |) /_,--.,--| |-.| |-.| |,---.,--. ,--,--.--.,--,--.,---.
| .-. | || | .-. | .-. | | .-. | |.'.| | .--' ,-. | .-. |
| '--' ' '' | `-' | `-' | \ --| .'. | | \ '-' | '-' '
`------' `----' `---' `---'`--'`----'--' '--`--' `--`--| |-'
`--'
(node:1798) ExperimentalWarning: The fs.promises API is experimental
cli ERROR The given androidSdk isn't correct.
It shows error cli Eroor The given androidSdk isn't correct
Here is my file structure is
- application
|
|- jdk
|- android-sdk
|- my-app
My ~/.bubblewrap/config.json contains
{
"jdkPath":"~/application/jdk",
"androidSdkPath":"~/application/andoid-sdk"
}
My OS
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

I got Bubblewrap to work with the latest version of command-line tools downloaded from https://developer.android.com/studio. (currently 6858069)
The downloaded archive file will have a directory called cmdline-tools, this needs to be renamed tools and put inside the android-sdk directory.
Your directory structure should look like
./android-sdk/
./android-sdk/tools
./android-sdk/tools/bin
./android-sdk/tools/lib
Then Bubblewrap asks for android-sdk I used the absolute path to the android-sdk directory, instead of using the ~ (tilde) character. I do not believe ~ is supported by Bubblewrap, as it is a node application and node does not support ~ by default according to https://github.com/nodejs/node/issues/684 . This should be true for your jdk path also.
Bubblewrap will continue and ask you to accept the licenses. After, it will start downloaded extra files to the android-sdk directory so read-write access is also needed.

*Modify the folder of the android-sdk to look like this: android-sdk -> tools -> bin, lib
Optional Commands:
bubblewrap updateConfig --androidSdkPath [your android-sdk path]
run bubblewrap doctor to check all
Hope it will help.

to solve when you can not execute and you get the address error message. just duplicate and copy the sdk/tools/bin/sdkmanager.bat to sdk/tools/bin/sdkmanager, in the same directory.

NPM Cache Step not working in Azure DevOps

I followed the following doc from Microsoft to configure the npm cache step for android app i am trying to build in azure and instead of package.json-Lock i am using package.json.
https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops#nodejsnpm
I am able to upload the cache dependency file in the post-cache step and upload that file in the beginning correctly when running the pipeline for 2nd time but even after the npm cache data is downloaded in the workspace the npm install step is still calling the remote libraries and downloading the remote dependencies.
I have also tried to run npm install --prefer-offline for the npm install step but did work. Please let me know if i am missing anything more.
Thankyou.

Use the Cache task to cache your application's node_modules folder. Use the cache hit variable (cacheHitVar) to store the result of the cache restoration. It will be set to true when the cache is restored (cache hit), otherwise set to false.
Then use a condition for the task that installs your dependencies (e.g. npm ci). Only install them on a cache miss.
steps:
- task: Cache#2
displayName: Cache node_modules
inputs:
key: 'npm | "$(Agent.OS)" | $(Build.SourcesDirectory)/package-lock.json'
path: $(Build.SourcesDirectory)/node_modules
cacheHitVar: CACHE_RESTORED
- task: Npm#1
displayName: 'Install the dependencies'
inputs:
command: custom
verbose: false
customCommand: 'ci'
condition: ne(variables.CACHE_RESTORED, 'true')
You'll see the following output in the pipeline execution when the cache was restored successfully.

Please check the following official recommendation from Microsoft regarding node modules caching https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops#nodejsnpm
variables:
npm_config_cache: $(Pipeline.Workspace)/.npm
steps:
- task: Cache#2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
path: $(npm_config_cache)
displayName: Cache npm
- script: npm ci
Because npm ci deletes the node_modules folder to ensure that a consistent, repeatable set of modules is used, you should avoid caching node_modules when calling npm ci.

This isn't much of an answer, because i have the exact same problem however this is my setup.
- task: Cache#2
displayName: Cache npm
inputs:
key: 'npm | "$(Agent.OS)" | $(Build.SourcesDirectory)/XX/package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
path: $(npm_config_cache)
- task: Npm#1
displayName: Npm restore dependencies
inputs:
command: 'custom'
workingDir: '$(clientapps)'
customCommand: 'install --cache $(npm_config_cache)'
adding --cache sets npm's cache file to a specific location. i'm now running a build with --prefer-offline and see if that helps. i'll answer here if that helps.

Path to repository in travis

I am not able to locate the path to my repo (that is cloned) in travis. When I perform ls, there is no output. I have tried ls ~/username/repo and ls ~/repo but get no such file or directory
Some background: I am integrating my android app with travis-ci. I want to create a new .java file, before build in travis. Since this file is private and added to .gitignore and is on build machines only. I intend to add content to this file by using encrypted environment variable feature.
Any custom command doesn't show the output. Not even echo. There is an environment variable $TRAVIS_BUILD_DIR but not sure how to use it

I guess it's a syntax issue, be sure you use it like here, it normally works:
TRAVIS_BUILD_DIR: The absolute path to the directory where the
repository being built has been copied on the worker.
- ${TRAVIS_BUILD_DIR}/gradle/caches/
- ls ${TRAVIS_BUILD_DIR}/gradle/caches/
Also check the lines below from here that links to this build and travis.yml (it works as shown here):
env:
global:
- ANDROID_HOME=${TRAVIS_BUILD_DIR}/android-sdk
- PATH=${ANDROID_HOME}/:${ANDROID_HOME}/tools/:${ANDROID_HOME}/platform-tools/:${PATH}
before_install:
- cp -R /usr/local/android-sdk-23.0.2 ./android-sdk
... You could create a ${TRAVIS_BUILD_DIR}/.travis-ci folder and save
there all the stuff so we can custom it.
# Comment out the lines below to show system image properties
- 'echo ${ANDROID_HOME}/system-images/${AVD_TARGET_DIR}/${AVD_TAG}/${AVD_ABI}'
- 'ls ${ANDROID_HOME}/system-images/${AVD_TARGET_DIR}/${AVD_TAG}/${AVD_ABI} -al || true'
- 'cat ${ANDROID_HOME}/system-images/${AVD_TARGET_DIR}/${AVD_TAG}/${AVD_ABI}/build.prop || true'
- echo 'DEFAULT SCRIPTS'
# Comment out the lines below to check default android scripts and PATH
- echo "$PATH"
- ls /usr/local/bin -Al
- cat /usr/local/bin/android-accept-licenses
- cat /usr/local/bin/android-update-sdk
- cat /usr/local/bin/android-wait-for-emulator

How to display HTML-formatted output files in Travis?

I'm trying to deploy an Android library on Bintray using Travis-CI. But when I upload my repo... I got this:
Ran lint on variant release: 6 issues found
Ran lint on variant debug: 6 issues found
Wrote HTML report to file:///home/travis/build/leandroBorgesFerreira/MoreCLoseButton/app/build/outputs/lint-results-debug.html
Wrote XML report to file:///home/travis/build/leandroBorgesFerreira/MoreCLoseButton/app/build/outputs/lint-results-debug.xml
:app:lint FAILED
Normally I would go to my project out put and read the lint-results-debug.html... But I don't know how to access this file in Travis.
So, How can I access outputs/lint-results-debug.html in Travis??
Any help is welcome!
Edit
my .travis.yml:
language: android
jdk: oraclejdk8
sudo: false
addons:
apt:
packages:
- lynx
android:
components:
- platform-tools
- tools
- build-tools-25.0.0
- android-25
- extra-android-m2repository
script:
- if [ -f /home/travis/build/leandroBorgesFerreira/MoreCLoseButton/app/build/outputs/lint-results-debug.html ]; then lynx -dump /home/travis/build/leandroBorgesFerreira/MoreCLoseButton/app/build/outputs/lint-results-debug.html; fi
- ./gradlew -PbintrayUser="${bintrayUser}" -PbintrayKey="${bintrayKey}" build
bintrayUpload --stacktrace --info
env:
global:
- secure: [...]
- secure: [...]

You can use lynx -dump to dump a plain-text rendering of any HTML file output from a Travis run.
To make Travis install lynx -dump: To the top of your .travis.yml, add this:
addons:
apt:
packages:
- lynx
Assuming the HTML file is an error log of some kind, you can make Travis show the output by putting something like the following in the script part of your .travis.yml:
after_failure:
- if [ -f /home/travis/build/…/foo.html ]; then lynx -dump /home/travis/build/…/foo.html; fi

While sideshowbarker gave a generic answer, I'd like to point out that Android lint has an option for console output, so you can do this in your build.gradle:
android {
lintOptions {
textReport = true
//textOutput "stdout" // default location, perfect for travis
}
}
Which removes the need for an extra dependency, and an extra script; plus it's reproducible on local machine easily.
One can take this a step further (in case spamming console on local machine is to be avoided) and do
android {
lintOptions {
textReport = project.property("lint.output.console").toBoolean()
}
}
and in gradle.properties: lint.output.console=false
and in .travis.yml: gradlew -Plint.output.console=true build