How to cache Android NDK in Azure devops pipeline? - android

While creating a task in Azure devops pipeline, I am requred to cache the NDK to reduce the build time. Hence, I am trying to write a task for the same.
For example here is what I am using for caching the gradle and konan repos
variables:
GRADLE_USER_HOME: $(Pipeline.Workspace)/.gradle
KONAN_USER_HOME: /Users/runner/.konan
steps:
- bash: env
displayName: env vars
- task: Cache#2
inputs:
key: 'gradle | "$(Agent.OS)"'
restoreKeys: gradle
path: $(GRADLE_USER_HOME)
displayName: Gradle build cache
- task: Cache#2
inputs:
key: 'konan | "$(Agent.OS)" | cache'
restoreKeys: konan
path: $(KONAN_USER_HOME)
displayName: Konan build cache
they work just fine!
But for NDK, how do I know the relative repository path for my NDK and cache it, assuming the task will be something like this
- task: Cache#2
inputs:
key: 'ndk | "$(Agent.OS)"'
restoreKeys: ndk
path: $(NDK_PATH)
displayName: NDK build cache
Any help in this is highly helpful.

How do I know the relative repository path for my NDK?
You can use the environment variable ANDROID_NDK_PATH to get the path for the NDK.
If you are using the ndk that pre-installed in Microsoft hosted agent windows-latest, the value of ANDROID_NDK_PATH is C:\Program Files (x86)\Android\android-sdk\ndk-bundle.

So, I figured this out, forgot to post here and this is what I found.
1 - to get the relative path of android-ndk from devops setup (I dont have those permissions).
2 - Then run the caching task to cache into that path
1 - get relative env vars with this bash output. CREDITS So In steps I write:
steps:
- bash: env
displayName: env vars
prints all the env variables available example gist with some vars intentionally removed
from that I picked up the path relevant to my use
2021-01-19T16:17:44.4273060Z ANDROID_HOME=/Users/runner/Library/Android/sdk
2021-01-19T16:17:44.4276800Z ANDROID_SDK_ROOT=/Users/runner/Library/Android/sdk
2021-01-19T16:17:44.4320010Z ANDROID_NDK_18R_PATH=/Users/runner/Library/Android/sdk/ndk/18.1.5063045
2021-01-19T16:17:44.4328690Z ANDROID_NDK_HOME=/Users/runner/Library/Android/sdk/ndk-bundle
now the most relevant path was to ANDROID_SDK_ROOT as this makes sense to then append /ndk to it.
then comes my task
2 the caching task:
create variable for path NDK_HOME: /Users/runner/Library/Android/sdk/ndk
and add it in the cache task which completely looks like this
variables:
GRADLE_USER_HOME: $(Pipeline.Workspace)/.gradle
KONAN_USER_HOME: /Users/runner/.konan
NDK_HOME: /Users/runner/Library/Android/sdk/ndk
pool:
vmImage: 'macos-latest'
name: $(date:yyyy).$(Month)$(rev:.r)
steps:
- bash: env
displayName: env vars
- task: Cache#2
inputs:
key: 'gradle | "$(Agent.OS)"'
restoreKeys: gradle
path: $(GRADLE_USER_HOME)
displayName: Gradle build cache
- task: Cache#2
inputs:
key: 'konan | "$(Agent.OS)" | cache'
restoreKeys: konan
path: $(KONAN_USER_HOME)
displayName: Konan build cache
- task: Cache#2
inputs:
key: 'ndk | "$(Agent.OS)"'
restoreKeys: ndk
path: $(NDK_HOME)/ndk
displayName: NDK build cache
Works like a charm 🤞
One other way to dynamically create your variable from this answer. Thank you
then my task looks like
steps:
- bash: env
displayName: env vars
- bash: echo "##vso[task.setvariable variable=ANDROID_SDK_ROOT;]$ANDROID_SDK_ROOT"
- task: Cache#2
inputs:
key: 'ndk | "$(Agent.OS)"'
restoreKeys: ndk
path: $(ANDROID_SDK_ROOT)/ndk
displayName: NDK build cache
works in same manner

Related

Azure Pipeline cmdLine error Bash exited with code '1'

I am going through the azure pipeline for my Android project. Issue is that local.properties file is in .gitignore so getting error local.properties not found for this i added its in secure file and now in YAML file i am downloading and installing taht secure file.
But I am stuck on the CmdLine step. line 18 error ##[error]Bash exited with code '1'.,
As for the pipeline yaml, I am using this template for android
# Android
# Build your Android project with Gradle.
# Add steps that test, sign, and distribute the APK, save build artifacts, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/android
trigger:
- dev_pooja
pool:
vmImage: 'macos-latest'
steps:
- task: DownloadSecureFile#1
name: localProperties
displayName: 'Download Local Properties file'
inputs:
secureFile: 'local.properties'
- script: |
echo Installing $(localProperties.secureFilePath) to the trusted directory...
sudo chown root:root $(localProperties.secureFilePath)
sudo chmod a+r $(localProperties.secureFilePath)
sudo ln -s -f /etc/ssl/certs/ $(localproperties.secureFilePath)
- task: JavaToolInstaller#0
inputs:
versionSpec: '11'
jdkArchitectureOption: 'x64'
jdkSourceOption: 'PreInstalled'
- task: Gradle#2
inputs:
workingDirectory: ''
gradleWrapperFile: 'gradlew'
gradleOptions: '-Xmx3072m'
publishJUnitResults: false
testResultsFiles: '**/TEST-*.xml'
tasks: 'assembleDebug'
- task: CopyFiles#2
inputs:
contents: "**/*.apk"
targetFolder: "$(build.artifactsStagingDirectory)"
- task: PublishBuildArtifacts#1
The error line : sudo ln -s -t /etc/ssl/certs/ $(localproperties.secureFilePath).
Try removing -t flag. In addition, you may need -f flag: If the target file already exists, then unlink it so that the link may occur.
Good luck!

Azure DevOps iOS and Android React Native Build Failing on Pod Install and Gradle Build Steps

I am trying to create CI/CD pipeline, for iOS and android, in Azure Devops for a React-Native app.
With the android pipeline, whether I choose to create a yaml or use the classic editor and use pre-configured android tasks, the pipeline always fails on the build task (bundleRelease). The error I receive is:
FAILURE: Build failed with an exception.
Where:
Settings file '/home/vsts/work/1/s/app-rn/android/settings.gradle' line: 12
What went wrong:
A problem occurred evaluating settings 'iEquos'.
Could not read script '/home/vsts/work/1/s/app-rn/node_modules/#react-native-community/cli-platform-android/native_modules.gradle' as it does not exist.
Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org
BUILD FAILED in 12s
Error: The process '/home/vsts/work/1/s/app-rn/android/gradlew' failed with exit code 1
at ExecState._setResult (/home/vsts/work/_tasks/Gradle_8d8eebd8-2b94-4c97-85af-839254cc6da4/2.200.2/node_modules/azure-pipelines-task-lib/toolrunner.js:944:25)
at ExecState.CheckComplete (/home/vsts/work/_tasks/Gradle_8d8eebd8-2b94-4c97-85af-839254cc6da4/2.200.2/node_modules/azure-pipelines-task-lib/toolrunner.js:927:18)
at ChildProcess. (/home/vsts/work/_tasks/Gradle_8d8eebd8-2b94-4c97-85af-839254cc6da4/2.200.2/node_modules/azure-pipelines-task-lib/toolrunner.js:840:19)
at ChildProcess.emit (events.js:198:13)
at maybeClose (internal/child_process.js:982:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
##[error]Error: The process '/home/vsts/work/1/s/app-rn/android/gradlew' failed with exit code 1
Finishing: Gradle
This is line being referred to in android/settings.gradle:
apply from: file("../node_modules/#react-native-community/cli-platform-android/native_modules.gradle"); applyNativeModulesSettingsGradle(settings, "../")
The node modules folder is in the 'app-rn' directory, at the same level as the "android" folder, so only using one "../" is correct?
I can build the android solution locally through a termianl or using Android studio so I am completely clueless to why this is occurring in DevOps.
A similar issue is occurring with my pipeline for iOS. The issue is occurring when istalling Cocoa Pods:
DevOps Install Cocoa Pods Error
Here is an image of my PodFile, located in the 'iOS' folder. The 'iOS' folder is located at the same level as 'node-modules', both inside a folder 'app-rn':
Podfile
Here is the yaml for android:
# Android
# Build your Android project with Gradle.
# Add steps that test, sign, and distribute the APK, save build artifacts, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/android
variables:
- group: DriverApp
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
name: $(date:yyyy).$(Month)$(rev:.r)
steps:
- script: yarn install
- task: Gradle#2
inputs:
gradleWrapperFile: 'app-rn/android/gradlew'
workingDirectory: 'app-rn/android/'
options: '-PversionName=$(Build.BuildNumber) -PversionCode=$(Build.BuildId)'
tasks: 'bundleRelease'
publishJUnitResults: false
javaHomeOption: 'JDKVersion'
gradleOptions: '-Xmx3072m'
sonarQubeRunAnalysis: false
- task: AndroidSigning#3
inputs:
apkFiles: '**/*.aab'
apksignerKeystoreFile: 'keystore.jks'
apksignerKeystorePassword: '$(AndroidKeyStorePassword)'
apksignerKeystoreAlias: '$(AndroidKeyAlias)'
apksignerKeyPassword: '$(AndroidKeyAliasPassword)'
zipalign: false
- task: PublishBuildArtifacts#1
inputs:
# PathtoPublish: 'android/app/build/outputs/apk/release'
PathtoPublish: 'android/app/build/outputs/'
ArtifactName: 'drop'
publishLocation: 'Container'
The yaml for iOS:
# trigger:
# branches:
# include:
# - master
variables:
- group: DriverApp
pool:
vmImage: 'macos-latest'
steps:
- checkout: self
persistCredentials: true
clean: true
- task: NodeTool#0
displayName: 'Install Node'
inputs:
versionSpec: '12.19.0' # you can use your desired version here
# workingDirectory: 'app-rn/'
- script: yarn install
displayName: Install Dependencies
- task: InstallAppleCertificate#2
displayName: Install Apple Certificate
inputs:
certSecureFile: 'AppleDistributionCertificate.p12'
certPwd: '$(AppleCertificatePassword)'
keychain: 'temp'
deleteCert: true
- task: InstallAppleProvisioningProfile#1
displayName: 'Install Apple Provisioning Profile'
inputs:
provisioningProfileLocation: 'secureFiles'
provProfileSecureFile: 'iEquos_App_Store.mobileprovision'
removeProfile: true
- task: CocoaPods#0
displayName: 'Install CocoaPods'
inputs:
workingDirectory: 'app-rn/ios'
- task: Xcode#5
displayName: 'Build IPA'
inputs:
actions: 'build'
configuration: 'Release'
sdk: 'iphoneos'
xcWorkspacePath: 'app-rn/ios/iEquos.xcworkspace'
scheme: 'iEquos'
packageApp: true
exportPath: 'output'
signingOption: 'manual'
signingIdentity: '$(APPLE_CERTIFICATE_SIGNING_IDENTITY)'
provisioningProfileUuid: '$(APPLE_PROV_PROFILE_UUID)'
- task: CopyFiles#2
displayName: 'Copy IPA'
inputs:
contents: '**/*.ipa'
targetFolder: '$(build.artifactStagingDirectory)'
overWrite: true
flattenFolders: true
- task: PublishBuildArtifacts#1
displayName: 'Publish IPA to artifacts'
inputs:
PathtoPublish: '$(build.artifactStagingDirectory)'
ArtifactName: 'ios'
publishLocation: 'Container'
I have searched stackoverflow for similar issues, but only seem to find people running into these issues locally, nto within DevOps. As I mentioned before, I can build the android and iOS app locally.
I am fairly new to posting on stack overflow so please let me know if I should provide anymore information.
Any help would be greatly appreciated.
Have you tried this in azure-pipelines.yml file:
- script: /usr/local/bin/pod deintegrate
workingDirectory: 'app-rn/ios'
displayName: 'pod deintegrate'
- script: /usr/local/bin/pod install
workingDirectory: 'app-rn/ios'
displayName: 'pod install'
insted of:
- task: CocoaPods#0
displayName: 'Install CocoaPods'
inputs:
workingDirectory: 'app-rn/ios'

Flutter Azure Devops Firebase: error: unknown option '--app'

I am trying to setup a pipeline that adds the apk to the app distribution in firebase. I have the following yml:
# Flutter build
variables:
app_token: 1:403...
firebase_token: 1//dsads...
trigger:
- develop
jobs:
- job: Android
pool:
vmImage: 'windows-latest'
steps:
- task: FlutterInstall#0
inputs:
channel: 'stable'
version: 'latest'
- task: FlutterBuild#0
inputs:
target: apk
- task: CopyFiles#2
inputs:
contents: '**/*.apk'
targetFolder: '$(build.artifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
artifactName: 'myartifactname'
pathtoPublish: '$(Build.ArtifactStagingDirectory)'
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- task: CmdLine#2
inputs:
script: 'npm install -g firebase-tools'
workingDirectory: '$(Agent.ToolsDirectory)'
displayName: 'install firebase tools'
- task: Bash#3
displayName: "Upload to firebase app distribution"
inputs:
targetType: "inline"
script: |
npm i -g firebase-tools
firebase appdistribution: distribute android/app/build/outputs/apk/release/app-release.apk \
--app "$(app_token)" \
--release-notes "From Azure Devops" \
--groups "testgroup" \
--token "$(firebase_token)"
This is the log:
C:\npm\prefix\firebase -> C:\npm\prefix\node_modules\firebase-tools\lib\bin\firebase.js
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents#~2.3.2 (node_modules\firebase-tools\node_modules\chokidar\node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents#2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
+ firebase-tools#9.19.0
updated 1 package in 11.855s
ls: cannot access 'D:a1a/build/app/outputs/flutter-apk': No such file or directory
error: unknown option '--app'
##[error]Bash exited with code '1'.
I don't find that what is wrong and how to fix it. Any help is appreciated.
Mostly code but not sure if you guys need additional informations. Thanks in advance.

How do you cache sdkmanager in AzureDevops?

I do the following in my pipeline
- bash: |
$ANDROID_SDK_ROOT/tools/bin/sdkmanager 'ndk;20.0.5594570' >/dev/null
$ANDROID_SDK_ROOT/tools/bin/sdkmanager 'ndk;21.0.6113669' >/dev/null
displayName: "install NDK"
Which takes about 3 minutes
I was wondering if I can cache this to speed things up. However, I am not sure where it puts the files.
It seems you want to cache ndk. You may check Pipeline caching:
https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops
Caching is added to a pipeline using the Cache task. Pipeline caching can help reduce build time by allowing the outputs or downloaded dependencies from one run to be reused in later runs, thereby reducing or avoiding the cost to recreate or redownload the same files again. Caching is especially useful in scenarios where the same dependencies are downloaded over and over at the start of each run.
As noted in a previous answer, the Cache#2 task is supposed to be used. And as per the comment path where the ndk is downloaded to is in ANDROID_SDK_ROOT. The thing to note is ANDROID_SDK_ROOT is not a variable that is immediately accessible. It needs to be exposed.
So to put it all together with a working example:
steps:
- bash: |
echo "##vso[task.setvariable variable=ANDROID_SDK_ROOT;]$ANDROID_SDK_ROOT"
- task: Cache#2
inputs:
key: 'ndk | "$(Agent.OS)"'
path: $(ANDROID_SDK_ROOT)/ndk
- bash: |
$ANDROID_SDK_ROOT/tools/bin/sdkmanager 'ndk;20.0.5594570' >/dev/null
$ANDROID_SDK_ROOT/tools/bin/sdkmanager 'ndk;21.0.6113669' >/dev/null
displayName: "install NDK"
This technique can be used in other places so that you'd get an agent pool agnostic script where the files are places in the home directory. Applies to both macOS and ubuntu pools.
steps:
- bash: |
echo "##vso[task.setvariable variable=HOME_DIRECTORY;]$HOME"
- task: Cache#2
inputs:
key: 'gradlew | "$(Agent.OS)"'
# this won't work on macOS
# path: /home/vsts/.gradle/wrapper
# but this will
path: $(HOME_DIRECTORY)/.gradle/wrapper
- task: Cache#2
inputs:
key: 'gradle | "$(Agent.OS)"'
path: $(HOME_DIRECTORY)/.gradle/caches

NPM Cache Step not working in Azure DevOps

I followed the following doc from Microsoft to configure the npm cache step for android app i am trying to build in azure and instead of package.json-Lock i am using package.json.
https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops#nodejsnpm
I am able to upload the cache dependency file in the post-cache step and upload that file in the beginning correctly when running the pipeline for 2nd time but even after the npm cache data is downloaded in the workspace the npm install step is still calling the remote libraries and downloading the remote dependencies.
I have also tried to run npm install --prefer-offline for the npm install step but did work. Please let me know if i am missing anything more.
Thankyou.
Use the Cache task to cache your application's node_modules folder. Use the cache hit variable (cacheHitVar) to store the result of the cache restoration. It will be set to true when the cache is restored (cache hit), otherwise set to false.
Then use a condition for the task that installs your dependencies (e.g. npm ci). Only install them on a cache miss.
steps:
- task: Cache#2
displayName: Cache node_modules
inputs:
key: 'npm | "$(Agent.OS)" | $(Build.SourcesDirectory)/package-lock.json'
path: $(Build.SourcesDirectory)/node_modules
cacheHitVar: CACHE_RESTORED
- task: Npm#1
displayName: 'Install the dependencies'
inputs:
command: custom
verbose: false
customCommand: 'ci'
condition: ne(variables.CACHE_RESTORED, 'true')
You'll see the following output in the pipeline execution when the cache was restored successfully.
Please check the following official recommendation from Microsoft regarding node modules caching https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops#nodejsnpm
variables:
npm_config_cache: $(Pipeline.Workspace)/.npm
steps:
- task: Cache#2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
path: $(npm_config_cache)
displayName: Cache npm
- script: npm ci
Because npm ci deletes the node_modules folder to ensure that a consistent, repeatable set of modules is used, you should avoid caching node_modules when calling npm ci.
This isn't much of an answer, because i have the exact same problem however this is my setup.
- task: Cache#2
displayName: Cache npm
inputs:
key: 'npm | "$(Agent.OS)" | $(Build.SourcesDirectory)/XX/package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
path: $(npm_config_cache)
- task: Npm#1
displayName: Npm restore dependencies
inputs:
command: 'custom'
workingDir: '$(clientapps)'
customCommand: 'install --cache $(npm_config_cache)'
adding --cache sets npm's cache file to a specific location. i'm now running a build with --prefer-offline and see if that helps. i'll answer here if that helps.

Categories

Resources