Develop Reference

Custom lint checks stopped running for multi-module Android project after upgrading to Android Gradle Plugin 4.2.2

Last year we successfully implemented some custom lint rules for our multi-module Android project by following https://proandroiddev.com/implementing-your-first-android-lint-rule-6e572383b292. At some point the lint rules stopped being applied, which I discovered by chance. By using git bisect on my repository, I was able to determine that the rules stopped being applied in a commit which upgraded the following:
Gradle in gradle-wrapper.properties: distributionUrl=https\://services.gradle.org/distributions/gradle-6.5-all.zip => gradle-6.7.1-all.zip
Android Gradle plugin in our top-level build.gradle: classpath "com.android.tools.build:gradle:4.1.3"' => gradle:4.2.2
Someone asked a similarly worded question 3 years ago but in their case they did not isolate a potential cause as the upgrade of Gradle or Android Gradle plugin.
Update 12/29/2021
I figured out that this has to do with multi-module projects. I compared the call stack that calls my custom lint Detector.GradleScanner and discovered that it changed from Android Gradle Plugin 4.1.3 to 4.2.2:
com.android.tools.build:gradle:4.1.3
at com.locuslabs.lintrules.GradleLibraryNonStableVersionDetector.checkDslPropertyAssignment(GradleLibraryNonStableVersionDetector.kt:22)
...
at com.android.tools.lint.client.api.LintDriver.checkBuildScripts(LintDriver.kt:1203)
at com.android.tools.lint.client.api.LintDriver.runFileDetectors(LintDriver.kt:1151)
at com.android.tools.lint.client.api.LintDriver.checkProject(LintDriver.kt:926)
at com.android.tools.lint.client.api.LintDriver.analyze(LintDriver.kt:446)
at com.android.tools.lint.LintCliClient.run(LintCliClient.kt:256)
at com.android.tools.lint.LintCliClient.run(LintCliClient.kt:237)
at com.android.tools.lint.gradle.LintGradleClient.run(LintGradleClient.kt:255)
at com.android.tools.lint.gradle.LintGradleExecution.runLint(LintGradleExecution.kt:259)
at com.android.tools.lint.gradle.LintGradleExecution.lintAllVariants(LintGradleExecution.kt:318)
at com.android.tools.lint.gradle.LintGradleExecution.analyze(LintGradleExecution.kt:68)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.android.tools.lint.gradle.api.ReflectiveLintRunner.runLint(ReflectiveLintRunner.kt:38)
at com.android.build.gradle.tasks.LintBaseTask.runLint(LintBaseTask.java:117)
at com.android.build.gradle.tasks.LintGlobalTask.lint(LintGlobalTask.java:51)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
...
com.android.tools.build:gradle:4.2.2
at com.locuslabs.lintrules.GradleLibraryNonStableVersionDetector.checkDslPropertyAssignment(GradleLibraryNonStableVersionDetector.kt:22)
...
at com.android.tools.lint.client.api.LintDriver$checkBuildScripts$2.run(LintDriver.kt:1490)
at com.android.tools.lint.client.api.LintClient.runReadAction(LintClient.kt:1780)
at com.android.tools.lint.client.api.LintDriver$LintClientWrapper.runReadAction(LintDriver.kt:2634)
at com.android.tools.lint.client.api.LintDriver.checkBuildScripts(LintDriver.kt:1483)
at com.android.tools.lint.client.api.LintDriver.runFileDetectors(LintDriver.kt:1401)
at com.android.tools.lint.client.api.LintDriver.checkProject(LintDriver.kt:1151)
at com.android.tools.lint.client.api.LintDriver.checkProjectRoot(LintDriver.kt:628)
at com.android.tools.lint.client.api.LintDriver.access$checkProjectRoot(LintDriver.kt:153)
at com.android.tools.lint.client.api.LintDriver$analyzeOnly$1.invoke(LintDriver.kt:442)
at com.android.tools.lint.client.api.LintDriver$analyzeOnly$1.invoke(LintDriver.kt:435)
at com.android.tools.lint.client.api.LintDriver.doAnalyze(LintDriver.kt:502)
at com.android.tools.lint.client.api.LintDriver.analyzeOnly(LintDriver.kt:434)
at com.android.tools.lint.LintCliClient$analyzeOnly$1.invoke(LintCliClient.kt:241)
at com.android.tools.lint.LintCliClient$analyzeOnly$1.invoke(LintCliClient.kt:241)
at com.android.tools.lint.LintCliClient.run(LintCliClient.kt:283)
at com.android.tools.lint.LintCliClient.run$default(LintCliClient.kt:266)
at com.android.tools.lint.LintCliClient.analyzeOnly(LintCliClient.kt:241)
at com.android.tools.lint.Main.run(Main.java:1536)
at com.android.tools.lint.Main.run(Main.java:269)
...
So the analyze() function was changed to start calling a new function checkProjectRoot(). I investigated when the function checkProjectRoot() was introduced by going though the Android Gradle Plugin source code. The command git log -S checkProjectRoot indicates the following commit introduced checkProjectRoot() which implies that multi-module analysis changed:
base[mirror-goog-studio-main]10:06:10 git log -S checkProjectRoot
commit 1d01e75bf984568ecdf198cd35bfe05c8b0cce9f
Author: Tor Norbye <tnorbye#google.com>
Date: Tue Jan 19 16:18:19 2021 -0800
178514993: Support module-independent analysis in Lint
This CL adds support into lint for "provisional reporting"; this is a
big architectural change which is key to significantly speeding up
the performance (and incrementality) of lint in Gradle.
The key idea is that, for lint embedding clients which support it such
as Gradle, modules are analyzed one at a time, independently (and
possibly in parallel), without access to sources from any upstream
dependencies. In some cases that means they can't conclusively report
issues (for example, if they depend on the consuming module's
minSdkVersion), but the detectors record enough state such that in a
merging phase (reporting), they can look at their previously computed
results and map it into definite warnings.
This requires many detectors to be updated to explicitly handle this new
mode. There are a handful of typical scenarios where a detector needs to
be updated: accessing the minSdkVersion of the consuming module, or
depending on the type of the consuming mode (e.g. is it also a library?
or an Android module?).
To simplify this, most lint checks can simply pass in one of a number of
built-in constraints, e.g.
context.report(incident, minSdkAtLeast(minSdk))
There are several mechanisms included to help catch detectors that need
to be updated. First, various lint APIs are not valid to be called when
in analysis rather than reporting mode, and if a detector calls them,
lint will emit a warning message.
Private.java: Error: The lint detector
com.android.tools.lint.checks.PrivateResourceDetector
called context.getMainProject() during module analysis.
This does not work correctly when running in Lint Unit Tests.
In particular, there may be false positives or false negatives
[...]
Second, there's a new TestMode for lint's unit tests which causes all
unit tests to be analyzed both with and without module-independent
analysis. To catch common problems, in addition to running the
infrastructure with module-independent analysis enabled and in separate
and analysis phases, it also varies the minSdkVersion between the two
phases to ensure that the lint checks aren't just basing their analysis
on the module's minSdkVersion.
NOTE: This CL only adds support for module-independent analysis in lint
itself, and the detectors, and the unit testing infrastructure. It does
not hook this up to for example AGP, which will be done separately.
Test: Existing
Bug: 178514993
Change-Id: Iffa25707792e416eab3ce446c1f8d9d8cc830295
Does anyone know what changes are needed to make custom lint rules apply to all modules of a project, not just the top-level? Currently, I've implemented my rule as follows:
internal const val PRIORITY = 10
#Suppress("UnstableApiUsage")
class GradleLibraryNonStableVersionDetector : Detector(), Detector.GradleScanner {
override fun checkDslPropertyAssignment(
context: GradleContext,
property: String,
value: String,
parent: String,
parentParent: String?,
propertyCookie: Any,
valueCookie: Any,
statementCookie: Any
) {
super.checkDslPropertyAssignment(context, property, value, parent, parentParent, propertyCookie, valueCookie, statementCookie)
if (property == "implementation"
&& (value.contains("-alpha")
|| value.contains("-beta")
|| value.contains("-rc"))) {
context.report(ISSUE,
statementCookie, context.getLocation(statementCookie),
"Use only stable versions of the library")
}
}
companion object {
#JvmField
val ISSUE: Issue = Issue.create("NonStableLibraryVersion",
"Marks usage of unstable version of dependency library.",
"It's not recommended to use alpha, beta or release candidate library versions in production",
CORRECTNESS, PRIORITY, Severity.ERROR,
Implementation(GradleLibraryNonStableVersionDetector::class.java,
Scope.GRADLE_SCOPE))
}
}
And I expect when I call the lint tool as follows, it should apply to all build.gradle files across all modules:
./gradlew -Pci --console=plain :sdk:lint -PbuildDir=lint
Update 1/7/2022
I reported a similar issue on the Google Sample project for lint rules.

What is `encode()` function detected in JaCoCo coverage report (for Kolin)?

Recently I upgrade my gradle from 6.5.1 to 7.0 i.e.
distributionUrl=https\://services.gradle.org/distributions/gradle-7.0-all.zip
After the upgrade, I notice my test coverage dropped.
After investigation, I found out that in Gradle 6.5.1, my JaCoCo test coverage report shows some of my class has an encode() function, but it no longer exists in Gradle 7.0.
I dig out the simplest example below. The code is as below
In Gradle 6.5, my JaCoCo report is as below (notice there's an additional encode() function.
However, in Gradle 7.0, my JaCoCo report is as below
Because of this missing covered function, hence my coverage dropped. Nonetheless, it looks like in Gradle 7.0, that is more correct, as my real code doesn't have encode().
I'm just trying to understand where encode() function is there in the first place, and why it is no longer in Gradle 7.0? And is it correct that my assumption that Gradle 7.0 result is correct?

Different versions of Gradle come with different values of jacoco.toolVersion which you also can change in your build.gradle, and which controls version of JaCoCo to use.
Gradle 6.5 by default uses JaCoCo 0.8.5, and starting from Gradle 6.7 default is 0.8.6.
And here is a list of changes in JaCoCo version 0.8.6 - https://www.jacoco.org/jacoco/trunk/doc/changes.html
Most likely your interface ContextData contains method encode with default implementation.
JVM supports default methods in interfaces starting from version 8, however Kotlin supports compilation into bytecode of version 6. To do so Kotlin compiler does a trick for such methods - it generates method in classes implementing this interface that merely delegates execution to default implementation:
for the following Example.kt
interface ContextData {
fun encode() = { /* something */ }
}
data class SearchRefinementModalOpenData(
val userAction: String?
) : ContextData
execution of
kotlin-compiler-1.4.32/bin/kotlinc Example.kt
javap -v -p SearchRefinementModalOpenData.class
shows following bytecode
public final class SearchRefinementModalOpenData implements ContextData
{
public kotlin.jvm.functions.Function0<kotlin.Unit> encode();
descriptor: ()Lkotlin/jvm/functions/Function0;
flags: (0x0001) ACC_PUBLIC
Code:
stack=1, locals=1, args_size=1
0: aload_0
1: invokestatic #29 // Method ContextData$DefaultImpls.encode:(LContextData;)Lkotlin/jvm/functions/Function0;
4: areturn
JaCoCo starting from version 0.8.6 filters out such methods since they are compiler artifacts and not presented in original source code.

It looks like you are using kotlinx.Serialization.
That generates a few encodeToString and similar functions for your objects. I guess Gradle 7 now ignores those?

Android: after Gradle update to 6.7.1 ClassLoader in JUnit test no longer lists all resources

I need to iterate over specific classes from main package in my android unit test, to check some of their properties.
For this I use standard approach, using ClassLoader:
val classLoader = Thread.currentThread().contextClassLoader
val resources: Enumeration<URL> = classLoader.getResources("com/models/package")
assert(resources.hasMoreElements()) // Fails from CL, works in AS
Before the Gradle update (had Gradle 5.6.4) that worked. Now the behaviour is as follows: it works when test is run from Android Studio, but fails (returns empty enumeration) when run from command line with gradlew.
I wonder what might be the difference in this respect between the two Gradle versions? And why it still works when run from Studio?
Some considerations and things I have tried:
Referencing these classes in unit test works ok, and also classLoader.findClass("com.models.package.MyModel") and
classLoader.loadClass("com.models.package.MyModel") from unit test is working. But even after that classLoader.getResources("com/models/package") returns empty enumeration.
Using other references to ClassLoader, like MyModel::class.java.classLoader and ClassLoader.getSystemClassLoader() didn't make any difference.
Gradle build from command line contains the warning "OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended", but as far as I can tell it's not connected to my issue.
If I put some of the classes from 'com/models/package' to the unit test /test folder, they are getting returned in enumeration.
This might be connected with some new optimisation setting that makes ClassLoaders omit registering some of the classes, in different root directories, but as it still works in AS there might be some setting to turn this optimisation off in a command line build also?
Thank you for any suggestions on this.

In Gradle 6.7.1 I had to include the directory with the code to the test sourceSets. Afterwards the classloader from junit started to see the classes and return them in Enumeration.
sourceSets {
test {
java.srcDirs += ['src/main']
}
}

How to generate OpenAPI sources from gradle when building Android app

What I'm trying to achieve
I'm trying to generate my REST API client for Android using OpenAPI Generator from the build.gradle script. That way, I wouldn't have to run the generator command line every time the specs change. Ideally, this would be generated when I build/assemble my app, and the sources would end up in the java (generated) folder, where generated sources are then accessible from the code (this is what happens with the BuildConfig.java file for example).
What I've tried so far
Following this link from their official GitHub, here's the build.gradle file I ended up with:
apply plugin: 'com.android.application'
apply plugin: 'org.openapi.generator'
...
openApiValidate {
inputSpec = "$rootDir/app/src/main/openapi/my-api.yaml"
recommend = true
}
openApiGenerate {
generatorName = "java"
inputSpec = "$rootDir/app/src/main/openapi/my-api.yaml"
outputDir = "$buildDir/generated/openapi"
groupId = "$project.group"
id = "$project.name-openapi"
version = "$project.version"
apiPackage = "com.example.mypackage.api"
invokerPackage = "com.example.mypackage.invoker"
modelPackage = "com.example.mypackage.model"
configOptions = [
java8 : "true",
dateLibrary : "java8",
library : "retrofit2"
]
}
...
First, I've never managed to get the API generated with the build/assemble task, even when I tried adding:
compileJava.dependsOn tasks.openApiGenerate
or
assemble.dependsOn tasks.openApiGenerate
The only way I could generate the sources was by manually triggering the openApiGenerate task:
Then, when I do generate my sources this way, they end up in the build folder but aren't accessible from my code, and aren't visible in the java (generated) folder:
I then have to manually copy/paste the generated source files to my project sources in order to use the API.
Even though I'm able to work around these issues by adding manual procedures, it would be way more maintainable if the whole process was simply automatic. I was able to achieve a similar result with another tool, Protobuf. Indeed, my gradle task gets triggered every time I build the app, and the sources end up in the java (generated) folder, so I don't have to do any additional work. The task is much simpler though, so I assume the main work that I'm not able to replicate with OpenAPI Generator is handled by the Protobuf plugin itself.

You have to specify path to the generated sources as a custom source set for your Gradle module, which is app in this case, as described here – https://developer.android.com/studio/build/build-variants#configure-sourcesets. That way Gradle will treat your sources as accessible from your code.
Something like this:
android {
...
sourceSets {
main {
java.srcDirs = ['build/generated/openapi/src/main/java']
}
}
...
}

I solved the issue you described like this, I'm using gradle.kts however.
See my build.gradle.kts
plugins {
// Your other plugins
id("org.openapi.generator") version "5.3.0"
}
openApiGenerate {
generatorName.set("kotlin")
inputSpec.set("$rootDir/app/src/main/openapi/my-api.yaml")
outputDir.set("$buildDir/generated/api")
// Your other specification
}
application {
// Your other code
sourceSets {
main {
java {
// TODO: Set this path according to what was generated for you
srcDir("$buildDir/generated/api/src/main/kotlin")
}
}
}
}
tasks.compileKotlin {
dependsOn(tasks.openApiGenerate)
}
You need to build the application at least once for the IDE to detect the library (at least this is the case for me in Intellij)

Your build should automatically generate the open api classes , to refer the generated classes in your java project you should add the generated class path to your source directory like it was mentioned in the other answers
https://developer.android.com/studio/build/build-variants#configure-sourcesets
As far as the task dependency goes , in android tasks are generated after configuration thus for gradle to recognize the task , wrap it inside afterEvaluate block like
afterEvaluate {
tasks.compileDebugJavaWithJavac.dependsOn(tasks.openApiGenerate)
}

I had this issue, and this answer https://stackoverflow.com/a/55646891/14111809 led me to a more informative error:
error: incompatible types: Object cannot be converted to Annotation
#java.lang.Object()
Taking a look at the generated files that were causing this error, noticed:
import com.squareup.moshi.Json;
After including a Moshi in the app build.gradle, the build succeeded and the generated code was accessible.
implementation("com.squareup.moshi:moshi-kotlin:1.13.0")

Is there anyway to prepend a jar to the unmanagedClasspath in sbt

I am using the android-sbt-plugin with the sbt, and I would like to add an unmanaged jar to the test classpath. The reason being android.jar contains stub functions for the org.json libraries and results in exceptions being thrown for unit tests. This is what I am doing
unmanagedClasspath in Test <+= (baseDirectory) map { base =>
Attributed.blank(base/"test-libs"/"json.jar")
}
Because of the order of the jars this file is ignored during when i run the test command within the sbt. If I type the command the order clearly shows the android.jar as the first jar
show test:unmanaged-classpath
[info] ArrayBuffer(Attributed(/home/rohit/Projects/android-sdk-linux/platforms/android- 17/android.jar), Attributed(/home/rohit/Projects/barfrendz/trunk/src/buzze/test-libs/json.jar))
If I create a lib folder and let sbt pick up the json jar the order is reversed the tests now run, but I can no longer create an android package due to conflicts with the org.json namespace in android.jar. Here is the exception
[error] (Buzze/android:proguard) java.io.IOException: Can't read [/home/rohit/Projects/barfrendz/trunk/src/buzze/lib/json.jar(;;;;!META-INF/MANIFEST.MF,!**/R.class,!**/R$*.class,!**/TR.class,!**/TR$.class,!**/library.properties)] (Can't process class [org/json/CDL.class] (Unsupported version number [51.0] for class format))
Is there anyway I can change the order of the jars in the classpath for the unit tests?

Instead of using <+=, use <<=, get unmanagedClasspath itself as a dependency, and then modify it as desired. The documentation has such an example with resolvers:
resolvers <<= resolvers {rs =>
val localMaven = "Local Maven Repository" at "file://"+Path.userHome.absolutePath+"/.m2/repository"
localMaven +: rs
}
This way, localMaven ends up first in resolvers.
According to the API docs, the unmanagedClasspath is a Task of type Classpath. Note that when you use that syntax, you are changing the Classpath, not the Task.
The API doc for the classpath is here -- it's a type, and it points to Seq[Attributed[File]], so you can manipulate it with any Seq command. I tried out the snippet here and it works:
$ cat build.sbt
unmanagedClasspath in Test <<= (unmanagedClasspath in Test, baseDirectory) map { (uc, base) =>
Attributed.blank(base/"test-libs"/"json.jar") +: uc
}
Daniel#DANIEL-PC /c/scala/Programas/sbtTest
$ sbt
[info] Set current project to default-60c6f9 (in build file:/C:/scala/Programas/sbtTest/)
> show test:unmanaged-classpath
[info] ArrayBuffer(Attributed(C:\scala\Programas\sbtTest\test-libs\json.jar))
[success] Total time: 0 s, completed 30/08/2013 13:32:42
>

Maybe overriding the unmanagedJars instead of the unmanagedClasspath would allow you to do this:
http://www.scala-sbt.org/0.12.3/docs/Detailed-Topics/Library-Management.html