Develop Reference

Android: after Gradle update to 6.7.1 ClassLoader in JUnit test no longer lists all resources

I need to iterate over specific classes from main package in my android unit test, to check some of their properties.
For this I use standard approach, using ClassLoader:
val classLoader = Thread.currentThread().contextClassLoader
val resources: Enumeration<URL> = classLoader.getResources("com/models/package")
assert(resources.hasMoreElements()) // Fails from CL, works in AS
Before the Gradle update (had Gradle 5.6.4) that worked. Now the behaviour is as follows: it works when test is run from Android Studio, but fails (returns empty enumeration) when run from command line with gradlew.
I wonder what might be the difference in this respect between the two Gradle versions? And why it still works when run from Studio?
Some considerations and things I have tried:
Referencing these classes in unit test works ok, and also classLoader.findClass("com.models.package.MyModel") and
classLoader.loadClass("com.models.package.MyModel") from unit test is working. But even after that classLoader.getResources("com/models/package") returns empty enumeration.
Using other references to ClassLoader, like MyModel::class.java.classLoader and ClassLoader.getSystemClassLoader() didn't make any difference.
Gradle build from command line contains the warning "OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended", but as far as I can tell it's not connected to my issue.
If I put some of the classes from 'com/models/package' to the unit test /test folder, they are getting returned in enumeration.
This might be connected with some new optimisation setting that makes ClassLoaders omit registering some of the classes, in different root directories, but as it still works in AS there might be some setting to turn this optimisation off in a command line build also?
Thank you for any suggestions on this.

In Gradle 6.7.1 I had to include the directory with the code to the test sourceSets. Afterwards the classloader from junit started to see the classes and return them in Enumeration.
sourceSets {
test {
java.srcDirs += ['src/main']
}
}

AspectJ - Android IllegalStateException after adding firebase dependency

Using Ibotta, an AOP gradle plugin for android, seems not working after adding below dependency
implementation 'com.google.firebase:firebase-auth:20.0.3'
Actual Result:
Build was successful and seems not creating all dex
Runtime Error:
Application class not found in dex path list.
AOP Log showing below error :
java.lang.IllegalStateException: Expecting .,<, or ;, but found - while unpacking <MessageType:Lcom/google/android/gms/internal/firebase-auth-api/zzaaa<TMessageType;TBuilderType;>;BuilderType:Lcom/google/android/gms/internal/firebase-auth-api/zzzw<TMessageType;TBuilderType;>;>Lcom/google/android/gms/internal/firebase-auth-api/zzyj<TMessageType;TBuilderType;>;
at org.aspectj.util.GenericSignatureParser.parseClassTypeSignature(GenericSignatureParser.java:204)
at org.aspectj.util.GenericSignatureParser.parseFieldTypeSignature(GenericSignatureParser.java:155)
at org.aspectj.util.GenericSignatureParser.parseFormalTypeParameter(GenericSignatureParser.java:130)
at org.aspectj.util.GenericSignatureParser.parseAsClassSignature(GenericSignatureParser.java:51)
at org.aspectj.weaver.UnresolvedType.forGenericTypeSignature(UnresolvedType.java:275)
at org.aspectj.weaver.bcel.BcelWorld.addSourceObjectType(BcelWorld.java:479)
at org.aspectj.weaver.bcel.BcelWorld.addSourceObjectType(BcelWorld.java:453)
at org.aspectj.weaver.bcel.BcelWeaver.addAspectsFromJarFile(BcelWeaver.java:265)
at org.aspectj.weaver.bcel.BcelWeaver.addLibraryJarFile(BcelWeaver.java:238)
Below added the required code to reproduce this.
gradle-wrapper.properties
distributionUrl=https\://services.gradle.org/distributions/gradle-6.5-all.zip
project build.gradle
repositories {
maven { url "https://plugins.gradle.org/m2/" }
}
dependencies {
classpath "com.ibotta:plugin:1.1.0"
}
app build.gradle
apply plugin: "com.ibotta.gradle.aop"
dependencies {
//AspectJ
implementation "org.aspectj:aspectjrt:1.9.6"
implementation 'com.google.firebase:firebase-auth:20.0.3'
}
AspectLogging class
#Aspect
class AOPLog {
#Pointcut("within(com.example.aopdemo..*) && execution(* *(..))")
fun allMethods() {
}
#Around("allMethods()")
#Throws(Throwable::class)
open fun onMethodAdvice(joinPoint: JoinPoint?): Any? {
val methodSignature = joinPoint!!.signature as MethodSignature
val methodName = methodSignature.name
val startTime = System.currentTimeMillis();
val result = (joinPoint as ProceedingJoinPoint).proceed()
val endTime = System.currentTimeMillis() - startTime
Log.e("LoggingVM", "$methodName ---> $endTime")
return result
}
}
Note: Firebase dependency is just added and not using anywhere in the demo app.
com.example.aopdemo package just have a launcher activity, and invoking some methods on onCreate method. It was tracking all the methods before firebase added. After adding, getting above errors.

Using Ibotta, an AOP library for android
It is not an AOP library but a Gradle plugin for people wishing to weave AspectJ aspects into their Android target apps or libraries, see here.
What strikes me as odd in your error message is the package name com/google/android/gms/internal/firebase-auth-api/zzaaa - please note the hyphens. Actually, hyphen ("-") characters in package names are illegal, see Java naming conventions.
If Firebase itself or maybe some kind of code converter or obfuscator uses firebase-auth-api instead of something like firebase_auth_api or firebase.auth.api (depending on what it looks like in the original code base), it is no surprise at all that it derails the AspectJ signature parser. I wonder which compiler even permits that nowadays.
Fix the package name, then I guess the AspectJ problem will disappear too. The error reported by AspectJ is valid, the problem is outside of AspectJ.
Update: I looked at the AAR file you added to your class path and indeed it contains an illegal package name with obfuscated classes, probably on purpose in order to make it more difficult to work with and manipulate those classes:
$ unzip firebase-auth-20.0.3.aar -d firebase-auth
Archive: firebase-auth-20.0.3.aar
inflating: firebase-auth/AndroidManifest.xml
inflating: firebase-auth/R.txt
inflating: firebase-auth/classes.jar
inflating: firebase-auth/proguard.txt
inflating: firebase-auth/third_party_licenses.json
inflating: firebase-auth/third_party_licenses.txt
$ unzip -l firebase-auth/classes.jar | grep firebase-auth-api | head -n 10
1209 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zza.class
1032 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaa.class
10997 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaaa.class
4291 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaab.class
172 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaac.class
275 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaad.class
192 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaae.class
525 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaaf.class
2430 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaag.class
373 2021-02-26 09:54 com/google/android/gms/internal/firebase-auth-api/zzaah.class
Unfortunately, this also stops legal use cases from working. You may want to raise an issue or alternatively make sure to exclude Firebase or at least its internal packages when using AspectJ, e.g. via !within(com.google.android.gms.internal..*). Or do your aspects actually want to modify anything in Firebase?
Update 2: Inside Ibotta, there is a source code comment as follows:
A Gradle plugin that performs AOP weaving using a technique recommended by Gradle reps. It taps into Android's bytecode manipulation pipeline which is a far more logical approach.
The basic idea is:
Change the Kotlin and Java compile output directories.
Copy the Kotlin/Java compiled output to one directory.
Weave the combined Kotlin/Java classes.
Be registered as a bytecode generator so that Android recognizes the custom AOP weaving as a formal step in the build pipeline.
Maybe in using this approach instead of calling the stand-alone AspectJ compiler, somehow the byte code weaver has to load and see all classes, even the ones the normal AspectJ compiler would ignore due to the within(com.example.aopdemo..*) in your sample aspect or the safe-guarding !within(com.google.android.gms.internal..*) I suggested. Either way, that it works after you changed the Gradle build plugin tells me that you should raise an issue for the Ibotta project (not for Firebase like I initially thought). Actually, it seems that you already did that, see issue #5.
Update 3: A while ago, the Ibotta maintainer said that you should use the tool's own filtering feature and use something like
aopWeave {
filter = "com/example/aopdemo"
}

How to use sbt-android to build scala android projects with RenderScript files?

It seems that sbt-android is ignoring the presence of the RenderScript files and is not generating the ScriptC_xxx java classes that are normally generated by gradle builds/Android Studio [UPDATE: this is false, se the update note below]. Because of this issue, the sbt-android build is failing because to use the scripts we need to reference the generated ScriptC_xxx classes, which gives the following error when building:
[error] apackage/YYYClass.java:55: cannot find symbol
[error] symbol: class ScriptC_xxx
[error] location: class apackage.YYYClass
[error] ScriptC_xxx xxxScript = new ScriptC_xxx(rs);
The generated .sbt file from the existing project (which compiles normally by gradle) has the apparent necessary configuration for RenderScript, generated from the build.gradle file:
rsTargetApi in Android := "18",
rsSupportMode in Android := rsSupportMode.value || true
What I am missing to be able to compile renderscript-containing android projects with sbt-android ?
UPDATE: I realized that the java classes for the scripts are being generated correctly, but somehow the classes that use those generated classes cannot find them, so maybe I have a classpath configuration problem.
This is the current file structure:
./asubmodule/src/main/rs/xxx.rs
(using package declaration #pragma rs java_package_name(apackage.scripts))
./asubmodle/src/main/java/apackage/YYYClass.java
./asubmodule/target/android/generated/source/android/support/v7/appcompat/R.java
./asubmodule/target/android/generated/source/apackage/scripts/ScriptC_xxx.java

Unfortunately, it seems com.android.builder.core.AndroidBuilder no longer generates .d files for processed renderscript sources. Bypassing AndroidBuilder to use com.android.sdklib.build.RenderScriptProcessor in sbt-android could restore the .d files that are required to determine the generated files. But that would be a solution for a future version of sbt-android.
To work around your problem for now, you can add the following into your build.sbt:
sourceGenerators in Compile += Def.task {
implicit output = outputLayout.value
val layout = projectLayout.value
(layout.generatedSrc ** "ScriptC_*.java").get ++ (layout.generatedSrc ** "*BitCode.java").get
}.taskValue
Since it sounds like you're using sbt-android-gradle you can just create a new file build.sbt and throw the above in (do not edit the 00-gradle-generated.sbt)
Update: this will be fixed in the next version of sbt-android https://github.com/scala-android/sbt-android/commit/ec09a55233aabd50e7df0085b10c567e38b616d3

Android Studio Gradle - JavaExec classpath configuration - Java Reflection - class access

Im currently working on an android project where i have to process .java-files to possibly generate another .java-files which should then be compiled and packed into the .apk-file.
Lets assume i have 2 files which will be processed by my library, FILE_A.java and FILE_B.java.
Now i need to access these files within my library via reflection, e.g. with:
Class.forName("com.test.entities.FILE_A");
Class.forName("com.test.entities.FILE_B");
The problem is that i'm not able to access the class files, i think because of the missing classpath configuration. Currently i use this task to call my .jar-file:
task (mytask, type: org.gradle.api.tasks.JavaExec) {
classpath(files('libs/myjar.jar'))
main('com.test.TestMain')
}
preBuild.dependsOn mytask
I found some ressources on the web, but they all don't work.
I tried to add the following to the classpath:
sourceSets.main.runtimeClasspath (main is unknown)
android.sourceSets.main.runtimeClasspath (runtimeClasspath is unkown).
So how can i access the class files in my library?

Try this
task execute(dependsOn: ['compileReleaseJavaWithJavac'], type:JavaExec) {
main = 'com.geniml.Main'
classpath(files('build/intermediates/classes/release',"${android.getSdkDirectory().getAbsolutePath() + '/platforms/' + android.compileSdkVersion + '/android.jar'}"))
}
android gralde is 1.5.
As to build dir, you can use rootProject.getBuildDir(). But normally build dir is a convention. A static way is ok.

Custom Class Loading in Dalvik with Gradle (Android New Build System)

As per the introduction of Custom Class Loading in Dalvik by Fred Chung on the Android Developers Blog:
The Dalvik VM provides facilities for developers to perform custom
class loading. Instead of loading Dalvik executable (“dex”) files from
the default location, an application can load them from alternative
locations such as internal storage or over the network.
However, not many developers have the need to do custom class loading. But those who do and follow the instructions on that blog post, might have some problems mimicking the same behavior with Gradle, the new build system for Android introduced in Google I/O 2013.
How exactly one can adapt the new build system to perform the same intermediary steps as in the old (Ant based) build system?

My team and I recently reached the 64K method references in our app, which is the maximum number of supported in a dex file. To get around this limitation, we need to partition part of the program into multiple secondary dex files, and load them at runtime.
We followed the blog post mentioned in the question for the old, Ant based, build system and everything was working just fine. But we recently felt the need to move to the new build system, based on Gradle.
This answer does not intend to replace the full blog post with a complete example. Instead, it will simply explain how to use Gradle to tweak the build process and achieve the same thing. Please note that this is probably just one way of doing it and how we are currently doing it in our team. It doesn't necessarily mean it's the only way.
Our project is structured a little different and this example works as an individual Java project that will compile all the source code into .class files, assemble them into a single .dex file and to finish, package that single .dex file into a .jar file.
Let's start...
In the root build.gradle we have the following piece of code to define some defaults:
ext.androidSdkDir = System.env.ANDROID_HOME
if(androidSdkDir == null) {
Properties localProps = new Properties()
localProps.load(new FileInputStream(file('local.properties')))
ext.androidSdkDir = localProps['sdk.dir']
}
ext.buildToolsVersion = '18.0.1'
ext.compileSdkVersion = 18
We need the code above because although the example is an individual Java project, we still need to use components from the Android SDK. And we will also be needing some of the other properties later on... So, on the build.gradle of the main project, we have this dependency:
dependencies {
compile files("${androidSdkDir}/platforms/android-${compileSdkVersion}/android.jar")
}
We are also simplifying the source sets of this project, which might not be necessary for your project:
sourceSets {
main {
java.srcDirs = ['src']
}
}
Next, we change the default configuration of the build-in jar task to simply include the classes.dex file instead of all .class files:
configure(jar) {
include 'classes.dex'
}
Now we need to have new task that will actually assemble all .class files into a single .dex file. In our case, we also need to include the Protobuf library JAR into the .dex file. So I'm including that in the example here:
task dexClasses << {
String protobufJarPath = ''
String cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.bat' : ''
configurations.compile.files.find {
if(it.name.startsWith('protobuf-java')) {
protobufJarPath = it.path
}
}
exec {
commandLine "${androidSdkDir}/build-tools/${buildToolsVersion}/dx${cmdExt}", '--dex',
"--output=${buildDir}/classes/main/classes.dex",
"${buildDir}/classes/main", "${protobufJarPath}"
}
}
Also, make sure you have the following import somewhere (usually at the top, of course) on your build.gradle file:
import org.apache.tools.ant.taskdefs.condition.Os
Now we must make the jar task depend on our dexClasses task, to make sure that our task is executed before the final .jar file is assembled. We do that with a simple line of code:
jar.dependsOn(dexClasses)
And we're done... Simply invoke Gradle with the usual assemble task and your final .jar file, ${buildDir}/libs/${archivesBaseName}.jar will contain a single classes.dex file (besides the MANIFEST.MF file). Just copy that into your app assets folder (you can always automate that with Gradle as we've done but that is out of scope of this question) and follow the rest of the blog post.
If you have any questions, just shout in the comments. I'll try to help to the best of my abilities.

The Android Studio Gradle plugin now provides native multidex support, which effectively solves the Android 65k method limit without having to manually load classes from a jar file, and thus makes Fred Chung's blog obsolete for that purpose. However, loading custom classes from a jar file at runtime in Android is still useful for the purpose of extensibility (e.g. making a plugin framework for your app), so I'll address that usage scenario below:
I have created a port of the original example app on Fred Chung's blog to Android Studio on my github page over here using the Android library plugin rather than the Java plugin. Instead of trying to modify the existing dex process to split up into two modules like in the blog, I've put the code which we want to go into the jar file into its own module, and added a custom task assembleExternalJar which dexes the necessary class files after the main assemble task has finished.
Here is relevant part of the build.gradle file for the library. If your library module has any dependencies which are not in the main project then you will probably need to modify this script to add them.
apply plugin: 'com.android.library'
// ... see github project for the full build.gradle file
// Define some tasks which are used in the build process
task copyClasses(type: Copy) { // Copy the assembled *.class files for only the current namespace into a new directory
// get directory for current namespace (PLUGIN_NAMESPACE = 'com.example.toastlib')
def namespacePath = PLUGIN_NAMESPACE.replaceAll("\\.","/")
// set source and destination directories
from "build/intermediates/classes/release/${namespacePath}/"
into "build/intermediates/dex/${namespacePath}/"
// exclude classes which don't have a corresponding .java entry in the source directory
def remExt = { name -> name.lastIndexOf('.').with {it != -1 ? name[0..<it] : name} }
eachFile {details ->
def thisFile = new File("${projectDir}/src/main/java/${namespacePath}/", remExt(details.name)+".java")
if (!(thisFile.exists())) {
details.exclude()
}
}
}
task assembleExternalJar << {
// Get the location of the Android SDK
ext.androidSdkDir = System.env.ANDROID_HOME
if(androidSdkDir == null) {
Properties localProps = new Properties()
localProps.load(new FileInputStream(file('local.properties')))
ext.androidSdkDir = localProps['sdk.dir']
}
// Make sure no existing jar file exists as this will cause dx to fail
new File("${buildDir}/intermediates/dex/${PLUGIN_NAMESPACE}.jar").delete();
// Use command line dx utility to convert *.class files into classes.dex inside jar archive
String cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.bat' : ''
exec {
commandLine "${androidSdkDir}/build-tools/${BUILD_TOOLS_VERSION}/dx${cmdExt}", '--dex',
"--output=${buildDir}/intermediates/dex/${PLUGIN_NAMESPACE}.jar",
"${buildDir}/intermediates/dex/"
}
copyJarToOutputs.execute()
}
task copyJarToOutputs(type: Copy) {
// Copy the built jar archive to the outputs folder
from 'build/intermediates/dex/'
into 'build/outputs/'
include '*.jar'
}
// Set the dependencies of the build tasks so that assembleExternalJar does a complete build
copyClasses.dependsOn(assemble)
assembleExternalJar.dependsOn(copyClasses)
For more detailed information see the full source code for the sample app on my github.

See my answer over here. The key points are:
Use the additionalParameters property on the dynamically created dexCamelCase tasks to pass --multi-dex to dx and create multiple dex files.
Use the multidex class loader to use the multiple dex files.