Develop Reference

Difference between D8 and R8 android

As android studio introduced two new tools D8 and R8.
As per google documentation D8 is a dex tool and R8 is a progourd tool but as their explanation both are doing almost same thing like below:
D8 is a dexer that converts java byte code to dex code.
R8 is a java program shrinking and minification tool that converts java byte code to optimized dex code.
It seems both converts java byte code to dex code literally. So, Whats actually they are doing internally in case of converting dex code?

D8 dexer and R8 shrinker
D8->D8 is a dexer that converts java byte code to dex code.
R8->R8 is a java program shrinking and minification tool that converts java byte code to optimized dex code.
Android developers know that dex compilation is a key step in building an APK. This is the process of transforming .class bytecode into .dex bytecode for the Android Runtime (or Dalvik, for older versions of Android). The dex compiler mostly works under the hood in your day-to-day app development, but it directly impacts your app's build time, .dex file size, and runtime performance.
The R8 project uses depot_tools from the chromium project to manage dependencies. Install depot_tools and add it to your path before proceeding.
The R8 project uses Java 8 language features and requires a Java 8 compiler and runtime system.
New version number scheme following the SDK Tools revision number.
Support for true debug build. No need to change the value of debuggable in
the Android Manifest.
Incremental build will automatically insert debuggable==true while using
the "export signed/unsigned application package" will not.
If debuggable=true is set, then release builds will actually do a debug build.
Automatic Proguard support in release builds. Only need to have a proguard.config
property in default.properties that points to a proguard config file.
Completely rewritten Visual Layout Editor. This is very much a work in progress.
full drag and drop from palette to layout for all Layout classes.
Move widgets inside a Layout view, from one Layout view to another and from one layout file to another.
Contextual menu with enum/flag type properties.
New zoom controls.

I think the introduction of this blogpost is a great resource to answer that question: https://jakewharton.com/r8-optimization-staticization
R8 is a version of D8 that also performs optimization. It’s not a separate tool or codebase, just the same tool operating in a more advanced mode. Where D8 first parses Java bytecode into its own intermediate representation (IR) and then writes out the Dalvik bytecode, R8 adds optimization passes over the IR before its written out.

D8 processes each Java class file individually without looking at the entire program. This makes the conversion to DEX fast, since classes can be processed in parallel, and during development it allows fast recompilation when the code of a class is modified.
On the other hand, R8 (like ProGuard) reads in the entire application and makes changes and optimizations (e.g. inlining) that require knowing the entire class hierarchy. For instance, R8 will remove unused classes and methods ("tree shaking") and rename classes, methods and fields (except for the application's entry points).
In Android Studio 3.1, D8 has replaced DX as the tool that converts Java class files to DEX, but R8 has not been enabled yet.

Compile with D8

I want to use the recently introduced D8 compiler from Google to build a dex file.
Unfortunately, I didn't find the jar executable in the build-tools folder (like the dx tool).
Does anyone know where it's located instead?
Thanks in advance.

There is no D8.jar anywhere in build-tools. D8 is distributed as a maven artifact instead. It can be downloaded from https://maven.google.com. The artifact is named com.android.tools:r8 because D8 is a part of the R8 project. The jar contains both R8 and D8 commandline interfaces, so, in theory, it can be downloaded and used from a commandline like:
java -cp r8.jar:<...dependency jars> com.android.tools.r8.D8 <D8 arguments here>
java -cp r8.jar:<...dependency jars> com.android.tools.r8.R8 <R8 arguments here>
However this artifact jar has a lot of dependencies and if you want to use it outside of the gradle-based build then it is easier to build r8/d8.jar executable jars from the sources by yourself. The instruction how to do it is at https://r8.googlesource.com/r8/. The resulting jars are self-contained (all deps bundled in).
The Android Gradle plugin (and, therefore, Android Studio) doesn't depend on the com.android.tools:r8. R8/D8 classes are bundled into com.android.tools.build:builder (one of the libraries that gradle plugin consists of) instead.

D8 dexer has become the default compiler in #AndroidStudio 3.1
Expect faster and smarter app compilation
Faster, smarter app compilation is always a goal for the Android tools teams. That's why we previously announced D8, a next-generation dex compiler. D8 runs faster and produces smaller .dex files with equivalent or better runtime performance when compared to the historic compiler - DX
Please see comparition beetween DX vs D8 :
You can always revert to DX for now via this setting in your project's gradle.properties file:
android.enableD8=false

Share a Kotlin module with an Android and Desktop project

I have a game that I am working on that uses the LibGDX game framework. Currently the platforms I am targeting are Desktop (PC, Mac, Linux) via a platform independent jar and Android.
The project is hosted at https://github.com/NoxHarmonium/project-whiplash Feel free to take a look if you need to.
The bulk of the code is in a module called core and is written entirely in Kotlin. This module is linked into both the Desktop and Android projects.
This works fine for Android versions 7.1+ and for desktop. For all other versions of Android I get a pile of java.lang.NoClassDefFoundError exceptions on anonymous functions such as this:
val objectObservable = this.observableCache.computeIfAbsent(assetRef, fun(assetRef: AssetRef): Observable<T> {
return Async.start(fun(): T {
...
}).observeOn(this.eventLoopScheduler)
})
Exception Sample:
java.lang.NoClassDefFoundError: com.projectwhiplash.utils.assets.LibGdxDataManager$objectMapFromYaml$objectMapObservable$1
It seems to be caused by an incompatibility with the JVM that Kotlin targets by default (1.8) and the JVM level that older versions of Android support (1.6). I could be wrong but this explains why the latest version of Android works as it supports a later version of the JVM.
The solution should be as simple as forcing Kotlin to emit JVM byte code as version 1.6 but I can't seem to work it out. If you compile Kotlin directly into an Android, this seems to be handled by using the kotlin-android Gradle plugin. Unfortunately I can't use this plugin for the core module because it should not have any Android dependencies.
I tried to override the JVM version using the build settings mentioned at https://kotlinlang.org/docs/reference/using-gradle.html#compiler-options like this:
compileKotlin {
kotlinOptions {
jvmTarget = "1.6"
}
}
However, it did not seem to work no matter which Gradle file I placed it in. In fact I get a "Cannot resolve symbol 'kotlinOptions'" error shown by Intellij when I try it. It is possible that the Kotlin team have changed something and the documentation has not been updated.
I can override the Kotlin settings manually in the Intellij module settings but it gets overridden every time I sync the gradle project and is not a good long term solution. The project is designed to be IDE independent.
Does anyone know how I could set up the core module for max compatibility with older versions of Android?
I currently have the minimum API level set to 9 as this is the current LibGDX default but I'm willing to set this higher if it would be too difficult to target such a low API level.
Edit 1:
I just extracted the jar file produced by the core module and examined the class files using the javap tool.
I ran the following command on a random class file
java -verbose foo.class
and it output text with the following text
...
minor version: 0
major version: 50
...
using this question List of Java class file format major version numbers? I determined that the class file is actually targeting JVM 1.6.
Therefore my original theory is wrong and there is another reason why older Android versions cannot load classes generated by Kotlin lambdas.

It looks like you are using functionality that only exists within the JDK 8 library. Specifically the computeIfAbsent() method on the Map class.
Because of this, even though your code has been compiled down to JVM 1.6 compatibility, the underlying implementation on Android devices is missing that functionality and thus the reason for the NoClassDefFoundError exception you were seeing.
Updated: You can see in the javadoc located at https://docs.oracle.com/javase/8/docs/api/java/util/Map.html#computeIfAbsent-K-java.util.function.Function- that the computeIfAbsent() has only been around since JDK 8

How Jack (Java Android Compiler Kit) will affect Scala developers

Now with announcement of Jack Google clarified the foreseeable future of Java in relation to Android. But what are the implications to Scala and other JVM-based languages developers. In particular:
Scala does it's magic due to own compiler that produces Java bytecode. But Jack toolchain doesn't deal with bytecode. Will generated bytecode gain any optimization benefits of Jack processing?
Starting from Scala 12 only Java 8+ is supported. That is the generated bytecode is Java 8+ too. Can Jack utilize Java 8 bytecode (without or with limitations)?
Can newly supported Java 8 features be used for developing for older Android versions (minSdkVersion < 'N') or i should maintain separate branch for each Java version? (it is not clear from documentation).
All these questions boil down to one: Can Scala be used for Android development in future without sacrificing the benefits of new Scala features and new Android tool chain?
Related reading:
Java 8 Language Features
Jack (Java Android Compiler Kit)
New Android Tool Chain - Jack and Jill
The Jack and Jill build system (slides)
Hello World, meet our new experimental toolchain
Scala 2.12 roadmap
please share related links in comments or answers
Related questions:
Android N Java 8 features (Jack compiler) and Kotlin interop
Related:
scala-on-android group discussion
Please vote for Jack tool feature request:
Support for Java 8 bytecode (libraries) by Jill
Support Java 8 language features
EDIT:
I'm trying to reason about (NOT to answer) my question hoping that experts will correct me if i'm wrong.
Below is a hypothetical flow of Jack build with some extra blocks that was added basing on my logic and what i have learned from available docs.
Base assumption is that Dalvik supports up to Java 7 bytecode instructions. If that is correct Java 8 instructions can not be directly passed to Dalvik, they should be somehow transformed to Java 7. (May be something similar to that Scala compiler always does).
Than the question is where is that transformation happens? Seems Jill can't process Java 8 bytecode as for now, so that possibly happens in block (3) of hypothetical flow. If that is correct than only Java source project files are subject to transformation and the answer to 2-nd question is - No. Java 8 classes from libraries can not be utilized until Jill will be able to do it (if it's possible at all). That is we can not use Scala 12+.
If all code optimization is performed in block (6) than the answer to 1-st question is - Yes. Scala code being converted to library .jar can benefit from Jack optimizations. But preliminarily it should be transformed to .jayce (AST-like representation) that will increase build time.
And finally Jack produces .dex Dalvik bytecode to maintain compatibility with older Dalvik runtimes (ART consumes Dalvik bytecode too). So the answer to 3-d question is: Yes, Java 8 features can be used. But only in project Java sources. App is still compatible with any runtime. But Java 8 advantages are dropped due to converting to Java 7 (Dalvik bytecode).

It’s important to understand that there is 2 tool introduced:
Jack: A new compiler to replace the complicated javac + proguard + dx
Jill: A library linker that will be able to link currently compiled libraries (.class) and more.
See http://tools.android.com/tech-docs/jackandjill
So it sounds like there is 2 separate problem here:
Scala compatibility:
Scala will not be supported by Jack, as Jack compiles Java source code.
However Scala 2.11 compiles to Java 1.6 bytecode and therefore Jill will be able to pick that code and convert to jack files to feed the Jack compiler.
See Android N Java 8 features (Jack compiler) and Kotlin interop (Kotlin as the same issue as Scala as it’s a JVM language)
Java 8, and therefore Scala 2.12+, compatibility:
This part is under development, if Jack/Jill supports Java 8, then it will also support Scala 2.12+ (through Jill). If not, Java 8 developers are in the same boat as Scala 2.12 developers.
In the case Jack supports Java 8 but not Jill, then Java 8 libraries developers will be in the same boat as Scala 2.12 developers.
See https://www.guardsquare.com/blog/DroidconLondon2015

Joan is correct, but I think Jill will have support for Java 8 at some point to, otherwise it will be impossible to use Java 8 in android libraries that will be consumed by android apps, as they package their code in a jar files inside of aar and I don't see this format change happening anywhere soon. Anyway we can only guess, as Google is currently a blackbox in respect to those kinds of changes.
Revising since new info is out in 2017: jack toolchain is now deprecated and old dex/javac stack will receive java8 support so nothing will change for scala now.

Google just announced that the Jack toolchain will deprecate the Jack toolchain and Android adds "support for Java 8 language features directly into the current javac and dx set of tools"
Source: https://android-developers.googleblog.com/2017/03/future-of-java-8-language-feature.html
We know how much our Android developer community cares about good support for Java 8 language features, and we're changing the way we support them.
and:
We've decided to add support for Java 8 language features directly into the current javac and dx set of tools, and deprecate the Jack toolchain.

Is jack and jill for android compatible with byte code weaving?

I just read the new blog post of #eric lafortune about Jack and Jill's new compiler on Android.
http://www.saikoa.com/blog/the_upcoming_jack_and_jill_compilers_in_android
And as I am working intensively on a bunch of new technologies built on byte code weaving for Android, I am wondering if anyone has any idea if byte code weaving would still work with this new tool chain.

From the article you mentioned:
Since the toolchain still supports both java source code and java
bytecode, existing libraries can be used without problems. Even tools
that generate or process bytecode during the build process, such as
Dagger, should be fine, with the proper integration.
Bytecode weaving is just bytecode generation/processing, so I think the answer to your question is "yes".

I'm wondering this myself, but it seems that Kotlin will be a second-class citizen with the new jack and jill compiler unless JetBrains makes an explicit statement that it won't: Android N Java 8 features (Jack compiler) and Kotlin interop
EDIT: no longer true because Kotlin is a first class citizen and the Jack & Jill project are dead.