Android R8 obfuscation with flexJson duplicate key issue - android

While obfuscating the android app using R8 and minifyEnabled true in build.gradle it adds duplicate key like below in one of webservice response.
Response: {"key1":"value1", ......., "key1":"value1"} it adds "key1" multiple time and flexJson throws exception and crashes the app
Caused by: flexjson.JSONException: Duplicate key "key1"
at flexjson.JSONTokener.putOnce(JSONTokener.java:498)
at flexjson.JSONTokener.parseObject(JSONTokener.java:471)
at flexjson.JSONTokener.nextValue(JSONTokener.java:357)
at flexjson.JSONTokener.parseObject(JSONTokener.java:471)
at flexjson.JSONTokener.nextValue(JSONTokener.java:357)
at flexjson.JSONDeserializer.deserialize(JSONDeserializer.java:197)
Everything works fine without obfuscation(minifyEnabled false).
Gradle Plugin Version used: 3.4.2, Also flexJson is used by one of the library included in the project.

In general you should ensure that all fields which are used for generating JSON through reflection are covered by a keep rule. Otherwise the name in the JSON can change from build to build. Also R8 use the property that the JVM and Android runtimes allow fields of different types to have the same name, you can end up in the situation described here.
One option could be to annotate all classes which are serialized, and use a keep rule like this:
-keep class #MyAnnotation ** {
<fields>;
}
or if all these classes are in a separate package:
-keep class com.example.mypackage.serialized_classes.** {
<fields>;
}

Related

Proguard rules for Jetpack GameActivity

I'm trying to use Jetpack GameActivity in my project, but I encounter a strange issue with Proguard. My project has 2 modules, app and my-lib:
my-lib module has MyActivity which extends GameActivity:
api 'androidx.games:games-activity:1.2.1'
app module has MainActivity which extends MyActivity from my-lib module
implementation project(":my-lib")
When I built my project in Release build and had Proguard enabled, I got a native crash:
Abort message: 'Unable to find method setWindowFlags'
=> I tried inspecting GameActivity class from my APK, setWindowFlags method didn't exist at all, which means Proguard has removed it, that's why the native part of the GameActivity library couldn't find the method and it threw an error.
Then, I tried to fix it, by adding Proguard rule like this:
-keep, includedescriptorclasses class com.google.androidgamesdk.GameActivity { *; }
But unfortunately, I got another native crash:
Abort message: 'Unable to find field left'
=> Why does this happen? I tried inspecting the APK again, but have no idea why
Anyway, how can I deal with this situation? Does Jetpack GameActivty need to add Proguard rules to keep methods that will be used by the native part?
There is a know issue in version 1.2.1 and before: the java functions called by native code only were stripped out by proguard for the release build. Version 1.2.2-alpha01 has fixed that and a few other important potential issues, including the static library release in the AAR. To use the 1.2.2-alpha01+:
Add the latest version to the dependency: 1.2.2-alpha01+
Use C/C++ static lib release in the AAR.
add the following or something similar to your project's existing CMakeLists.txt
find_package(game-activity REQUIRED CONFIG)
add_library(${PROJECT_NAME} SHARED sth.cpp) #<=== this is your own.
target_link_libraries(${PROJECT_NAME} game-activity::game-activity_static
# optional: does not hurt to add it; refer to the official doc for update.
set(CMAKE_SHARED_LINKER_FLAGS
"${CMAKE_SHARED_LINKER_FLAGS} -u \
Java_com_google_androidgamesdk_GameActivity_initializeNativeCode")
If want to use the source code directly(not recommended), do something similar to this:
# Find GameActivity sources to build.
find_package(game-activity REQUIRED CONFIG)
get_target_property(GAME_ACTIVITY_INCLUDE_PATH
game-activity::game-activity
INTERFACE_INCLUDE_DIRECTORIES)
string(REPLACE "include" "" GAME_ACTIVITY_SRCS ${GAME_ACTIVITY_INCLUDE_PATH})
file(GLOB_RECURSE GAME_ACTIVITY_SRCS
${GAME_ACTIVITY_INCLUDE_PATH}/*.c*)
add_library(${PROJECT_NAME} SHARED
sth.cpp. #<== this is your own file
${GAME_ACTIVITY_SRCS})
target_link_libraries(${PROJECT_NAME} PUBLIC
android log
game-activity::game-activity
# other libs
)
Note: it is still just 3 source files now, and you can unzip the latest AAR and list them explicitly like:
add_library(${PROJECT_NAME} SHARED
sth.cpp. #<== this is your own file
${GAME_ACTIVITY_INCLUDE_PATH}/game-activity/GameActivity.cpp
${GAME_ACTIVITY_INCLUDE_PATH}/game-activity/native_app_glue/android_native_app_glue.c
${GAME_ACTIVITY_INCLUDE_PATH}/game-text-input/gametextinput.cpp)
The glob way might be better for future compatibility reasons; but it is not cmake recommended way to do things.
Handle the the "back" button press to exit app either from Kotlin/Java side(handle KEYCODE_BACK in onKeyDown()) or native code (AKEYCODE_BACK in input key event processing).
Referring to the documentation page and probably the list above, you should be able to get your project going. The simple fact is this: all of the C/C++ things are inside AAR, under the prefab sub-directory, you even can copy it out and directly put into your own source tree. This Pull Request might help (but not totally sure).
If you see something or like some new things, please create a bug.
I had the same issues as you and I solved this by adding the following line to my proguard-rules.pro file:
-keep class com.google.androidgamesdk.** { *; }
Of course then I had other proguard related errors so I had to add these
additional lines:
-keep class androidx.core.graphics.Insets { *; }
-keep class androidx.core.view.** { *; }
-keep class org.fmod.** { *; }
Of course you will have different errors in your build, but hopefully this will help you.

How can I obfuscate my sdk coded with kotlin (and get rid of Metadata)

I'm developing a SDK (Android library), and I have to obfuscate a large part of my code so the customer may not try and play with internal code.
My lib is coded in kotlin, and I used proguard to obfuscate the code. Problem is that there are still #kotlin.Metadata (runtime) annotations inside the code after compile and obfuscation. With those annotations, it's really easy to retrieve the java code that originated this "(not-so-)obfuscated" bytecode.
I first thought it was my fault, and my project had too many entropy sources that might have induced this behaviour, so I made a sample project to prove that the problem does not come from my sdk implementation.
I created a new project with AS, then a lib module with 2 files:
facade.kt is my facade class, the one that I do not wish to obfuscate, so the customer may use it:
package com.example.mylibrary
class MyFacade(val internalClass:InternalClass) {
fun doSomething() {
internalClass.doSomething(
firstArgument=1,
secondArgument=2
)
}
}
and in this sample, internal.kt holds the classes that I want to obfuscate:
package com.example.mylibrary
class InternalClass {
fun doSomething(firstArgument: Int, secondArgument: Int) {
System.out.println("Arguments are : $firstArgument, $secondArgument")
}
}
The proguard rules are injected into gradle project with this release closure:
buildTypes {
release {
minifyEnabled true
proguardFiles 'proguard-rules.pro'
}
}
And here is proguard-rules.pro (only one line, nothing more) :
-keep class com.example.mylibrary.MyFacade {*;}
The result: when I ./gradlew clean myLib:assembleRelease, I do obtain an aar in which my facade is kept, and my internal class has been renamed in 'a', with one method 'a', except that the class is still annotated with kotlin #Metadata, which holds every information that helps the decompiler retrieve the original class name, the method, attribute and argument names, etc...
So my code is not so obfuscated at all...
#Metadata(
mv = {1, 1, 7},
bv = {1, 0, 2},
k = 1,
d1 = {"\u0000\u001a\n\u0002\u0018\u0002\n\u0002\u0010\u0000\n\u0002\b\u0002\n\u0002\u0010\u0002\n\u0000\n\u0002\u0010\b\n\u0002\b\u0002\u0018\u00002\u00020\u0001B\u0005¢\u0006\u0002\u0010\u0002J\u0016\u0010\u0003\u001a\u00020\u00042\u0006\u0010\u0005\u001a\u00020\u00062\u0006\u0010\u0007\u001a\u00020\u0006¨\u0006\b"},
d2 = {"Lcom/example/mylibrary/InternalClass;", "", "()V", "doSomething", "", "firstArgument", "", "secondArgument", "mylibrary_release"}
)
public final class a {
...
}
So my question: is it possible to get rid of those annotations, am I the only one facing this problem, or have I missed something?
Finally, I found a way to delete Kotlin metadata annotations.
In order to hide Kotlin metadata annotations, you need to enable R8 full mode.
Here is the information about my environment.
Environment
OS: macOS 10.15.1
Android Studio: 3.5.1
Gradle: 5.4.1
Android Gradle Tool: 3.5.2
What you have to do is just add properties to gradle.properties like below
gradle.properties
android.enableR8.fullMode=true
And here is my Proguard Rules
proguard-rules.pro
-dontwarn kotlin.**
-assumenosideeffects class kotlin.jvm.internal.Intrinsics {
static void checkParameterIsNotNull(java.lang.Object, java.lang.String);
}
FYI, R8 full mode is still testing now, so sometimes it doesn't work well. However, for me, it works perfectly for now.
There is some plugin dedicated for this kind of this request : https://github.com/oliver-jonas/unmeta
This plugin allows removing all Kotlin #Metadata / #DebugMetadata annotations from generated class files. This is safe to do as long as:
you do not intend to use the resulting binaries as a Kotlin library (#Metadata annotations are used to determine Kotlin function definitions),
you are not using Kotlin Reflection (certain reflection functionality depends on the presence of the #Metadata annotations).
Must be careful because when removing the metadata kotlin may your application or your library will not work.

ProGuard configuration for AndroidPlot

Since building a release version of my app with ProGuard enabled, my plot style is reset to the default and I see many warnings in Logcat informing me of unsupported parameters:
Error inflating XML: Setter for field "[...]" does not exist.
I've pinpointed this to be coming from AndroidPlot's Configurator, but haven't found any official ProGuard configuration for this project.
The mechanism through which AndroidPlot sets the configuration parameters relies heavily on reflection, and in that light I've decided it's useless to try to obfuscate anything inside this library:
-keep class com.androidplot.** { *; }
In my case, I had been using proguard for debug builds and it worked fine. Then I ran a release build (which adds obfuscation to the proguard configuration) and that crashed when it tried to inflate XYPlot in a view.
Binary XML file line #12: Binary XML file line #12: Error inflating class com.androidplot.xy.XYPlot
To fix it, I just configured proguard to not obfuscate the names of any androidplot objects:
-keepnames class com.androidplot.**
That did not work for me yet. For troubleshooting I set the switches -dontshrink -dontoptimize -dontobfuscate in the first step (if that doesn't help, the reason is probably not to be found in ProGuard).
After that you can step by step exclude single groups of classes, e.g. "-keep, includedescriptorclasses, includecode class my.path.to.R*{*;}". In my case the resource classes created by Android Studio had to be "-keep'ed":
-keep, includedescriptorclasses, includecode class com.androidplot.** {*;}
-keepclassmembers class **.R$* {
public static <fields>;
}
see https://www.guardsquare.com/en/products/proguard/manual/examples
"We're keeping the static fields of referenced inner classes of auto-generated R classes, just in case your code is accessing those fields by introspection. Note that the compiler already inlines primitive fields, so ProGuard can generally remove all these classes entirely anyway (because the classes are not referenced and therefore not required)."

Removing uses of TestFlight SDK with ProGuard

How can I automatically remove all uses of TestFlight SDK from my app? For example, all passCheckpoint calls:
TestFlight.passCheckpoint("FreemiumDialog opened");
My (ProGuard-obfuscated) release builds should not even attempt to send any info to TestFlight, yet I do not want to manually toggle between having TestFlight jar & its uses in my codebase.
This ProGuard configuration removes all calls to TestFlight SDK:
# Remove all TestFlight SDK calls, e.g. TestFlight.takeOff( ... );
# and TestFlight.passCheckpoint( ... );
-assumenosideeffects class com.testflightapp.lib.TestFlight { *; }
-dontwarn org.msgpack.**
Note: without the last line, ProGuard fails with a bunch of warnings like:
Warning: org.msgpack.template.builder.BeansBuildContext: can't find
referenced class javassist.CtClass
Warning: org.msgpack.util.json.JSONUnpacker: can't find referenced class > org.json.simple.parser.JSONParser
This is because the TestFlightLib jar (at least version 1.3) contains and uses a library called MessagePack. We need to either -keep it or mute the warnings. Above I'm muting warnings, since the whole point here is not to use TestFlight in release builds.

Dagger cannot create object graph although it can produce dot file

I'm struggling with the setup of Dagger (1.0.1), in a existing application. It was configured to use ProGuard but I disabled it for this test with -dontobfuscate.
When I enable dagger-compiler it's able to successfully generate a dot file with the dependencies graph, but when I remove the compiler and build the app in Release mode it crashes during startup, complaining that it's unable to create the object graph.
java.lang.RuntimeException: Unable to start activity
ComponentInfo{com.corp.myapp/com.corp.myapp.ui.activity.MainActivity}:
java.lang.IllegalStateException: Errors creating object graph:
No injectable members on com.corp.myapp.core.services.ConnectionMonitor. Do
you want to add an injectable constructor? required by
com.corp.myapp.core.services.ConnectionMonitor
com.corp.myapp.ui.activity.MyAppBaseActivity.connectionManager
No injectable members on com.corp.myapp.ui.crouton.CroutonManager. Do you want
to add an injectable constructor? required by
com.corp.myapp.ui.crouton.CroutonManager
com.corp.myapp.ui.activity.MyAppBaseActivity.croutonManager
No injectable members on com.corp.core.assembler.ResourceAssembler. Do you want
to add an injectable constructor? required by
com.corp.core.assembler.ResourceAssembler
com.corp.myapp.ui.activity.MyAppBaseActivity.resourceAssembler
I see MyAppBaseActivity and it's dependencies with CroutonManager or ConnectionMonitor being displayed in the generated dot file, so according to this comment I expected this to work. AFAIK if there was something wrong it should be detected by the compiler-enabled build that I used to generate the dot file.
UPDATE:
I previously stated that
In Debug mode it never fails
but it's not really true after further testing: In Debug mode it doesn't fail because ProGuard is disabled, whereas in Release mode it is enabled by default. If I build the app in Release mode but skip ProGuard, I don't get the errors either and the app successfully starts. So the problem is definitely related to my ProGuard configuration.
Dagger relies a lot on reflection and class names that are hard-coded and manipulated as strings. This makes the code difficult to shrink/optimize/obfuscate.
The following configuration works for the sample dagger/examples/simple in Dagger 1.1.0:
-keepattributes *Annotation*
-keepclassmembers,allowobfuscation class * {
#javax.inject.* *;
#dagger.* *;
<init>();
}
-keep class **$$ModuleAdapter
-keep class **$$InjectAdapter
-keep class **$$StaticInjection
-keepnames !abstract class coffee.*
-keepnames class dagger.Lazy
The configuration keeps all fields and methods with javax.inject or dagger annotations, and all parameterless constructors. ProGuard might otherwise remove them if they appear unused, but Dagger is actually injecting/accessing them through reflection. This is similar to RoboGuice.
It also has to keep all adapter classes generated by Dagger.
It also has to keep all class names related to these adapter classes, so the names still match. In this sample, those are almost all classes in the package coffee, so the easiest way is to use a wild-card. This line will be different for other applications.
Finally, it also has to keep the name of the class dagger.Lazy, since its name is hard-coded as a string in the generated code.
I got the app to start after adding -dontshrink to the ProGuard config file. Having -dontobfuscate at the beginning was not enough.
In fact, if I remove -dontobfuscate it also works.
I definitely need finer control for this but it's a starting point. My current ProGuard setup for Dagger is:
#############
# Dagger #
#############
-keep class dagger.** { *; }
-dontwarn dagger.internal.codegen.**
Dagger doesn't require #Inject to be on a class to be passed into graph.inject(myActivity) because some activities may not have any injections to make. However, these seem like upstream dependencies, which means that they need to be provided to ComponentInfo, and therefore need to be provisioned by Dagger. It cannot do this if it cannot create these classes, and it can't do so if these are not annotated, unless it provides them via a #Provides method.
So, you either need to create an #Module-annotated class which returns these types from #Provides-annotated methods, or you need to add #Inject to their constructor.
-keep class * extends dagger.internal.Binding
That said, in this case, are you using proguard in "release" mode? And not proguarding in debug mode? If so, I suspect Proguard to be stripping away annotations. You'll need to do some variant of:
-keep class javax.inject.** { *; }
... to ensure that Proguard doesn't remove the annotations.
I ended up burning a week+ trying to get this to work. In the end I failed and decided to give DexGuard a shot. It worked beautifully right out of the box. Yes its a commercial product but DexGuard has great support and because of such we were able to finally ship. Id definitely recommend DexGuard if you absolutely need to solve this issue.

Categories

Resources