We need to store and retrieve the content that users generate with our app online. To do so, we decided to use Android Studio's integrated Google Cloud Endpoints template to quickly create an API (official usage example here).
It works fine in debug, but in release mode, with Proguard enabled, it fails. Worse still, I've failed to find any documentation or samples about using Proguard with the Android Studio's Endpoints templates.
After an hour or so of poking around and trying to make it work, the proguard-rules.pro now looks like this:
-keep class com.google.api.** { public *; }
-dontwarn com.google.api.**
-keep class com.google.common.** { public *; }
-dontwarn com.google.common.**
# Not allowed to post company and app names, but this line is correct in the real file
-keep class com.companyname.appname.application.backend.** { *; }
With this configuration, I'm getting a class cast exception in my ArrayAdapter:
java.lang.ClassCastException: com.google.api.client.util.ArrayMap cannot be cast to com.companyname.appname.application.backend.messageApi.model.Message
It seems the conversion of returned data isn't performed somewhere and, instead of a List of Message objects, I get a List of com.google.api.client.util.ArrayMap objects (they do contain valid data, by the way).
I COULD check whether the app is running in release mode and do the conversion manually, however, it's a hacky way and I'd prefer to do it properly. So, can someone please tell me what I'm missing in the Proguard configuration file?
I do similar things with endpoints in one of my apps. I had some problems with Proguard as well (can't remember exactly what).
This section of my Proguard rules seems applicable:
# Needed by google-api-client to keep generic types and #Key annotations accessed via reflection
-keepclassmembers class * {
#com.google.api.client.util.Key <fields>;
}
-keepattributes Signature,RuntimeVisibleAnnotations,AnnotationDefault
I don't know if it is necessary, but I also have this section:
# Play Services
-dontwarn com.google.android.gms.**
-dontwarn com.google.common.cache.**
-dontwarn com.google.common.primitives.**
-keep class * extends java.util.ListResourceBundle {
protected Object[][] getContents();
}
-keep public class com.google.android.gms.common.internal.safeparcel.SafeParcelable {
public static final *** NULL;
}
-keepnames #com.google.android.gms.common.annotation.KeepName class *
-keepclassmembernames class * {
#com.google.android.gms.common.annotation.KeepName *;
}
Hope it helps.
Related
Sometimes, hackers may use automation tools, such as Robotium、Espresso、UiAutomator2、Appium, to automatically operate my customers' apps, which will integrate our SDK, to earn profits.
So how to detect if current app is being operating by such automated testing tools (except checking the service and process) without editting the app itself? Thanks~
The best way to do it is to remove all IDs, text descriptions and other helpers from the app. Even if there are IDs have multiple builds with different IDs or dynamic IDs. Other methods are:
To re-skin the app regularly to avoid automation using image comparisons.
Have more captcha or other security methods to avoid automation.
Well, the easiest way is the strict Proguard configuration when you make a release build.
If you are assuming somebody will create a script using Appium+Espresso/UiAutmator to communicate with your app, make sure you don't have the following in your proguard rules:
-dontwarn com.google.android.material.**
-keep class com.google.android.material.** { *; }
-dontwarn androidx.**.
-keep class androidx.** { *; }
-keep interface androidx.** { *; }
-dontwarn android.support.v4.**
-keep class android.support.v4.** { *; }
-dontwarn android.support.v7.**
-keep class android.support.v7.** { *; }
In this case, your app simply won't be suitable for test automation.
Also, if you do have your own tests, make sure there is no way to start them for release APK. Mostly like no harm, but I have seen it once :)
EDIT: For future devs who may be suffering from the same issues, I'm afraid I can't be of much use. The problem apparently solved itself during future versions of the app, but I have been unable to ascertain the cause.
I've been testing an application through the Google Play closed beta testing system, using my friends as beta testers. After pushing an update where the .apk was obfuscated using ProGuard, two testers started reporting crashes. The crashes are unpredictable and sporadic, and they have been unable to ascertain any reason or pattern for them. To make things worse, the crash reports I'm getting in the Dev Console contain no information at all, simply stating "Native crash at (no location available)", no stack trace, not even any device information.
I do know the device information through just asking the testers in question, and both devices are Sony phones, an Xperia Z3 and an Xperia Z3 Compact. However, I also have a Z3 Compact available for testing and have been unable to reproduce the crash while running the same obfuscated apk. We are running the same OS version as well.
I have sent one of the testers and unobfuscated apk, and he was unable to reproduce the crash running that version. This, and the fact that the crash first occurred after a patch that introduced ProGuard obfuscation, leads me to believe that to be the cause.
At first, I ran ProGuard using only the default configuration file from the SDK:
# This is a configuration file for ProGuard.
# LINK REMOVED
-dontusemixedcaseclassnames
-dontskipnonpubliclibraryclasses
-verbose
# Optimization is turned off by default. Dex does not like code run
# through the ProGuard optimize and preverify steps (and performs some
# of these optimizations on its own).
-dontoptimize
-dontpreverify
# Note that if you want to enable optimization, you cannot just
# include optimization flags in your own project configuration file;
# instead you will need to point to the
# "proguard-android-optimize.txt" file instead of this one from your
# project.properties file.
-keepattributes *Annotation*
-keep public class com.google.vending.licensing.ILicensingService
-keep public class com.android.vending.licensing.ILicensingService
# For native methods, see LINK REMOVED
-keepclasseswithmembernames class * {
native <methods>;
}
# keep setters in Views so that animations can still work.
# see http://proguard.sourceforge.net/manual/examples.html#beans
-keepclassmembers public class * extends android.view.View {
void set*(***);
*** get*();
}
# We want to keep methods in Activity that could be used in the XML attribute onClick
-keepclassmembers class * extends android.app.Activity {
public void *(android.view.View);
}
# For enumeration classes, see http://proguard.sourceforge.net/manual/examples.html#enumerations
-keepclassmembers enum * {
public static **[] values();
public static ** valueOf(java.lang.String);
}
-keepclassmembers class * implements android.os.Parcelable {
public static final android.os.Parcelable$Creator CREATOR;
}
-keepclassmembers class **.R$* {
public static <fields>;
}
# The support library contains references to newer platform versions.
# Don't warn about those in case this app is linking against an older
# platform version. We know about them, and they are safe.
-dontwarn android.support.**
I have since added the following lines to proguard-project.txt, which is also included in the proguard configuration variable for the project:
-keep class * extends java.util.ListResourceBundle {
protected java.lang.Object[][] getContents();
}
# Keep SafeParcelable value, needed for reflection. This is required to support backwards
# compatibility of some classes.
-keep public class com.google.android.gms.common.internal.safeparcel.SafeParcelable {
public static final *** NULL;
}
# Keep the names of classes/members we need for client functionality.
-keepnames #com.google.android.gms.common.annotation.KeepName class *
-keepclassmembernames class * {
#com.google.android.gms.common.annotation.KeepName *;
}
# Needed for Parcelable/SafeParcelable Creators to not get stripped
-keepnames class * implements android.os.Parcelable {
public static final ** CREATOR;
}
# Needed when building against the Marshmallow SDK
-dontwarn org.apache.http.**
# Needed when building against pre-Marshmallow SDK.
-dontwarn android.security.NetworkSecurityPolicy
-keep public class * extends android.app.Activity
-keep public class * extends android.app.Application
-keep public class * extends android.app.Service
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider
-libraryjars D:/Docs/Android/spaceprog/spaceprogr/libs/android-support-v4.jar
-keep class com.viewpagerindicator.** { *; }
-keep class com.google.android.gms.** { *; }
-keep class com.google.example.games.basegameutils.** { *; }
-printmapping out.map
-renamesourcefileattribute SourceFile
-keepattributes SourceFile,LineNumberTable
The upper part (with comments) of these additions comes directly from the README of the Google Play Services as a recommendation to be added whenever using the library. The rest I added myself while trying to research and debug the problem.
If anyone notices any omissions or other signs of trouble in my ProGuard configuration I'd be happy to know. I'd also be greatful for any suggestions on how to further debug the problem. Keep in mind that full module testing where only one module is obfuscated at a time is impossible in this case, because I do not have a device that can reproduce the issue and I cannot ask volunteer testers to install 30+ different versions of the .apk and play until it either crashes or until they are confident it will not.
I've seen a couple questions regarding this issue, but they are for older versions of Netty.
I have tried their answers, switching org.jboss.netty out with io.netty, but the same error occurs.
I'm trying to compile an Android app that uses Netty 5.0.0Alpha2 (build #16) with Proguard enabled.
Without Proguard, the app runs fine.
As soon as I enable Proguard, I get this exception when it tries to use Netty:
java.lang.IllegalStateException: unknown type parameter 'I': class io.netty.channel.SimpleChannelInboundHandler
at io.netty.util.internal.TypeParameterMatcher.find0(Unknown Source)
at io.netty.util.internal.TypeParameterMatcher.find(Unknown Source)
at io.netty.channel.SimpleChannelInboundHandler.<init>(Unknown Source)
at io.netty.channel.SimpleChannelInboundHandler.<init>(Unknown Source)
...
This is my Proguard config:
# billing
-keep class com.android.vending.billing.**
# butterknife
-dontwarn butterknife.internal.**
-keep class **$$ViewInjector {
*;
}
-keepnames class * {
#butterknife.InjectView *;
}
# admob
-keep public class com.google.android.gms.ads.** {
public *;
}
-keep public class com.google.ads.** {
public *;
}
# logging
-assumenosideeffects class android.util.Log
# netty (partial)
-dontwarn io.netty.**
-dontwarn sun.**
I have tested it without the -dontwarn options to see if the warnings would point me in the right direction, but it's all missing optional dependencies like slf4j and Tomcat.
I have also tried excluding all the Netty classes like so:
-keep class io.netty.** {
*;
}
...but that does not appear to fix it either.
I have fixed this issue with some carefully* applied Proguard rules after reading through parts of the rather huge Netty sources:
-keepattributes Signature,InnerClasses
-keepclasseswithmembers class io.netty.** {
*;
}
-keepnames class io.netty.** {
*;
}
My original exception was caused by the type variables being removed from the bytecode, which Netty uses via reflection. Signature in -keepattributes keeps this information.
You get a slightly different exception if you only do Signature on -keepattributes - adding InnerClasses fixes this one by bringing back even more information in the class files.
Later, I got java.lang.NoSuchFieldException: ctl; that's what -keepnames is for. This way, the field is still called ctl like Netty expects.
Finally, some members (like ctl, seen earlier) were being removed by Proguard because Netty only uses them via reflection. The final rule, -keepclasseswithmembers, makes sure Proguard doesn't remove them.
If you take this approach, I strongly recommend you use only the Netty jars you need, instead of the -all jar. Switching from -all to just the required Netty jars brought my method count way down after I had gone past the 65k limit. Reducing your jars requires bit of trial-and-error though as the separation is unclear and there's not really any resources saying what's what.
* not carefully at all, I just slapped rules into the file and removed them if they did nothing. There's probably a better way to do this that doesn't keep this information in the entire program, but instead just Netty.
Una's answer keep too many classes, which makes my app 1MB larger than usual. So I use the rules below:
# netty
-keepclassmembernames class io.netty.buffer.AbstractByteBufAllocator {
*;
}
-keepclassmembernames class io.netty.buffer.AdvancedLeakAwareByteBuf {
*;
}
-keep public class io.netty.util.ReferenceCountUtil {
*;
}
I'm configuring Proguard for an app that uses 3rd party libraries. Is it "best practice" (in order to avoid future hard-to-find bugs) to include the line:
-keep class 3rd_party_lib_name.** {*;}
for every single 3rd party open source library that doesn't have specific Proguard instructions from its developer?
Also, a related question: is there a general guideline for which cases I should use
-keep class
and in which cases i should use
-keep public class
many thanks
The major problem with proguard and code obfuscation in general is that classname, methods and fields name are modified. ( i.e. myExplicitMethodName() became a() )
When a classname, method name or a field is modified, you cannot access it using the reflection API (i.e. Class.classForName(...) , ... )
Knowing that, it's a best practice to -keep all classes and libraries that can be invoked using the reflection API.
For 3rd party libraries, if you don't know if they use or not the reflection API : then -keep
For your own code: hopefully, you know in which classes you use it. So use -keep for those classes.
Note that some popular framework like dagger or jackson use the reflection API on your own classes, so if you use them, be careful!
The fewer -keep options you can use, the better your results will be, in terms of optimization and obfuscation. If you don't have the time to find an optimal configuration, you can take a more conservative approach. The most conservative solution is to preserve all classes, fields, and methods in the library, so any internal reflection will continue to work:
-keep class 3rd_party_lib_name.** {*;}
Slightly less conservative, but typically sufficient: preserve all public API:
-keep public class 3rd_party_lib_name.** {
public *;
}
Even less conservative: only preserve public classes, but not necessarily their fields or methods:
-keep public class 3rd_party_lib_name.**
Some experimentation can go along way.
As ben75 mentions, this doesn't account for third party libraries performing reflection on your own code.
Since some libraries use reflection or json conversion for some classes, if you do not keep library classes, your app will not work properly. For a sample case,
I used honeywell rfid library with proguard. When some classes and enums are not kept, a strange case occurred. When trying to write an rfid tag, even if it has failed, library was returning that it was a successfull writing. All other methods was working properly. So what to do for protecting your own code.
Open the third party library file by double clicking it in Android Studio. Enter into the classes.jar file and determine which packages are used. Then keep these packages in the proguard file.
As an example :
[![third party packages to include][1]][1]
[1]: https://i.stack.imgur.com/lr2fb.png
proguard-rules.pro file must look like this.
-keep class com.honeywell.** { *; }
-keep class com.silionmodule.** { *; }
-keep class com.bth.** { *; }
-keep class com.communication.** { *; }
-keep class com.thingmagic.** { *; }
-keep class com.tool.** { *; }
-keep enum com.honeywell.** { *; }
-keep enum com.silionmodule.** { *; }
-keep enum com.bth.** { *; }
-keep enum com.communication.** { *; }
-keep enum com.thingmagic.** { *; }
-keep enum com.tool.** { *; }
Background
I am developing an Android application that relies on multiple external libraries (8 added as library project dependencies, 14 added as jar dependencies).
Some of these jar libraries are closed source an have already been obfuscated and some of them rely pretty heavily on reflection.
The application uses ZXing for QR code scanning/recognition and, without Proguard optimization, ZXing is quite slow (at least on Android).
At first, I only needed to optimize the com.google.zxing.** package using Proguard. In order to do that I've added the following Proguard options in my config file (the best I could figure out from this question):
-keep class !com.google.zxing.** { *; }
-keep interface !com.google.zxing.** { *; }
-keep enum !com.google.zxing.** { *; }
-dontwarn !com.google.zxing.**
I exported my application and it works like a charm.
Problem
Now, I want to use Proguard to obfuscate the application's classes.
I've tried changing the above to:
-keep class !(com.google.zxing.**, com.example.app.**) { *; }
-keep interface !(com.google.zxing.**, com.example.app.**) { *; }
-keep enum !(com.google.zxing.**, com.example.app.**) { *; }
-dontwarn !(com.google.zxing.**, com.example.app.**)
-keep com.example.app.activities.** { *; }
-keep com.example.app.receivers.** { *; }
-keep com.example.app.services.** { *; }
-keep com.example.app.views.** { *; }
The problem is that Proguard does not accept !(package.one.**, second.package.**) { *; } as a valid option for a -keep rule.
Another approach would be to put a -keep rule for every package in my application.
This approach has two big disadvantages:
adding or swapping libraries would require changing the Proguard config file
it makes updating libraries a pain, as some of them are obfuscated and, when recompiled by the library's developer, will change package names.
Obviously, I would like to avoid this approach as much as possible (because of the high number of external libraries).
Question
Is it possible to use Proguard to obfuscate just two packages, without defining a -keep rule for each of the other packages in my app? If so, how can I do this?
The correct syntax is a comma-separated list without any parentheses:
-keep class !com.google.zxing.**,!com.example.app.** { *; }
See the ProGuard manual > Usage > Filters.
Note that this single line already implies the two other lines for interfaces and enums. You can imply the -keep options for all subpackages by not letting the last wildcard match subpackages:
-keep class !com.google.zxing.**,!com.example.app.* { *; }