Would any of you be so kind as to rephrase (in your own words) the explanations for some of the proguard keywords that are written in their manual? I have hard time understanding in full what some of them mean, and what changes if they are not there in the .cfg file.
The keywords I'm interested are:
1) -dontskipnonpubliclibraryclasses and -dontskipnonpubliclibraryclassmembers
The second is being explained as:
Specifies not to ignore package visible library class members (fields and methods). By default, ProGuard skips these class members while parsing library classes, as program classes will generally not refer to them. Sometimes however, program classes reside in the same packages as library classes, and they do refer to their package visible class members. In those cases, it can be useful to actually read the class members, in order to make sure the processed code remains consistent.
First of all, does it refere only in the context of external jars? Second, what is the difference between those flags reside in the .cfg file vs not being there?
2) -libraryjars, I'm lost for that one. What is the purpose of this keyword? On proguard manual page it reads:
Specifies the library jars (or aars, wars, ears, zips, apks, or directories) of the application to be processed.
So does it mean, that if I don't use this flag, then those jars are not put under the whole obfuscation process? But if that's the case, then why when I don't use this keyword, there are a lot of warnings for classes in those jars in the proguard output?
Next it says:
The files in these jars will not be included in the output jars.
What does it mean exactly? It means, that if this flag is set, then all other files aside .class files will not be included in the parent's application jar?
After hours of reading I think I got my answers. Especialy what helped me was reading many of the creator of ProGuard answers here at StackOverflow.
Let me start with the jars topic. Libraryjars is usually the platfrom jar, the application is build against, so android.jar is a good example here. This jar will not be processed, it's classes will not reside in the output apk, because they will be all on the clients device. They will not be obfuscated or shrunk, because a) even if they were they would not be coppied into the output apk anyway, and b) if they were obfuscated then it would crash application due to the fact that say Activity during obfuscation would have the name changed to "a", but on the clients device the android API is not changed.
So libraryjars is used for all the jars that proguard needs when processing our app, but which jars will not be included (or it's class files) in the final apk.
Injars on the other hand are all the jars that we want to be shrunk/obfuscated etc (unless we use keep* keywords).
Now the reason that I had so much difficulty was because there were conflicting information about those keywords all over the place. Some people said to use -injars, some said to use -libraryjars, some said neighter. What I found out later on, is that the last answer is correct. No -libraryjars or -injars keyword is needed because ADT does all this for the developer, and it uses the -injars keyword with all the jars residing in the /libs folder.
That is also the reason why I found many people using the "keep" keywords with the packages of one of the jars to ignore it's obfuscation/shrinking. The reason for it is that because ADT uses -injars keyword for those jars by default (and not libraryjars which would essentialy do the same in this context) then those jars are marked to be processed (obfuscated/shrunk). To negate this effect, people use -keep keywords for the packages of those jars.
As for the #1 question:
First of all, does it refere only in the context of external jars? The answer is no. It reffers to all the libraries even referenced inside the attached jars
Second, what is the difference between those flags reside in the .cfg file vs not being there? From what I found out it's for helping the ProGuard with processing of those libraries.
Related
I created two aar-libraries: com.example:test1:1.0.0 and com.example:test2:1.0.0. I added these libs to my main app and after build Android-Studio throws error:
Duplicate class com.example.utils.a found in modules classes.jar (com.example:test1:1.0.0) and classes.jar (com.example:test2:1.0.0)
I find out that this error happens because both libraries have classes in same package (com.example.utils) and after obfuscation R8 creates classes (a.class) with same full name (I saw this in classes.jar of aar). If I disable R8 in properties of my libs this error has gone.
'a' is not library class: after obfuscation all library classe names remain unchanged and a.class was added in package additionaly by R8.
I read R8 documentation and found nothing about this problem.
Are there any ways to solve this issue without ranaming the package in one of my libs?
When creating two libraries it is best practice to use two different namespaces, as otherwise there will be the possibility of duplicate classes even without using R8 when "accidentally" a class with the same name is added to both. So in your case use com.example.test1 and com.example.test2.
Depending on you use case, it might also be a better option to not apply R8 to the libraries, but only apply R8 to the final app including the two libraries. Shrinking libraries are mainly to make distribution size smaller, and rename internals to avoid library users (accidentally or knowingly) depend on internals which might change between library versions.
When shrinking libraries you also want to consider the option -keeppackagenames to make sure that all renamed classes stay within the package of the library. Otherwise you might end up with class e.g. a.a.a.a.class in multiple libraries.
If this issue happens for libraries that you don't have control over tools like shadow can be used to relocate.
You should always prefix all of your code in Java or other JVM languages with unique packages to create a unique fully qualified name because any two classes with the exact same fully qualified name will cause a build error. This happens because the JVM only uses the fully qualified name string, saved initially in a single table, to instantiate all the classes and interfaces in the system. If the table will have more than one entry for a class/interface, it won't know which one to choose. You can read more about it here.
As I describe in my answer here, the best solution for obfuscation related collisions is to use -repackageclasses com.example:test#.ofs in the proguard-rules file of each library while replaceing # with 1 and 2 respectivly. This will move all the obfuscated classes into the com.example:test#.ofs package while all the non-obfuscated classes will remain in their original packages and you're guaranteed to have no collisions.
As the Proguard documentation states:
-repackageclasses [package_name]
Specifies to repackage all class files that are renamed, by moving them into the single given package.
According to the documentation -dontusemixedcaseclassnames turns off the feature that causes files to self distruct if extracted on windows. surely this is a good thing when trying to hide your code. Why is it enabled, is there a downside to not using it?
-dontusemixedcaseclassnames
Specifies not to generate mixed-case class names while obfuscating. By
default, obfuscated class names can contain a mix of upper-case
characters and lower-case characters. This creates perfectly
acceptable and usable jars. Only if a jar is unpacked on a platform
with a case-insensitive filing system (say, Windows), the unpacking
tool may let similarly named class files overwrite each other. Code
that self-destructs when it's unpacked! Developers who really want to
unpack their jars on Windows can use this option to switch off this
behavior. Obfuscated jars will become slightly larger as a result.
Only applicable when obfuscating.
Dalvik bytecode works fine with similar mixed-case class names. I suspect the configuration in the Android SDK contains the option to avoid confusion for developers who inspect their own compiled code.
History/Context
I have a project[1] where size really matters - recently I moved stuff to a shared lib[2] and thought proguard will take care and remove the unused classes because I had a config that was drastically reducing the size but by using the lib i came over the magic 100kb mark so I investigated: classes which I do not use for sure are in the resulting dex file - and even with full name ( not shortened to single-char ) - e.g. I see the SquareView in the dex which I in no way use in the App.
Question
Surprisingly I found in the proguard documentation the following:
The library jars themselves always remain unchanged.
Can I somehow tell/trick proguard (in)to process them? I find this really strange especially because I expect more stuff to be removeable in the lib than in the App itself..
[1] https://github.com/ligi/FAST
[2] https://github.com/ligi/AndroidHelper
The Eclipse/Ant/Gradle build processes in the Android SDK automatically specify your code (from bin/classes) and its libraries (from libs) with the option -injars. This means that the complete application is compacted, optimized, and obfuscated (in release builds, assuming ProGuard is enabled).
The build processes only specify the Android runtime android.jar with the option -libraryjars. It is necessary to process the code, but it should not end up in the processed apk, since it is already present on the device.
So it should all work out automatically. You may still see entire libraries with their original names in processed apks, if your configuration proguard-project.txt contains lines like -keep class org.mylibrary.** { *; }. Such configuration is typically a conservative solution to account for reflection. With some research and experimentation, you can often refine the configuration and get better results. You can figure out why classes are being kept with the option -whyareyoukeeping.
I believe you have to use -injars:
-injars class_path
Specifies the input jars (or wars, ears, zips, or directories) of the application to be processed. The class files in these jars will be
processed and written to the output jars. By default, any non-class
files will be copied without changes. Please be aware of any temporary
files (e.g. created by IDEs), especially if you are reading your input
files straight from directories. The entries in the class path can be
filtered, as explained in the filters section. For better readability,
class path entries can be specified using multiple -injars options.
Source: http://proguard.sourceforge.net/index.html#manual/usage.html
In android applications we include many library files such as google-play-services-lib and Facebook-SDK etc. But we never really use all features and classes from those libraries, so my question is when .apk file gets created does all those classes are included or the only classes we use are included in our application? If yes then is there a way we can get around that? ie can we remove or do something to avoid inclusion of all classes?
Thank You...
You should keep your Android application as small as possible. Therefore you should only include classes in the jars which you really need.
http://www.vogella.com/blog/2010/02/11/java-library-jar-android/
It is better to obfuscate your code using Proguard.
ProGuard is a Java class file shrinker, optimizer, obfuscator, and preverifier. The shrinking step detects and removes unused classes, fields, methods, and attributes. The optimization step analyzes and optimizes the bytecode of the methods
Obfuscation also secures your code to an extent.
To enable ProGuard in your project, edit project.properties
# Project target.
proguard.config=${sdk.dir}/tools/proguard/proguard-android.txt
target=Google Inc.:Google APIs:16
android.library.reference.1=../actionbarsherlock
http://developer.android.com/tools/help/proguard.html
http://proguard.sourceforge.net/index.html#manual/introduction.html
In the case, if I will
build a project
clean up all binaries
build it again (no source/resources and etc has changed).
Does Proguard guarantee to provide the same mapping.txt file?
ProGuard is deterministic: for the same input, it will generate the same output.
There is one subtlety though: if the operating system lists input files in a directory (notably class files that are not inside an archive) in a different order, then they may be processed in a different order, and the output can be different.
It might actually happen, but i don't think proguard guarantees that.
I found this in the Proguard documentation that will allow you to reuse your mapping.txt to avoid changes on the mappings
-applymapping filename
Specifies to reuse the given name mapping that was printed out in a previous obfuscation run of ProGuard. Classes and class members that are listed in the mapping file receive the names specified along with them. Classes and class members that are not mentioned receive new names. The mapping may refer to input classes as well as library classes. This option can be useful for incremental obfuscation, i.e. processing add-ons or small patches to an existing piece of code. If the structure of the code changes fundamentally, ProGuard may print out warnings that applying a mapping is causing conflicts. You may be able to reduce this risk by specifying the option -useuniqueclassmembernames in both obfuscation runs. Only a single mapping file is allowed. Only applicable when obfuscating.
If you want a guarantee then you have to use the mappings file as input to the obfuscation process. But then you carefully have to check all warnings about conflicts relating to that mapping file. If you ignore that, you may get subtle errors, when working with reflection.