I'm compiling my (fairly simple, just 5 files with few hundred LOC) app from command line on OSX using:
ant debug
It works. But it works slowly:
BUILD SUCCESSFUL
Total time:
26 seconds
Why is that? It takes this much time even if I change only one line in one java file. Most of this time is spent in dex stage (about 20 seconds), which is AFAIK creating Dalvik bytecode. But my friend that also works on the same project on Windows using Eclipse says that compiling takes only a second or two on his machine. Is there anything I can do to speed up this proccess?
I finally found a solution for this! It's a bit of a hack, but it works.
First, go to your ANDROID-SDK/platform-tools directory, then rename dx app to something else, like dextool, and finally create new dx file with contents:
#!/bin/sh
shift
dextool --dex --incremental --no-optimize $#
Replace "dextool" with the name you chose before. This will prepend (undocumented) --incremental attribute to every dex invocation, which will massively decrease build times by dexing only classes that have changed between builds. Now it looks like this:
[dx] Merged dex A (1 defs/11,3KiB) with dex B (359 defs/1253,2KiB). Result is 359 defs/1519,3KiB. Took 0,5s
0.5s instead of 20s is a huge difference!
Edit - few remarks:
you have to compile your project at least once before using this, because it uses previous classes.dex file
you can run into problems when using other Android toolchains than ant
UPDATE:
Google released SDK Tools 21.0, which renders above tweak absolete, because it does supports pre-dexing. Finally!
Even in 21.1.1 with the --incremental --no-optimize added in the original dex.bat it is slow, so I went on to figure something out, the result is: if you order the .jar files passed to dex by size you get better performance.
Watch https://code.google.com/p/android/issues/detail?id=79166 for updates, I hope they agree and this goes into vNext.
#!/usr/bin/perl
use strict;
use warnings;
#use Data::Dump qw(dump);
use List::Util qw(first), qw(sum);
# size of the argument, -s for files, -s on **/*.class for folders
sub size {
if (-d $_) {
# directory size is sum of all class files in the dir recursively
# account for pre-dexing and compression with a 25% decrease
return sum(map { size($_) * 0.25 } <$_/*.class>) || 0;
}
return -s $_; # use built-in size operator
}
my $dx_args_with_args =
qr/^--(output|positions|(no-)?optimize-list|dump-(to|width|method)|num-threads|main-dex-list|input-list)$/;
my $nArgs = $#ARGV;
# last argument like --blah, those are for dx
my $lastArg = $nArgs - first { $ARGV[$nArgs - $_] =~ /^--/ } 0..$nArgs;
if ($lastArg != -1 && $ARGV[$lastArg] =~ /$dx_args_with_args/) {
$lastArg += 1;
}
my #inputs = map { $_->[1] }
sort { $a->[0] <=> $b->[0] }
map { [size(), $_] }
#ARGV[$lastArg + 1 .. $nArgs];
print join(" ", #ARGV[0..$lastArg], #inputs);
exit 0;
Usage
have Perl on your path
copy the above perl script to ANDROID-SDK/build-tools/v.v.v/dx.pl
rename dx in ANDROID-SDK/build-tools/v.v.v/
Unix: rename dx to dx-orig
Windows: rename dx.bat to dx-orig.bat
add a new replacement dx which calls through:
Windows: dx.bat
#echo off
setlocal
set args=%*
for /f "delims=" %%i in ('perl "%~dp0dx.pl" %args%') do set args=%%i
call "%~dp0dx-orig.bat" %args%
endlocal
Unix: dx
#!/bin/sh
dx-orig `perl dx.pl $#`
Related
Trying to build libwally-core C library for Android on Windows in Cygwin with supplied autotools scripts:
libwally-core
After running
bash tools/build_android_libraries.sh
or
bash tools/autogen.sh
I get the following error:
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: error: AC_CONFIG_MACRO_DIRS([build-aux/m4]) conflicts with
ACLOCAL.AMFLAGS=-I build-aux/m4
autoreconf-2.69: libtoolize failed with exit status: 1
I tried the following things to no avail:
Re-saved all scripts with Unix line-endings (LF only)
Commenting out "ACLOCAL_AMFLAGS = -I tools/build-aux/m4" in Makefile.am
The error happens at the following place in libtool's source in libtoolize.in:
macrodir="$ac_macrodir"
test -z "$macrodir" && macrodir="$am_macrodir"
if test -n "$am_macrodir" && test -n "$ac_macrodir"; then
test "$am_macrodir" = "$ac_macrodir" \
|| func_fatal_error "AC_CONFIG_MACRO_DIR([$ac_macrodir]) conflicts with ACLOCAL_AMFLAGS=-I $am_macrodir."
fi
I assume that the above makes sure that AC_CONFIG_MACRO_DIR and value after "-I" in ACLOCAL_AMFLAGS are identical (checked for identical line endings with hex editor too). The values are identical in both configure.ac and Makefile.am. However, even if I comment out setting ACLOCAL_AMFLAGS in Makefile.am, the error persits.
I would like to compile the library and generate libwallycore.so. Any insight would be much appreciated.
I am trying to compile chromium for android on Debian 9 and I get this error, can someone help me on this?
dacod#phoenix:~/Devel/chromium/src$ ninja -C out/Default cast_shell_apk
ninja: Entering directory `out/Default'
[0/21317] ACTION //third_party/WebKit/Source/bindings/modules/v8:generate...snapshot_external_references(//build/toolchain/android:android_clang_arm)
ninja: error:
WriteFile(__third_party_WebKit_Source_bindings_modules_v8_generate_v8_context_snapshot_external_references___build_toolchain_android_android_clang_arm__rule.rsp):
Unable to create file. File name too long
ninja: build stopped: .
The file system you are building on does not support name sizes long enough to accommodate the 150 character filename:
__third_party_WebKit_Source_bindings_modules_v8_generate_v8_context_snapshot_external_references___build_toolchain_android_android_clang_arm__rule.rsp
Typical file systems will support 255 character file names. Here is a reference:
https://serverfault.com/questions/9546/filename-length-limits-on-linux
However, eCryptfs only supports 143 character file names if it is also configured to encrypt the file names themselves [source]. Are you building in your home folder and is it encrypted? If so, this may be the reason. To check the file name size, run the following in your build folder:
$ getconf NAME_MAX .
143
If it is not 255 you may be building on an ecryptfs partition. Try building in /tmp instead, as it may not be encrypted. You can check this with:
$ getconf NAME_MAX /tmp
255
In my previous Windows/Eclipse/ant project development method I had a means of producing a time-limited release apk from the command line. I would like to reproduce this functionality with my new Ubuntu/Android Studio/Gradle setup.
This is how the old system worked:
I had a .bat file which ran a runnable jar which I wrote (and can also create on my new machine). That jar takes two arguments expiry date and where to put the output file. The jar's output is called Timekey.java and it looks like:
package uk.co.myname.timekey;
public final class Timekey{
public static final String EXPIRY_DATE =
"the encrypted string";
public String getExpiryDate() {
return EXPIRY_DATE;
}
}
// Plain date : 2020-01-01.00_00_00
I have my build.xml checking for the presence of this file
<target name="-check-timekey">
<echo>"${timelimit_src}/Timekey.java"</echo>
<available file="${timelimit_src}/Timekey.java" property="timekey.present" />
</target>
and if present it sets the src
<if condition="${timekey.present}">
<src path="${timelimit_src}" />
Thus I can produce an apk which will only run up to the date entered as a parameter to the batch file. The encryption is not military grade but good enough to defeat amateurs and should stop the de-obfuscation fiends.
Any ideas on how to implement this with gradle will be most welcome. I know how to run the jar from a bash script but swapping source directories, just for command line release builds has me stumped
I did manage to solve this in the end. I created the build type 'release' which was not previously necessary,as 'main' sufficed. I also reated the build type 'timelimited'. Timekey.java was removed from main and placed in debug, release and timelimited's src/java folders (including the class stucture hierarchical folders).
This script completed the process
#!/bin/bash
# Script to build time limited apk NBT 2nd March 2016
# Must be run from one leve below AndroidStudioProjects folder
# Must have one argument of expiry date in YYYY-MM-DD format
CURRENT_DIR=`pwd`
case "$CURRENT_DIR" in
*AndroidStudioProjects/*) ;;
*) echo "Quitting because of wrong starting directory name"
echo "You must be one folder below ~/AndroidStudioProjects to run this script"
exit 1 ;;
esac
OUTDIR_SUFFIX=/app/src/timelimited/java/uk/co/myname/timekey/
OUTDIR=$CURRENT_DIR$OUTDIR_SUFFIX
echo "Directory to write in: "$OUTDIR
if [ -z "$1" ]
then
echo "argument 1 required, as expiry date in YYYY-MM-DD format, so quitting"
exit 1
fi
EXDATE=$1
case "$EXDATE" in
????-??-??) ;;
*) echo "Quitting because of bad date format on parameter 1"
exit 1 ;;
esac
# the encrypt5.jar requires these two arguments
java -jar ~/runnablejars/encrypt5.jar $EXDATE $OUTDIR
echo " "
echo " "
gradlew assembleTimelimited
cat $OUTDIR/Timekey.java
echo " "
echo "All done, now IF you saw Timekey.java printed out, then"
echo "the time limited apk is built. Run gradlew InstallTimelimited"
echo "to install it on a running device"
while true; do
read -p "Do you wish to install this apk on running device y/n [enter]? " yn
case $yn in
[Yy]* ) gradlew InstallTimelimited; break;;
[Nn]* ) exit;;
* ) echo "Please answer yes or no.";;
esac
done
I hope this might prove useful. There are probably less convoluted ways of doing it but I don't know one.
I'm familiar with building large applications using make, but now I have begun using Android Studio and I want to understand how to do things I already do in a Makefile.
Here are a example that might help you frame an answer:
Makefile example: (minimalist)
INK=inkscape
INKFLAGS=--export-area-page
# Resolve *.png dependencies
drawable-mdpi/%.png: %.svg
$(INK) $< --export-png=$# -w 48 -h 48 $(INKFLAGS)
drawable-hdpi/%.png: %.svg
$(INK) $< --export-png=$# -w 72 -h 72 $(INKFLAGS)
drawable-xdpi/%.png: %.svg
$(INK) $< --export-png=$# -w 96 -h 96 $(INKFLAGS)
More simple example:
drawable-mdpi/ic_launcher.png: ic_launcher.svg
inkscape ic_launcher.svg --export-png=ic_launcher.png -w 48 -h 48 --export-area-page
drawable-hdpi/ic_launcher.png: ic_launcher.svg
inkscape ic_launcher.svg --export-png=ic_launcher.png -w 72 -h 72 --export-area-page
How to do that in Gradle?
I want to resolve external dependencies such as mentioned in the above example. Actually I'm doing it via 'make', but i want to completely remove this extra step.
It is possible to run external commands from Grandle and integrate those into your build process. My example runs inkscape.exe on Windows and defines its parameters in the build script, you can also just run a shell script this way.
The following code goes into the app\build.gradle file. Task convertDrawable is written in Groovy syntax and accomplishes the following (tl;dr it is an implementation of your "simple example"):
It looks through all the *.svg files in a custom folder art/drawable
Within each of those *.svg files, looks through all the drawable-* folders in your resources folder
Based on drawable-* folder name, determine the target resolution.
Then calls inkscape.exe to convert each *.svg to *.png with the required size.
Code:
task convertDrawables() {
def ink = 'C:\\Program Files (x86)\\Inkscape\\inkscape.exe'
// look for *.svg files in app/src/art/drawable folder
new File('app\\src\\art\\drawable').eachFileMatch(~/.*\.svg/) { file ->
// look for destination folders
new File('app\\src\\main\\res').eachFileMatch(~/drawable-.*/) { outputDir ->
// define size based on folder name
def size = ''
switch (outputDir.getAbsolutePath()) {
case ~/.*-ldpi/:
size = '36'
break
case ~/.*-mdpi/:
size = '48'
break
case ~/.*-hdpi/:
size = '72'
break
case ~/.*-xhdpi/:
size = '96'
break
case ~/.*-xxhdpi/:
size = '144'
break
case ~/.*-xxxhdpi/:
size = '192'
break
}
def cmd = ink + ' ' + file.getCanonicalPath() + ' --export-png=' + outputDir.getAbsolutePath() + '\\ic_launcher2.png -w ' + size + ' -h ' + size + ' --export-area-page'
def process = cmd.execute();
process.waitFor();
}
}
}
// make sure the convertDrawable task is executed somewhere in the make process
gradle.projectsEvaluated {
preBuild.dependsOn(convertDrawable)
}
Here are the resources I used:
Gradle Doc Chapter 15: More about Tasks
Gradle Doc Chapter 58. Writing Custom Task Classes
Gradle Doc Exec function
Groovy execute()
Stackoverflow Run task before compilation using Android Gradle plugin
Groovy Doc for File
I've been asked to automate the process of building a 'secondary dex' that can be loaded by a DexClassLoader. The idea is that instead of manually exporting to a .jar file and then running the dx script to add classes.dex to a new .jar in the main project's assets directory, normal compilation of the secondary project would automatically copy a dynamically loadable .jar to the main project.
It does look like I can do this by marking the secondary project as an Android Library (so that it compiles to a .jar) and adding an Eclipse Builder that calls the dx script. I've got a bash script and a .launch wrapper for it that passes ${project_loc}/bin/${resource_name}.jar to the script; if I place the bash script in the main project's root directory, %0 tells me where to copy the loadable .jar.
Of course, dx is not on the path. I figured I'd do a `which adb` and then use find ... but adb is not on the path, either!
So, my question: Is there an Eclipse string variable that contains the location of the Android SDK?
A coworker pointed out that ${project_classpath} contains a path to annotations.jar in the SDK directory. That let me write this script:
#!/bin/bash
#####
##### Get the directory this script is running in.
##### We will write our output to ${client}/assets
#####
client=${0%/*}
#####
##### Get the input parameter; calculate the output
#####
input=$1
output=${client}/assets/${input##*/}
#####
##### Use the classpath parameter to find the latest dx script:
#####
# Turn : into \n
classpath=${2//:/\\n}
# Find the line that refers to annotations.jar
annotations=`echo -e $classpath | grep annotations`
# Strip the filename
path=${annotations%/*}
# There may be multiple copies. We get access time and path.
# We output : instead of \n because bash "Word Splitting" is complicated.
jars=`find $path -name dx.jar -printf "%A# %p:"`
until [ "$jars" ]
do
path=${path}/..
jars=`find $path -name dx.jar -printf "%A# %p:"`
done
# Turn : into \n
jars=${jars//:/\\n}
# We want the last accessed jar
sorted=`echo -e "$jars" | sort -nrk 1,2`
# Get the jar filename
last_used=`echo -e "$sorted" | grep -om 1 "/.*$"`
# Strip the filename
last_path=${last_used%/*}
# Find the dx script
dx=`find $last_path -name dx`
until [ "$dx" ]
do
last_path=${last_path}/..
dx=`find $last_path -name dx`
done
#####
##### Add classes.dex to input jar
#####
$dx --dex --keep-classes --output $output $input
Here's it's .launch file, with the project name and path redacted:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.ui.externaltools.ProgramBuilderLaunchConfigurationType">
<booleanAttribute key="org.eclipse.debug.core.ATTR_REFRESH_RECURSIVE" value="false"/>
<stringAttribute key="org.eclipse.debug.core.ATTR_REFRESH_SCOPE" value="${working_set:<?xml version="1.0" encoding="UTF-8"?>
<resources>
<item path="/MAIN_PROJECT_NAME/assets" type="2"/>
</resources>}"/>
<booleanAttribute key="org.eclipse.debug.ui.ATTR_LAUNCH_IN_BACKGROUND" value="false"/>
<booleanAttribute key="org.eclipse.ui.externaltools.ATTR_BUILDER_ENABLED" value="true"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_LOCATION" value="${project_loc}/../MAIN_PROJECT_DIRECTORY/secondary-dexxer"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_RUN_BUILD_KINDS" value="incremental,auto,"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_TOOL_ARGUMENTS" value="${project_loc}/bin/${resource_name}.jar
${project_classpath}"/>
<booleanAttribute key="org.eclipse.ui.externaltools.ATTR_TRIGGERS_CONFIGURED" value="true"/>
</launchConfiguration>
Not exactly optimal, and definitely not something I can add to the source tree (we have people who use Windows for some reason) but a start: painful though some of the bash "Word Splitting" issues were, they presumably do pale next to the learning curve that will be involved in turning this into a 'proper' Eclipse plugin, in Java.
Fwiw, I never did find "an Eclipse string variable that contains the location of the Android SDK". Instead, what I did was
Read the dx script and notice that it just passed arguments to dx.jar.
Add the most recent available dx.jar to my Builder project, and simply call it directly, from my Java code. The source was not hard to find. It turned out that the main main(String[]) simply dispatched to one of several internal main(String[]) methods, so I just called the right one directly.
Split my Builder into two pieces: a) a 'stand-alone dexxer' which is just a jar file that can be called from an ant action, and which calls into dx.jar; and b) an Eclipse Builder which contains the build control logic and a bit of Eclipse console code, and which calls into the stand-alone dexxer.
This worked fine. We figured that if Google ever released a dx.jar that somehow offered significantly better dexification, we could just rebuild the Builder with the new jar; this hasn't been an issue, yet.