Develop Reference

Building with autotools: AC_CONFIG_MACRO_DIRS conflicts with ACLOCAL.AMFLAGS

Trying to build libwally-core C library for Android on Windows in Cygwin with supplied autotools scripts:
libwally-core
After running
bash tools/build_android_libraries.sh
or
bash tools/autogen.sh
I get the following error:
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: error: AC_CONFIG_MACRO_DIRS([build-aux/m4]) conflicts with
ACLOCAL.AMFLAGS=-I build-aux/m4
autoreconf-2.69: libtoolize failed with exit status: 1
I tried the following things to no avail:
Re-saved all scripts with Unix line-endings (LF only)
Commenting out "ACLOCAL_AMFLAGS = -I tools/build-aux/m4" in Makefile.am
The error happens at the following place in libtool's source in libtoolize.in:
macrodir="$ac_macrodir"
test -z "$macrodir" && macrodir="$am_macrodir"
if test -n "$am_macrodir" && test -n "$ac_macrodir"; then
test "$am_macrodir" = "$ac_macrodir" \
|| func_fatal_error "AC_CONFIG_MACRO_DIR([$ac_macrodir]) conflicts with ACLOCAL_AMFLAGS=-I $am_macrodir."
fi
I assume that the above makes sure that AC_CONFIG_MACRO_DIR and value after "-I" in ACLOCAL_AMFLAGS are identical (checked for identical line endings with hex editor too). The values are identical in both configure.ac and Makefile.am. However, even if I comment out setting ACLOCAL_AMFLAGS in Makefile.am, the error persits.
I would like to compile the library and generate libwallycore.so. Any insight would be much appreciated.

Call an Android build-tool from an Eclipse "builder"?

I've been asked to automate the process of building a 'secondary dex' that can be loaded by a DexClassLoader. The idea is that instead of manually exporting to a .jar file and then running the dx script to add classes.dex to a new .jar in the main project's assets directory, normal compilation of the secondary project would automatically copy a dynamically loadable .jar to the main project.
It does look like I can do this by marking the secondary project as an Android Library (so that it compiles to a .jar) and adding an Eclipse Builder that calls the dx script. I've got a bash script and a .launch wrapper for it that passes ${project_loc}/bin/${resource_name}.jar to the script; if I place the bash script in the main project's root directory, %0 tells me where to copy the loadable .jar.
Of course, dx is not on the path. I figured I'd do a `which adb` and then use find ... but adb is not on the path, either!
So, my question: Is there an Eclipse string variable that contains the location of the Android SDK?

A coworker pointed out that ${project_classpath} contains a path to annotations.jar in the SDK directory. That let me write this script:
#!/bin/bash
#####
##### Get the directory this script is running in.
##### We will write our output to ${client}/assets
#####
client=${0%/*}
#####
##### Get the input parameter; calculate the output
#####
input=$1
output=${client}/assets/${input##*/}
#####
##### Use the classpath parameter to find the latest dx script:
#####
# Turn : into \n
classpath=${2//:/\\n}
# Find the line that refers to annotations.jar
annotations=`echo -e $classpath | grep annotations`
# Strip the filename
path=${annotations%/*}
# There may be multiple copies. We get access time and path.
# We output : instead of \n because bash "Word Splitting" is complicated.
jars=`find $path -name dx.jar -printf "%A# %p:"`
until [ "$jars" ]
do
path=${path}/..
jars=`find $path -name dx.jar -printf "%A# %p:"`
done
# Turn : into \n
jars=${jars//:/\\n}
# We want the last accessed jar
sorted=`echo -e "$jars" | sort -nrk 1,2`
# Get the jar filename
last_used=`echo -e "$sorted" | grep -om 1 "/.*$"`
# Strip the filename
last_path=${last_used%/*}
# Find the dx script
dx=`find $last_path -name dx`
until [ "$dx" ]
do
last_path=${last_path}/..
dx=`find $last_path -name dx`
done
#####
##### Add classes.dex to input jar
#####
$dx --dex --keep-classes --output $output $input
Here's it's .launch file, with the project name and path redacted:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.ui.externaltools.ProgramBuilderLaunchConfigurationType">
<booleanAttribute key="org.eclipse.debug.core.ATTR_REFRESH_RECURSIVE" value="false"/>
<stringAttribute key="org.eclipse.debug.core.ATTR_REFRESH_SCOPE" value="${working_set:<?xml version="1.0" encoding="UTF-8"?>
<resources>
<item path="/MAIN_PROJECT_NAME/assets" type="2"/>
</resources>}"/>
<booleanAttribute key="org.eclipse.debug.ui.ATTR_LAUNCH_IN_BACKGROUND" value="false"/>
<booleanAttribute key="org.eclipse.ui.externaltools.ATTR_BUILDER_ENABLED" value="true"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_LOCATION" value="${project_loc}/../MAIN_PROJECT_DIRECTORY/secondary-dexxer"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_RUN_BUILD_KINDS" value="incremental,auto,"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_TOOL_ARGUMENTS" value="${project_loc}/bin/${resource_name}.jar
${project_classpath}"/>
<booleanAttribute key="org.eclipse.ui.externaltools.ATTR_TRIGGERS_CONFIGURED" value="true"/>
</launchConfiguration>
Not exactly optimal, and definitely not something I can add to the source tree (we have people who use Windows for some reason) but a start: painful though some of the bash "Word Splitting" issues were, they presumably do pale next to the learning curve that will be involved in turning this into a 'proper' Eclipse plugin, in Java.

Fwiw, I never did find "an Eclipse string variable that contains the location of the Android SDK". Instead, what I did was
Read the dx script and notice that it just passed arguments to dx.jar.
Add the most recent available dx.jar to my Builder project, and simply call it directly, from my Java code. The source was not hard to find. It turned out that the main main(String[]) simply dispatched to one of several internal main(String[]) methods, so I just called the right one directly.
Split my Builder into two pieces: a) a 'stand-alone dexxer' which is just a jar file that can be called from an ant action, and which calls into dx.jar; and b) an Eclipse Builder which contains the build control logic and a bit of Eclipse console code, and which calls into the stand-alone dexxer.
This worked fine. We figured that if Google ever released a dx.jar that somehow offered significantly better dexification, we could just rebuild the Builder with the new jar; this hasn't been an issue, yet.

Time cannot run brunch?

I'm having an issue with the time command.
I use /usr/bin/time so i can use the -f and -o modifiers.
My script is:
make clean
repo sync -j5
. ./platform_manifest/manual_add_kernel_manifest.sh \
&& . build/envsetup.sh \
&& /usr/bin/time -o log$day$month.log -f "Buildtime: %E" brunch aokp_mako-eng
The script is part of an automated build system for an android ROM.
When it gets to the time part, i get:
/usr/bin/time: cannot run brunch: No such file or directory
Command exited with non-zero status 127
Buildtime: 0:00.00
Brunch works fine with the regular time command, but I cannot route its output to a file, which is why I am using /usr/bin/time in the first place. If there is a way to do this, that is fine for me as I can trim off the real: header with | awk '{ print $2 }'.
Any help is appreciated!
Ubuntu 12.04, all updates installed

I don't know what brunch is, but the most likely reason that a non-builtin time program can't find it, even when the built-in time can, is that it's a shell function or a Bash alias.
Whether or not that's the case . . .
[…] I cannot route [the regular time command's] output to a file […]
You can write:
{ time brunch aokp_mako-eng ; } 2>"log$day$month.log"
to wrap time (and everything else) into a command-list whose STDERR has already been redirected to the file.

Android ant and bash scripting

I've had 0 exposure to BASH scripting and this is something I would love to learn. I can't figure out how to run a conditional statement based on the output of ant debug on an Android build.
I would like to essentially say something like
if(`ant debug` == SUCCESS) {
// EXECUTE THESE COMMANDS
} else {
// EXECUTE THESE COMMANDS
}
How can I determine if the ant debug has passed or failed in shell script?
SOLUTION
Okay here is what I have:
ant clean
if ant debug; then
echo "success"
else
echo "failure"
fi

I'll give a quick summary for you.
In Bash, conditionals are based around the exit codes of programs. An exit code of 0 is accepted as true, while everything else is accepted as false.
For example, the true program always exits with an exit code of 0, which means that something like this is possible:
if true;
echo "It is true"
fi
Most commands honor this system, but not every program does. The first thing to check is what exit code ant returns on success and failure. You can check the exit code of the previous command with $?.
Here is an example:
$ true
$ echo $?
0
$ false
$ echo $?
1
If ant does honor the exit code system properly, then something like the following should be possible:
if ant debug; then
echo success
else
echo failure
fi

I know nothing about Ant debugging, but there are two approaches to doing what you want to do in Bash. The first is to test output like you've shown:
if test $(ant debug) == 'SUCCESS'; then
# do stuff
else
# do other stuff
fi
You can make your shell script portable to other variants on the Bourne shell by using backticks instead of $(.....) like you wrote in your question, but that starts to become a hassle if your commands later involve nested quotes.
The second way, which is a little more robust, is to test the exit value of the commands instead of their output. This depends on Ant (or whatever) having exit codes that are documented and stable, but it means that if the output of the commands changes, your scripts will continue to work. For example, the POSIX standard says that if a programs succeeds in doing whatever it's supposed to do, it should exit() with a value of zero:
ant debug > /dev/null
ant_exit_code=$?
# other commands can safely go here now that $? is captured
if test $ant_exit_code -eq 0; then
# do stuff
else
# do other stuff
fi
And yes, Bourne shell really does end an if block with "fi". :-)

A quick man ant shows that ant debug invokes Ant with the debug task. Ant's tasks are kinda of user-defined XML scripts, and by default Ant searches a build.xml file in the current directory. You can generate the file with the android tools, however a template is kept in android-sdk/tools/ant and you can view it online (line 1005 defines the debug target).
So ant debug isn't really a command, and should not be put in a script toolchain. However, if you find your output to be stable and feel brave, you can always compare string. This is the definitve guide.
if [ `ant debug` = $SOMETHING ]; then
echo Success
else
echo Failure
fi

Compiling Android project from command line is slow

I'm compiling my (fairly simple, just 5 files with few hundred LOC) app from command line on OSX using:
ant debug
It works. But it works slowly:
BUILD SUCCESSFUL
Total time:
26 seconds
Why is that? It takes this much time even if I change only one line in one java file. Most of this time is spent in dex stage (about 20 seconds), which is AFAIK creating Dalvik bytecode. But my friend that also works on the same project on Windows using Eclipse says that compiling takes only a second or two on his machine. Is there anything I can do to speed up this proccess?

I finally found a solution for this! It's a bit of a hack, but it works.
First, go to your ANDROID-SDK/platform-tools directory, then rename dx app to something else, like dextool, and finally create new dx file with contents:
#!/bin/sh
shift
dextool --dex --incremental --no-optimize $#
Replace "dextool" with the name you chose before. This will prepend (undocumented) --incremental attribute to every dex invocation, which will massively decrease build times by dexing only classes that have changed between builds. Now it looks like this:
[dx] Merged dex A (1 defs/11,3KiB) with dex B (359 defs/1253,2KiB). Result is 359 defs/1519,3KiB. Took 0,5s
0.5s instead of 20s is a huge difference!
Edit - few remarks:
you have to compile your project at least once before using this, because it uses previous classes.dex file
you can run into problems when using other Android toolchains than ant
UPDATE:
Google released SDK Tools 21.0, which renders above tweak absolete, because it does supports pre-dexing. Finally!

Even in 21.1.1 with the --incremental --no-optimize added in the original dex.bat it is slow, so I went on to figure something out, the result is: if you order the .jar files passed to dex by size you get better performance.
Watch https://code.google.com/p/android/issues/detail?id=79166 for updates, I hope they agree and this goes into vNext.
#!/usr/bin/perl
use strict;
use warnings;
#use Data::Dump qw(dump);
use List::Util qw(first), qw(sum);
# size of the argument, -s for files, -s on **/*.class for folders
sub size {
if (-d $_) {
# directory size is sum of all class files in the dir recursively
# account for pre-dexing and compression with a 25% decrease
return sum(map { size($_) * 0.25 } <$_/*.class>) || 0;
}
return -s $_; # use built-in size operator
}
my $dx_args_with_args =
qr/^--(output|positions|(no-)?optimize-list|dump-(to|width|method)|num-threads|main-dex-list|input-list)$/;
my $nArgs = $#ARGV;
# last argument like --blah, those are for dx
my $lastArg = $nArgs - first { $ARGV[$nArgs - $_] =~ /^--/ } 0..$nArgs;
if ($lastArg != -1 && $ARGV[$lastArg] =~ /$dx_args_with_args/) {
$lastArg += 1;
}
my #inputs = map { $_->[1] }
sort { $a->[0] <=> $b->[0] }
map { [size(), $_] }
#ARGV[$lastArg + 1 .. $nArgs];
print join(" ", #ARGV[0..$lastArg], #inputs);
exit 0;
Usage
have Perl on your path
copy the above perl script to ANDROID-SDK/build-tools/v.v.v/dx.pl
rename dx in ANDROID-SDK/build-tools/v.v.v/
Unix: rename dx to dx-orig
Windows: rename dx.bat to dx-orig.bat
add a new replacement dx which calls through:
Windows: dx.bat
#echo off
setlocal
set args=%*
for /f "delims=" %%i in ('perl "%~dp0dx.pl" %args%') do set args=%%i
call "%~dp0dx-orig.bat" %args%
endlocal
Unix: dx
#!/bin/sh
dx-orig `perl dx.pl $#`