Develop Reference

Why can't BIRTReportGenerator find my source file?

I'm trying to build a fortify report using BIRTReportGenerator as outlined in the SCA user guide. The scans work fine and generate an fpr file, but when I run the report generator I get the following error.
FPR source file not found or not readable.
Here are the commands I'm using. These were cut and pasted straight from the user guide. The only thing modified is the paths.
sourceanalyzer -b myproject -clean
sourceanalyzer -b myproject -cp /Users/ginger.mcmurray/Mobuyle-Android-New-Ui/MobuyleCore/libs -Dcom.fortify.sca.SuppressLowSeverity=true -Dcom.fortify.sca.LowSeverityCutoff=10.0 -jdk 1.6 MobuyleCore/src
sourceanalyzer -b myproject -scan -f results.fpr
BIRTReportGenerator -template "OWASP Top 10" -source results.fpr -format PDF -showSuppressed --Version "OWASP Top 10 2013" --UseFortifyPriorityOrder -output MyOWASP_Top10_Report.pdf
If I use ReportGenerator instead, everything works. However, I need the ability to create BIRT reports for our security department.
This is for an android java project, just in case that changes anything.
Also, I get a lot of unknown function and reference issues for things that are inside my jar files, despite including the path in the command.
Output from BIRTReportGenerator with -debug option.
Start VM: -Xms40m
-Xmx1088m
-XX:MaxPermSize=320m
-XX:-UseCompressedOops
-Xdock:icon=../Resources/Awb.icns
-XstartOnFirstThread
-Dorg.eclipse.swt.internal.carbon.smallFonts
-Dcom.fortify.InstallRoot=../../../../../../..
-Djava.awt.headless=true
-Dcom.fortify.InstallRoot=/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/bin/..
-Xmx1000M
-XX:MaxPermSize=256m
-Djava.class.path=/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/Auditworkbench.app/Contents/MacOS//../../../plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar
-os macosx
-ws cocoa
-arch x86_64
-launcher /Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/Auditworkbench.app/Contents/MacOS/eclipse
-name HPE Security Fortify Report Generation
--launcher.library /Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/Auditworkbench.app/Contents/MacOS//../../../plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.1.200.v20150204-1316/eclipse_1607.so
-startup /Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/Auditworkbench.app/Contents/MacOS//../../../plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar
--launcher.appendVmargs
-application com.hp.fortify.birt.report.generator.console.Application
-data /Users/ginger.mcmurray/.fortify/BIRT16.10/workspace
-configuration /Users/ginger.mcmurray/.fortify/BIRT16.10/configuration442
-template OWASP Top 10
-source results.fpr
-format PDF
-showSuppressed
--Version OWASP Top 10 2013
--UseFortifyPriorityOrder
-debug
-output MyOWASP_Top10_Report.pdf
-consoleLog
-vm /Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/jre/lib/jli/libjli.dylib
-vmargs
-Xms40m
-Xmx1088m
-XX:MaxPermSize=320m
-XX:-UseCompressedOops
-Xdock:icon=../Resources/Awb.icns
-XstartOnFirstThread
-Dorg.eclipse.swt.internal.carbon.smallFonts
-Dcom.fortify.InstallRoot=../../../../../../..
-Djava.awt.headless=true
-Dcom.fortify.InstallRoot=/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/bin/..
-Xmx1000M
-XX:MaxPermSize=256m
-Djava.class.path=/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/Auditworkbench.app/Contents/MacOS//../../../plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar
Configuration location:
file:/Users/ginger.mcmurray/.fortify/BIRT16.10/configuration442/
Configuration file:
file:/Users/ginger.mcmurray/.fortify/BIRT16.10/configuration442/config.ini loaded
Install location:
file:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/
Configuration file:
file:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/configuration/config.ini loaded
Loading timestamp file from:
file:/Users/ginger.mcmurray/.fortify/BIRT16.10/configuration442/ .baseConfigIniTimestamp
No timestamp file found
Timestamps found:
config.ini in the base: 1458848541000
remembered -1
Shared configuration location:
file:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/configuration/
Framework located:
file:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/plugins/org.eclipse.osgi_3.10.2.v20150203-1939.jar
Loading extension: reference:file:org.eclipse.osgi.compatibility.state_1.0.1.v20140709-1414.jar
eclipse.properties not found
Framework classpath:
file:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/plugins/org.eclipse.osgi_3.10.2.v20150203-1939.jar
file:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/plugins/
file:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/plugins/org.eclipse.osgi.compatibility.state_1.0.1.v20140709-1414.jar
Debug options:
file:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_16.10/Core/private-bin/awb/eclipse/Auditworkbench.app/Contents/MacOS/.options not found
Time to load bundles: 5
Starting application: 864
FPR source file not found or not readable.

There seems to be a bug in the BIRTReportGenerator in versions 16.10/16.20 when on the MacOS having to do with relative paths.
This has been fixed in 17.10 (current version as of Oct 2017).
I do not know of a workaround, you can try contacting Fortify Technical Support (fortifytechsupport#hpe.com) and see if they have a workaround.

Cross compiling "Hello World" on Mac for Android

I'm trying to build a standard "Hello, World!" command-line executable for Android. The executable is to be run via adb shell.
0. The Go (Golang) Source
package main
import (
"fmt"
)
func main() {
fmt.Println("Hello, world!")
}
1A. The Build Command
$ CGO_ENABLED=0 GOOS=android GOARCH=arm GOARM=7 go build .
1B. The Output (Line Breaks Rearranged to Prevent Scrollbars)
# github.com/asukakenji/cross
warning: unable to find runtime/cgo.a
/usr/local/go/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: warning: ignoring file
/var/folders/dd/6k6vkzbd6d5803xj9zkjdhmh0000gn/T/go-link-150305609/go.o,
file was built for unsupported file format
( 0x7F 0x45 0x4C 0x46 0x01 0x01 0x01 0x00
0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 )
which is not the architecture being linked (x86_64):
/var/folders/dd/6k6vkzbd6d5803xj9zkjdhmh0000gn/T/go-link-150305609/go.o
Undefined symbols for architecture x86_64:
"_main", referenced from:
implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
1C. The Build Command, Again
The following command gives the same result:
$ env CGO_ENABLED=0 GOOS=android GOARCH=arm GOARM=7 go build .
2. The Build Command (Verbose)
I've tried using "-v" as mentioned like this:
$ CGO_ENABLED=0 GOOS=android GOARCH=arm GOARM=7 go build \
-x -ldflags "-extldflags -v" .
It gives me more than 100 lines of messages, so I don't post it here unless it's necessary. The go build command seems to try compiling the source with the clang bundled with Xcode.
3A. The Build Command (Successful, but...)
Given the hint that the wrong compiler is found, I tried to set $CC like this:
$ CGO_ENABLED=0 GOOS=android GOARCH=arm GOARM=7 \
CC=/path/to/arm-linux-androideabi/bin/clang go build .
arm-linux-androideabi is the output from make_standalone_toolchain.py (or make-standalone-toolchain.sh).
3B. The Output
The executable (named cross) is successfully built, with the following messages:
# github.com/asukakenji/cross
warning: unable to find runtime/cgo.a
I tried adb push it and run it with adb shell on Android, it worked fine.
My Questions
Why does it need a C compiler? Doesn't Go cross-compile out-of-the-box?
When building for Linux (instead of Android), the compilation works fine:
$ CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build .
Why?
The go build command keeps looking for runtime/cgo.a, even when I didn't use CGO in the source code, and even when I set CGO_ENABLED=0. How can I get rid of the warning? How is it harmful not having one?

if you run that code you find the android as an official target platform
listed as GOOS/GOARCH
$go tool dist list

Android isn't official target platform for cross-compilation. If all you need are command-line executables then you can set GOOS=linux because android is a linux under the hood, else take a look at https://github.com/golang/go/wiki/Mobile

The cgo requirement might be because go requires libc for DNS lookups on Android: https://github.com/golang/go/issues/8877

You need to use Android NDK to compile for android, you can download it from the link, or from Android Studio:
Then you can find the compilerlink in the route as below:
Last login: Fri Sep 4 09:25:16 on console
The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
Hasans-Air:~ hajsf$ pwd
/Users/hajsf
Hasans-Air:~ hajsf$ cd Library
Hasans-Air:Library hajsf$ cd android
Hasans-Air:android hajsf$ cd sdk
Hasans-Air:sdk hajsf$ cd ndk
Hasans-Air:ndk hajsf$ ls
21.3.6528147
Hasans-Air:ndk hajsf$ cd 21.3.6528147
Hasans-Air:21.3.6528147 hajsf$ cd toolchains
Hasans-Air:toolchains hajsf$ cd llvm
Hasans-Air:llvm hajsf$ cd prebuilt
Hasans-Air:prebuilt hajsf$ ls
darwin-x86_64
Hasans-Air:prebuilt hajsf$ cd darwin-x86_64
Hasans-Air:darwin-x86_64 hajsf$ cd bin
Hasans-Air:bin hajsf$ pwd
/Users/hajsf/Library/android/sdk/ndk/21.3.6528147/toolchains/llvm/prebuilt/darwin-x86_64/bin
Then you can use the one reflectiong:
Archeticture you want to build your android app for, like aarch64
Android API you want to compile for, like Android 30
You can see full list of availbe options there, and you can seect the one you want like aarch64-linux-android30-clang
If you are at Windows 10 you'll find it at:
"C:\Users\${user}\AppData\Local\Android\Sdk\ndk\${NKD_version}\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android30-clang"
There are 4 avaiailble linkers which are:
//CC_FOR_TARGET=/Users/hajsf/Library/android/sdk/ndk/21.3.6528147/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android30-clang
//CC_FOR_TARGET=/Users/hajsf/Library/android/sdk/ndk/21.3.6528147/toolchains/llvm/prebuilt/darwin-x86_64/bin/i686-linux-android30-clang
//CC_FOR_TARGET=/Users/hajsf/Library/android/sdk/ndk/21.3.6528147/toolchains/llvm/prebuilt/darwin-x86_64/bin/x86_64-linux-android30-clang
//CC_FOR_TARGET=/Users/hajsf/Library/android/sdk/ndk/21.3.6528147/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi30-clang
After that only you can do cross copiling, as:
$ CGO_ENABLED=1
$ GOOS=android
$ GOARCH=arm64
$ CC_FOR_TARGET=/Users/hajsf/Library/android/sdk/ndk/21.3.6528147/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android30-clang
$ go build -buildmode=c-shared -o lib-aarch64-android30.so lib.go
And a simple cgo file lib.go could be:
package main
import "C"
import "fmt"
//export HelloWorld
func HelloWorld() {
fmt.Printf("hello world from GO\n")
}
//export GetKey
func GetKey() *C.char {
theKey := "123-456-789"
return C.CString(theKey)
}
func main() {}
As shown copilation completed successfuly, and both lib-aarch64-android30.so and ib-aarch64-android30.h had been generated without any error.
Quick note, not to go far from the scope of the question, if you return a string, then the return value from this function must be explicitly freed in the C code if if you call it from C code, but as you call it from garbage collector environment, Java/Kotlin you do not want to worry about it.
If freeing the allocated buffer isn't convenient, its common to fill a buffer provided by the caller:
func GetKey(buff *C.char, n int) int
If you can allocate the memory but don't want to handle C strings, you can insert the buffer into a pointer and return the size.
func GetKey(buff **C.char) int

Option -no-engine causes problems with OpenSSL build for Android

I have to build the OpenSSL 1.0.1j libraries for Android, following the instructions at http://wiki.openssl.org/index.php/Android, on a Debian 7 system.
My configuration options are
./Configure dist -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine
The build fails due to the error
make[2]: *** No rule to make target `../../include/openssl/engine.h', needed by `rsa_lib.o'.
(Remark: Using linux-generic64 instead of dist made no difference)
Providing the option -no-rsa leads to complaints from dsa_lib.o. It
also does not make sense to disable RSA and DSA, does it?
I read the NEWS file, http://wiki.openssl.org/ and questions here on SO,
but could not find a solution.
Any suggestions?
Besides that: What is the actual meaning of -no-engine? According to my understanding,
ENGINE is the interface to the crypto algorithms of openssl. Why should it be possible to disable it at all?

Option -no-engine causes problems with OpenSSL build for Android ...
./Configure dist -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine
You can safely omit the no-engine option. The option was used to reduce the size of the binary.
What is the actual meaning of -no-engine? According to my understanding, ENGINE is...
That's a good point, and I can't answer it. But I can say I've used the procedures on the wiki page for a few years, and I know OpenSSL still works (compiles/links/runs) when the no-engine option is used.
Maybe something has changed for 1.0.1j. I did not upgrade (meaning I did not build 1.0.1j for Android and iOS) because I'm not interested in that Downgrade SCSV to accommodate the browsers and their broken-shit, insecure practices of retrying with SSLv3.
Using linux-generic64 instead of dist made no difference...
The cross-compile script (setenv-android.sh) sets the paths to the Android NDK tools AND it sets a few key environmental variables. Of them, CROSS_COMPILE are ANDROID_DEV are critical. From the tail of setenv-android.sh:
# For the Android toolchain
# https://android.googlesource.com/platform/ndk/+/ics-mr0/docs/STANDALONE-TOOLCHAIN.html
export ANDROID_SYSROOT="$ANDROID_NDK_ROOT/platforms/$_ANDROID_API/$_ANDROID_ARCH"
export SYSROOT="$ANDROID_SYSROOT"
export NDK_SYSROOT="$ANDROID_SYSROOT"
export ANDROID_NDK_SYSROOT="$ANDROID_SYSROOT"
export ANDROID_API="$_ANDROID_API"
# CROSS_COMPILE and ANDROID_DEV are DFW (Don't Fiddle With). Its used by OpenSSL build system.
# export CROSS_COMPILE="arm-linux-androideabi-"
export ANDROID_DEV="$ANDROID_NDK_ROOT/platforms/$_ANDROID_API/$_ANDROID_ARCH/usr"
export HOSTCC=gcc
Configuration for Android is picked up through SYSTEM and ARCH. Once Android kicks in, CROSS_COMPILE and ANDROID_DEV are utilized.
Because of the environmental variables, all you need to do is configure no-ssl2 no-ssl3 ....

A symbolic link to engine.h is not created when building OpenSSL with no-engine. I just added
(cd include/openssl ; ln -s ../../crypto/engine/engine.h .)
to my build process.

Autoconf triplet for Android x86_64?

I wanted to examine the issue from this question: Android NDK for x86_64 has no reference for bcopy and index. In the question, the poster asked about Android x86_64 and bcopy in relation to the Lame sound library.
I set up my cross-compile environment, set the paths for tools and sysroot, and exported the usual suspects like CC, CXX CFLAGS, CXXFLAGS, etc. For example:
$ echo $CC
x86_64-linux-android-gcc
$ which x86_64-linux-android-gcc
/opt/android-ndk-r10d/toolchains/x86_64-4.9/prebuilt/darwin-x86_64/bin/x86_64-linux-android-gcc
And:
$ export CFLAGS="--sysroot=$ANDROID_SYSROOT"
$ echo $CFLAGS
--sysroot=/opt/android-ndk-r10d/platforms/android-21/arch-x86_64
The above is usually enough to get things started. Everything is set correctly, and I can use the script to build other libraries, like Crypto++ and OpenSSL (with some minor adjustments).
However, I have not been able to configure for Android x86_64 (see below). I can't seem to find the right triplet.
What triplet does Android x86_64 use with Autoconf?
Under Autoconf, AC's "build" is what most people consider "host", and AC's "host" is what most people consider "target". So the options are actually correct when cross compiling.
$ ./configure --build=`./config.guess` --host=x86_64-linux-androideabi
checking build system type... i686-apple-darwin12.5.0
checking host system type... Invalid configuration
`x86_64-linux-androideabi': system `androideabi' not recognized
configure: error: /bin/sh ./config.sub x86_64-linux-androideabi failed
$ ./configure --build=`./config.guess` --host=x86_64-linux-android
checking build system type... i686-apple-darwin12.5.0
checking host system type... Invalid configuration
`x86_64-linux-android': system `android' not recognized
configure: error: /bin/sh ./config.sub x86_64-linux-android failed
$ ./configure --build=`./config.guess`
checking build system type... i686-apple-darwin12.5.0
checking host system type... i686-apple-darwin12.5.0
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... x86_64-linux-android-gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... configure: error: in
`/Users/android/lame-3.99.5':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

I fetched autoconf-2.69.tar.xz from the GNU FTP site. I found the latest config.guess and config.sub in the build-aux directory and used it for the LAME MP3 encoder project.
It appears the triplet is x86_64-linux-android.
The project has a broken link though. The invocation of the compiler driver through libtool that creates the shared object lacks CFLAGS, so --sysroot is not passed to the linker. That causes a bunch of link errors due to missing system libraries, like crtbegin_so.o, crtend_so.o, -lc and -ldl.
The fix for that is to manually run the the failed command from the libmp3lame directory. Prepend the command with $CC $CFLAGS <rest of project's command>.

Building OpenSSL for Android (ARMv7) on Win32

How can I build OpenSSL for Android ARM v7 (using Android NDK) on Win32?

Until the OpenSSL's wiki and setenv-android.sh are updated accordingly, I'll publish the recipe here. The required fixes to the process are:
Update setenv-android.sh to support Windows.
Update PATH to use Android NDK's (mingw) GNU make (rather than Cygwin's).
Invoke make with a Windows-style path to Cygwin's perl.
This recipe will be a strange hybrid of Cygwin and mingw (since Android NDK gcc toolchains for win32 are mingw). I'm assuming a Windows x86_64 build of the Android NDK unpacked into c:\android-ndk-r9d, and that you wish to use a gcc 4.8 toolchain.
Install Android NDK (duh!).
Install Cygwin -- make sure to include perl
Start Cygwin shell as an administrator to make sure native symlinks will work.
Within the console, run the following script to set the variables:
export \
CYGWIN=winsymlinks:native \
ANDROID_API=android-14 \
ANDROID_DEV=c:/android-ndk-r9d/platforms/android-14/arch-arm/usr \
PATH=/cygdrive/c/android-ndk-r9d/prebuilt/windows-x86_64/bin:/cygdrive/c/android-ndk-r9d/toolchains/arm-linux-androideabi-4.8/prebuilt/windows-x86_64/bin:$PATH \
MACHINE=armv7 \
SYSTEM=android \
ARCH=arm \
CROSS_COMPILE=arm-linux-androideabi-
Now, unpack openssl:
tar xzfv openssl-1.0.1i.tar.gz (or whatever your tarball is)
cd openssl-1.0.1i (or whatever your version is)
Make sure you have actual native Win32 (!) symlinks in include/openssl:
cmd /c "dir include\openssl"
You should see something like:
13-Aug-14 05:59 PM <SYMLINK> aes.h [..\..\crypto\aes\aes.h]
13-Aug-14 05:59 PM <SYMLINK> asn1.h [..\..\crypto\asn1\asn1.h]
(etc.)
Now it's time to configure:
./config shared -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine --openssldir=/foo/bar
Ignore the failure to build (due to failure to find perl). We'll rectify this right away. Do this:
make PERL=$(cygpath -w $(which perl))
Now wait for a few minutes until it builds, and presto, you have your libcrypto.so etc.

Just a couple of comments on my experience with this:
Executing this statement:
PERL=$(cygpath -w $(which perl))
in the cygwin shell allows the shell to interpret the backslashes as escape characters and the build process chokes. To solve this I did the following:
$ echo $(cygpath -w $(which perl))
which produced the windows formatted path to the perl executable:
C:\cygwin64\bin\perl
Then I added this line to the export shown above:
PERL=c:/cygwin64/bin/perl \
There are other ways of doing this, but it worked and headed off the problem with the ./config statement documented above (not finding perl).
Second issue was the -no- statements. After running the configure, the script reports that you'll have to run make depend. I wanted to exclude MD5 (i.e. -no-md5) and when I did the make depend, it errored out with a report that MD5 was disabled. Uhhh, yes, that was kind of the idea, but I just won't use MD5 hashes. I did use the -no-ssl2 and didn't get any complaints after the make depend.
Third issue and this is a mystery. The build broke on compiling crypto because it could not find a symbol that is supposed to be defined in /crypto/objects/obj_xref.h. When I looked at the file, it was empty. Something in the perl script I suppose, but no time to debug right now, since I'm at proof of concept phase. I placed a copy from a patch that I picked up at https://github.com/devpack/openssl-android
After that, my build ran to completion. I've done no testing with this and it is not a trustworthy solution, but it did compile and produce the static libraries that I need for proof of concept for my client.
Just as an update, my shared library built with these libraries loaded fine on my target.