I have found a bug in the Android NDK compiler (r16b). Code snippet is shown below:
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
int frameWidth = 208;
int main(int argc, char* argv[])
{
int32_t count = (argc > 1) ? atoi(argv[1]) : -1;
int width = frameWidth;
int rows = count / (width * sizeof(int16_t));
if (rows >= 3) {
printf("rows=%u\n", rows);
} else {
printf("count=%d rows=%d\n", count, rows);
}
}
The problem is the gcc/llvm compiler is treating the divide as an unsigned divide. You can see this by compiling the code to a cmdline executable and running with no arguments or a negative number and/or generating the assembly code and looking at the arm source:
#DEBUG_VALUE: main:count <- %R4
.loc 1 11 22 is_stmt 0 # ./main.c:11:22
mov r0, r4
.Ltmp8:
bl __aeabi_uidiv
mov r2, r0
.Ltmp9:
#DEBUG_VALUE: main:rows <- %R2
.loc 1 12 9 is_stmt 1 # ./main.c:12:9
cmp r2, #3
blt .LBB0_5
Where do I report this bug so it can be fixed? I can imagine there are other places in our code where a signed divide is expected.
You can report issues in the official NDK GitHub repository:
https://github.com/android-ndk/ndk/issues
Related
There are a couple of questions already on SO assessing the possibility of executing self generated code. The generic answer is that it's possible, it's being done in JIT compilers and with dynamic library loading.
The application is in optimisation of sparse convolutions where the sparse structure should be embedded in the code, since skipping multiplication by zeros conditionally is slower than multiplying by zeros. Neither it's feasible in terms of performance to encode the structure in data:
do { // actually: don't do this
auto offset = *encoded_offset++;
auto coeff = *encoded_coefficient++;
accum += data[offset] * coeff;
} while (end_of_data); // e.g. offset < 0, or offset == previous offset
Vs.
...
ldr q5, [x1], 16 ; load data
fmla v0.4s, v5.4s, v8.s[0]
fmla v1.4s, v5.4s, v8.s[1]
fmla v2.4s, v5.4s, v8.s[2]
// fmla v3.4s, v5.4s, v8.s[3] ; omitted, when |weight| < threshold
fmla v4.4s, v5.4s, v8.s[3] ; we reuse the v8.s[3] with a weight that matters
ldr q5, [x1], 16
// fmla v0.4s, v5.4s, v9.s[0] ; omitted, when |weight| < threshold
// fmla v1.4s, v5.4s, v9.s[0] ; omitted
fmla v2.4s, v5.4s, v9.s[0]
fmla v3.4s, v5.4s, v9.s[1]
fmla v4.4s, v5.4s, v9.s[2]
; + several kilobytes of instructions
It's possible to generate the code by templates, but it would be far more efficient (and marginally more secure reverse engineering in mind) if the code was generated on the fly.
Now the question is, how to exactly call the generated function in some of the most important mobile ecosystems: Ios, android and possibly webassembly (using off course WASM (SIMD) bytecode).
What exact system calls need to be issued to be able to execute the block of memory filled with the code -- legally?
The only problem you will stumble upon is that "normal" memory is usually protected against execution. You just need to set on that memory the execute permission.
After that you can set any sort of instructions you want. See for example the top fizzbuzz bandwidth winner, who uses a simple JIT machine.
https://codegolf.stackexchange.com/a/236630/108147
But the basic code is this:
#include <string>
#include <iostream>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <string.h>
uint8_t code[] = { 0x89, 0xf8, // mov eax, edi
0x0f, 0xaf, 0xc6, // imul eax, esi
0xc3 }; // ret
int main() {
using multifn = int( int a, int b);
size_t pagesize = getpagesize();
size_t codesize = ((sizeof(code)-1)/pagesize+1)*pagesize;
uint64_t codeaddr = uint64_t(code);
codeaddr -= codeaddr % pagesize;
mprotect((void*)codeaddr, codesize, PROT_READ | PROT_WRITE |PROT_EXEC);
multifn* m = reinterpret_cast<multifn*>(code);
std::cout << "Result:" << m(2,3) << std::endl;
return 0;
}
I just ran it in my Linux box as a regular user and the result is:
Program stdout
Result:6
I assume that as Android is basically Linux, the result will be the same. As per IOS I have to speculate here.
Also Godbolt: https://godbolt.org/z/hsMKWsjM9
If you are really picky about not messing up with regular memory, you can create a separate page for your generated code
#include <string>
#include <iostream>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <string.h>
void* createExecPage( size_t size ) {
size_t pagesize = getpagesize();
size = ((size-1)/pagesize+1)*pagesize;
std::cout << "Pagesize:" << pagesize << " size:" << size << std::endl;
void* temp = nullptr;
int res = posix_memalign(&temp, pagesize, size );
if ( res!=0 ) {
perror("posix_memalign");
}
res = mprotect(temp, size, PROT_READ | PROT_WRITE |PROT_EXEC);
if ( res!=0 ) {
perror( "mprotect" );
}
return temp;
}
uint8_t code[] = { 0x89, 0xf8, // mov eax, edi
0x0f, 0xaf, 0xc6, // imul eax, esi
0xc3 }; // ret
int main()
{
using multifn = int( int a, int b);
void* xpage = createExecPage(sizeof(code));
memcpy( xpage, code, sizeof(code));
multifn* m = reinterpret_cast<multifn*>(xpage);
std::cout << "Result:" << m(2,3) << std::endl;
return 0;
}
https://godbolt.org/z/ebW76o1KP
I'm actually developing a C app on Android Studio using the SDL 2, a C graphic library. But the problem doesn't come from the library itself but from the #include of my own files.
I created source files in which I put my functions, I associated to them header files with the declarations of the functions. But when I call the functions in my main source file, I get the error:
error: undefined reference to 'draw_render_rectangle' for example.
It doesn't work for any of my files or functions. But it works for the SDL.
So I try to #include.
Main source file "main_graphics.c":
#include "main_graphics.h"
void disp_main_menu(int selected) {
POINT A, B;
A.x = 0;
A.y = 0;
B.x = WIDTH;
B.y = HEIGHT;
draw_render_rectangle(A, B, 70, 70, 70);
}
Main header file "main_graphics.h":
#ifndef ANDROID_GAME_MAIN_GRAPHICS_H
#define ANDROID_GAME_MAIN_GRAPHICS_H
#include "../SDL2/include/SDL.h"
#include "../SDL2/include/SDL.h"
#include "../SDL2_ttf/SDL_ttf.h"
#include "../SDL2_image/SDL_image.h"
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include "graphics_mobile.h"
int WIDTH = 1080;
int HEIGHT = 1920;
void disp_main_menu(int selected);
#endif //ANDROID_GAME_MAIN_GRAPHICS_H
"graphics_mobile.c" :
#include "graphics_mobile.h"
typedef struct point {int x,y;} POINT;
void draw_render_rectangle(POINT p1, POINT p2, Uint8 r, Uint8 g, Uint8 b)
{
SDL_Rect rectangle;
int xmin, xmax;
int ymin, ymax;
int i,j;
if (p1.x < p2.x) {xmin=p1.x; xmax=p2.x;} else{xmin=p2.x; xmax=p1.x;}
if (p1.y < p2.y) {ymin=p1.y; ymax=p2.y;} else{ymin=p2.y; ymax=p1.y;}
rectangle.x = xmin;
rectangle.y = HEIGHT - (ymax-ymin) - ymin;
rectangle.w = xmax-xmin;
rectangle.h = ymax-ymin;
SDL_SetRenderTarget(renderer, texture);
SDL_RenderDrawRect(renderer,&rectangle);
SDL_SetRenderDrawColor(renderer, r, g, b, 0x00);
SDL_RenderFillRect(renderer, &rectangle);
SDL_SetRenderTarget(renderer, NULL);
SDL_RenderCopy(renderer, texture, NULL, NULL);
}
"graphics_mobile.h"
#ifndef ANDROID_GAME_GRAPHICS_MOBILE_H
#define ANDROID_GAME_GRAPHICS_MOBILE_H
#include "main_graphics.h"
typedef struct point {int x,y;} POINT;
void draw_render_rectangle(POINT p1, POINT p2, Uint8 r, Uint8 g, Uint8 b);
#endif //ANDROID_GAME_GRAPHICS_MOBILE_H
One thing that is weird is that the autocompletion is working, when I type the function, it actually recognize it and tells me in the little window of the autocompletion that it is defined in "graphics_mobile.h".
Can the problem come from the compilator?
I tried ndk-bundle_10e and ndk-bundle_16b and I get the same error.
So I did some tests and I got a working code :
you have to put in "graphics_mobile.h":
#ifndef ANDROID_GAME_GRAPHICS_MOBILE_H
#define ANDROID_GAME_GRAPHICS_MOBILE_H
#include "main_graphics.h"
#include "graphics_mobile.c"
#endif //ANDROID_GAME_GRAPHICS_MOBILE_H
and remove the declarations of the functions
I get error like this when I do ndk-build:
/Users/../handle.c:80: error: 'struct _AR2Tracking2DParamT' has no member named 'template'
/Users/../handle.c:100: error: 'struct _AR2Tracking2DParamT' has no member named 'template'
I didn't know about "template", so searched about it on google. Finally, I know it's the qualifier of C++.
But handle.c is a C file. Why does ndk-build incorrectly guess that its a C++ file?
Is there any way to specify it is a C file in Android.mk?
thanks.
2013/11/22 edit:
Sorry I put another error code. Relevant error code is the following:
-- template.h --
int ar2FreeTemplate( AR2TemplateT *template );
-- ndk-build says --
/Users/.../template.h:63: error: expected ',' or '...' before 'template'
so ndk-build incorrectly guesses that template.h is C++ header file.
If it understands that it is a C header file, it should analyze template as a variable, not a keyword.
2013/11/25 edit:
here is template.h.
I replaced "template" to "temp". So i fixed this issue for the moment.
#ifndef AR2_TEMPLATE_H
#define AR2_TEMPLATE_H
#include <AR/ar.h>
#include <AR2/config.h>
#include <AR2/imageSet.h>
#include <AR2/featureSet.h>
#ifdef __cplusplus
extern "C" {
#endif
(An omission)
AR2TemplateT *ar2GenTemplate ( int ts1, int ts2 );
int ar2FreeTemplate( AR2TemplateT *temp );
int ar2SetTemplateSub ( ARParamLT *cparamLT, float trans[3][4], AR2ImageSetT *imageSet,
AR2FeaturePointsT *featurePoints, int num,
AR2TemplateT *temp );
int ar2GetBestMatching ( ARUint8 *img, ARUint8 *mfImage, int xsize, int ysize, int pixFormat,
AR2TemplateT *mtemp, int rx, int ry,
int search[3][2], int *bx, int *by, float *val);
int ar2GetBestMatching2(void);
int ar2GetResolution( ARParamLT *cparamLT, float trans[3][4], float pos[2], float dpi[2] );
int ar2GetResolution2( ARParam *cparam, float trans[3][4], float pos[2], float dpi[2] );
int ar2SelectTemplate( AR2TemplateCandidateT *candidate, AR2TemplateCandidateT *prevFeature, int num,
float pos[4][2], int xsize, int ysize );
#ifdef __cplusplus
}
#endif
#endif
Unfortunately, template is a reserved word in C++. Therefore, you cannot use this as identifier in C++ context, even in an h file included from C++, even if you wrap it with extren "C", as
extern "C" {
#include "template.h"
}
The best solution would be to change the name of parameter in template.h. If the file is carved in stone, here is a workaround:
extern "C" {
#define template notemplate
#include "template.h"
#undef template
}
I'm very newbie in Android world and I have to write an streaming video quality checker application on Android. I have to use the native StageFright media framework to play videos. As far as I understand there is an native API of render statistics, but I need advice how I can get it. Thank you.
There is an ADB command to print playback framerate.
Procedure
Open console on windows (or linux) on the host. Make sure that required drivers have been installed for USB connectivity with the device (android phone or board)
Run the following commands
$> adb kill-server
$> adb shell setprop debug.video.showfps 1
Run the video playback. If the video is being run using Android Media player stack, then you will see prints reporting frame rate achieved.
You're welcome to use this as well, call it at the beginning and end of each frame rendered. It's a slightly altered version of some sample code from the NDK:
stats.c:
#include <sys/time.h>
#include <string.h>
#include <android/log.h>
#include <stdio.h>
#include "stats.h"
#define LOG_TAG "[STATS]"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO,LOG_TAG,__VA_ARGS__)
#define LOGW(...) __android_log_print(ANDROID_LOG_WARN,LOG_TAG,__VA_ARGS__)
#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR,LOG_TAG,__VA_ARGS__)
#define STATS_DUMP(...) __android_
double now_ms()
{
struct timeval tv;
gettimeofday(&tv, NULL);
return tv.tv_sec*1000. + tv.tv_usec/1000.;
}
void stats_init(Stats* s)
{
s->lastTime = now_ms();
s->firstTime = 0.;
s->firstFrame = 0;
s->numFrames = 0;
s->dump = malloc(128);
memset(s->dump,0,128);
}
void stats_startFrame(Stats* s)
{
s->frameTime = now_ms();
}
void stats_endFrame(Stats* s)
{
double now = now_ms();
double renderTime = now - s->frameTime;
double frameTime = now - s->lastTime;
int nn;
if (now - s->firstTime >= MAX_PERIOD_MS) {
if (s->numFrames > 0) {
double renderTime = now - s->frameTime;
double frameTime = now - s->lastTime;
int nn;
double minRender, maxRender, avgRender;
double minFrame, maxFrame, avgFrame;
int count;
nn = s->firstFrame;
minRender = maxRender = avgRender = s->frames[nn].renderTime;
minFrame = maxFrame = avgFrame = s->frames[nn].frameTime;
for (count = s->numFrames; count > 0; count-- ) {
nn += 1;
if (nn >= MAX_FRAME_STATS)
nn -= MAX_FRAME_STATS;
double render = s->frames[nn].renderTime;
if (render < minRender) minRender = render;
if (render > maxRender) maxRender = render;
double frame = s->frames[nn].frameTime;
if (frame < minFrame) minFrame = frame;
if (frame > maxFrame) maxFrame = frame;
avgRender += render;
avgFrame += frame;
}
avgRender /= s->numFrames;
avgFrame /= s->numFrames;
sprintf(s->dump,"Frames per second - [AVG:%.1f] [MIN:%.1f] [MAX:%.1f]Rendering time ms - [AVG:%.1f] [MIN:%.1f] [MAX:%.1f]", 1000./avgFrame, 1000./maxFrame, 1000./minFrame, avgRender, minRender, maxRender);
//LOGI("Frames per second - [AVG:%.1f] [MIN:%.1f] [MAX:%.1f]Rendering time ms - [AVG:%.1f] [MIN:%.1f] [MAX:%.1f]", 1000./avgFrame, 1000./maxFrame, 1000./minFrame, avgRender, minRender, maxRender);
}
s->numFrames = 0;
s->firstFrame = 0;
s->firstTime = now;
}
nn = s->firstFrame + s->numFrames;
if (nn >= MAX_FRAME_STATS)
nn -= MAX_FRAME_STATS;
s->frames[nn].renderTime = renderTime;
s->frames[nn].frameTime = frameTime;
if (s->numFrames < MAX_FRAME_STATS) {
s->numFrames += 1;
} else {
s->firstFrame += 1;
if (s->firstFrame >= MAX_FRAME_STATS)
s->firstFrame -= MAX_FRAME_STATS;
}
s->lastTime = now;
}
stats.h:
#include <jni.h>
#define MAX_FRAME_STATS 120
#define MAX_PERIOD_MS 5000
typedef struct{
double renderTime;
double frameTime;
} FrameStats;
typedef struct{
double firstTime;
double lastTime;
double frameTime;
int firstFrame;
int numFrames;
FrameStats frames[ MAX_FRAME_STATS ];
char* dump;
} Stats;
extern double now_ms();
extern void stats_init(Stats *);
extern int stats_dump(Stats *);
extern void stats_startFrame(Stats *);
extern void stats_endFrame(Stats *);
I wrote a simple loop to aid in billboarding that will check if a pixel is white. if so, it will set it to 100% transparency. i wrote it in native code because the java equivalent of this loop took 19 seconds to run for a 256x256 bitmap, too slow.
when compiling:
#include "org_me_renderscene_Billboard.h"
#include <stdio.h>
#include <stdlib.h>
JNIEXPORT jintArray JNICALL Java_org_me_renderscene_Billboard_NativeSetAlphaWhereWhite
(JNIEnv *envptr, jclass jClass, jintArray pixels, jint length)
{
int *mPixels = (*int)malloc(length * 4);
static int currentcolor;
static int writecolor;
static int red, green, blue;
for(int x = 0; x < length; x++)
{
currentcolor = pixels[x];
red = currentcolor << 16;
green = currentcolor << 8;
blue = currentcolor;
if((red == 0) && (green == 0) && (blue == 0))
{
mPixels[x] = 0x00000000;
}
else
{
mPixels[x] = currentcolor;
}
}
return mPixels;
}
the auto-generated stub for which is:
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class org_me_renderscene_Billboard */
#ifndef _Included_org_me_renderscene_Billboard
#define _Included_org_me_renderscene_Billboard
#ifdef __cplusplus
extern "C" {
#endif
/*
* Class: org_me_renderscene_Billboard
* Method: NativeSetAlphaWhereWhite
* Signature: ([II)[I
*/
JNIEXPORT jintArray JNICALL Java_org_me_renderscene_Billboard_NativeSetAlphaWhereWhite
(JNIEnv *, jclass, jintArray, jint);
#ifdef __cplusplus
}
#endif
#endif
i get these errors:
thomas#THOMASDESKLINUX:~/Documents/LinuxProgramming/EclipseWorkspace/RenderScene$ /home/thomas/Documents/LinuxProgramming/AndroidSDKs/android-ndk-r4b/ndk-build
Compile thumb : Billboardlib <= /home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/jni/org_me_renderscene_Billboard.c
/home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/jni/org_me_renderscene_Billboard.c: In function 'Java_org_me_renderscene_Billboard_NativeSetAlphaWhereWhite':
/home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/jni/org_me_renderscene_Billboard.c:9: error: expected expression before 'int'
/home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/jni/org_me_renderscene_Billboard.c:9: error: expected ',' or ';' before 'malloc'
/home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/jni/org_me_renderscene_Billboard.c:15: error: 'for' loop initial declarations are only allowed in C99 mode
/home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/jni/org_me_renderscene_Billboard.c:15: note: use option -std=c99 or -std=gnu99 to compile your code
/home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/jni/org_me_renderscene_Billboard.c:17: warning: dereferencing 'void *' pointer
/home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/jni/org_me_renderscene_Billboard.c:17: error: void value not ignored as it ought to be
make: *** [/home/thomas/Documents/LinuxProgramming/EclipseWorkspace/RenderScene/obj/local/armeabi/objs/Billboardlib/org_me_renderscene_Billboard.o] Error 1
why is this happening? my C code should be fine, and these errors dont make much sense.
Try (int*) instead of (*int)
what flags are you using in you Android.mk ?
did you set LOCAL_CFLAGS := -std=c99
also
you need to change to this
int *mPixels = (int*)malloc(length * 4);
int *mPixels = (*int)malloc(length * 4);
should be
int *mPixels = (int*)malloc(length * 4);
or even better
int *mPixels = (int*)malloc(length * sizeof(int));
and also note that this will not properly separate red, green, and blue:
red = currentcolor << 16;
green = currentcolor << 8;
blue = currentcolor;
Given that you're just checking for zero, and you don't really care about the individual RGB values, you can probably just get away with:
if ( (currentcolor & 0x00FFFFFF) == 0)
This will zero out the Alpha from the pixel, leaving behind just the RGB portion. If that whole thing is zero, each color must be zero, so there's no need to check each color individually.
Final thought:
I haven't done much with Android specifically, but isn't 0x000000 black and 0xFFFFFF white? So you're actually matching on black instead of white here.