I should rotate every yuv image buffer, which i receive from camera, on 90 degrees counterclockwise. I found this post where using a java. This code works fine.
But I've tried to do a native method, because I wanted to do method which has the same logic but works faster.
JNIEXPORT jbyteArray JNICALL Java_com_ndk_example_utils_NativeUtils_rotateFrameBackward
(JNIEnv *env, jobject obj, jbyteArray arr, jint w, jint h){
jint arrSize = w*h*3/2;
jbyte *data,*yuv;
data = (*env)->GetByteArrayElements(env, arr, JNI_FALSE);
yuv = (*env)->GetByteArrayElements(env, arr, JNI_FALSE);
int x,y,i = 0;
for(x = 0; x < w; x++){
for(y = h-1;y >= 0;y--){
yuv[i] = data[y*w+x];
i++;
}
}
i = arrSize - 1;
for(x = w-1;x > 0;x=x-2)
{
for(y = 0;y < h/2;y++)
{
yuv[i] = data[(w*h)+(y*w)+x];
i--;
yuv[i] = data[(w*h)+(y*w)+(x-1)];
i--;
}
}
(*env)->ReleaseByteArrayElements(env, arr, yuv, JNI_ABORT);
yuv = 0;
data = 0;
return arr;
}
When i launched this method on my htc 816(v5.1) it works fine, but when I launched the app on Samsung S3(v4.3) and Lenovo P-70(v4.4.2), the app is crashes. And in Android monitor tab in Android Studio, i saw that
memory usage is always increasing until my app is crashes. In my htc i don't have problems with it. Any ideas?
You do a double GetByteArrayElements for arr using data and yuv, then only release yuv. You also don't check if a copy was made using the last parameter, you just give in JNI_FALSE. You shouldn't do that, you should use a boolean parameter to receive the value, not try to tell the system whether to copy.
You should therefore release both pointers in the end.
Also if this code works, it means that copies are in fact made since you are reading and writing from the same memory area and that would cause corruption of the image.
Related
In my native code I generate a vector of float and need to send this to java part by converting it to byte array (using little endian scheme). Later I resend this byte array and need to convert it back to original float vector. I could not find exact example and wrote below code, that will take 4 byte values at a time and will convert it to float and add it to final vector of float. I will not be modifying any of the data, just perform some calculations, so need it to be fast and if possible avoid memory copy wherever possible.
Currently it is giving me warning that "Using unsigned char for signed value of type jbyte". Can someone guide me how to proceed?
JNIEXPORT jfloat JNICALL Java_com_xyzxyzxcyzxczxczc(JNIEnv *env, jclass type, jlong hEngineHandle, jbyteArray feature1){
try {
PeopleCounting *obj = (PeopleCounting *) hEngineHandle;
jbyte *f1 = (jbyte *)env->GetByteArrayElements(feature1, NULL);
if(obj->faceRecognitionByteArraySize == 0){ // Setting it once for future use as it not going to change for my use case
obj->faceRecognitionByteArraySize = env->GetArrayLength(feature1);
}
union UStuff
{
float f;
unsigned char c[4];
};
UStuff f1bb;
std::vector<float> f1vec;
//Convert every 4 bytes to float using a union
for (int i = 0; i < obj->faceRecognitionByteArraySize; i+=4){
//Going backwards - due to endianness
// Warning here. // Using unsigned char for signed value of type jbyte
f1bb.c[3] = f1[i];
f1bb.c[2] = f1[i+1];
f1bb.c[1] = f1[i+2];
f1bb.c[0] = f1[i+3];
f1vec.push_back(f1bb.f);
}
// release it
env->ReleaseByteArrayElements(feature1, f1, 0 );
// Work with f1vec data
}
UPDATES:
As suggested by #Alex both consumer and producer of byte array will be the C++ then there is no need for any endianess. So the approach I am going to take is as below:
A) Java end I initialize a byte[] of needed length (4 * number of float values)
B) Pass this as jbyteArray to JNI function
Now, How to fill this byteArray from C++ end?
JNIEXPORT void JNICALL Java_com_xyz_FaceRecognizeGenerateFeatureData(JNIEnv *env, jclass type, jlong hEngineHandle, jlong addrAlignedFaceMat, jbyteArray featureData){
try {
PeopleCounting *obj = (PeopleCounting *) hEngineHandle;
Mat *pMat = (Mat *) addrAlignedFaceMat;
vector<float> vecFloatFeatureData = obj->faceRecognizeGenerateFeatureData(*pMat);
void* data = env->GetDirectBufferAddress(featureData); // How to fill the byteArray with values from vecFloatFeatureData? (If requied I can have a constant having the length of the array or number of actual float values i.e. len of array/4
C) Now, later on I need to consume this data again by passing this from Java to C++. So passing the jbyteArray to the JNI function
JNIEXPORT jfloat JNICALL Java_com_xyz_ConsumeData(JNIEnv *env, jclass type, jlong hEngineHandle, jbyteArray feature1){
try {
PeopleCounting *obj = (PeopleCounting *) hEngineHandle;
void* data = env->GetDirectBufferAddress(featureData);
float *floatBuffer = (float *) data1;
vector<float> vecFloatFeature1Data(floatBuffer, floatBuffer + obj->_faceRecognitionByteArraySize); // _faceRecognitionByteArraySize contains the byte array size i.e. 4*no. of floats
Would this work?
Unfortunately, the updated code won't work either.
But first, let's address the answer you gave to #Botje.
java end just stores the data in the database and maybe send this data further to servers
These are two signoficant restrictions. First, if your database interface takes only byte arrays as blobs, this would prevent you from using DirectByteBuffer. Second, if the array may be sent to a different machine, you must be sure that the floating point values are stored there exactly as on the machine that produced the byte array. Mostly, this won't be a problem, but you should better check before deploying your distributed solution.
Now, back to your JNI code. There is actually no need to preallocate an array on Java side. The FaceRecognizeGenerateFeatureData() method can simply return the new byte array it creates:
JNIEXPORT jbyteArray JNICALL Java_com_xyz_FaceRecognizeGenerateFeatureData(JNIEnv *env, jclass type,
jlong hEngineHandle, jlong addrAlignedFaceMat) {
PeopleCounting *obj = (PeopleCounting *) hEngineHandle;
Mat *pMat = (Mat *) addrAlignedFaceMat;
vector<float> vecFloatFeatureData = obj->faceRecognizeGenerateFeatureData(*pMat);
jbyte* dataBytes = reinterpret_cast<jbyte*>(vecFloatFeatureData.data());
size_t dataLen = vecFloatFeatureData.size()*sizeof(vecFloatFeatureData[0]);
jbyteArray featureData = env->NewByteArray(dataLen);
env->SetByteArrayRegion(featureData, 0, dataLen, dataBytes);
return featureData;
}
Deserialization can use the complementary GetByteArrayRegion() and avoid double copy:
JNIEXPORT jfloat JNICALL Java_com_xyz_ConsumeData(JNIEnv *env, jclass type,
jlong hEngineHandle, jbyteArray featureData) {
PeopleCounting *obj = (PeopleCounting *) hEngineHandle;
size_t dataLen = env->GetArrayLength(featureData);
vector<float> vecFloatFeatureDataNew(dataLen/sizeof(float));
jbyte* dataBytes = reinterpret_cast<jbyte*>(vecFloatFeatureDataNew.data());
env->GetByteArrayRegion(featureData, 0, dataLen, dataBytes);
…
Note that with this architecture, you could gain a bit from using DirectByteBuffer instead of byte array. Your PeopleCounting engine produces a vector which cannot be mapped to an external buffer; on the other side, you can wrap the buffer to fill the vecFloatFeatureDataNew vector without copy. I believe this optimization would not be significant, but lead to less more cumbersome code.
From https://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/types.html I see that jbyte is a signed 8 bits type. So it would make sense to use signed char in your union. The warnings should be gone then. After that is fixed, there is the issue of if floats are represented on the native side the same way they are on the java side.
I have two functions and I'm getting a ReferenceTable overflow.
The summary of consumed array entries is:
1 of byte[] (3 elements)
446 of byte[] (75 elements) (2 unique instances)
576 of byte[] (147 elements) (2 unique instances)
1 of int[] (25 elements)
I really checked the code to find any mistake, but didn't found it. Im releasing arrays after getting them. The only thing is that these functions are called thousands of times, can this be the cause?
Here is all my native code:
called once:
JNIEXPORT void JNICALL Java_ar_com_teasoft_Image_nativeUnlock(
JNIEnv *env, jclass clazz, jobject bitmap) {
AndroidBitmap_unlockPixels(env, bitmap);
}
called once:
JNIEXPORT jlong JNICALL Java_ar_com_teasoft_Image_nativeLock(
JNIEnv *env, jclass clazz, jobject bitmap) {
int ret;
AndroidBitmapInfo info;
if ((ret = AndroidBitmap_getInfo(env, bitmap, &info)) < 0) {
LOGE("AndroidBitmap_getInfo() failed ! error=%d", ret);
return 0;
}
if (info.format != ANDROID_BITMAP_FORMAT_RGBA_8888) {
LOGE("Bitmap format is not RGBA_8888!");
return 0;
}
void* bitmapPixels;
if ((ret = AndroidBitmap_lockPixels(env, bitmap, &bitmapPixels)) < 0) {
LOGE("AndroidBitmap_lockPixels() failed ! error=%d", ret);
return 0;
}
return (jlong) bitmapPixels;
}
called lots of times:
JNIEXPORT void JNICALL Java_ar_com_teasoft_Image_nativeCopyPixels(
JNIEnv *env, jclass clazz, jlong dataRef, jintArray sourceIndexes,
jintArray targetIndexes, jint count) {
argb* sourcePixels;
argb* targetPixels;
jint *sourceArray = env->GetIntArrayElements(sourceIndexes, NULL);
jint *targetArray = env->GetIntArrayElements(targetIndexes, NULL);
for (int i = 0; i < count; i++) {
sourcePixels = (argb*)((char*) dataRef + sourceArray[i] * 4);
targetPixels = (argb*)((char*) dataRef + targetArray[i] * 4);
(*targetPixels) = (*sourcePixels);
}
env->ReleaseIntArrayElements(sourceIndexes, sourceArray, JNI_ABORT);
env->ReleaseIntArrayElements(targetIndexes, targetArray, JNI_ABORT);
}
called lots of times:
JNIEXPORT void JNICALL Java_ar_com_teasoft_Image_nativeGetRGB(
JNIEnv *env, jclass clazz, jlong dataRef, jintArray indexes,
jbyteArray destRgb) {
jint *array = env->GetIntArrayElements(indexes, NULL);
jbyte *dstarray = env->GetByteArrayElements(destRgb, NULL);
int size = env->GetArrayLength(indexes);
char* sourcePixels;
int dstCount = 0;
for (int i = 0; i < size; i++) {
sourcePixels = (char*) dataRef + array[i] * 4;
dstarray[dstCount++] = (*(sourcePixels + 1));
dstarray[dstCount++] = (*(sourcePixels + 2));
dstarray[dstCount++] = (*(sourcePixels + 3));
}
env->ReleaseIntArrayElements(indexes, array, JNI_ABORT);
env->ReleaseByteArrayElements(destRgb, dstarray, JNI_COMMIT);
}
Based on the summary, it looks like the one that is not released, is of byte[], so it has to be the one in function nativeGetRGB. But i cannot find where the mistake is.
Please Help!
Regards,
Juan Ignacio
Java_ar_com_teasoft_Image_nativeGetRGB():
As far as I can see, you'd need to commit and free any temporary array copy by passing 0 instead of JNI_COMMIT to ReleaseByteArrayElements(). The second argument of Get*ArrayElements() is a pointer to a boolean, which will be set to true, if the returned array is a copy, instead of pinned memory.
Java_ar_com_teasoft_Image_nativeCopyPixels():
You might also want to pass 0 instead of JNI_ABORT, which discards everything, here:
env->ReleaseIntArrayElements(targetIndexes, targetArray, JNI_ABORT);
The tricky thing with arrays is, that the release mode applies to copied arrays only, since pinned memory gets modified directly. There's no way to force either array copy or pinning.
I am having a bit of trouble with the following code that appears to be causing a segmentation fault at the indicated line. I'm trying to create an array of 8 bit unsigned integers in order to instantiate an OpenCV Mat object with, however the segfault occurs partway through the loop that populates the array.
It appears to happen at a different iteration each time, leading me to suspect that something is getting deallocated by GC, but I can't determine what.
SignDetector.c
JNIEXPORT void JNICALL Java_org_xxx_detectBlobs(JNIEnv *env, jclass clazz, jintArray in)
{
jint *contents = (*env)->GetIntArrayElements(env, in, NULL);
threshold(contents, PIXEL_SAMPLE_RATE);
detectBlobs(contents);
(*env)->ReleaseIntArrayElements(env, in, contents, 0);
}
BlobDetector.cpp
void detectBlobs(jint *contents)
{
LOGD("Call to detectBlobs in BlobDetector.cpp");
uint8_t *thresholded = (uint8_t*) malloc(frame_size);
int i;
for(i = 0; i < frame_size - 1; i++)
thresholded[i] = (contents[i] == WHITE) ? 0 : 1; // Segfaults partway through this loop.
frame_size is simply the number of pixels in an image, which is also equivalent to the length of the jintArray that the image is passed to native code in.
Any suggestions?
Managed to resolve this by simply restarting my AVD on the Android emulator. Problem appears to not occur on a real device either, so I can only conclude that something exploded in the virtual device's RAM.
i am trying to do real time image processing in android using jni. I have a native method to decode image data and i call this method for every frame. After a few seconds later i get out of memory and my app terminats.
LOG OUTPUT:
12-03 20:54:19.780: E/dalvikvm-heap(8119): Out of memory on a 3686416-byte allocation.
MY NATIVE METHOD:
JNIEXPORT jintArray JNICALL Java_net_oyunyazar_arcc_data_FrameManager_processImage(JNIEnv* env, jobject javaThis, jint width, jint height, jbyteArray arr) {
jint *convertedData;
convertedData = (jint*)malloc((width*height) * sizeof(jint));
jintArray result = (*env)->NewIntArray(env, width*height);
jint y,x;
jbyte grey;
jsize len = (*env)->GetArrayLength(env, arr);
jbyte *YUVData = (*env)->GetByteArrayElements(env, arr, 0);
for (y = 0; y < height; y++){
for (x = 0; x < width; x++){
grey = YUVData[y * width + x];
convertedData[y*width+x] =(jint) grey & 0xff;
}
}
LOGD("Random [%d]",len);
(*env)->SetIntArrayRegion(env, result, 0, (width*height),convertedData );
free(convertedData);
(*env)->ReleaseByteArrayElements(env, YUVData, (jbyte*)arr, 0);
return result;
}
Thanks for any help.
I have the same problem as yours.
In your specific case, while you are using pixel (and probably bitmap) you can send a bitmap instead of your bytearray and modify it
void *pixel_bm;
int retValue;
AndroidBitmapInfo info;
if ((retValue = AndroidBitmap_getInfo(env, bitmap, &info)) < 0) return 0;
if ((retValue = AndroidBitmap_lockPixels(env, bitmap, &pixel_bm)) < 0) return 0;
// you can now read an write into pixel_bm
AndroidBitmap_unlockPixels(env, bitmap);
If you find a solution to correctly free a GetByteArrayElement result, I'm instrested by the solution !!!
I have solved this problem by releasing the parameters.
(*env)->ReleaseByteArrayElements(env, arr, YUVData, 0);
It works great now.
I am trying to perform matrix multiplication which involves two dimensional double arrays. I followed an article on the SUN website but still couldn't get it right. Here's a brief description of my problem and the way I handled it:
I've two 2-dimensional double arrays which I am passing to JNI from my java function. I passed these arrays as JObectArray. In my JNI function, I created two local arrays of type double and copied the JObjectArrays in these local arrays. I then performed matrix multiplication and stored the result in another local array called result.
Now I want to return this result array back to Java function but I don't know how to do this.
Can someone please help me out with my query?
Here's my code:
JNIEXPORT jobjectArray JNICALL Java_prj_anyapp_JNIForMFCC_compute(JNIEnv *env, jclass jClass, jint Am, jint Bm, jint Bn, jobjectArray A, jobjectArray B)
{
jobjectArray retC;
int i,j,k;
double sum;
int localAm, localBm, localBn;
localAm = Am;
localBm = Bm;
localBn = Bn;
double localArrayCopyB[localBm][localBn];
double localArrayCopyA[localAm][localBm];
double localArrayCopyC[localAm][localBn];
for(i=0; i<localAm; i++) {
jdoubleArray oneDimA=
(jdoubleArray)(*env)->GetObjectArrayElement(env,A, i);
jint *elementA = (*env)->GetIntArrayElements(env,oneDimA, 0);
for(j=0; j<localBm; j++) {
localArrayCopyA[i][j]= elementA[j];
}
for(i=0; i<localBm; i++) {
jdoubleArray oneDimB=
(jdoubleArray)(*env)->GetObjectArrayElement(env,B, i);
jint *elementB = (*env)->GetIntArrayElements(env,oneDimB, 0);
for(j=0; j<localBn; j++) {
localArrayCopyB[i][j]= elementB[j];
}
}
for(i=0; i<localAm; i++){
for(j=0; j<localBm; j++){
for(k=0; k<localBn; k++){
sum += localArrayCopyA[i][k] * localArrayCopyB[k][j];
}
localArrayCopyC[i][j] = sum;
}
}
}
}