Develop Reference

JNI how to manage the life cycle of a c++ buffer

I have an image buffer allocated on the heap in my c++ code that I would like to share with some other c++ objects as well as Java objects through JNI. On the native side im using shared_ptr and I was wondering what is the best way to do so ? my thought is to allocate the buffer on the heap once and share a reference everywhere. I'm taking advantage of smart pointers so that the buffer will be deallocated as soon as all the references go out of scope, but I'm facing an issue when sharing a reference to the java side.
how can I ensure that my java object has a valid reference to the buffer all the time ? how can c++ determine that the reference counter reaches 0 when java object is done using its reference. My concern is to avoid memory leak and also ensure that buffer doesn't get destroyed too soon before getting processed by the java class.
thanks for the help

The general answer is "make the Java object's lifetime influence the lifetime of the C++ object".
Start with the following Java class:
class Refholder implements AutoCloseable {
private long ptr; // the actual pointer
private long shared_ptr; // a pointer to a shared_ptr keeping `ptr` alive
public Refholder(long ptr, long shared_ptr) {
this.ptr = ptr;
this.shared_ptr = shared_ptr;
}
public native void close();
public void finalize() { close(); }
// Other methods to access the contents of `ptr` go here.
};
This will contain both the actual pointer and a pointer to a shared_ptr.
When you want to hand a reference to Java, use the following:
jobject passToJava(JNIEnv *env, std::shared_ptr<Foo> instance) {
jclass cls_Refholder = env->FindClass("Refholder");
jmethodID ctr_Refholder = env->GetMethodID(cls_Refholder, "<init>", "(JJ)V");
// This line increases the reference count and remembers where we put the copy
std::shared_ptr<Foo> *copy = new std::shared_ptr<Foo>(std::move(instance));
jobject ret = env->NewObject(cls_Refholder, ctr_Refholder, copy->get(), copy);
return ret;
}
Finally, the close method is responsible for extracting the shared_ptr and deallocating it:
JNIEXPORT void Java_Refholder_close(JNIEnv *env, jobject obj) {
jclass cls_Refholder = env->GetObjectClass(obj);
jfieldID fld_Refholder_ptr = env->GetFieldID(cls_Refholder, "ptr", "J");
jfieldID fld_Refholder_shared_ptr = env->GetFieldID(cls_Refholder, "shared_ptr", "J");
std::shared_ptr<Foo> *copy = (std::shared_ptr<Foo>*)env->GetLongField(obj, fld_Refholder_shared_ptr);
if (!copy)
return;
env->SetLongField(obj, fld_Refholder_ptr, 0);
env->SetLongField(obj, fld_Refholder_shared_ptr, 0);
delete copy;
}
I have decided to implement both AutoCloseable and finalize because I do not know how your Java code plans to use the reference. If you need deterministic destruction of the C++ object you need to use try-with-resources or explicit calls to the close method. If you do not, at last the finalize will close it.

Invalid indirect reference 0x416f68a0 in decodeIndirectRef

I've seen plenty of questions about exactly the same error, but none of them seems to be trying to do this simple thing and still fail.
I have in my class header, as private members:
static JNIEnv* env;
static jclass copterServiceClass;
static jmethodID mavlinkMsgMethod;
Then in the source for that class:
JNIEnv* JU_Calls::env = 0;
jclass JU_Calls::copterServiceClass = 0;
jmethodID JU_Calls::mavlinkMsgMethod = 0;
bool JU_Calls::setupJNICalls() {
if (cached_jvm->GetEnv(reinterpret_cast<void**>(&env), JNI_VERSION_1_6) != JNI_OK) {
__android_log_print(ANDROID_LOG_ERROR, TAG, "Unable to get Java Env from cached JavaVM");
return -1;
}
jclass dataClass = env->FindClass("eu/deye/copterdroidair/copterdroidair/Services/CopterService");
copterServiceClass = (jclass) env->NewGlobalRef(dataClass);
mavlinkMsgMethod = env->GetMethodID(copterServiceClass, "MavlinkMsg", "(Ljava/lang/String;)V");
jobject javaObjectRef = env->NewObject(copterServiceClass, mavlinkMsgMethod);
jstring msg = env->NewStringUTF("aaaa");
env->CallVoidMethod(javaObjectRef, mavlinkMsgMethod, msg);
return true;
}
Note: cached_jvm is assigned at JNI_OnLoad.
It fails with the aforementioned error when executing NewObject.
Invalid indirect reference 0x416f68a0 in decodeIndirectRef
I tried passing NULL to the CallVoidMethod as the message, as seen on other's questions, but as it's logical, the problem is before, so useless.
Hope you can help me, as always, great SO masters ;)
EDIT: While I think Bangyno answer is the right one, what I've eventually done to solve the problem as quickest as possible, was to declare the Java Methods that would be called from C++ as static. That way I don't have to call the constructor, and everything makes a lot more sense. Because the Java class to which I was calling, was an Android Service, thus calling the constructor was erroneous for sure.
Apart from declaring the Java method static, the resulting C++ code results as follows:
jclass copterServiceClass = env->FindClass("eu/deye/copterdroidair/copterdroidair/Services/CopterService");
jmethodID mavlinkMsgMethod = env->GetStaticMethodID(copterServiceClass, "MavlinkMsg", "(ILjava/lang/String;)V");
jstring msg = env->NewStringUTF(str);
env->CallStaticVoidMethod(copterServiceClass, mavlinkMsgMethod, severity, msg);
env->DeleteLocalRef(msg);
Is very important not to forget the last line, because otherwise it will fill your JNI table and crash.

Here I want to discuss is the error message:
Invalid indirect reference 0x416f68a0 in decodeIndirectRef
It means you give the wrong argument, the second one "mavlinkMsgMethod".
In my experience, if you change "mavlinkMsgMethod" to a number such as "5", the 0x416f68a0 will change to 0x5.
The right way to use newObject is use to invoke the constructor. It should look like this, just a sample:
jclass dataClass = env->FindClass("eu/deye/copterdroidair/copterdroidair/Services/CopterService");
mavlinkMsgMethod = env->GetMethodID(copterServiceClass, "<init>", "(Ljava/lang/String;)V");
jstring str = env->NewStringUTF("testing");
jobject javaObjectRef = env->NewObject(copterServiceClass, mavlinkMsgMethod, str);

Android JNI ReleaseByteArrayElements does not work

i have got a problem at using Android JNI. I call a java method from native C. Everything works fine, BUT after a few seconds the APP crashes, because the maximum of 512 entrys of JNI refs is full (or the memory is full).
Here is my code:
int jniBluetoothSend( TJNIAdapter *pAdapter, byte *data, int dwLength )
{
JNIEnv *pEnv;
JavaVM *pVm = NULL;
jclass cls;
jmethodID methodId;
int nRet;
jbyteArray aData;
if ( pAdapter )
{
pVm = pAdapter->pVm;
( *pVm )->AttachCurrentThread( pVm, &pEnv, NULL );
if ( pAdapter->pClasses != NULL && pAdapter->pClasses->hgs_bluetooth != NULL )
{
// get function
methodId = ( *pEnv )->GetMethodID( pEnv, pAdapter->pClasses->hgs_bluetooth, "write", "([BI)I" );
if ( methodId )
{
aData = ( *pEnv )->NewByteArray( pEnv, dwLength);
( *pEnv )->SetByteArrayRegion( pEnv, aData, 0, dwLength, data);
// write Data to device
nRet = ( *pEnv )->CallIntMethod( pEnv, g_hgsBthObject, methodId, aData, dwLength );
// and delete Reference -> so GC can cleanup
( *pEnv )->DeleteLocalRef( pEnv, aData );
}
}
//( *pVm )->DetachCurrentThread( pVm ); -> // crashes as soon as getting called
}
return nRet;
}
Everytime "ReleaseByteArrayElements()" gets called a warning shows up in Android Log from dalvik:
JNI: unpinPrimitiveArray(0x428e84a8) failed to find entry (valid=1)
So, i think, the problem is, that the created array is not getting freed. But I don't know, how to do this in the right way.
Can anyone help me with this?
Thanks!
EDIT
I have done a several tests.
If i add the DetachCurrentThread function to the bottom of the second if(), the APP crashes, if DetachCurrentThread is getting called.
But i have added DeleteLocalRef in an other function, and the APP does not crash anymore.
So my question is: Do i have to call DetachCurrentThread in every function i call AttachCurrentThread or is it enough if i call that once to the end of the APP?
Also updated the Code

Everytime ReleaseByteArrayElements() gets called a warning shows up in Android Log from dalvik:
JNI: unpinPrimitiveArray(0x428e84a8) failed to find entry (valid=1)
ReleaseByteArrayElements is meant to be paired with a previous call to GetByteArrayElements. From Oracle's documentation:
Release<PrimitiveType>ArrayElements Routines
void Release<PrimitiveType>ArrayElements(JNIEnv *env,
ArrayType array, NativeType *elems, jint mode);
A family of functions that informs the VM that the native code no longer needs access to elems. The elems argument is a pointer derived from array using the corresponding Get<PrimitiveType>ArrayElements() function.
Since you didn't call GetByteArrayElements you also shouldn't call ReleaseByteArrayElements.
Note that SetByteArrayRegion() function is not one of the functions that pin an array, therefore it does not require a release.

Linux mutexes must be global?

Most of the examples I'm looking at on the Web have pthread_mutex_t sitting at the top of the file in the global space and I think I read somewhere that Linux mutexes have to be global. Is this true?
edit:
I have some Win32 multithreading code that I'm porting over to Linux. For the windows code, there are several wrapper functions that encapsulate things like mutex creation and locking/unlocking. My understanding is that every synchronization primitive that's created through one of the Create() API calls in Windows returns a HANDLE that can be stored in an instance field and then used later. In this case, it's used in the Lock() function, which is wrapper around WaitForSingleObject(). For Linux, could I simply store the mutex in an instance field and call pthread_mutex_lock()/pthread_cond_wait() in the Lock() function and expect the same behavior as on Windows?
Nv_Mutex::Nv_Mutex(Nv_XprocessID name)
{
#if defined(WIN32)
if((handle = ::CreateMutexA(0, false, name)) == NULL)
{
throw Nv_EXCEPTION(XCPT_ResourceAllocationFailure, GetLastError());
}
isCreator = !(::GetLastError() == ERROR_ALREADY_EXISTS);
#else
if (name == Nv_XprocessID_NULL) {
/*
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; // Fast
pthread_mutex_t recmutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP; // Recursive
pthread_mutex_t errchkmutex = PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP; // Errorcheck
*/
mutex = PTHREAD_MUTEX_INITIALIZER;
// attributes??
if (pthread_mutex_init(&mutex, NULL) != 0) {
throw Nv_EXCEPTION(XCPT_ResourceAllocationFailure, GetLastError());
}
}
else {
// insert code for named mutex (needed for shared mutex across processes) here.
}
//isCreator = !(GetLastError() == EBUSY);
#endif
}
bool
Nv_Mutex::Lock(const char *f, int l, Nv_uint32 timeout)
{
switch(WaitForSingleObject(handle, timeout))
{
case WAIT_OBJECT_0:
file = f;
line = l;
return true;
case WAIT_TIMEOUT:
return false;
}
throw Nv_EXCEPTION(XCPT_WaitFailed, GetLastError());
}

No, they can scoped. There is nothing special about the actual mutex pointer.

You have the requirement a bit wrong. Mutexes do not need to be global, however, you cannot statically initialize a non-static mutex. But you do not need to statically initialize a mutex prior to calling pthread_mutex_init on it, because that initializes it. So just don't use static initializers and instead call pthread_mutex_init.
It will actually work, but this is by luck due to the details of the implementation. Please don't rely on an implementation detail.
Static initialization is legal only for statically ALLOCATED storage[.] ... Although C syntax allows using the static initialization macros on "automatic" variables, this is specifically prohibited by the POSIX standard. It's not correct, and it's not portable. - David Butenhof

Why can't my C code find my Java functions?

My application makes calls to a native library, and then within the native library, makes calls back to the Java code. I have the calls TO the native library working correctly ( I'm pretty sure ), but not its telling me that it can't find the Java functions when trying to call them from the C file.
The application doesn't even crash, there is no pop up telling me it closed unexpectedly, it just flashes a black screen and nothing happens.
The code I am trying to reproduce was originally made in C for Palm Pilot, and is being transferred to an Android device. This is the original call in C I am trying to duplicate...
InitRelay(Changeit,getit,putit,flushit,delayit);
The parameters in the call above are function names, the functions are here below:
void Changeit(WORD baud){
ChangeRate(baud);
}
extern Boolean ChangeRate(UInt32 baud)
{...}
Int16 getit(UInt16 t) {...}
void putit( BYTE p ) {...}
void flushit(void){...}
extern void delayit( UInt16 wait ) {...}
This is InitRelay shown in my .c file:
BYTE __stdcall InitRelay(fp_setbaud _setbaud, fp_get _get, fp_put _put, fp_flush _flush, fp_delay _delay){
RelayAPI_SetBaud=_setbaud;
RelayAPI_get=_get;
RelayAPI_put=_put;
RelayAPI_flush=_flush;
RelayAPI_delay=_delay;
....
}
The parameters shown in above are typedefed in my .h file:
typedef void (__stdcall *fp_setbaud)(WORD);
typedef short (__stdcall *fp_get)(WORD);
typedef void (__stdcall *fp_put)(BYTE);
typedef void (__stdcall *fp_flush)(void);
typedef void (__stdcall *fp_delay)(WORD);
The WORD/BYTE/DWORD types are defined in a separate .h file shown here:
typedef unsigned char BYTE;
typedef unsigned short WORD;
typedef unsigned long DWORD;
Now, in my Android code, I call a function I made called InitRelayJava(), all of my data for the application is stored in a separate class called RelayAPIModel. I created a nested class within that to store all of my native functions. I did this so that I can access these functions the same way no matter what Activity the application is currently in.
public class RelayAPIModel {
....
public static class NativeCalls {
static {
System.loadLibrary( "RelayAPI" );
}
public native static byte InitRelayJava();
public native static void FreeRelayJava();
public static void changeItJavaWrapper( short l ) {
mModelService.changeitJava( l );
}
public static void flushItJavaWrapper() {
mModelService.flushitJava();
}
public static void putItJavaWrapper( byte[] p ) {
mModelService.putitJava( p );
}
public static void delayItJavaWrapper( short wait ) {
mModelService.delayitJava( wait );
}
public static short getItJavaWrapper( short s ) {
return mModelService.getitJava( s );
}
}
}
The calls made inside the *Wrapper functions go to a separate class that I have to handle all of the applications bluetooth capabilities. I do not think that those are needed for this problem.
This is what InitRelayJava looks like in my C code...
BYTE Java_my_eti_commander_RelayAPIModel_00024NativeCalls_InitRelayJava( JNIEnv *env, jobject obj ) {
myEnv = (env);
bluetoothClass = (*env)->GetObjectClass( env, obj );
myObject = obj;
changeID = (*myEnv)->GetMethodID( myEnv, myObject, "changeitJavaWrapper", "(S)V" );
getID = (*myEnv)->GetMethodID( myEnv, myObject, "getitJavaWrapper" , "(S)S" );
putID = (*myEnv)->GetMethodID( myEnv, myObject, "putitJavaWrapper" , "(B)V" );
flushID = (*myEnv)->GetMethodID( myEnv, myObject, "flushitJavaWrapper" , "()V" );
delayID = (*myEnv)->GetMethodID( myEnv, myObject, "delayitJavaWrapper" , "(S)V" );
...
}
This is the LogCat I am receiving...
08-02 10:27:32.406: D/dalvikvm(28376): Trying to load lib /data/data/my.eti.commander/lib/libRelayAPI.so 0x40515430
08-02 10:27:32.406: D/dalvikvm(28376): Added shared lib /data/data/my.eti.commander/lib/libRelayAPI.so 0x40515430
08-02 10:27:32.406: D/dalvikvm(28376): No JNI_OnLoad found in /data/data/my.eti.commander/lib/libRelayAPI.so 0x40515430, skipping init
08-02 10:27:32.406: D/dalvikvm(28376): GetMethodID: method not found: Lmy/eti/commander/RelayAPIModel$NativeCalls;.changeitJavaWrapper:(S)V
08-02 10:27:32.413: D/dalvikvm(28376): GetMethodID: method not found: Lmy/eti/commander/RelayAPIModel$NativeCalls;.getitJavaWrapper:(S)S
08-02 10:27:32.413: E/dalvikvm(28376): Class lookup Ljava/lang/NoSuchMethodError; attempted while exception Ljava/lang/NoSuchMethodError; pending

InitRelayJava is a static method - that means your second parameter (obj) is a class object pointer, not a this pointer. So get rid of the following line:
bluetoothClass = (*env)->GetObjectClass( env, obj );
and instead pass obj to GetMethodID(), like this:
changeID = (*myEnv)->GetMethodID( myEnv, obj, "changeitJavaWrapper", "(I)Z" );
EDIT: also, your parameter/return type signatures are wrong. Short is not the same as int, so the signature for changeitJavaWrapper is "(S)Z", for getitJavaWrapper is "()I" as it takes no parameters and returns an int. Be more careful please; it does not take an advanced knowledge of C to get those right, just some self-checking. This project of yours is inching into What have you tried? territory.
Cheat sheet on JNI type codes here.
Let me try and anticipate your next question - you cannot call those methods. Of course you cannot, it's not possible to call a nonstatic method from a static one. Your Java callbacks are all nonstatic, while the native method is static. Either make them static, or make the native method nonstatic and insert the GetObjectClass back.
EDIT2: so you changed your Java methods to static without telling. Now instead of (*env)->GetMethodID() you need to call (*env)->GetStaticMethodID() to get the method ID. Same parameters.

Your method signatures are wrong. Use javap -s classname to get the signatures of the class, for example: javap -s java.lang.String.
You can find javap in your local JDK.