Develop Reference

Replacing WCHAR and wchar_t with something generic

I'm trying to convert some VS 2015 C++ code to Android Studio C++ code.
My function looks like this:
int size = 0;
int len = 0;
fread(&size,sizeof(int),1,g_File);
#ifdef VERBOSE
printf("fullSentences size = %d\n",size);
#endif
int i1 = 0;
int i2 = 0;
int i3 = 0;
for(int i = 0; i < size; i++)
{
fread(&len,sizeof(int),1,g_File);
wchar_t *buff = new WCHAR[len+1];
fread(buff,sizeof(WCHAR),len,g_File);
buff[len]=0;
fread(&i1,sizeof(int),1,g_File);
fread(&i2,sizeof(int),1,g_File);
fread(&i3,sizeof(int),1,g_File);
fullSentences.Add(buff,i1,i2,i3);
delete buff;
#ifdef VERBOSE
FullSentence fs = fullSentences.Content().back();
printf("%s\t%d\t%d\n",fs.Text.c_str(),fs.ByteStart,fs.ByteCount);
#endif
}
I would like to get rid of the WCHAR and wchar_t in order to make the porting easier.
Can anybody suggestion a replacement for these?
I would like to avoid having to tell Android Studio what wchar_t is, if possible.
Thank you.
Edit:
Here is the class information:
//
// -------------------- clsFullSentences -----------------------------
//
vector<FullSentence> &clsFullSentences::Content()
{
return m_content;
}
vector<wstring> &clsFullSentences::CleanLower()
{
return m_sCleanLower;
}
void clsFullSentences::LoadSerializedFullSentences(string uFile)
{
if (!fileExists(stringToWString(uFile)))
{
DebugBreak();
}
FILE* inFile = fopen(uFile.c_str(), "rb");
wchar_t signature[2];
fread(signature, sizeof(wchar_t), 1, inFile);
wstring wline;
//read how many possibleresults we have
getLineW(inFile, wline);
unsigned int count=_wtoi(wline.c_str());
for (unsigned int i = 0; i < count; i++)
{
FullSentence st;
getLineW(inFile,wline);
st.Text = wline;
//getLineW(inFile,wline);
st.Emotion =0;// _wtoi(wline.c_str());
getLineW(inFile,wline);
st.ByteStart = _wtoi(wline.c_str());
getLineW(inFile,wline);
st.ByteCount = _wtoi(wline.c_str());
m_content.push_back(st);
}
fclose(inFile);
}
void clsFullSentences::Add(wstring text, int i1, int i2, int i3)
{
FullSentence fs;
fs.Text = text;
fs.Emotion = i1;
fs.ByteStart = i2;
fs.ByteCount = i3;
m_content.push_back(fs);
wstring sClean;
sClean=StripPuncToLower(text);
m_sCleanLower.push_back(sClean);
}
bool getLineW(FILE *inFile, wstring &result)
{
wchar_t data[2] = { 0, 0 };
result = L"";
do
{
fread(data, sizeof(wchar_t), 1, inFile);
if (data[0] > 0)
{
if (data[0] != 13)
{
if (data[0] != 10)
{
result += data;
}
else
{
break;//10 is the end of the line
}
}
}
} while (!feof(inFile));
if (result.size() > 0)
{
return true;
}
else
{
return false;
}
}

If you want this code to work on Windows and Android builds then you will find that on windows wchar_t is 2 bytes, while on android it is 4bytes. So you will not be able to write a file on windows, and later read it properly on android. In such case you would have to use char16_t on android and convert it to 'your
choosen' string type under android.
(edit: even better, if you can - make sure all the files are written as utf8 strings)
As for the 'choosen string type' I would suggest to use utf8, so instead of std::wstring use std::string (encoded using utf8). From my experience NDK team was discouraging use of wchar_t from the very begining (missing w* functions from c library etc.) I am not sure how it is now.
I work on a project which was originally coded with MFC, and then ported to android. We used from the very begining a TCHAR macro. Which as you know resolves to char on non unicode builds and to wchar_t on unicode builds. So the idea is to use TCHAR everywhere then under android TCHAR should resolve to char, while on windows use wchar_t (I assume you use unicode build under windows?).
I am not saying this is a best solution to share code between windows and android platform, there are lots of problems like conversions to utf8 on android, these are done with if-defs.

Use std::string that contains text in UTF-8 encoding. You may also want some lightweight library that helps to deal with checks and conversions of such text. Avoid using wchar_t and std::wstring in code that is meant to be portable.

Android JNI - Reliable way to convert jstring to wchar_t

In my Android JNI code, I need to convert jstring to wchar_t. The closest reference I found was How do I convert jstring to wchar_t *.
One can obtain jchar* and the length using the following code:
const jchar *raw = env->GetStringChars(string, 0);
jsize len = env->GetStringLength(string);
wchar_t* wStr = new wchar_t[len+1];
It seems I cannot use wcncpy to copy "raw" into "wStr." Although jchar is 2-bytes long, wchar_t is 4 bytes long on all modern versions of Android OS.
One option is to copy one character at a time in a for loop:
for(int i=0;i<len;i++) {
wStr[i] = raw[i];
}
wStr[len] = 0;
The other option would be to call env->GetStringUTFChars() and use iconv_* routines to convert to wchar_t type.
Can someone please confirm if option 1 is valid? Hope I don't have to resort to option 2. Is there a better option? Regards.

wchar_t specifies an element size but not a character set or encoding. Since you are asking about a 32-bit element, can we assume you want to use Unicode/UTF-32? Regardless, once you decide which encoding you want, standard Java libraries are up to the task.
Use a String.getBytes() overload to get an array of bytes. (It is easier to do this in Java rather than JNI, if you have a choice.) Once you have a jbyteArray, you can copy it to a C buffer and cast to wchar_t *.
On Android, you might want Unicode/UTF-8. But that has an 8-bit code-unit so you probably wouldn't be asking about wchar_t. (BTW-a character in UTF-8 can need 1 or more bytes.)

One way would be to use String.getBytes("UTF-32LE"). Note this is making the ASSUMPTION that wchar_t is 4 bytes and little-endian, but this should be a fairly safe assumption to make.
Here's an example that passes a String from Java to C++, where it is converted to std::wstring, reversed, and passed back to Java:
class MyClass {
private native byte[] reverseString(byte[] arr);
String reverseString(String s) {
try {
return new String(reverseString(s.getBytes("UTF-32")), "UTF-32");
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return "";
}
}
}
On the C++ side you have:
std::wstring toWStr(JNIEnv *env, jbyteArray s)
{
const wchar_t *buf = (wchar_t*) env->GetByteArrayElements(s, NULL);
int n = env->GetArrayLength(s) / sizeof(wchar_t);
// First byte is BOM (0xfeff), so we skip it, hence the "buf + 1".
// There IS NO null-terminator.
std::wstring ret(buf + 1, buf + n);
env->ReleaseByteArrayElements(s, (jbyte*) buf, 0);
return ret;
}
jbyteArray fromWStr(JNIEnv *env, const std::wstring &s)
{
jbyteArray ret = env->NewByteArray((s.size()+1)*sizeof(wchar_t));
// Add the BOM in front.
wchar_t bom = 0xfeff;
env->SetByteArrayRegion(ret, 0, sizeof(wchar_t), (const jbyte*) &bom);
env->SetByteArrayRegion(ret, sizeof(wchar_t), s.size()*sizeof(wchar_t), (const jbyte*) s.c_str());
return ret;
}
extern "C" JNIEXPORT jbyteArray JNICALL Java_MyClass_reverseString(JNIEnv *env, jobject thiz, jbyteArray arr)
{
std::wstring s= toWStr(env, arr);
std::reverse(s.begin(), s.end());
return fromWStr(env, s);
}
I tested it both on my phone, which has Android 4.1.2 and ARM CPU, and on the Android Emulator - Android 4.4.2 and x86 CPU, and this code:
MyClass obj = new MyClass();
Log.d("test", obj.reverseString("hello, здравствуйте, 您好, こんにちは"));
Gave this output:
06-04 17:18:20.605: D/test(8285): はちにんこ ,好您 ,етйувтсвардз ,olleh

As long as all your data is UCS2, you can use option 1. Please see wchar_t for UTF-16 on Linux? for a similar discussion. Note that C++11 provides std::codecvt_utf16 to deal with the situation.

No need to convert. Cast const jchar to (wchar_t *). jni.h define jchar as typedef uint16_t jchar; /* unsigned 16 bits */ which is eventually wchar_t.
You can try this, it worked for me in old project.

Android: Find out which core the thread is running on

I'm trying to write code for Android which would give me some kind of information (id?) of the processor and the core on which a thread is running on.
I've google'd and grep'ed the sources for some inspiration but with no luck. All I know is, that most likely I will need some C/C++ calls.
What I have working is the following:
#include <jni.h>
int getCpuId() {
// missing code
return 0;
}
int getCoreId() {
// missing code
return 0;
}
JNIEXPORT int JNICALL Java_com_spendoptima_Utils_getCpuId(JNIEnv * env,
jobject obj) {
return getCpuId();
}
JNIEXPORT int JNICALL Java_com_spendoptima_Utils_getCoreId(JNIEnv * env,
jobject obj) {
return getCoreId();
}
The whole project compiles and runs just fine. I'm able to call the functions from within Java and I get the proper responses.
Is here anybody who could fill in the blanks?

This is what seems to work for me:
//...
#include <sys/syscall.h>
//...
int getCpuId() {
unsigned cpu;
if (syscall(__NR_getcpu, &cpu, NULL, NULL) < 0) {
return -1;
} else {
return (int) cpu;
}
}
//...

The good news is, the necessary library and system call are defined on Android (sched_getcpu() and __getcpu()). The bad news is, they aren't part of the NDK.
You can roll your own syscall wrapper and library call, using the method shown in this answer.
Another approach is to read /proc/self/stat and parse out the processor entry. The proc(5) man page describes the format:
/proc/[pid]/stat
Status information about the process. This is used by ps(1).
It is defined in /usr/src/linux/fs/proc/array.c.
...
processor %d (since Linux 2.2.8)
CPU number last executed on.
This is much slower, and the "file" format may change if the kernel is updated, so it's not the recommended approach.

Access array in native code (NDK) from Java (SDK) via JNI

I want to access an array that is created and updated in the native C code efficiently. If need be, i could send a pointer or reference from Java code to the native side and have the C-code populate it so that i can read it from SDK side when it's ready to be consumed.
Currently, this is how i am doing it. But i think there can be better ways to do it, since i am doing one copy in C-side and then there an object that is created every time i issue a read on the Java-side.
My Java code:
double[] valuesFromNative = getValues();
public static native double[] getValues();
static { System.loadLibrary("test-jni"); }
My native (C and not C++) code:
#define LEN 18
double testDoubleArr[LEN];
jdoubleArray Java_com_test_testActivity_getValues(JNIEnv *env, jclass clazz) {
jboolean isCopy;
int i;
jdoubleArray result = (*env)->NewDoubleArray(env, LEN);
jdouble* destArrayElems = (*env)->GetDoubleArrayElements(env, result, &isCopy);
for (i = 0; i < LEN; i++) {
destArrayElems[i] = testDoubleArr[i];
}
if(isCopy == JNI_TRUE) {
// isCopy should NEVER be JNI_TRUE in this case, right?
// so, i could as well replace this condition with
// assert(isCopy == JNI_FALSE)?
}
return result;
}
This code snippet works - so, i am looking at more efficient or rather correct way to achieve the same thing.
Thanks for sharing your thoughts.

I think SetDoubleArrayRegion() would be faster. Less code and less JNI calls, that's for sure.
jdoubleArray result = (*env)->NewDoubleArray(env, LEN);
(*env)->SetDoubleArrayRegion(env, result, 0, LEN, testDoubleArr);
You don't even have to create the array on the C++ side. Declare the method like this:
public static native void getValues(double[] a);
Implement like this:
void Java_com_test_testActivity_getValues(JNIEnv *env, jclass clazz, jdoubleArray a)
{//...
Create the array on the Java side, cache it in an instance variable or something, and pass it to JNI to be filled whenever needed. Make sure the assumptions about array size are the same on the Java side and on the C side.

Android -> iOS Cipher migration

I have to migrate an Android app to iOS that uses Cipher. So here is the Android code:
PaddedBufferedBlockCipher cipher = new PaddedBufferedBlockCipher(new CBCBlockCipher(new DESedeEngine()));
...
byte[] result = Hex.encode(output, 0, output.length);
String resultS = new String(Str.toChars(result));
I tried with lot of stuff for objective-c but can't find a way to get the same string as i get on Java. I used the iOS code from here http://dotmac.rationalmind.net/2009/02/aes-interoperability-between-net-and-iphone/ (and a lot more but all do the same).
And then to get the string on iOS use something like:
NSString* resultS = [encryptedData base64Encoding]
but result strings don't match. Maybe the problem is how i handle the encoding for the NSData (seems that the Java version don't use base64, i'm ok?)
Any ideas?
EDIT 1:
Ok, i made some progress (i hope). Checking the java code they use a block size of 8 and DES/CBC with a key of 24 chars. So i change the code from CocoaFu to this:
- (NSData *)doCipher:(NSData *)dataIn
key:(NSData *)symmetricKey
context:(CCOperation)encryptOrDecrypt
{
CCCryptorStatus ccStatus = kCCSuccess;
size_t cryptBytes = 0; // Number of bytes moved to buffer.
NSMutableData *dataOut = [NSMutableData dataWithLength:dataIn.length + kCCBlockSizeDES];
uint8_t iv[kChosenCipherBlockSize];
memset((void *) iv, 0x0, (size_t) sizeof(iv));
ccStatus = CCCrypt( encryptOrDecrypt,
kCCAlgorithmDES,
kCCOptionPKCS7Padding,
symmetricKey.bytes,
kCCKeySize3DES,
(const void *)iv,
dataIn.bytes,
dataIn.length,
dataOut.mutableBytes,
dataOut.length,
&cryptBytes);
if (ccStatus != kCCSuccess) {
NSLog(#"CCCrypt status: %d", ccStatus);
}
dataOut.length = cryptBytes;
return dataOut;
}
When i try to encode the "test" message in java i get "f69d7c299597c880" but on iOS (using same key of course) i get "< 91864397 > < 41434eaa >" for 3DES and "< ed660859 > < 4bad6f7f >" for DES. Any ideas on what else could i change?

This is a difficult problem because it either works or not with very little in clues. The best way to proceed is by starting as simply as possible and building up.
The first thing is to learn exactly what the java code is doing. With CBC mode there will be an iv (initialization value) but none is explicitly specified in the java code posted. You need to find out what the java code is using. Also post the full java code.
From the code PaddedBufferedBlockCipher I infer that there is block padding, this might be PKCS5 or PKCS7, both are the same from a padding point of view, the iOS equivalent is kCCOptionPKCS7Padding. Make sure about this.
Make sure the key length are the same, for AES the options are 128, 192 and 256 bits, unless there are specific reasons use 128.
The code Hex.encode seems to imply that the output is being hex encoded, you will need to do the same on iOS, this is not the same as Base64.
The other main issue is getting all the parameters the same on both sides. Of particular interest are:
encryption key value and size
mode: CBC, ECB, etc. (you should probably be using CBC)
initialization vector (iv) is needed for most modes
padding method: PKCS7, etc. (AES is a block cypher and needs input in a multiple of block size)
Any post encryption processing, hex or Base64 encoding.
Start as simply as possible, iv of all 0, data of one block size, no padding, simple key, no post processing. Get the key, iv and test data from a file that can be shared between systems, this will prevent some errors such as c string jul termination, etc.
Here is the iOS code I use:
#import <CommonCrypto/CommonCryptor.h>
+ (NSData *)doCipher:(NSData *)dataIn
iv:(NSData *)iv
key:(NSData *)symmetricKey
context:(CCOperation)encryptOrDecrypt
{
CCCryptorStatus ccStatus = kCCSuccess;
size_t cryptBytes = 0; // Number of bytes moved to buffer.
NSMutableData *dataOut = [NSMutableData dataWithLength:dataIn.length + kCCBlockSizeAES128];
ccStatus = CCCrypt( encryptOrDecrypt,
kCCAlgorithmAES128,
kCCOptionPKCS7Padding,
symmetricKey.bytes,
kCCKeySizeAES128,
iv.bytes,
dataIn.bytes,
dataIn.length,
dataOut.mutableBytes,
dataOut.length,
&cryptBytes);
if (ccStatus != kCCSuccess) {
NSLog(#"CCCrypt status: %d", ccStatus);
}
dataOut.length = cryptBytes;
return dataOut;
}
Also add Security.framework to your project.
If the security is important consider having someone with security experience create the code and protocol. If the security is not important, just send the password in the clear.
A few bugs in an app is not that bad, the app still basically works, one bug in security and all security is lost.
Good security is not as easy as one might think--or as my wife says: "If crypto was easy everyone would do it" but she really means correctly.

Ok, i made some progress (i hope). Checking the java code they use a block size of 8 and DES/CBC with a key of 24 chars. So i change the code from CocoaFu to this:
- (NSData *)doCipher:(NSData *)dataIn
key:(NSData *)symmetricKey
context:(CCOperation)encryptOrDecrypt
{
CCCryptorStatus ccStatus = kCCSuccess;
size_t cryptBytes = 0; // Number of bytes moved to buffer.
NSMutableData *dataOut = [NSMutableData dataWithLength:dataIn.length + kCCBlockSizeDES];
uint8_t iv[kChosenCipherBlockSize];
memset((void *) iv, 0x0, (size_t) sizeof(iv));
ccStatus = CCCrypt( encryptOrDecrypt,
kCCAlgorithmDES,
kCCOptionPKCS7Padding,
symmetricKey.bytes,
kCCKeySize3DES,
(const void *)iv,
dataIn.bytes,
dataIn.length,
dataOut.mutableBytes,
dataOut.length,
&cryptBytes);
if (ccStatus != kCCSuccess) {
NSLog(#"CCCrypt status: %d", ccStatus);
}
dataOut.length = cryptBytes;
return dataOut;
}
When i try to encode the "test" message in java i get "f69d7c299597c880" but on iOS (using same key of course) i get "< 91864397 > < 41434eaa >" for 3DES and "< ed660859 > < 4bad6f7f >" for DES. Any ideas on what else could i change?

You're trying to use CBC mode on your cipher, but iOS doesn't support CBC mode, only ECB. Look here