I am building a forensic tool, that shall read and write from file slacks. At the moment I use JNI to call the C functions read() and write() to write a buffer to the FAT SD card (also EXT4 internal memory would be nice). I use those because you can pass the length to read/write thus ignoring EOF.
I already tried writing by using a standard write and then truncate it in order to write in the file slack, which works on ubuntu 14.04 but not on Android (API 21).
Reading more from a file than the actuall file size does not reflect the flie slack on the SD-card.
There are tools e.g. "bmap" or "slacker.exe" (for NTFS) that manage to access file slacks. I am in need of a way to ignore EOF or handle it myself. I would prefere not to change existing file system drivers.
I appreciate any suggestions.
here is some sample code (that does not work yet):
jstring
Java_com_example_hellojni_HelloJni_write(JNIEnv *env, jobject thiz, jstring path)
{
char *cpath = (*env)->GetStringUTFChars(env, path, NULL);
int raw_file_descriptor = open(cpath,O_WRONLY,0);
lseek(raw_file_descriptor,0,SEEK_END);
char block_buffer [4096] = "hidden";
int buffer_length = 6;
int wret = write(raw_file_descriptor,block_buffer,buffer_length); // increases file size and moves EOF - I don't want that
LOGD(" char written: %d", wret);
free(cpath);
return (*env)->NewStringUTF(env, "write ok ");
}
jbyteArray
Java_com_example_hellojni_HelloJni_read2(JNIEnv *env, jobject thiz, jstring path) {
char *cpath = (*env)->GetStringUTFChars(env, path, NULL);
LOGD("open %s with ", cpath);
int raw_file_descriptor = open(cpath,O_RDONLY,0);
char buffer [4096];
int readretval = read(raw_file_descriptor, buffer, 50); // stops at EOF - I want it to continue reading all 50 bytes from the SD-card - This gives me the file content until EOF and then I get some random characters from the uninitialized buffer.
LOGD("read (%d) buffer with length %d: %s", readretval, sizeof(buffer), buffer);
int i; // Debug code
for (i=0; i < 49; ++i) {
LOGD("%c",buffer[i]);
}
close(raw_file_descriptor);
free(cpath);
return NULL;
}
Apparently this way is impossible. But with root it is possible to write file slack via mounting a loop back device. Still, ignoring EOF is not possible without changing the driver or implementing your own kernel module.
Related
This simple program to write into a file is running fine in my emulator as I am getting a text file generated inside the sdcard(text file has HELLO WORLD in it) along with the given message on emulator screen.
jstring
Java_com_example_hellojni_HelloJni_stringFromJNI( JNIEnv* env,
jobject thiz )
{
FILE* file = fopen("/sdcard/hello.txt","w+");
if (file != NULL)
{
fputs("HELLO WORLD!\n", file);
fflush(file);
fclose(file);
}
return (*env)->NewStringUTF(env, "Hello from JNI (with file io)!");
}
But when I try to read the file using this program I am facing problems. It outputs nothing on the emulator screen or on logcat. As seen from the above program there definitely is no problem with the location of sdcard or with using FILE operations. How to fix it?
JNIEXPORT jstring JNICALL Java_com_example_myproject_MainActivity_doSomething( JNIEnv* env, jobject thiz)
{
// Counts the characters inside file
char ch;
int cnt= 0;
FILE* fptr = fopen("/sdcard/hello.txt","r");
while((ch=fgetc(fptr))!=EOF)
cnt++;
fclose(fptr);
LOGE("cnt: %d",cnt);
return (*env)->NewStringUTF(env, "Hello from JNI (with file io)!");
}
Thanks
EDIT:
This above code to read file works well in linux gcc but has errors working in Android. Changing it as mentioned by Andrew solves it.
fgetc() returns int, not char, so this loop never terminates:
while((ch=fgetc(fptr))!=EOF)
cnt++;
because ch can never equal EOF.
In my Android JNI code, I need to convert jstring to wchar_t. The closest reference I found was How do I convert jstring to wchar_t *.
One can obtain jchar* and the length using the following code:
const jchar *raw = env->GetStringChars(string, 0);
jsize len = env->GetStringLength(string);
wchar_t* wStr = new wchar_t[len+1];
It seems I cannot use wcncpy to copy "raw" into "wStr." Although jchar is 2-bytes long, wchar_t is 4 bytes long on all modern versions of Android OS.
One option is to copy one character at a time in a for loop:
for(int i=0;i<len;i++) {
wStr[i] = raw[i];
}
wStr[len] = 0;
The other option would be to call env->GetStringUTFChars() and use iconv_* routines to convert to wchar_t type.
Can someone please confirm if option 1 is valid? Hope I don't have to resort to option 2. Is there a better option? Regards.
wchar_t specifies an element size but not a character set or encoding. Since you are asking about a 32-bit element, can we assume you want to use Unicode/UTF-32? Regardless, once you decide which encoding you want, standard Java libraries are up to the task.
Use a String.getBytes() overload to get an array of bytes. (It is easier to do this in Java rather than JNI, if you have a choice.) Once you have a jbyteArray, you can copy it to a C buffer and cast to wchar_t *.
On Android, you might want Unicode/UTF-8. But that has an 8-bit code-unit so you probably wouldn't be asking about wchar_t. (BTW-a character in UTF-8 can need 1 or more bytes.)
One way would be to use String.getBytes("UTF-32LE"). Note this is making the ASSUMPTION that wchar_t is 4 bytes and little-endian, but this should be a fairly safe assumption to make.
Here's an example that passes a String from Java to C++, where it is converted to std::wstring, reversed, and passed back to Java:
class MyClass {
private native byte[] reverseString(byte[] arr);
String reverseString(String s) {
try {
return new String(reverseString(s.getBytes("UTF-32")), "UTF-32");
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return "";
}
}
}
On the C++ side you have:
std::wstring toWStr(JNIEnv *env, jbyteArray s)
{
const wchar_t *buf = (wchar_t*) env->GetByteArrayElements(s, NULL);
int n = env->GetArrayLength(s) / sizeof(wchar_t);
// First byte is BOM (0xfeff), so we skip it, hence the "buf + 1".
// There IS NO null-terminator.
std::wstring ret(buf + 1, buf + n);
env->ReleaseByteArrayElements(s, (jbyte*) buf, 0);
return ret;
}
jbyteArray fromWStr(JNIEnv *env, const std::wstring &s)
{
jbyteArray ret = env->NewByteArray((s.size()+1)*sizeof(wchar_t));
// Add the BOM in front.
wchar_t bom = 0xfeff;
env->SetByteArrayRegion(ret, 0, sizeof(wchar_t), (const jbyte*) &bom);
env->SetByteArrayRegion(ret, sizeof(wchar_t), s.size()*sizeof(wchar_t), (const jbyte*) s.c_str());
return ret;
}
extern "C" JNIEXPORT jbyteArray JNICALL Java_MyClass_reverseString(JNIEnv *env, jobject thiz, jbyteArray arr)
{
std::wstring s= toWStr(env, arr);
std::reverse(s.begin(), s.end());
return fromWStr(env, s);
}
I tested it both on my phone, which has Android 4.1.2 and ARM CPU, and on the Android Emulator - Android 4.4.2 and x86 CPU, and this code:
MyClass obj = new MyClass();
Log.d("test", obj.reverseString("hello, здравствуйте, 您好, こんにちは"));
Gave this output:
06-04 17:18:20.605: D/test(8285): はちにんこ ,好您 ,етйувтсвардз ,olleh
As long as all your data is UCS2, you can use option 1. Please see wchar_t for UTF-16 on Linux? for a similar discussion. Note that C++11 provides std::codecvt_utf16 to deal with the situation.
No need to convert. Cast const jchar to (wchar_t *). jni.h define jchar as typedef uint16_t jchar; /* unsigned 16 bits */ which is eventually wchar_t.
You can try this, it worked for me in old project.
From the examples I got the basic idea of this code.
However I am not sure, what I am missing, as muxing.c demuxing.c and decoding_encoding.c
all use different approaches.
The process of converting an audio file to another file should go roughly like this:
inputfile -demux-> audiostream -read-> inPackets -decode2frames->
frames
-encode2packets-> outPackets -write-> audiostream -mux-> outputfile
However I found the following comment in demuxing.c:
/* Write the raw audio data samples of the first plane. This works
* fine for packed formats (e.g. AV_SAMPLE_FMT_S16). However,
* most audio decoders output planar audio, which uses a separate
* plane of audio samples for each channel (e.g. AV_SAMPLE_FMT_S16P).
* In other words, this code will write only the first audio channel
* in these cases.
* You should use libswresample or libavfilter to convert the frame
* to packed data. */
My questions about this are:
Can I expect a frame that was retrieved by calling one of the decoder functions, f.e.
avcodec_decode_audio4 to hold suitable values to directly put it into an encoder or is
the resampling step mentioned in the comment mandatory?
Am I taking the right approach? ffmpeg is very asymmetric, i.e. if there is a function
open_file_for_input there might not be a function open_file_for_output. Also there are different versions of many functions (avcodec_decode_audio[1-4]) and different naming
schemes, so it's very hard to tell, if the general approach is right, or actually an
ugly mixture of techniques that where used at different version bumps of ffmpeg.
ffmpeg uses a lot of specific terms, like 'planar sampling' or 'packed format' and I am having a hard time, finding definitions for these terms. Is it possible to write working code, without deep knowledge of audio?
Here is my code so far that right now crashes at avcodec_encode_audio2
and I don't know why.
int Java_com_fscz_ffmpeg_Audio_convert(JNIEnv * env, jobject this, jstring jformat, jstring jcodec, jstring jsource, jstring jdest) {
jboolean isCopy;
jclass configClass = (*env)->FindClass(env, "com.fscz.ffmpeg.Config");
jfieldID fid = (*env)->GetStaticFieldID(env, configClass, "ffmpeg_logging", "I");
logging = (*env)->GetStaticIntField(env, configClass, fid);
/// open input
const char* sourceFile = (*env)->GetStringUTFChars(env, jsource, &isCopy);
AVFormatContext* pInputCtx;
AVStream* pInputStream;
open_input(sourceFile, &pInputCtx, &pInputStream);
// open output
const char* destFile = (*env)->GetStringUTFChars(env, jdest, &isCopy);
const char* cformat = (*env)->GetStringUTFChars(env, jformat, &isCopy);
const char* ccodec = (*env)->GetStringUTFChars(env, jcodec, &isCopy);
AVFormatContext* pOutputCtx;
AVOutputFormat* pOutputFmt;
AVStream* pOutputStream;
open_output(cformat, ccodec, destFile, &pOutputCtx, &pOutputFmt, &pOutputStream);
/// decode/encode
error = avformat_write_header(pOutputCtx, NULL);
DIE_IF_LESS_ZERO(error, "error writing output stream header to file: %s, error: %s", destFile, e2s(error));
AVFrame* frame = avcodec_alloc_frame();
DIE_IF_UNDEFINED(frame, "Could not allocate audio frame");
frame->pts = 0;
LOGI("allocate packet");
AVPacket pktIn;
AVPacket pktOut;
LOGI("done");
int got_frame, got_packet, len, frame_count = 0;
int64_t processed_time = 0, duration = pInputStream->duration;
while (av_read_frame(pInputCtx, &pktIn) >= 0) {
do {
len = avcodec_decode_audio4(pInputStream->codec, frame, &got_frame, &pktIn);
DIE_IF_LESS_ZERO(len, "Error decoding frame: %s", e2s(len));
if (len < 0) break;
len = FFMIN(len, pktIn.size);
size_t unpadded_linesize = frame->nb_samples * av_get_bytes_per_sample(frame->format);
LOGI("audio_frame n:%d nb_samples:%d pts:%s\n", frame_count++, frame->nb_samples, av_ts2timestr(frame->pts, &(pInputStream->codec->time_base)));
if (got_frame) {
do {
av_init_packet(&pktOut);
pktOut.data = NULL;
pktOut.size = 0;
LOGI("encode frame");
DIE_IF_UNDEFINED(pOutputStream->codec, "no output codec");
DIE_IF_UNDEFINED(frame->nb_samples, "no nb samples");
DIE_IF_UNDEFINED(pOutputStream->codec->internal, "no internal");
LOGI("tests done");
len = avcodec_encode_audio2(pOutputStream->codec, &pktOut, frame, &got_packet);
LOGI("encode done");
DIE_IF_LESS_ZERO(len, "Error (re)encoding frame: %s", e2s(len));
} while (!got_packet);
// write packet;
LOGI("write packet");
/* Write the compressed frame to the media file. */
error = av_interleaved_write_frame(pOutputCtx, &pktOut);
DIE_IF_LESS_ZERO(error, "Error while writing audio frame: %s", e2s(error));
av_free_packet(&pktOut);
}
pktIn.data += len;
pktIn.size -= len;
} while (pktIn.size > 0);
av_free_packet(&pktIn);
}
LOGI("write trailer");
av_write_trailer(pOutputCtx);
LOGI("end");
/// close resources
avcodec_free_frame(&frame);
avcodec_close(pInputStream->codec);
av_free(pInputStream->codec);
avcodec_close(pOutputStream->codec);
av_free(pOutputStream->codec);
avformat_close_input(&pInputCtx);
avformat_free_context(pOutputCtx);
return 0;
}
Meanwhile I have figured this out and written an Android Library Project that does this
(for audio files). https://github.com/fscz/FFmpeg-Android
See the file /jni/audiodecoder.c for details
I write the Android application using libusb-1.0.9 and NDK (Android 4.0.4+) which has to read out audio-data from the USB-audiocard. The USB device from libusb opens successfully, and for it it is possible to receive all interfaces/EndPoint list. But when making ISOC transfer I faced with following unclear me error:
Debugging C code of making transfer:
static uint8_t buf[12];
static void cb_xfr(struct libusb_transfer *xfr)
{
LOGD("USB callback\n");
libusb_free_transfer(xfr);
}
JNIEXPORT jlong JNICALL
Java_com_usbtest_libusb_Libusb_makeISOCTransfer(JNIEnv *env, jobject this, jlong ms)
{
static struct libusb_transfer *xfr;
int num_iso_pack = 1;
unsigned char ep = 0x84;
xfr = libusb_alloc_transfer(num_iso_pack);
if (!xfr) {
LOGD("libusb_alloc_transfer: errno=%d\n", errno);
return -1000;
}
LOGD("libusb_fill_iso_transfer: ep=%x, buf=%d, num iso=%d\n", ep, sizeof(buf), num_iso_pack);
libusb_fill_iso_transfer(xfr, handle, ep, buf, sizeof(buf), num_iso_pack, cb_xfr, NULL, 0);
libusb_set_iso_packet_lengths(xfr, sizeof(buf)/num_iso_pack);
int res = libusb_submit_transfer(xfr);
LOGD("libusb_submit_transfer: %d, errno=%d\n", res, errno);
struct timeval tv;
tv.tv_sec = 1;
tv.tv_usec = 0;
libusb_handle_events_timeout(NULL, &tv);
return res;
}
After a call of libusb_submit_transfer I received a mistake: libusb_submit_transfer: -1, errno -2
And text messages:
need 1 32k URBs for transfer
first URB failed, easy peasy
EndPoint 0x84 is true for audio-in. The buf[] size = 12 - corresponds to the field wMaxPacketSize of this EndPoint. I request 1 transfer. Field of xfr->status = 0, but callback isn't caused.
I thought that it is necessary to increase the buf buffer to 32K - I increased, but it didn't help.
I tried to increase quantity of transfers. Same error.
Prompt me please in what there can be a error?
I found the solution of this problem.
In this topic http://en.it-usenet.org/thread/14995/14985/ there was a similar question, and the decision appeared to make detach for device HID USB interfaces.
I've used libusb_detach_kernel to detach the 2 hid drivers and submit
transfer returned 0!!
I made libusb_detach_kernel_driver(handle, interface) for all interfaces of the device (0..3) after device opening, and as a result of libusb_submit_transfer return SUCCESS of transfer.
I am trying to load a TGA file in Android NDK.
I open the file using AssetManager, read in the entire contents of the TGA file into a memory buffer, and then I try to extract the pixel data from it.
I can read the TGA header part of the file without any problems, but when I try to advance the memory pointer past the TGA header, the app crashes. If I don't try to advance the memory pointer, it does not crash.
Is there some sort of limitation in Android NDK for pointer arithmetic?
Here is the code:
This function opens the asset file:
char* GEAndroid::OpenAssetFile( const char* pFileName )
{
char* pBuffer = NULL;
AAssetManager* assetManager = m_pState->activity->assetManager;
AAsset* assetFile = AAssetManager_open(assetManager, pFileName, AASSET_MODE_UNKNOWN);
if (!assetFile) {
// Log error as 'error in opening the input file from apk'
LOGD( "Error opening file %s", pFileName );
}
else
{
LOGD( "File opened successfully %s", pFileName );
const void* pData = AAsset_getBuffer(assetFile);
off_t fileLength = AAsset_getLength(assetFile);
LOGD("fileLength=%d", fileLength);
pBuffer = new char[fileLength];
memcpy( pBuffer, pData, fileLength * sizeof( char ) );
}
return pBuffer;
}
And down here in my texture class I try to load it:
char* pBuffer = g_pGEAndroid->OpenAssetFile( fileNameWithPath );
TGA_HEADER textureHeader;
char *pImageData = NULL;
unsigned int bytesPerPixel = 4;
textureHeader = *reinterpret_cast<TGA_HEADER*>(pBuffer);
// I double check that the textureHeader is valid and it is.
bytesPerPixel = textureHeader.bits/8; // Divide By 8 To Get The Bytes Per Pixel
m_imageSize = textureHeader.width*textureHeader.height*bytesPerPixel; // Calculate The Memory Required For The TGA Data
pImageData = new char[m_imageSize];
// the line below causes the crash
pImageData = reinterpret_cast<char*>(pBuffer + sizeof( TGA_HEADER)); // <-- causes a crash
If I replace the line above with the following line (even though it is incorrect), the app runs, although obviously the texture is messed up.
pImageData = reinterpret_cast<char*>(pBuffer); // <-- does not crash, but obviously texture is messed up.
Anyone have any ideas?
Thanks.
Why reinterpret_cast? You're adding an integer to a char*; that operation produces a char*. No typecast necessary.
One caveat for pointer juggling on Android (and on ARM devices in general): ARM cannot read/write unaligned data from memory. If you read/write an int-sized variable, it needs to be at an address that's a multiple of 4; for short, a multiple of 2. Bytes can be at any address. This does not, as far as I can see, apply to the presented snippet. But do keep in mind. It does throw off binary format parsing occasionally, especially when ported from Intel PCs.
Simply assigning an unaligned value to a pointer does not crash. Dereferencing it might.
Sigh, I just realized the mistake. I allocate memory for pImageData, then set the point to the buffer. This does not sit well when I try to create an OpenGL texture with the pixel data. Modifying it so I memcpy the pixel data from (pBuffer + sizeof( TGA_HEADER) ) to pImageData fixes the problem.