Background
I need to parse some zip files of various types (getting some inner files content for one purpose or another, including getting their names).
Some of the files are not reachable via file-path, as Android has Uri to reach them, and as sometimes the zip file is inside another zip file. With the push to use SAF, it's even less possible to use file-path in some cases.
For this, we have 2 main ways to handle: ZipFile class and ZipInputStream class.
The problem
When we have a file-path, ZipFile is a perfect solution. It's also very efficient in terms of speed.
However, for the rest of the cases, ZipInputStream could reach issues, such as this one, which has a problematic zip file, and cause this exception:
java.util.zip.ZipException: only DEFLATED entries can have EXT descriptor
at java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:321)
at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:124)
What I've tried
The only always-working solution would be to copy the file to somewhere else, where you could parse it using ZipFile, but this is inefficient and requires you to have free storage, as well as remove the file when you are done with it.
So, what I've found is that Apache has a nice, pure Java library (here) to parse Zip files, and for some reason its InputStream solution (called "ZipArchiveInputStream") seem even more efficient than the native ZipInputStream class.
As opposed to what we have in the native framework, the library offers a bit more flexibility. I could, for example, load the entire zip file into bytes array, and let the library handle it as usual, and this works even for the problematic Zip files I've mentioned:
org.apache.commons.compress.archivers.zip.ZipFile(SeekableInMemoryByteChannel(byteArray)).use { zipFile ->
for (entry in zipFile.entries) {
val name = entry.name
... // use the zipFile like you do with native framework
gradle dependency:
// http://commons.apache.org/proper/commons-compress/ https://mvnrepository.com/artifact/org.apache.commons/commons-compress
implementation 'org.apache.commons:commons-compress:1.20'
Sadly, this isn't always possible, because it depends on having the heap memory hold the entire zip file, and on Android it gets even more limited, because the heap size could be relatively small (heap could be 100MB while the file is 200MB). As opposed to a PC which can have a huge heap memory being set, for Android it's not flexible at all.
So, I searched for a solution that has JNI instead, to have the entire ZIP file loaded into byte array there, not going to the heap (at least not entirely). This could be a nicer workaround because if the ZIP could be fit in the device's RAM instead of the heap, it could prevent me from reaching OOM while also not needing to have an extra file.
I've found this library called "larray" which seems promising , but sadly when I tried using it, it crashed, because its requirements include having a full JVM, meaning not suitable for Android.
EDIT: seeing that I can't find any library and any built-in class, I tried to use JNI myself. Sadly I'm very rusty with it, and I looked at an old repository I've made a long time ago to perform some operations on Bitmaps (here). This is what I came up with :
native-lib.cpp
#include <jni.h>
#include <android/log.h>
#include <cstdio>
#include <android/bitmap.h>
#include <cstring>
#include <unistd.h>
class JniBytesArray {
public:
uint32_t *_storedData;
JniBytesArray() {
_storedData = NULL;
}
};
extern "C" {
JNIEXPORT jobject JNICALL Java_com_lb_myapplication_JniByteArrayHolder_allocate(
JNIEnv *env, jobject obj, jlong size) {
auto *jniBytesArray = new JniBytesArray();
auto *array = new uint32_t[size];
for (int i = 0; i < size; ++i)
array[i] = 0;
jniBytesArray->_storedData = array;
return env->NewDirectByteBuffer(jniBytesArray, 0);
}
}
JniByteArrayHolder.kt
class JniByteArrayHolder {
external fun allocate(size: Long): ByteBuffer
companion object {
init {
System.loadLibrary("native-lib")
}
}
}
class MainActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
thread {
printMemStats()
val jniByteArrayHolder = JniByteArrayHolder()
val byteBuffer = jniByteArrayHolder.allocate(1L * 1024L)
printMemStats()
}
}
fun printMemStats() {
val memoryInfo = ActivityManager.MemoryInfo()
(getSystemService(Context.ACTIVITY_SERVICE) as ActivityManager).getMemoryInfo(memoryInfo)
val nativeHeapSize = memoryInfo.totalMem
val nativeHeapFreeSize = memoryInfo.availMem
val usedMemInBytes = nativeHeapSize - nativeHeapFreeSize
val usedMemInPercentage = usedMemInBytes * 100 / nativeHeapSize
Log.d("AppLog", "total:${Formatter.formatFileSize(this, nativeHeapSize)} " +
"free:${Formatter.formatFileSize(this, nativeHeapFreeSize)} " +
"used:${Formatter.formatFileSize(this, usedMemInBytes)} ($usedMemInPercentage%)")
}
This doesn't seem right, because if I try to create a 1GB byte array using jniByteArrayHolder.allocate(1L * 1024L * 1024L * 1024L) , it crashes without any exception or error logs.
The questions
Is it possible to use JNI for Apache's library, so that it will handle the ZIP file content which is contained within JNI's "world" ?
If so, how can I do it? Is there any sample of how to do it? Is there a class for it? Or do I have to implement it myself? If so, can you please show how it's done in JNI?
If it's not possible, what other way is there to do it? Maybe alternative to what Apache has?
For the solution of JNI, how come it doesn't work well ? How could I efficiently copy the bytes from the stream into the JNI byte array (my guess is that it will be via a buffer)?
I took a look at the JNI code you posted and made a couple of changes. Mostly it is defining the size argument for NewDirectByteBuffer and using malloc().
Here is the output of the log after allocating 800mb:
D/AppLog: total:1.57 GB free:1.03 GB used:541 MB (34%)
D/AppLog: total:1.57 GB free:247 MB used:1.32 GB (84%)
And the following is what the buffer looks like after the allocation. As you can see, the debugger is reporting a limit of 800mb which is what we expect.
My C is very rusty, so I am sure that there is some work to be done. I have updated the code to be a little more robust and to allow for the freeing of memory.
native-lib.cpp
extern "C" {
static jbyteArray *_holdBuffer = NULL;
static jobject _directBuffer = NULL;
/*
This routine is not re-entrant and can handle only one buffer at a time. If a buffer is
allocated then it must be released before the next one is allocated.
*/
JNIEXPORT
jobject JNICALL Java_com_example_zipfileinmemoryjni_JniByteArrayHolder_allocate(
JNIEnv *env, jobject obj, jlong size) {
if (_holdBuffer != NULL || _directBuffer != NULL) {
__android_log_print(ANDROID_LOG_ERROR, "JNI Routine",
"Call to JNI allocate() before freeBuffer()");
return NULL;
}
// Max size for a direct buffer is the max of a jint even though NewDirectByteBuffer takes a
// long. Clamp max size as follows:
if (size > SIZE_T_MAX || size > INT_MAX || size <= 0) {
jlong maxSize = SIZE_T_MAX < INT_MAX ? SIZE_T_MAX : INT_MAX;
__android_log_print(ANDROID_LOG_ERROR, "JNI Routine",
"Native memory allocation request must be >0 and <= %lld but was %lld.\n",
maxSize, size);
return NULL;
}
jbyteArray *array = (jbyteArray *) malloc(static_cast<size_t>(size));
if (array == NULL) {
__android_log_print(ANDROID_LOG_ERROR, "JNI Routine",
"Failed to allocate %lld bytes of native memory.\n",
size);
return NULL;
}
jobject directBuffer = env->NewDirectByteBuffer(array, size);
if (directBuffer == NULL) {
free(array);
__android_log_print(ANDROID_LOG_ERROR, "JNI Routine",
"Failed to create direct buffer of size %lld.\n",
size);
return NULL;
}
// memset() is not really needed but we call it here to force Android to count
// the consumed memory in the stats since it only seems to "count" dirty pages. (?)
memset(array, 0xFF, static_cast<size_t>(size));
_holdBuffer = array;
// Get a global reference to the direct buffer so Java isn't tempted to GC it.
_directBuffer = env->NewGlobalRef(directBuffer);
return directBuffer;
}
JNIEXPORT void JNICALL Java_com_example_zipfileinmemoryjni_JniByteArrayHolder_freeBuffer(
JNIEnv *env, jobject obj, jobject directBuffer) {
if (_directBuffer == NULL || _holdBuffer == NULL) {
__android_log_print(ANDROID_LOG_ERROR, "JNI Routine",
"Attempt to free unallocated buffer.");
return;
}
jbyteArray *bufferLoc = (jbyteArray *) env->GetDirectBufferAddress(directBuffer);
if (bufferLoc == NULL) {
__android_log_print(ANDROID_LOG_ERROR, "JNI Routine",
"Failed to retrieve direct buffer location associated with ByteBuffer.");
return;
}
if (bufferLoc != _holdBuffer) {
__android_log_print(ANDROID_LOG_ERROR, "JNI Routine",
"DirectBuffer does not match that allocated.");
return;
}
// Free the malloc'ed buffer and the global reference. Java can not GC the direct buffer.
free(bufferLoc);
env->DeleteGlobalRef(_directBuffer);
_holdBuffer = NULL;
_directBuffer = NULL;
}
}
I also updated the array holder:
class JniByteArrayHolder {
external fun allocate(size: Long): ByteBuffer
external fun freeBuffer(byteBuffer: ByteBuffer)
companion object {
init {
System.loadLibrary("native-lib")
}
}
}
I can confirm that this code along with the ByteBufferChannel class provided by Botje here works for Android versions before API 24. The SeekableByteChannel interface was introduced in API 24 and is needed by the ZipFile utility.
The maximum buffer size that can be allocated is the size of a jint and is due to the limitation of JNI. Larger data can be accommodated (if available) but would require multiple buffers and a way to handle them.
Here is the main activity for the sample app. An earlier version always assumed the the InputStream read buffer was was always filled and errored out when trying to put it to the ByteBuffer. This was fixed.
MainActivity.kt
class MainActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
}
fun onClick(view: View) {
button.isEnabled = false
status.text = getString(R.string.running)
thread {
printMemStats("Before buffer allocation:")
var bufferSize = 0L
// testzipfile.zip is not part of the project but any zip can be uploaded through the
// device file manager or adb to test.
val fileToRead = "$filesDir/testzipfile.zip"
val inStream =
if (File(fileToRead).exists()) {
FileInputStream(fileToRead).apply {
bufferSize = getFileSize(this)
close()
}
FileInputStream(fileToRead)
} else {
// If testzipfile.zip doesn't exist, we will just look at this one which
// is part of the APK.
resources.openRawResource(R.raw.appapk).apply {
bufferSize = getFileSize(this)
close()
}
resources.openRawResource(R.raw.appapk)
}
// Allocate the buffer in native memory (off-heap).
val jniByteArrayHolder = JniByteArrayHolder()
val byteBuffer =
if (bufferSize != 0L) {
jniByteArrayHolder.allocate(bufferSize)?.apply {
printMemStats("After buffer allocation")
}
} else {
null
}
if (byteBuffer == null) {
Log.d("Applog", "Failed to allocate $bufferSize bytes of native memory.")
} else {
Log.d("Applog", "Allocated ${Formatter.formatFileSize(this, bufferSize)} buffer.")
val inBytes = ByteArray(4096)
Log.d("Applog", "Starting buffered read...")
while (inStream.available() > 0) {
byteBuffer.put(inBytes, 0, inStream.read(inBytes))
}
inStream.close()
byteBuffer.flip()
ZipFile(ByteBufferChannel(byteBuffer)).use {
Log.d("Applog", "Starting Zip file name dump...")
for (entry in it.entries) {
Log.d("Applog", "Zip name: ${entry.name}")
val zis = it.getInputStream(entry)
while (zis.available() > 0) {
zis.read(inBytes)
}
}
}
printMemStats("Before buffer release:")
jniByteArrayHolder.freeBuffer(byteBuffer)
printMemStats("After buffer release:")
}
runOnUiThread {
status.text = getString(R.string.idle)
button.isEnabled = true
Log.d("Applog", "Done!")
}
}
}
/*
This function is a little misleading since it does not reflect the true status of memory.
After native buffer allocation, it waits until the memory is used before counting is as
used. After release, it doesn't seem to count the memory as released until garbage
collection. (My observations only.) Also, see the comment for memset() in native-lib.cpp
which is a member of this project.
*/
private fun printMemStats(desc: String? = null) {
val memoryInfo = ActivityManager.MemoryInfo()
(getSystemService(Context.ACTIVITY_SERVICE) as ActivityManager).getMemoryInfo(memoryInfo)
val nativeHeapSize = memoryInfo.totalMem
val nativeHeapFreeSize = memoryInfo.availMem
val usedMemInBytes = nativeHeapSize - nativeHeapFreeSize
val usedMemInPercentage = usedMemInBytes * 100 / nativeHeapSize
val sDesc = desc?.run { "$this:\n" }
Log.d(
"AppLog", "$sDesc total:${Formatter.formatFileSize(this, nativeHeapSize)} " +
"free:${Formatter.formatFileSize(this, nativeHeapFreeSize)} " +
"used:${Formatter.formatFileSize(this, usedMemInBytes)} ($usedMemInPercentage%)"
)
}
// Not a great way to do this but not the object of the demo.
private fun getFileSize(inStream: InputStream): Long {
var bufferSize = 0L
while (inStream.available() > 0) {
val toSkip = inStream.available().toLong()
inStream.skip(toSkip)
bufferSize += toSkip
}
return bufferSize
}
}
A sample GitHub repository is here.
You can steal LWJGL's native memory management functions. It is BSD3 licensed, so you only have to mention somewhere that you are using code from it.
Step 1: given an InputStream is and a file size ZIP_SIZE, slurp the stream into a direct byte buffer created by LWJGL's org.lwjgl.system.MemoryUtil helper class:
ByteBuffer bb = MemoryUtil.memAlloc(ZIP_SIZE);
byte[] buf = new byte[4096]; // Play with the buffer size to see what works best
int read = 0;
while ((read = is.read(buf)) != -1) {
bb.put(buf, 0, read);
}
Step 2: wrap the ByteBuffer in a ByteChannel.
Taken from this gist. You possibly want to strip the writing parts out.
package io.github.ncruces.utils;
import java.nio.ByteBuffer;
import java.nio.channels.NonWritableChannelException;
import java.nio.channels.SeekableByteChannel;
import static java.lang.Math.min;
public final class ByteBufferChannel implements SeekableByteChannel {
private final ByteBuffer buf;
public ByteBufferChannel(ByteBuffer buffer) {
if (buffer == null) throw new NullPointerException();
buf = buffer;
}
#Override
public synchronized int read(ByteBuffer dst) {
if (buf.remaining() == 0) return -1;
int count = min(dst.remaining(), buf.remaining());
if (count > 0) {
ByteBuffer tmp = buf.slice();
tmp.limit(count);
dst.put(tmp);
buf.position(buf.position() + count);
}
return count;
}
#Override
public synchronized int write(ByteBuffer src) {
if (buf.isReadOnly()) throw new NonWritableChannelException();
int count = min(src.remaining(), buf.remaining());
if (count > 0) {
ByteBuffer tmp = src.slice();
tmp.limit(count);
buf.put(tmp);
src.position(src.position() + count);
}
return count;
}
#Override
public synchronized long position() {
return buf.position();
}
#Override
public synchronized ByteBufferChannel position(long newPosition) {
if ((newPosition | Integer.MAX_VALUE - newPosition) < 0) throw new IllegalArgumentException();
buf.position((int)newPosition);
return this;
}
#Override
public synchronized long size() { return buf.limit(); }
#Override
public synchronized ByteBufferChannel truncate(long size) {
if ((size | Integer.MAX_VALUE - size) < 0) throw new IllegalArgumentException();
int limit = buf.limit();
if (limit > size) buf.limit((int)size);
return this;
}
#Override
public boolean isOpen() { return true; }
#Override
public void close() {}
}
Step 3: Use ZipFile as before:
ZipFile zf = new ZipFile(ByteBufferChannel(bb);
for (ZipEntry ze : zf) {
...
}
Step 4: Manually release the native buffer (preferably in a finally block):
MemoryUtil.memFree(bb);
Related
I need to pass the FFMPEG 'raw' data back to my JAVA code in order to display it on the screen.
I have a native method that deals with FFMPEG and after that calls a method in java that takes Byte[] (so far) as an argument.
Byte Array that is passed is read by JAVA but when doing BitmapFactory.decodeByteArray(bitmap, 0, bitmap.length); it returns null. I have printed out the array and I get 200k of elements (which are expected), but cannot be decoded. So far what I'm doing is taking data from AvFrame->data casting it to unsigned char * and then casting that to jbyterArray. After all the casting, I pass the jbyteArray as argument to my JAVA method. Is there something I'm missing here? Why won't BitmapFactory decode the array into an image for displaying?
EDIT 1.0
Currently I am trying to obtain my image via
public void setImage(ByteBuffer bmp) {
bmp.rewind();
Bitmap bitmap = Bitmap.createBitmap(1920, 1080, Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(bmp);
runOnUiThread(() -> {
ImageView imgViewer = findViewById(R.id.mSurfaceView);
imgViewer.setImageBitmap(bitmap);
});
}
But I keep getting an exception
JNI DETECTED ERROR IN APPLICATION: JNI NewDirectByteBuffer called with pending exception java.lang.RuntimeException: Buffer not large enough for pixels
at void android.graphics.Bitmap.copyPixelsFromBuffer(java.nio.Buffer) (Bitmap.java:657)
at void com.example.asmcpp.MainActivity.setSurfaceImage(java.nio.ByteBuffer)
Edit 1.1
So, here is the full code that is executing every time there is a frame incoming. Note that the ByteBuffer is created and passed from within this method
void VideoClientInterface::onEncodedFrame(video::encoded_frame_t &encodedFrame) {
AVFrame *filt_frame = av_frame_alloc();
auto frame = std::shared_ptr<video::encoded_frame_t>(new video::encoded_frame_t,
[](video::encoded_frame_t *p) { if (p) delete p; });
if (frame) {
frame->size = encodedFrame.size;
frame->ssrc = encodedFrame.ssrc;
frame->width = encodedFrame.width;
frame->height = encodedFrame.height;
frame->dataType = encodedFrame.dataType;
frame->timestamp = encodedFrame.timestamp;
frame->frameIndex = encodedFrame.frameIndex;
frame->isKeyFrame = encodedFrame.isKeyFrame;
frame->isDroppable = encodedFrame.isDroppable;
frame->data = new char[frame->size];
if (frame->data) {
memcpy(frame->data, encodedFrame.data, frame->size);
AVPacket packet;
av_init_packet(&packet);
packet.dts = AV_NOPTS_VALUE;
packet.pts = encodedFrame.timestamp;
packet.data = (uint8_t *) encodedFrame.data;
packet.size = encodedFrame.size;
int ret = avcodec_send_packet(m_avCodecContext, &packet);
if (ret == 0) {
ret = avcodec_receive_frame(m_avCodecContext, m_avFrame);
if (ret == 0) {
m_transform = sws_getCachedContext(
m_transform, // previous context ptr
m_avFrame->width, m_avFrame->height, AV_PIX_FMT_YUV420P, // src
m_avFrame->width, m_avFrame->height, AV_PIX_FMT_RGB24, // dst
SWS_BILINEAR, nullptr, nullptr, nullptr // options
);
auto decodedFrame = std::make_shared<video::decoded_frame_t>();
decodedFrame->width = m_avFrame->width;
decodedFrame->height = m_avFrame->height;
decodedFrame->size = m_avFrame->width * m_avFrame->height * 3;
decodedFrame->timeStamp = m_avFrame->pts;
decodedFrame->data = new unsigned char[decodedFrame->size];
if (decodedFrame->data) {
uint8_t *dstSlice[] = {decodedFrame->data,
0,
0};// outFrame.bits(), outFrame.bits(), outFrame.bits()
const int dstStride[] = {decodedFrame->width * 3, 0, 0};
sws_scale(m_transform, m_avFrame->data, m_avFrame->linesize,
0, m_avFrame->height, dstSlice, dstStride);
auto m_rawData = decodedFrame->data;
auto len = strlen(reinterpret_cast<char *>(m_rawData));
if (frameCounter == 10) {
jobject newArray = GetJniEnv()->NewDirectByteBuffer(m_rawData, len);
GetJniEnv()->CallVoidMethod(m_obj, setSurfaceImage, newArray);
frameCounter = 0;
}
frameCounter++;
}
} else {
av_packet_unref(&packet);
}
} else {
av_packet_unref(&packet);
}
}
}
}
I am not entirely sure I am even doing that part correctly. If you see any errors in this, feel free to point them out.
You cannot cast native byte arrays to jbyteArray and expect it to work. A byte[] is an actual object with length field, a reference count, and so on.
Use NewDirectByteBuffer instead to wrap your byte buffer into a Java ByteBuffer, from where you can grab the actual byte[] using .array().
Note that this JNI operation is relatively expensive, so if you expect to do this on a per-frame basis, you might want to pre-allocate some bytebuffers and tell FFmpeg to write directly into those buffers.
I'm trying to retrieve metadata in Android using FFmpeg, JNI and a Java FileDescriptor and it isn't' working. I know FFmpeg supports the pipe protocol so I'm trying to emmulate: "cat test.mp3 | ffmpeg i pipe:0" programmatically. I use the following code to get a FileDescriptor from an asset bundled with the Android application:
FileDescriptor fd = getContext().getAssets().openFd("test.mp3").getFileDescriptor();
setDataSource(fd, 0, 0x7ffffffffffffffL); // native function, shown below
Then, in my native (In C++) code I get the FileDescriptor by calling:
static void wseemann_media_FFmpegMediaMetadataRetriever_setDataSource(JNIEnv *env, jobject thiz, jobject fileDescriptor, jlong offset, jlong length)
{
//...
int fd = jniGetFDFromFileDescriptor(env, fileDescriptor); // function contents show below
//...
}
// function contents
static int jniGetFDFromFileDescriptor(JNIEnv * env, jobject fileDescriptor) {
jint fd = -1;
jclass fdClass = env->FindClass("java/io/FileDescriptor");
if (fdClass != NULL) {
jfieldID fdClassDescriptorFieldID = env->GetFieldID(fdClass, "descriptor", "I");
if (fdClassDescriptorFieldID != NULL && fileDescriptor != NULL) {
fd = env->GetIntField(fileDescriptor, fdClassDescriptorFieldID);
}
}
return fd;
}
I then pass the file descriptor pipe # (In C) to FFmpeg:
char path[256] = "";
FILE *file = fdopen(fd, "rb");
if (file && (fseek(file, offset, SEEK_SET) == 0)) {
char str[20];
sprintf(str, "pipe:%d", fd);
strcat(path, str);
}
State *state = av_mallocz(sizeof(State));
state->pFormatCtx = NULL;
if (avformat_open_input(&state->pFormatCtx, path, NULL, &options) != 0) { // Note: path is in the format "pipe:<the FD #>"
printf("Metadata could not be retrieved\n");
*ps = NULL;
return FAILURE;
}
if (avformat_find_stream_info(state->pFormatCtx, NULL) < 0) {
printf("Metadata could not be retrieved\n");
avformat_close_input(&state->pFormatCtx);
*ps = NULL;
return FAILURE;
}
// Find the first audio and video stream
for (i = 0; i < state->pFormatCtx->nb_streams; i++) {
if (state->pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO && video_index < 0) {
video_index = i;
}
if (state->pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_AUDIO && audio_index < 0) {
audio_index = i;
}
set_codec(state->pFormatCtx, i);
}
if (audio_index >= 0) {
stream_component_open(state, audio_index);
}
if (video_index >= 0) {
stream_component_open(state, video_index);
}
printf("Found metadata\n");
AVDictionaryEntry *tag = NULL;
while ((tag = av_dict_get(state->pFormatCtx->metadata, "", tag, AV_DICT_IGNORE_SUFFIX))) {
printf("Key %s: \n", tag->key);
printf("Value %s: \n", tag->value);
}
*ps = state;
return SUCCESS;
My issue is avformat_open_input doesn't fail but it also doesn't let me retrieve any metadata or frames, The same code works if I use a regular file URI (e.g file://sdcard/test.mp3) as the path. What am I doing wrong? Thanks in advance.
Note: if you would like to look at all of the code I'm trying to solve the issue in order to provide this functionality for my library: FFmpegMediaMetadataRetriever.
Java
AssetFileDescriptor afd = getContext().getAssets().openFd("test.mp3");
setDataSource(afd.getFileDescriptor(), afd.getStartOffset(), fd.getLength());
C
void ***_setDataSource(JNIEnv *env, jobject thiz,
jobject fileDescriptor, jlong offset, jlong length)
{
int fd = jniGetFDFromFileDescriptor(env, fileDescriptor);
char path[20];
sprintf(path, "pipe:%d", fd);
State *state = av_mallocz(sizeof(State));
state->pFormatCtx = avformat_alloc_context();
state->pFormatCtx->skip_initial_bytes = offset;
state->pFormatCtx->iformat = av_find_input_format("mp3");
and now we can continue as usual:
if (avformat_open_input(&state->pFormatCtx, path, NULL, &options) != 0) {
printf("Metadata could not be retrieved\n");
*ps = NULL;
return FAILURE;
}
...
Even better, use <android/asset_manager.h>, like this:
Java
setDataSource(getContext().getAssets(), "test.mp3");
C
#include <android/asset_manager_jni.h>
void ***_setDataSource(JNIEnv *env, jobject thiz,
jobject assetManager, jstring assetName)
{
AAssetManager* assetManager = AAssetManager_fromJava(env, assetManager);
const char *szAssetName = (*env)->GetStringUTFChars(env, assetName, NULL);
AAsset* asset = AAssetManager_open(assetManager, szAssetName, AASSET_MODE_RANDOM);
(*env)->ReleaseStringUTFChars(env, assetName, szAssetName);
off_t offset, length;
int fd = AAsset_openFileDescriptor(asset, &offset, &length);
AAsset_close(asset);
Disclaimer: error checking was omitted for brevity, but resources are released correctly, except for fd. You must close(fd) when finished.
Post Scriptum: note that some media formats, e.g. mp4 need seekable protocol, and pipe: cannot help. In such case, you may try sprintf(path, "/proc/self/fd/%d", fd);, or use the custom saf: protocol.
Thks a lot for this post.
That help me a lot to integrate Android 10 and scoped storage with FFmpeg using FileDescriptor.
Here the solution I'm using on Android 10:
Java
URI uri = ContentUris.withAppendedId(
MediaStore.Audio.Media.EXTERNAL_CONTENT_URI,
trackId // Coming from `MediaStore.Audio.Media._ID`
);
ParcelFileDescriptor parcelFileDescriptor = getContentResolver().openFileDescriptor(
uri,
"r"
);
int pid = android.os.Process.myPid();
String path = "/proc/" + pid + "/fd/" + parcelFileDescriptor.dup().getFd();
loadFFmpeg(path); // Call native code
CPP
// Native code, `path` coming from Java `loadFFmpeg(String)`
avformat_open_input(&format, path, nullptr, nullptr);
OK, I spent a lot of time trying to transfer media data to ffmpeg through Assetfiledescriptor. Finally, I found that there may be a bug in mov.c. When mov.c parsed the trak atom, the corresponding skip_initial_bytes was not set. I have tried to fix this problem.
Detail please refer to FFmpegForAndroidAssetFileDescriptor, demo refer to WhatTheCodec.
FileDescriptor fd = getContext().getAssets().openFd("test.mp3").getFileDescriptor();
Think you should start with AssetFileDescripter.
http://developer.android.com/reference/android/content/res/AssetFileDescriptor.html
I am looking since some time for a way to play MIDI in Delphi XE5 with Android targeted. Several of my questions before were related to this "quest" :-). I have filed two requests to embarcadero: #119422 to add MIDI support to TMediaPlayer and #119423 to add a MIDI framework to Firemonkey, but that did not help. I have succeeded at last. As I know there are some more people who were looking for MIDI on Android I post this question with answer for documentation.
The Android system has an internal MIDI synthesizer. You can access it via the Android NDK. I have described this in an article containing some downloads. This answer is a short description of this article. What you'll see here is a Proof of Concept. It will show how to play MIDI notes on an Android system but needs improvement. Suggestions for improvement are welcome :-)
Use Eclipse to interface with the Java project. I presume you have Delphi XE5 with the Mobile pack, which gives you two things already installed: the Android SDK and NDK. Do not reinstall these by downloading the complete Android SDK from Google. Download and install the Eclipse Android Development Tools (ADT) plugin and follow the installation instructions. This allows you to use the Android SDK/NDK environment already installed by Delphi XE5 (you will find the paths in Delphi, Options | Tools | SDK Manager). In this way Delphi and Eclipse will share the same SDK and NDK.
I used the MIDI library developed by Willam Farmer. He also has the full SoniVox documentation available which I couldn't get elsewhere. His driver comes with a full example (Java) program. I created my own project with and changed the package name to org.drivers.midioutput, so all functions are prefixed by Java_org_drivers_midioutput_MidiDriver_ (see code below).
When you wish to compile the midi.c jus open a command window, and call ndk-build in the project directory. Some error messages are ok. The mips and x86 libraries were not built in my case.
There is one point though you should be aware of: the path to the ndk may not contain spaces. As you let the Delphi installer install Delphi there is bound to be a space in it: the subdirectory Rad Studio in that terrible long file name where Delphi installs the SDK and NDK. In order to work around this problem, create an empty directory on drive C:, call it C:\ndk. Use MKLINK to link this directory to the ndk directory. This can only be done from an elevated command prompt and as you do so you'll lose your network connections. The link is persistent so just close the command prompt and open another, unelevated one, and all should work now. Now you can really use ndk-build.
midi.c - the NDK interface with the SoniVox
////////////////////////////////////////////////////////////////////////////////
//
// MidiDriver - An Android Midi Driver.
//
// Copyright (C) 2013 Bill Farmer
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//
// Bill Farmer william j farmer [at] yahoo [dot] co [dot] uk.
//
///////////////////////////////////////////////////////////////////////////////
// Some slight modifications by Arnold Reinders. Added a test function and changed
// the package to org.drivers.midioutput. The original copyright still applies
#include
// for EAS midi
#include "eas.h"
#include "eas_reverb.h"
// determines how many EAS buffers to fill a host buffer
#define NUM_BUFFERS 4
// EAS data
static EAS_DATA_HANDLE pEASData;
const S_EAS_LIB_CONFIG *pLibConfig;
static EAS_PCM *buffer;
static EAS_I32 bufferSize;
static EAS_HANDLE midiHandle;
// This function is added to test whether the functionality of this NDK code can be accesses
// without needing to access the MIDI system. Added for testing purposes
jint
Java_org_drivers_midioutput_MidiDriver_version (JNIEnv *env, jobject clazz)
{
return 3;
}
// init EAS midi
jint
Java_org_drivers_midioutput_MidiDriver_init(JNIEnv *env,
jobject clazz)
{
EAS_RESULT result;
// get the library configuration
pLibConfig = EAS_Config();
if (pLibConfig == NULL || pLibConfig->libVersion != LIB_VERSION)
return 0;
// calculate buffer size
bufferSize = pLibConfig->mixBufferSize * pLibConfig->numChannels *
NUM_BUFFERS;
// init library
if ((result = EAS_Init(&pEASData)) != EAS_SUCCESS)
return 0;
// select reverb preset and enable
EAS_SetParameter(pEASData, EAS_MODULE_REVERB, EAS_PARAM_REVERB_PRESET,
EAS_PARAM_REVERB_CHAMBER);
EAS_SetParameter(pEASData, EAS_MODULE_REVERB, EAS_PARAM_REVERB_BYPASS,
EAS_FALSE);
// open midi stream
if (result = EAS_OpenMIDIStream(pEASData, &midiHandle, NULL) !=
EAS_SUCCESS)
{
EAS_Shutdown(pEASData);
return 0;
}
return bufferSize;
}
// midi config
jintArray
Java_org_drivers_midioutput_MidiDriver_config(JNIEnv *env,
jobject clazz)
{
jboolean isCopy;
if (pLibConfig == NULL)
return NULL;
jintArray configArray = (*env)->NewIntArray(env, 4);
jint *config = (*env)->GetIntArrayElements(env, configArray, &isCopy);
config[0] = pLibConfig->maxVoices;
config[1] = pLibConfig->numChannels;
config[2] = pLibConfig->sampleRate;
config[3] = pLibConfig->mixBufferSize;
(*env)->ReleaseIntArrayElements(env, configArray, config, 0);
return configArray;
}
// midi render
jint
Java_org_drivers_midioutput_MidiDriver_render(JNIEnv *env,
jobject clazz,
jshortArray shortArray)
{
jboolean isCopy;
EAS_RESULT result;
EAS_I32 numGenerated;
EAS_I32 count;
jsize size;
// jbyte* GetByteArrayElements(jbyteArray array, jboolean* isCopy)
// void ReleaseByteArrayElements(jbyteArray array, jbyte* elems,
// void* GetPrimitiveArrayCritical(JNIEnv*, jarray, jboolean*);
// void ReleasePrimitiveArrayCritical(JNIEnv*, jarray, void*, jint);
if (pEASData == NULL)
return 0;
buffer =
(EAS_PCM *)(*env)->GetShortArrayElements(env, shortArray, &isCopy);
size = (*env)->GetArrayLength(env, shortArray);
count = 0;
while (count < size) { result = EAS_Render(pEASData, buffer + count, pLibConfig->mixBufferSize, &numGenerated);
if (result != EAS_SUCCESS)
break;
count += numGenerated * pLibConfig->numChannels;
}
(*env)->ReleaseShortArrayElements(env, shortArray, buffer, 0);
return count;
}
// midi write
jboolean
Java_org_drivers_midioutput_MidiDriver_write(JNIEnv *env,
jobject clazz,
jbyteArray byteArray)
{
jboolean isCopy;
EAS_RESULT result;
jint length;
EAS_U8 *buf;
if (pEASData == NULL || midiHandle == NULL)
return JNI_FALSE;
buf = (EAS_U8 *)(*env)->GetByteArrayElements(env, byteArray, &isCopy);
length = (*env)->GetArrayLength(env, byteArray);
result = EAS_WriteMIDIStream(pEASData, midiHandle, buf, length);
(*env)->ReleaseByteArrayElements(env, byteArray, buf, 0);
if (result != EAS_SUCCESS)
return JNI_FALSE;
return JNI_TRUE;
}
// shutdown EAS midi
jboolean
Java_org_drivers_midioutput_MidiDriver_shutdown(JNIEnv *env,
jobject clazz)
{
EAS_RESULT result;
if (pEASData == NULL || midiHandle == NULL)
return JNI_FALSE;
if ((result = EAS_CloseMIDIStream(pEASData, midiHandle)) != EAS_SUCCESS)
{
EAS_Shutdown(pEASData);
return JNI_FALSE;
}
if ((result = EAS_Shutdown(pEASData)) != EAS_SUCCESS)
return JNI_FALSE;
return JNI_TRUE;
}
When the library is built by ndk-build this will prefix the compiled library with lib and replace the extension by .so. So midi.c will compile to libmidi.so. Compiled libraries are added to the download, so you needn't compile midi.c.
MidiDriver.Java declares an interface, an audioTrack and a thread to handle all these. I haven't taken the trouble to find how exactly this works. Because I didn't know how to handle an interface and such in Delphi I created a Java wrapper for MidiDriver: class MIDI_Output. This class is used to interface with Delphi.
Class MidiDriver is the interface between Java and the C-functions that call SoniVox functions. Class MIDI_Output is the interface between Java and Delphi. MIDI_Output creates an instance of MidiDriver.
Class MidiDriver - the interface with the NDK
////////////////////////////////////////////////////////////////////////////////
//
// MidiDriver - An Android Midi Driver.
//
// Copyright (C) 2013 Bill Farmer
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//
// Bill Farmer william j farmer [at] yahoo [dot] co [dot] uk.
//
///////////////////////////////////////////////////////////////////////////////
package org.drivers.midioutput;
import java.io.File;
import android.media.AudioFormat;
import android.media.AudioManager;
import android.media.AudioTrack;
import android.util.Log;
// MidiDriver
public class MidiDriver implements Runnable
{
private static final int SAMPLE_RATE = 22050;
private static final int BUFFER_SIZE = 4096;
private Thread thread;
private AudioTrack audioTrack;
private OnMidiStartListener listener;
private short buffer[];
// Constructor
public MidiDriver ()
{
Log.d ("midi", " *** MidiDriver started");
}
public void start ()
{
// Start the thread
thread = new Thread (this, "MidiDriver");
thread.start ();
} // start //
#Override
public void run ()
{
processMidi ();
} // run //
public void stop ()
{
Thread t = thread;
thread = null;
// Wait for the thread to exit
while (t != null && t.isAlive ())
Thread.yield ();
} // stop //
// Process MidiDriver
private void processMidi ()
{
int status = 0;
int size = 0;
// Init midi
Log.d ("midi", " *** processMIDI");
if ((size = init()) == 0)
return;
buffer = new short [size];
// Create audio track
audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, SAMPLE_RATE,
AudioFormat.CHANNEL_OUT_STEREO,
AudioFormat.ENCODING_PCM_16BIT,
BUFFER_SIZE, AudioTrack.MODE_STREAM);
if (audioTrack == null)
{
shutdown ();
return;
} // if
// Call listener
if (listener != null)
listener.onMidiStart();
// Play track
audioTrack.play();
// Keep running until stopped
while (thread != null)
{
// Render the audio
if (render (buffer) == 0)
break;
// Write audio to audiotrack
status = audioTrack.write (buffer, 0, buffer.length);
if (status < 0) break; } // while // Render and write the last bit of audio if (status > 0)
if (render(buffer) > 0)
audioTrack.write(buffer, 0, buffer.length);
// Shut down audio
shutdown();
audioTrack.release();
} // processMidi //
public void setOnMidiStartListener (OnMidiStartListener l)
{
listener = l;
} // setOnMidiStartListener //
public static void load_lib (String libName)
{
File file = new File (libName);
if (file.exists ())
{
System.load (libName);
} else
{
System.loadLibrary (libName);
}
} // Listener interface
public interface OnMidiStartListener
{
public abstract void onMidiStart ();
} // OnMidiStartListener //
// Native midi methods
public native int version ();
private native int init ();
public native int [] config ();
private native int render (short a []);
public native boolean write (byte a []);
private native boolean shutdown ();
// Load midi library
static
{
System.loadLibrary ("midi");
}
}
Class MIDI_Output - providing a wrap for class MidiDriver
package org.drivers.midioutput;
import java.io.File;
import java.io.FileDescriptor;
import java.io.FileInputStream;
import java.io.IOException;
import org.drivers.midioutput.MidiDriver.OnMidiStartListener;
import android.content.res.AssetFileDescriptor;
import android.media.MediaPlayer;
import android.os.Environment;
import android.util.Log;
public class MIDI_Output implements OnMidiStartListener
{
protected MidiDriver midi_driver;
protected MediaPlayer media_player;
public MIDI_Output ()
{
// Create midi driver
midi_driver = new MidiDriver();
Log.d ("midi", " *** midi_driver opened with version " +
String.valueOf (midi_driver.version ()));
// Set onmidistart listener to this class
if (midi_driver != null)
midi_driver.setOnMidiStartListener (this);
} // MIDI_Output () //
public int test_int (int n)
{
int sq = n * n;
// Log.d ("midi", " *** test_int computes " + String.valueOf (sq));
return n * n;
}
public void start ()
{
if (midi_driver != null)
{
midi_driver.start ();
Log.d ("midi", " *** midi_driver.start ()");
}
} // start //
public void stop ()
{
if (midi_driver != null)
{
midi_driver.stop ();
Log.d ("midi", " *** midi_driver.stop ()");
}
stopSong ();
} // stop //
// Listener for sending initial midi messages when the Sonivox
// synthesizer has been started, such as program change. Runs on
// the MidiDriver thread, so should only be used for sending midi
// messages.
#Override
public void onMidiStart()
{
Log.d ("midi", " *** onSMidiStart");
// TODO
}
// Sends a midi message
protected void putShort (int m, int n, int v)
{
if (midi_driver != null)
{
byte msg [] = new byte [3];
msg [0] = (byte) m;
msg [1] = (byte) n;
msg [2] = (byte) v;
Log.d ("midi", " *** putShort (" + String.valueOf (m) + ", " + String.valueOf (n) + ", " + String.valueOf (v) + ")");
midi_driver.write (msg);
} // if
} // putShort //
public boolean isPlayingSong ()
{
return media_player != null;
} // isPlayingSong //
public void playSong (String audioFilename)
{
String audioPath;
try
{
FileDescriptor fd = null;
audioFilename = "/Data/d/song.mid";
File baseDir = Environment.getExternalStorageDirectory ();
audioPath = baseDir.getAbsolutePath () + audioFilename;
Log.d ("midi", " *** Look for file: " + audioPath);
FileInputStream fis = new FileInputStream (audioPath);
fd = fis.getFD ();
if (fd != null)
{
Log.d ("midi", " *** Found file, trying to play: " + audioPath);
MediaPlayer mediaPlayer = new MediaPlayer ();
mediaPlayer.setDataSource (fd);
mediaPlayer.prepare ();
mediaPlayer.start ();
}
} catch (Exception e)
{
Log.d ("midi", " *** Exception while trying to play file: " + e.getMessage ());
}
}
public void stopSong ()
{
if (media_player != null)
{
media_player.stop ();
media_player.release ();
media_player = null;
} // if
} // stopSong //
} // Class: MIDI_Output //
From MidiDriver and MIDI_Output an Eclipse Android project was created, a MainActivity added and run. After eliminating a lot of bugs I got it up and running. A useful tool is the android debugger (adb). Open a command windows and run adb -d logcat. I have added a lot of log.d ('midi", " *** message') statements in the code in order to see where things went wrong. Remove them if you don't like them, but if you are unknown to Android (which I still am to a great extent) it is a useful way to see what happens in your application. Log works as wel in Delphi, see the Delphi Sources.
When the program compiles well you have a MIDI_Output.apk package in your project\bin directory. This package will be used by Delphi to run the Java methods.
Java can be accessed from Delphi by using JNI. A hands-on tutorial can be found on the site of RedTitan. The ideas of this tutorial were implemented in class TMIDI_Output_Device.
As you may see a constant string test_apk_fn is defined with the path to the MIDI_Output.apk Android package. This string provides JNI with the name where the Java library can be found. The string javaClassName provides the package name necessary to interface with Java. With these string the Delphi JNI is able to find the requested classes.
Class TMIDI_Output_Device - providing a Delphi wrap for Java class MIDI_Output
unit MIDI_Output_Device;
interface
uses
System.SysUtils,
FMX.Types,
Androidapi.JNIBridge,
Androidapi.JNI.JavaTypes,
Androidapi.Jni,
Androidapi.JNI.Dalvik,
Androidapi.JNI.GraphicsContentViewText;
const
test_apk_fn = '/storage/sdcard0/Data/d/MIDI_Output.apk';
type
TMIDI_Output_Device = class (TObject)
private
JavaEnv: PJNIEnv;
context: JContext;
CL: JDexClassLoader;
JavaObject: JObject;
JavaObjectID: JNIObject;
jTempClass: Jlang_Class;
jTemp: JObject;
oTemp: TObject;
jLocalInterface: ILocalObject;
optimizedpath_jfile: JFile;
dexpath_jstring, optimizedpath_jstring: JString;
fun_version: JNIMethodID;
fun_start: JNIMethodID;
fun_put_short: JNIMethodID;
fun_play_song: JNIMethodID;
public
constructor Create;
procedure setup_midi_output (class_name: string);
procedure put_short (status, data_1, data_2: integer);
procedure play_song (file_name: string);
end; // Class: MIDI_Output_Device //
implementation
uses
FMX.Helpers.Android;
constructor TMIDI_Output_Device.Create;
begin
setup_midi_output ('MIDI_Output');
end; // Create //
procedure TMIDI_Output_Device.setup_midi_output (class_name: string);
var
javaClassName: string;
ji: JNIInt;
jiStatus, jiData_1, jiData_2: JNIValue;
begin
javaClassName := Format ('org.drivers.midioutput/%s', [class_name]);
context := SharedActivityContext;
JavaEnv := TJNIResolver.GetJNIEnv;
Log.d ('Loading external library from "' + test_apk_fn + '"');
dexpath_jstring := StringToJString (test_apk_fn);
// locate/create a directory where our dex files can be put
optimizedpath_jfile := context.getDir (StringToJString ('outdex'), TJContext.javaclass.mode_private);
optimizedpath_jstring := optimizedpath_jfile.getAbsolutePath;
Log.d ('Path for DEX files = ' + JStringToString (optimizedpath_jstring));
Log.d ('APK containing target class = ' + JStringToString (dexpath_jstring));
CL := TJDexClassLoader.JavaClass.init (dexpath_jstring, optimizedpath_jstring, nil, TJDexClassLoader.JavaClass.getSystemClassLoader);
// Test whether the Dex class is loaded, if not, exit
if not assigned (CL) then
begin
Log.d ('?Failed to get DEXClassLoader');
exit;
end; // if
// Load the Java class
jTempClass := CL.loadClass (StringToJString (javaClassName));
if assigned (jTempClass) then
begin
jTemp := jTempClass; // N.B You could now import the entire class
if jTemp.QueryInterface (ILocalObject,jLocalInterface) = S_OK then
begin
// supports ilocalobject
JavaObject := jTempClass.newInstance;
oTemp := JavaObject as TObject;
JavaObjectID := tjavaimport (otemp).GetObjectID;
Log.d (oTemp.ClassName);
// try to access the version function from the midi_output class
fun_version := TJNIResolver.GetJavaMethodID ((jTempClass as ILocalObject).GetObjectID, 'version', '()I');
if not assigned (fun_version) then
begin
Log.d ('?fun_version not supported');
end else
begin
ji := JavaEnv^.CallIntMethodA (JavaEnv, JavaObjectID, fun_version, nil);
Log.d ('version returns ' + inttostr (ji));
end; // if
// try to access the start function from the midi_output class
fun_start := TJNIResolver.GetJavaMethodID ((jTempClass as ILocalObject).GetObjectID, 'start', '()V');
if not assigned (fun_start) then
begin
Log.d ('?fun_start not supported');
end else
begin
JavaEnv^.CallVoidMethodA (JavaEnv, JavaObjectID, fun_start, nil);
Log.d ('fun_start found');
end; // if
// try to access the putShort function from the midi_output class
fun_put_short := TJNIResolver.GetJavaMethodID ((jTempClass as ILocalObject).GetObjectID, 'putShort','(III)V');
if not assigned (fun_put_short) then
begin
Log.d ('?putShort not supported');
end else
begin
Log.d (Format (' ### putShort (%d, %d, %d)', [jiStatus.i, jiData_1.i, jiData_2.i]));
put_short ($90, 60, 127);
end; // if
// try to access the playSong function from the midi_output class
fun_play_song := TJNIResolver.GetJavaMethodID ((jTempClass as ILocalObject).GetObjectID, 'playSong', '(Ljava/lang/String)V');
if not assigned (fun_play_song) then
begin
Log.d ('?playSong not supported');
end else
begin
Log.d (' ### playSong found');
end; // if
end else
begin
Log.d ('?Could not derive ILOCALOBJECT');
end;
end else Log.d ('?'+javaClassname+' not found')
end; // setup_midi_output //
procedure TMIDI_Output_Device.put_short (status, data_1, data_2: integer);
var
jiStatus, jiData_1, jiData_2: JNIValue;
x: array of JNIOBJECT;
begin
jiStatus.i := status;
jiData_1.i := data_1;
jiData_2.i := data_2;
setLength (x, 3);
x [0] := jiStatus.l;
x [1] := jiData_1.l;
x [2] := jiData_2.l;
Log.d (Format ('putShort (%d, %d, %d)', [jiStatus.i, jiData_1.i, jiData_2.i]));
JavaEnv^.CallVoidMethodV (JavaEnv, JavaObjectID, fun_put_short, x);
end; // put_short //
procedure TMIDI_Output_Device.play_song (file_name: string);
var
x: array of JNIObject;
begin
SetLength (x, 1);
x [0] := StringToJNIString (JavaEnv, file_name);
Log.d ('playSong (' + file_name + ')');
JavaEnv^.CallVoidMethodV (JavaEnv, JavaObjectID, fun_play_song, x);
end; // playSong //
end. // Unit: MIDI_Output_Device //
Delphi now knows where to find the Java classes. In theory it should now be able to find libmidi.so because an Android package is a .zip file containing the necessary files to run the Java package. If you open MIDI_Output.apk with WinZip or WinRar then you see these files. In the archive you'll find a directory lib which contains libmidi.so for the ARM 5 and 7 platforms. When launching the program and having adb -d logcat running in a command window adb says so much as unpacking MIDI_Output.apk. Well, it might do so, but libmidi.so will not be found.
Libmidi.so should be added to the \usr\lib somewhere under the \platforms directory of the Android SDK. The complete link in my case is: C:\Users\Public\Documents\RAD Studio\12.0\PlatformSDKs\android-ndk-r8e\platforms\android-14\arch-arm\usr\lib. This should help as I found out some time ago.
Using the call chain as I have shown here one may call MIDI functions in Delphi generated Android code. There are some questions regarding this technique:
Wouldn't it be easier to call the NDK function directly? It is
possible to call NDK functions directly from Delphi in the same
way as DLL's. However, class MidiDriver adds a lot of functionality
which I do not understand at this moment. This functionality must be
programmed in C or Pascal when call the NDK functions directly.
In the code from Bill Farmer he uses the MediaPlayer to play MIDI
files. Alas the MediaPlayer can only be accessed from an Activity and
I do not know how to transfer the Delphi MainActivity to a JNI Java
function. So this functionality does not work as of yet.
Native libraries are packed into the .apk but not unpacked in such a
way that the JavaVM detects it. Now the libmidi.so has to be put
manually into \usr\lib.
Even worse is that a hard link must be added to the .apk package. The
package should be deployed automatically to the /data/app-lib of the
application, else creating an app with JNI classes and installing it
from the Play Store seems impossible.
Another way would have been to use a native Android version of BASS plus BASSMIDI plugin. There is good sample code coming with it. And Ian's support is excellent. You can find both of them here:
BASS lib for Android: http://www.un4seen.com/forum/?topic=13225
BASSMIDI plugin (d/l): http://www.un4seen.com/download.php?bassmidi24-linux
There is a .NET version on his site as well, and a 3rd party project on Sourceforge that exposes the API to Java. I'm not allowed to post more than two links (yet) but you can find it with a quick search for nativebass
Fairly late answer but it might still help someone else looking for a quicker way or one that works in Delphi alone.
I have only been able to find solutions dated 2010 and earlier. So I wanted to see if there was a more up-to-date stance on this.
I'd like to avoid using Java and purely use C++, to access files (some less-or-more than 1MB) stored away in the APK. Using AssetManager means I can't access files like every other file on every other operating system (including iOS).
If not, is there a method in C++ where I could somehow map fopen/fread to the AssetManager APIs?
I actually found pretty elegant answer to the problem and blogged about it here.
The summary is:
The AAssetManager API has NDK bindings. This lets you load assets from the APK.
It is possible to combine a set of functions that know how to read/write/seek against anything and disguise them as a file pointer (FILE*).
If we create a function that takes an asset name, uses AssetManager to open it, and then disguises the result as a FILE* then we have something that's very similar to fopen.
If we define a macro named fopen we can replace all uses of that function with ours instead.
My blog has a full write up and all the code you need to implement in pure C. I use this to build lua and libogg for Android.
Short answer
No. AFAIK mapping fread/fopen in C++ to AAssetManager is not possible. And if were it would probably limit you to files in the assets folder. There is however a workaround, but it's not straightforward.
Long Answer
It IS possible to access any file anywhere in the APK using zlib and libzip in C++.
Requirements : some java, zlib and/or libzip (for ease of use, so that's what I settled for). You can get libzip here: http://www.nih.at/libzip/
libzip may need some tinkering to get it to work on android, but nothing serious.
Step 1 : retrieve APK location in Java and pass to JNI/C++
String PathToAPK;
ApplicationInfo appInfo = null;
PackageManager packMgmr = parent.getPackageManager();
try {
appInfo = packMgmr.getApplicationInfo("com.your.application", 0);
} catch (NameNotFoundException e) {
e.printStackTrace();
throw new RuntimeException("Unable to locate APK...");
}
PathToAPK = appInfo.sourceDir;
Passing PathToAPK to C++/JNI
JNIEXPORT jlong JNICALL Java_com_your_app(JNIEnv *env, jobject obj, jstring PathToAPK)
{
// convert strings
const char *apk_location = env->GetStringUTFChars(PathToAPK, 0);
// Do some assigning, data init, whatever...
// insert code here
//release strings
env->ReleaseStringUTFChars(PathToAPK, apk_location);
return 0;
}
Assuming that you now have a std::string with your APK location and you have zlib on libzip working you can do something like this:
if(apk_open == false)
{
apk_file = zip_open(apk_location.c_str(), 0, NULL);
if(apk_file == NULL)
{
LOGE("Error opening APK!");
result = ASSET_APK_NOT_FOUND_ERROR;
}else
{
apk_open = true;
result = ASSET_NO_ERROR;
}
}
And to read a file from the APK:
if(apk_file != NULL){
// file you wish to read; **any** file from the APK, you're not limited to regular assets
const char *file_name = "path/to/file.png";
int file_index;
zip_file *file;
struct zip_stat file_stat;
file_index = zip_name_locate(apk_file, file_name, 0);
if(file_index == -1)
{
zip_close(apk_file);
apk_open = false;
return;
}
file = zip_fopen_index(apk_file, file_index, 0);
if(file == NULL)
{
zip_close(apk_file);
apk_open = false;
return;
}
// get the file stats
zip_stat_init(&file_stat);
zip_stat(apk_file, file_name, 0, &file_stat);
char *buffer = new char[file_stat.size];
// read the file
int result = zip_fread(file, buffer, file_stat.size);
if(result == -1)
{
delete[] buffer;
zip_fclose(file);
zip_close(apk_file);
apk_open = false;
return;
}
// do something with the file
// code goes here
// delete the buffer, close the file and apk
delete[] buffer;
zip_fclose(file);
zip_close(apk_file);
apk_open = false;
Not exactly fopen/fread but it gets the job done. It should be pretty easy to wrap this to your own file reading function to abstract the zip layer.
My question is pretty simple, but I am having a hard time finding any info about this online.
Is it possible to use ifstream to open a file from assets and/or resources using Android NDK?
For example, placing a test.txt file in /assets and trying the following does not work:
char pLine[256];
std::ifstream fin("/assets/test.txt");
if(!fin.fail())
{
LOGD( "test.txt opened" );
while( !fin.eof() )
{
fin.getline( pLine, 256 );
LOGD(pLine);
}
}
else
{
LOGD( "test.txt FAILED TO OPEN!" );
}
fin.close();
Nor does any variable of:
std::ifstream fin("assets/test.txt");
std::ifstream fin("test.txt");
Etc..., nor placing it in /res instead.
So, is it possible to use normal ifstream operators to access assets and or resource files?
It is right that std::ifstream cannot be used, but one can create an assetistream that could be used in a similar way. For example:
class asset_streambuf: public std::streambuf
{
public:
asset_streambuf(AAssetManager* manager, const std::string& filename)
: manager(manager)
{
asset = AAssetManager_open(manager, filename.c_str(), AASSET_MODE_STREAMING);
buffer.resize(1024);
setg(0, 0, 0);
setp(&buffer.front(), &buffer.front() + buffer.size());
}
virtual ~asset_streambuf()
{
sync();
AAsset_close(asset);
}
std::streambuf::int_type underflow() override
{
auto bufferPtr = &buffer.front();
auto counter = AAsset_read(asset, bufferPtr, buffer.size());
if(counter == 0)
return traits_type::eof();
if(counter < 0) //error, what to do now?
return traits_type::eof();
setg(bufferPtr, bufferPtr, bufferPtr + counter);
return traits_type::to_int_type(*gptr());
}
std::streambuf::int_type overflow(std::streambuf::int_type value) override
{
return traits_type::eof();
};
int sync() override
{
std::streambuf::int_type result = overflow(traits_type::eof());
return traits_type::eq_int_type(result, traits_type::eof()) ? -1 : 0;
}
private:
AAssetManager* manager;
AAsset* asset;
std::vector<char> buffer;
};
class assetistream: public std::istream
{
public:
assetistream(AAssetManager* manager, const std::string& file)
: std::istream(new asset_streambuf(manager, file))
{
}
assetistream(const std::string& file)
: std::istream(new asset_streambuf(manager, file))
{
}
virtual ~assetistream()
{
delete rdbuf();
}
static void setAssetManager(AAssetManager* m)
{
manager = m;
}
private:
static AAssetManager* manager;
};
void foo(AAssetManager* manager)
{
assetistream::setAssetManager(manager);
assetistream as("text/tmp.txt");
std::string s;
std::getline(as, s);
}
Improvements are very welcome.
No, you cannot. Assets are stored within the apk, a zip file. ifstream cannot read within the zip file.
To access these files you either need to access them in java and save them elsewhere or extract the contents of the apk to get to the assets.
Here is an example of doing the former.
http://www.itwizard.ro/android-phone-installing-assets-how-to-60.html
Here is an example of doing the latter.
http://www.anddev.org/ndk_opengl_-_loading_resources_and_assets_from_native_code-t11978.html