I want to allow user to convert .mid files to .wav files in my Android app.
Actually, there is not plenty of information on the Net about any kind of midi to wav conversion, and so there is very litlle info about doing so in Android.
What should I do? Where to go?
TL;DR
Dependless on the way featured in this answer you pick, you'll have to include some c code in your project. And also you'll have to include lib lisenced under LGPL-2.1. You should really pay attention to both of these. Read this to understand it better. Really. Do read it. It IS important to understand.
Lazy option - use my lib
Better option - set FluidSynth up following either of the links (official github page guide and\or Medium article), include this snippet of code into your project and you're good to go. This code should be included into your project like it is explained in the Medium article listed above.
Very bad option - use Timidity++.
More detailed version
Not recommended option
There are two options when you convert mid files: FluidSynth and Timidity++. I wouldn't recommend you to use Timidity++ for purpose of converting .mid to raw audio. The lib is kind of old and it's not supported; docs and community are imposible to find. FluidSynth is a much better choice: it's newer, it's supported, it's gotta plenty of API docs, its community is kinda lil more alive than Timidity's is. I couldn't get Timidity working on Android.
Anyway, here are some links in case you'd like to use Timidity anyway.
somewhat usable timidity lib
example of converting mid to wav
a working app featuring timidity (I couldn't get anything going on in this code, but this app uses timidity and it's able to convert mid to wav, it's a fact)
It's not a complete list of timidity resources. Also, there are a few mysterious repos on the web that claim to be either timidity or some kind of timidity lib or even lib for android... But personally I never understood what was going on there, and so I don't recommend to go for timidity.
Lazy option
The laziest option is to use my own library. All the instructions are in the Github readme. Using my lib is not recommended way to do it, probably, since my implementation can have some big performance issues.
You can see examples of using this approach in the lib's readme.
Better option
The best option to convert .mid to .wav file is to use FluidSynth software synthesizer. This way you'll have to do some c codding. I told you.
The official github wiki got the instruction on how to set FluidSynth for Android, but I'd suggest you to read this Medium article about configuring this synthesizer instead, 'cause it's a lot more easier to follow and understand.
After you've set this thing up, you can do some neat .mid to .wav conversion. Here is the official docs for that. Gonna leave the code here in case the link goes down.
fluid_settings_t* settings;
fluid_synth_t* synth;
fluid_player_t* player;
fluid_file_renderer_t* renderer;
settings = new_fluid_settings();
// specify the file to store the audio to
// make sure you compiled fluidsynth with libsndfile to get a real wave file
// otherwise this file will only contain raw s16 stereo PCM
fluid_settings_setstr(settings, "audio.file.name", "/path/to/output.wav");
// use number of samples processed as timing source, rather than the system timer
fluid_settings_setstr(settings, "player.timing-source", "sample");
// since this is a non-realtime scenario, there is no need to pin the sample data
fluid_settings_setint(settings, "synth.lock-memory", 0);
synth = new_fluid_synth(settings);
// *** loading of a soundfont omitted ***
player = new_fluid_player(synth);
fluid_player_add(player, "/path/to/midifile.mid");
fluid_player_play(player);
renderer = new_fluid_file_renderer (synth);
while (fluid_player_get_status(player) == FLUID_PLAYER_PLAYING)
{
if (fluid_file_renderer_process_block(renderer) != FLUID_OK)
{
break;
}
}
// just for sure: stop the playback explicitly and wait until finished
fluid_player_stop(player);
fluid_player_join(player);
delete_fluid_file_renderer(renderer);
delete_fluid_player(player);
delete_fluid_synth(synth);
delete_fluid_settings(settings);
... and basically that is it. You can start converting .mid to .wav files right now.
Here is the example on how to integrate that code in your project.
Here is the example how to use this function in your android code.
Related
What are the standard ways of reading and writing audio files on Android / Kotlin?
I am very confused. I've found plenty of posts that discuss this at some level, but they're all either giving a third party answer (someone's own implementation like https://medium.com/#rizveeredwan/working-with-wav-files-in-android-52e9500297e or https://stackoverflow.com/a/43569709/4959635 or https://gist.github.com/kmark/d8b1b01fb0d2febf5770) or using some Java class, of which I don't know how it's related to the Android SDK (https://stackoverflow.com/a/26598862/4959635, https://gist.github.com/niusounds/3e49013a8e942cdba3fbfe1c336b61fc, https://github.com/google/oboe/issues/548#issuecomment-502758633).
I cannot find a standard way from the Android documentation. Some answer said to use https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/read-bytes.html for reading, but I'm quite sure this doesn't parse the file header.
So what's the standard way of processing audio files on Android / Kotlin?
I'm already using dr_wav just fine on desktop, so I am actually thinking of just using that through NDK and maybe creating a wrapper to it.
Your use case is not clear from the question.
Assuming that you need to process raw audio data (PCM samples) - the standard way is to read the (compressed) input file using the MediaExtractor and decode the packets using the MediaCodec. Note that the documentation includes some example code.
The MediaCodec outputs ByteBuffers containing raw PCM samples. The binary format is described here.
Well, there is no strict standard.
In production, you usually choose stable third party library or your company's reusable internal solution for this kind of tasks. You still can implement it yourself, but it will cost you time, since most likely the implementation will consist of hundreds of lines of code and you probably will just create another variation of existing solution which is present on the internet.
I'm writing an Android app that has to perform audio processing (more specifically, MFCC). I have decided to use the TarsosDSP library, but it does not provide a way to pass a sound file to it, as opposed to microphone input. This means that we currently have to write a class that implements a specific interface: basically, it's a wrapper around a buffer of doubles.
Do you know of a way to get the samples as a double from a music file format that Android supports ? We've been scouring the net for a solution but we haven't found one that is generic enough.
Thanks for the help !
I had the same problem for a while and found this:
https://0110.be/posts/Decode_MP3s_and_other_Audio_formats_the_easy_way_on_Android
You can still use the pipe in android. The code in the link allows you pass audio files directly into the decoder. It also contains the ffmpeg binaries required for the assets.
the link takes you through the steps. Good luck.
I am working on a matlab project where I add effects to audio files (mp3, wav). Therefore, I load the files into arrays using the matlab function audioread(..).
Now, I want to export this to Android. I read that the best way is to use the Matlab Coder to export the matlab code to C/C++ (or Java) and then export it into android (more or less).
However, the function call audioplayer (and play) are Unsupported (that's what the code generation readiness issues says).
What can I do ? One idea was to play the sounds directly using c++ code (so after the code generation). But how to play sounds from arrays using c++ ?
Or if you guys have others ideas without touching c++ codes (so fixing the problem directly in matlab), I would be glad to hear it !
Thanks and have a good day !
Typically what I recommend in cases like this is to factor your code in two pieces:
The part that does the audio file I/O and audio playing (namely the OS-specific part)
The computational kernel for which you will generate code using MATLAB Coder. This piece usually takes numeric arrays representing the image or audio data as arguments.
I've used this approach to leverage MATLAB Coder generated code to do image filtering on Android.
To do part (1), as Navan says, you'll need to use Android APIs to read in audio files, write data back to files, and to play them as desired. Note, I haven't done extensive Android development, so doing these tasks may take some research or be difficult.
Once you have the data in a format suitable for the function(s) in (2), likely a numeric array, then you can call your generated code using JNI to add the desired effects. The generated code would return the data back to the Java code and you can then encode it, play it, or do as you please with it using the Android APIs.
Playing audio normally uses platform dependent libraries. In DSP System toolbox, there is an audio player object called dsp.AudioPlayer which supports C code generation. But I believe this uses platform dependent libraries in the generated code and it will not be straight forward to make it work in Android. You will be better off finding an audio player library for Android and hooking that in manually after generating code.
I have to modify the Http Live Streaming implementation of Android Media Player.
The implementation is under the stagefright library
http://androidxref.com/4.0.4/xref/frameworks/base/media/libstagefright/httplive/LiveDataSource.cpp
I think these library will compile to a libstagefright.so which should be part of the Android system.
My question is if I make some changes to this library and compile a new libstagefright.so.
If I load this new libstagefright.so in my new application and call up the media player, will it use the code in my new libstagefright.so?
You will not be able to replace the original library, since when you try to loadLibrary it will load the library from within /system/lib. So unless you replace that (which is not possible on unrooted devices), you won't be able to load your custom code.
https://github.com/android/platform_system_core/blob/66ed50af6870210ce013a5588a688434a5d48ee9/rootdir/init.environ.rc.in sets the LD_LIBRARY_PATH by default. And loads it from these paths if available. If not, then your application's lib directory will be searched; but not the other way around.
I tried this with libwebkit.so in the past on various mainstream devices and haven't had any luck getting it to load instead of the one in /system/lib.
You can learn more by looking at:
doLoad from here https://android.googlesource.com/platform/libcore/+/41d00b744b7772f9302fdb94dddadb165b951220/luni/src/main/java/java/lang/Runtime.java
findLibrary here http://developer.android.com/reference/dalvik/system/BaseDexClassLoader.html#findLibrary(java.lang.String)
I'm pretty sure you can't replace the default class loader either for security reasons.
What you can do, though, is a straightforward fork the Media Player and have it load your modified libstagefright-modified.so. There could be other solutions, haven't looked at Media Player's code.
Knowing that all you want to do is parse the data before it gets to the MediaPlayer, I suggest not trying to alter the Android libraries. As soulseekah mentioned, it's not going to work without a rooted device. There are other options, although they both have drawbacks.
1) If you are only targeting recent versions (4.2 or later, I believe), you can take a look at new classes added to the android.media package, like MediaExtractor and MediaCodec. I'm not greatly familiar with those because they aren't available on the hardware with which I work, but they could be useful in getting to the raw data. Here is a decent sample of using them to play video. The drawback is those classes aren't available in earlier versions.
2) The other option is to put a local proxy on the device. Connect the MediaPlayer to the proxy and make the request to the media server yourself. See my answer here for a little more info on that. With a proxy, you will see all the data that comes through, giving you a chance to parse the ID3 tags. There is the drawback that you will have to parse the TS packets to put together an elementary stream (essentially doing the demuxer's job), but it will work with any version of Android. TS streams aren't difficult to disassemble, and ID3 tags aren't time consuming to parse, so I think this is a reasonable approach.
I have a very basic question regarding Android and ffmpeg. I obtained ffmpeg from http://bambuser.com/opensource and was able to compile it for ARM.
The results are the binaries (ffmpeg) as well as several libsomething.so files.
My question is: Is this enough to decode videos? How do I actually use ffmpeg then?
To load the library I have:
static {
System.load("/data/data/com.package/lib/libavcodec.so");
}
It loads fine. But what then?
More explanation: I saw other projects where people had their ffmpeg source in a JNI directory in the project. They also created some Android.mk files and some C code along with it. Would I need this as well? Why would I create the .so files first and then copy the ffmpeg source code again?
I know the NDK and how it should work but I've never seen an example of how one would actually call ffmpeg functions using it, because people seem to be hiding their implementations (which is sort of understandable) but not even giving useful pointers or examples.
Let's just say I wanted to decode a video file. Which kind of native methods would I need to implement? How do I run the project? Which data types need to be passed? etc. There are certainly a few people here who have at least done that, I know this from searching for hours and hours.
For your first question;
Just building is not enough for the proper use of the ffmpeg libraries. You should also wrap those so files in the right order because these so files NEED other libraries in the link time. You can display header information of the so file, by using.
objdump -x libavcodec.so | grep NEEDED
So you need to wrap these so files through Android.mk. You may check this link.
The second one;
You only need the header files from the ffmpeg project. The implementation will linked from the so libraries. Thats perhaps because, developers didn't bother to filter header files.
And the last one;
your thoughts seems right for the time being, most of the current developers are struggling to use ffmpeg but they lack of documentation and sample codes.