I am developing a Unity3D app to be deployed on Android.
I need to read midi files (preferrably as .midi, but other endings would be fine too) and play them (get the parsed note events at the correct timing).
Actual sound is not a priority.
It is also very important for me to be able to change the tempo.
Can you recommend a framework/ plugin/ code for this?
All the stuff i found does not work on Android, regarding C#SynthProject i could not figure out how to change the tempo.
Getting the sound is somewhat problematic (as you have noticed) as you need to emulate hardware chip that used to do the GeneralMIDI synthesis in some form or another, that is no longer commonly mounted anywhere (even windows emulates this internally), but midi file format (also known as StandardMidiFile aka SMF) files themselves are fairly well documented standard and are not terribly complex (although two non-compatible variants are a pain sometimes)
While writing own .MID file parser is probably a lot of fun, I can recommend using this piece of code dated 2013, written by our Unity god - Keijiro Takahashi
https://github.com/keijiro/smflite
Related
What are the standard ways of reading and writing audio files on Android / Kotlin?
I am very confused. I've found plenty of posts that discuss this at some level, but they're all either giving a third party answer (someone's own implementation like https://medium.com/#rizveeredwan/working-with-wav-files-in-android-52e9500297e or https://stackoverflow.com/a/43569709/4959635 or https://gist.github.com/kmark/d8b1b01fb0d2febf5770) or using some Java class, of which I don't know how it's related to the Android SDK (https://stackoverflow.com/a/26598862/4959635, https://gist.github.com/niusounds/3e49013a8e942cdba3fbfe1c336b61fc, https://github.com/google/oboe/issues/548#issuecomment-502758633).
I cannot find a standard way from the Android documentation. Some answer said to use https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/read-bytes.html for reading, but I'm quite sure this doesn't parse the file header.
So what's the standard way of processing audio files on Android / Kotlin?
I'm already using dr_wav just fine on desktop, so I am actually thinking of just using that through NDK and maybe creating a wrapper to it.
Your use case is not clear from the question.
Assuming that you need to process raw audio data (PCM samples) - the standard way is to read the (compressed) input file using the MediaExtractor and decode the packets using the MediaCodec. Note that the documentation includes some example code.
The MediaCodec outputs ByteBuffers containing raw PCM samples. The binary format is described here.
Well, there is no strict standard.
In production, you usually choose stable third party library or your company's reusable internal solution for this kind of tasks. You still can implement it yourself, but it will cost you time, since most likely the implementation will consist of hundreds of lines of code and you probably will just create another variation of existing solution which is present on the internet.
I am working on a matlab project where I add effects to audio files (mp3, wav). Therefore, I load the files into arrays using the matlab function audioread(..).
Now, I want to export this to Android. I read that the best way is to use the Matlab Coder to export the matlab code to C/C++ (or Java) and then export it into android (more or less).
However, the function call audioplayer (and play) are Unsupported (that's what the code generation readiness issues says).
What can I do ? One idea was to play the sounds directly using c++ code (so after the code generation). But how to play sounds from arrays using c++ ?
Or if you guys have others ideas without touching c++ codes (so fixing the problem directly in matlab), I would be glad to hear it !
Thanks and have a good day !
Typically what I recommend in cases like this is to factor your code in two pieces:
The part that does the audio file I/O and audio playing (namely the OS-specific part)
The computational kernel for which you will generate code using MATLAB Coder. This piece usually takes numeric arrays representing the image or audio data as arguments.
I've used this approach to leverage MATLAB Coder generated code to do image filtering on Android.
To do part (1), as Navan says, you'll need to use Android APIs to read in audio files, write data back to files, and to play them as desired. Note, I haven't done extensive Android development, so doing these tasks may take some research or be difficult.
Once you have the data in a format suitable for the function(s) in (2), likely a numeric array, then you can call your generated code using JNI to add the desired effects. The generated code would return the data back to the Java code and you can then encode it, play it, or do as you please with it using the Android APIs.
Playing audio normally uses platform dependent libraries. In DSP System toolbox, there is an audio player object called dsp.AudioPlayer which supports C code generation. But I believe this uses platform dependent libraries in the generated code and it will not be straight forward to make it work in Android. You will be better off finding an audio player library for Android and hooking that in manually after generating code.
I'd like to load and play '.mid' MIDI files in Kivy but mainly on mobile platforms (i.e. IOS and Android).
I'm assuming that, I don't want to get into the intricacies of native-C/C++ based stuff (i.e. to avoid have to cross-compile, I'm not a seasoned Python dev), so the library, most probably, would need to be pure Python.
I'll start the bidding with some Frameworks I've found (some require C/C++), I've used none (0, zero), maybe someone could offer some feedback (or they might come in useful for the wider community):
BTW, it would be great if the/a MIDI library could also read the lyrics from a '.mid' file
Python Midi libraries
http://web.mit.edu/music21
https://github.com/cuthbertLab/music21
https://code.google.com/p/midiutil <- Pure Python, no lyrics (?)
https://code.google.com/p/mingus
http://mxm.dk/products/public/pythonmidi
http://das.nasophon.de/mididings
https://github.com/vishnubob/python-midi
https://gist.github.com/anonymous/202595 <- smidi.py
http://webcache.googleusercontent.com/search?q=cache%3Alarndham.net%2Fservice%2Fpys60%2Fsmidi.py
https://pypi.python.org/pypi/midi <- C for Alsa (!)
http://www.fluidsynth.org <- For pypi midi
http://sourceforge.net/p/fluidsynth <- C based
I'm using Fluidsynth to give me cross-platform MIDI output. It plays MIDI files with a SoundFont module for high quality audio. If you're happy with the LGPL licence, you can get Python bindings for it from pyFluidSynth (https://code.google.com/p/pyfluidsynth), but it's pretty straightforward to write your own bindings module using the standard Python ctypes module.
Normally, you would be expected to compile (and thus cross-compile) Fluidsynth from the supplied sources, however you can just take a pre-compiled version from another project, for instance MuseScore. This is what I'm doing for the moment, I'll switch to compiling Fluidsynth once (and if) the project I'm working on gets close to completion.
If you want to process MIDI files, e.g. to extract the lyrics, take a look at Mido (https://mido.readthedocs.org/en/latest). I've found it very useful for reading/writing MIDI files.
I have to modify the Http Live Streaming implementation of Android Media Player.
The implementation is under the stagefright library
http://androidxref.com/4.0.4/xref/frameworks/base/media/libstagefright/httplive/LiveDataSource.cpp
I think these library will compile to a libstagefright.so which should be part of the Android system.
My question is if I make some changes to this library and compile a new libstagefright.so.
If I load this new libstagefright.so in my new application and call up the media player, will it use the code in my new libstagefright.so?
You will not be able to replace the original library, since when you try to loadLibrary it will load the library from within /system/lib. So unless you replace that (which is not possible on unrooted devices), you won't be able to load your custom code.
https://github.com/android/platform_system_core/blob/66ed50af6870210ce013a5588a688434a5d48ee9/rootdir/init.environ.rc.in sets the LD_LIBRARY_PATH by default. And loads it from these paths if available. If not, then your application's lib directory will be searched; but not the other way around.
I tried this with libwebkit.so in the past on various mainstream devices and haven't had any luck getting it to load instead of the one in /system/lib.
You can learn more by looking at:
doLoad from here https://android.googlesource.com/platform/libcore/+/41d00b744b7772f9302fdb94dddadb165b951220/luni/src/main/java/java/lang/Runtime.java
findLibrary here http://developer.android.com/reference/dalvik/system/BaseDexClassLoader.html#findLibrary(java.lang.String)
I'm pretty sure you can't replace the default class loader either for security reasons.
What you can do, though, is a straightforward fork the Media Player and have it load your modified libstagefright-modified.so. There could be other solutions, haven't looked at Media Player's code.
Knowing that all you want to do is parse the data before it gets to the MediaPlayer, I suggest not trying to alter the Android libraries. As soulseekah mentioned, it's not going to work without a rooted device. There are other options, although they both have drawbacks.
1) If you are only targeting recent versions (4.2 or later, I believe), you can take a look at new classes added to the android.media package, like MediaExtractor and MediaCodec. I'm not greatly familiar with those because they aren't available on the hardware with which I work, but they could be useful in getting to the raw data. Here is a decent sample of using them to play video. The drawback is those classes aren't available in earlier versions.
2) The other option is to put a local proxy on the device. Connect the MediaPlayer to the proxy and make the request to the media server yourself. See my answer here for a little more info on that. With a proxy, you will see all the data that comes through, giving you a chance to parse the ID3 tags. There is the drawback that you will have to parse the TS packets to put together an elementary stream (essentially doing the demuxer's job), but it will work with any version of Android. TS streams aren't difficult to disassemble, and ID3 tags aren't time consuming to parse, so I think this is a reasonable approach.
I'm porting a rather large game engine written in C++ from Windows/Mac to Android. There is a lot of pre-existing code to read assets for games. In addition, there is quite a bit of code doing file system calls (stat'ing the files to make sure they exist, looking up all of the files and directories inside of a directory, etc.)
Right now, I'm focusing on just getting something up and running as quickly as possible, so I'd prefer not to have to rewrite a lot of this. What would be a good way of getting our game assets onto the device and accessing them with minimal changes to our existing standard C++ file system API usage?
I've got some basic support implemented already using the Asset Manager API, but that doesn't support the file system calls and I'm concerned that the 1 MB asset size limit is going to bite me at some point.
I've also looked at OBB, but the tools for creating an OBB file don't look like they are part of the current SDK/NDK. Otherwise, that looks like it would be perfect.
Is it a horrible idea to package up all of the files and just extract them on the SD Card the first time the app is run? Or is there some better way of dealing with this?
Update: I'm also not very concerned on being able to run on a broad range of devices, I am specifically looking at newish tablets, probably the 10.1" Samsung Galaxy tab.
We ran into a similar problem in developing our (data-file-heavy) app, and we ended up deciding to keep the APK tiny and simply download our data files on first run; they're going to have to be downloaded either way, but a small APK works much better on older devices without a lot of internal storage. Plus, you can potentially rig up a way for people to copy over the files directly from their computer if they have a limited data plan or a slow internet connection on their phone.
The "Downloader" sample app in apps-for-android (confusingly buried under "Samples") is almost a fully-implemented solution for this - you can pretty much just plug in the particulars of your data files and let it do the rest.
I wrote an app that relies on putting a good amount of native code into the Android filesystem. I did this by packaging the files into the APK as 'resources'. Instead of pushing them to the SD card, you can put then into the application's private namespace, I.E. /data/data/com.yourdomain.yourapp/nativeFolder.
For details on how to accomplish this, you can see my answer to this question.
It's fairly simple to package to just unpack them on the first run and never worry about them again. Also, since they're under the application's namespace, they should be deleted if/when someone were to decide to delete your app.
EDIT:
This method can be used to put anything into the app's private area; /data/data/com.yourdomain.yourapp/
However, as far as I know, your application has to be the one to create all the folders and sub-folders in this area. Luckily this is fairly easy to do. For example to have your app make a folder:
Process mkdir = Runtime.getRuntime().exec("mkdir " +localPath);
That works as it would in most linux shells. I walked through the assets folder I packaged into my APK, made the corresponding directories and copied all the native files to those directories.
What you might be more concerned with is the limited Android shell. There are many commands that you might want that aren't present. stat for example isn't available, so all of this may be moot if your native code can't make it's system calls.