I am working on a matlab project where I add effects to audio files (mp3, wav). Therefore, I load the files into arrays using the matlab function audioread(..).
Now, I want to export this to Android. I read that the best way is to use the Matlab Coder to export the matlab code to C/C++ (or Java) and then export it into android (more or less).
However, the function call audioplayer (and play) are Unsupported (that's what the code generation readiness issues says).
What can I do ? One idea was to play the sounds directly using c++ code (so after the code generation). But how to play sounds from arrays using c++ ?
Or if you guys have others ideas without touching c++ codes (so fixing the problem directly in matlab), I would be glad to hear it !
Thanks and have a good day !
Typically what I recommend in cases like this is to factor your code in two pieces:
The part that does the audio file I/O and audio playing (namely the OS-specific part)
The computational kernel for which you will generate code using MATLAB Coder. This piece usually takes numeric arrays representing the image or audio data as arguments.
I've used this approach to leverage MATLAB Coder generated code to do image filtering on Android.
To do part (1), as Navan says, you'll need to use Android APIs to read in audio files, write data back to files, and to play them as desired. Note, I haven't done extensive Android development, so doing these tasks may take some research or be difficult.
Once you have the data in a format suitable for the function(s) in (2), likely a numeric array, then you can call your generated code using JNI to add the desired effects. The generated code would return the data back to the Java code and you can then encode it, play it, or do as you please with it using the Android APIs.
Playing audio normally uses platform dependent libraries. In DSP System toolbox, there is an audio player object called dsp.AudioPlayer which supports C code generation. But I believe this uses platform dependent libraries in the generated code and it will not be straight forward to make it work in Android. You will be better off finding an audio player library for Android and hooking that in manually after generating code.
Related
What are the standard ways of reading and writing audio files on Android / Kotlin?
I am very confused. I've found plenty of posts that discuss this at some level, but they're all either giving a third party answer (someone's own implementation like https://medium.com/#rizveeredwan/working-with-wav-files-in-android-52e9500297e or https://stackoverflow.com/a/43569709/4959635 or https://gist.github.com/kmark/d8b1b01fb0d2febf5770) or using some Java class, of which I don't know how it's related to the Android SDK (https://stackoverflow.com/a/26598862/4959635, https://gist.github.com/niusounds/3e49013a8e942cdba3fbfe1c336b61fc, https://github.com/google/oboe/issues/548#issuecomment-502758633).
I cannot find a standard way from the Android documentation. Some answer said to use https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/read-bytes.html for reading, but I'm quite sure this doesn't parse the file header.
So what's the standard way of processing audio files on Android / Kotlin?
I'm already using dr_wav just fine on desktop, so I am actually thinking of just using that through NDK and maybe creating a wrapper to it.
Your use case is not clear from the question.
Assuming that you need to process raw audio data (PCM samples) - the standard way is to read the (compressed) input file using the MediaExtractor and decode the packets using the MediaCodec. Note that the documentation includes some example code.
The MediaCodec outputs ByteBuffers containing raw PCM samples. The binary format is described here.
Well, there is no strict standard.
In production, you usually choose stable third party library or your company's reusable internal solution for this kind of tasks. You still can implement it yourself, but it will cost you time, since most likely the implementation will consist of hundreds of lines of code and you probably will just create another variation of existing solution which is present on the internet.
I am developing a Unity3D app to be deployed on Android.
I need to read midi files (preferrably as .midi, but other endings would be fine too) and play them (get the parsed note events at the correct timing).
Actual sound is not a priority.
It is also very important for me to be able to change the tempo.
Can you recommend a framework/ plugin/ code for this?
All the stuff i found does not work on Android, regarding C#SynthProject i could not figure out how to change the tempo.
Getting the sound is somewhat problematic (as you have noticed) as you need to emulate hardware chip that used to do the GeneralMIDI synthesis in some form or another, that is no longer commonly mounted anywhere (even windows emulates this internally), but midi file format (also known as StandardMidiFile aka SMF) files themselves are fairly well documented standard and are not terribly complex (although two non-compatible variants are a pain sometimes)
While writing own .MID file parser is probably a lot of fun, I can recommend using this piece of code dated 2013, written by our Unity god - Keijiro Takahashi
https://github.com/keijiro/smflite
I am new to android. I have two files of same length, One is audio file and one is video file with no audio. I want to make a video with audio by combining these two files. Help me to achieve this task.
I assume you have native Android app and familiarity with Java (or know porting the code in native C) and are willing to use other open-source classes in your project.
This is what you might give a head-start: Since this project is not actively maintained now, you might have to fork and use their logic into your code.
https://github.com/tqnst/MP4ParserMergeAudioVideo
Another alternative is using ffmpeg port for Android (however I am not sure how this works natively).
https://github.com/WritingMinds/ffmpeg-android-java
I'm writing an Android app that has to perform audio processing (more specifically, MFCC). I have decided to use the TarsosDSP library, but it does not provide a way to pass a sound file to it, as opposed to microphone input. This means that we currently have to write a class that implements a specific interface: basically, it's a wrapper around a buffer of doubles.
Do you know of a way to get the samples as a double from a music file format that Android supports ? We've been scouring the net for a solution but we haven't found one that is generic enough.
Thanks for the help !
I had the same problem for a while and found this:
https://0110.be/posts/Decode_MP3s_and_other_Audio_formats_the_easy_way_on_Android
You can still use the pipe in android. The code in the link allows you pass audio files directly into the decoder. It also contains the ffmpeg binaries required for the assets.
the link takes you through the steps. Good luck.
I'd like to load and play '.mid' MIDI files in Kivy but mainly on mobile platforms (i.e. IOS and Android).
I'm assuming that, I don't want to get into the intricacies of native-C/C++ based stuff (i.e. to avoid have to cross-compile, I'm not a seasoned Python dev), so the library, most probably, would need to be pure Python.
I'll start the bidding with some Frameworks I've found (some require C/C++), I've used none (0, zero), maybe someone could offer some feedback (or they might come in useful for the wider community):
BTW, it would be great if the/a MIDI library could also read the lyrics from a '.mid' file
Python Midi libraries
http://web.mit.edu/music21
https://github.com/cuthbertLab/music21
https://code.google.com/p/midiutil <- Pure Python, no lyrics (?)
https://code.google.com/p/mingus
http://mxm.dk/products/public/pythonmidi
http://das.nasophon.de/mididings
https://github.com/vishnubob/python-midi
https://gist.github.com/anonymous/202595 <- smidi.py
http://webcache.googleusercontent.com/search?q=cache%3Alarndham.net%2Fservice%2Fpys60%2Fsmidi.py
https://pypi.python.org/pypi/midi <- C for Alsa (!)
http://www.fluidsynth.org <- For pypi midi
http://sourceforge.net/p/fluidsynth <- C based
I'm using Fluidsynth to give me cross-platform MIDI output. It plays MIDI files with a SoundFont module for high quality audio. If you're happy with the LGPL licence, you can get Python bindings for it from pyFluidSynth (https://code.google.com/p/pyfluidsynth), but it's pretty straightforward to write your own bindings module using the standard Python ctypes module.
Normally, you would be expected to compile (and thus cross-compile) Fluidsynth from the supplied sources, however you can just take a pre-compiled version from another project, for instance MuseScore. This is what I'm doing for the moment, I'll switch to compiling Fluidsynth once (and if) the project I'm working on gets close to completion.
If you want to process MIDI files, e.g. to extract the lyrics, take a look at Mido (https://mido.readthedocs.org/en/latest). I've found it very useful for reading/writing MIDI files.