I'm writing an Android app that has to perform audio processing (more specifically, MFCC). I have decided to use the TarsosDSP library, but it does not provide a way to pass a sound file to it, as opposed to microphone input. This means that we currently have to write a class that implements a specific interface: basically, it's a wrapper around a buffer of doubles.
Do you know of a way to get the samples as a double from a music file format that Android supports ? We've been scouring the net for a solution but we haven't found one that is generic enough.
Thanks for the help !
I had the same problem for a while and found this:
https://0110.be/posts/Decode_MP3s_and_other_Audio_formats_the_easy_way_on_Android
You can still use the pipe in android. The code in the link allows you pass audio files directly into the decoder. It also contains the ffmpeg binaries required for the assets.
the link takes you through the steps. Good luck.
Related
What are the standard ways of reading and writing audio files on Android / Kotlin?
I am very confused. I've found plenty of posts that discuss this at some level, but they're all either giving a third party answer (someone's own implementation like https://medium.com/#rizveeredwan/working-with-wav-files-in-android-52e9500297e or https://stackoverflow.com/a/43569709/4959635 or https://gist.github.com/kmark/d8b1b01fb0d2febf5770) or using some Java class, of which I don't know how it's related to the Android SDK (https://stackoverflow.com/a/26598862/4959635, https://gist.github.com/niusounds/3e49013a8e942cdba3fbfe1c336b61fc, https://github.com/google/oboe/issues/548#issuecomment-502758633).
I cannot find a standard way from the Android documentation. Some answer said to use https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/read-bytes.html for reading, but I'm quite sure this doesn't parse the file header.
So what's the standard way of processing audio files on Android / Kotlin?
I'm already using dr_wav just fine on desktop, so I am actually thinking of just using that through NDK and maybe creating a wrapper to it.
Your use case is not clear from the question.
Assuming that you need to process raw audio data (PCM samples) - the standard way is to read the (compressed) input file using the MediaExtractor and decode the packets using the MediaCodec. Note that the documentation includes some example code.
The MediaCodec outputs ByteBuffers containing raw PCM samples. The binary format is described here.
Well, there is no strict standard.
In production, you usually choose stable third party library or your company's reusable internal solution for this kind of tasks. You still can implement it yourself, but it will cost you time, since most likely the implementation will consist of hundreds of lines of code and you probably will just create another variation of existing solution which is present on the internet.
I want to allow user to convert .mid files to .wav files in my Android app.
Actually, there is not plenty of information on the Net about any kind of midi to wav conversion, and so there is very litlle info about doing so in Android.
What should I do? Where to go?
TL;DR
Dependless on the way featured in this answer you pick, you'll have to include some c code in your project. And also you'll have to include lib lisenced under LGPL-2.1. You should really pay attention to both of these. Read this to understand it better. Really. Do read it. It IS important to understand.
Lazy option - use my lib
Better option - set FluidSynth up following either of the links (official github page guide and\or Medium article), include this snippet of code into your project and you're good to go. This code should be included into your project like it is explained in the Medium article listed above.
Very bad option - use Timidity++.
More detailed version
Not recommended option
There are two options when you convert mid files: FluidSynth and Timidity++. I wouldn't recommend you to use Timidity++ for purpose of converting .mid to raw audio. The lib is kind of old and it's not supported; docs and community are imposible to find. FluidSynth is a much better choice: it's newer, it's supported, it's gotta plenty of API docs, its community is kinda lil more alive than Timidity's is. I couldn't get Timidity working on Android.
Anyway, here are some links in case you'd like to use Timidity anyway.
somewhat usable timidity lib
example of converting mid to wav
a working app featuring timidity (I couldn't get anything going on in this code, but this app uses timidity and it's able to convert mid to wav, it's a fact)
It's not a complete list of timidity resources. Also, there are a few mysterious repos on the web that claim to be either timidity or some kind of timidity lib or even lib for android... But personally I never understood what was going on there, and so I don't recommend to go for timidity.
Lazy option
The laziest option is to use my own library. All the instructions are in the Github readme. Using my lib is not recommended way to do it, probably, since my implementation can have some big performance issues.
You can see examples of using this approach in the lib's readme.
Better option
The best option to convert .mid to .wav file is to use FluidSynth software synthesizer. This way you'll have to do some c codding. I told you.
The official github wiki got the instruction on how to set FluidSynth for Android, but I'd suggest you to read this Medium article about configuring this synthesizer instead, 'cause it's a lot more easier to follow and understand.
After you've set this thing up, you can do some neat .mid to .wav conversion. Here is the official docs for that. Gonna leave the code here in case the link goes down.
fluid_settings_t* settings;
fluid_synth_t* synth;
fluid_player_t* player;
fluid_file_renderer_t* renderer;
settings = new_fluid_settings();
// specify the file to store the audio to
// make sure you compiled fluidsynth with libsndfile to get a real wave file
// otherwise this file will only contain raw s16 stereo PCM
fluid_settings_setstr(settings, "audio.file.name", "/path/to/output.wav");
// use number of samples processed as timing source, rather than the system timer
fluid_settings_setstr(settings, "player.timing-source", "sample");
// since this is a non-realtime scenario, there is no need to pin the sample data
fluid_settings_setint(settings, "synth.lock-memory", 0);
synth = new_fluid_synth(settings);
// *** loading of a soundfont omitted ***
player = new_fluid_player(synth);
fluid_player_add(player, "/path/to/midifile.mid");
fluid_player_play(player);
renderer = new_fluid_file_renderer (synth);
while (fluid_player_get_status(player) == FLUID_PLAYER_PLAYING)
{
if (fluid_file_renderer_process_block(renderer) != FLUID_OK)
{
break;
}
}
// just for sure: stop the playback explicitly and wait until finished
fluid_player_stop(player);
fluid_player_join(player);
delete_fluid_file_renderer(renderer);
delete_fluid_player(player);
delete_fluid_synth(synth);
delete_fluid_settings(settings);
... and basically that is it. You can start converting .mid to .wav files right now.
Here is the example on how to integrate that code in your project.
Here is the example how to use this function in your android code.
I am new to android. I have two files of same length, One is audio file and one is video file with no audio. I want to make a video with audio by combining these two files. Help me to achieve this task.
I assume you have native Android app and familiarity with Java (or know porting the code in native C) and are willing to use other open-source classes in your project.
This is what you might give a head-start: Since this project is not actively maintained now, you might have to fork and use their logic into your code.
https://github.com/tqnst/MP4ParserMergeAudioVideo
Another alternative is using ffmpeg port for Android (however I am not sure how this works natively).
https://github.com/WritingMinds/ffmpeg-android-java
I am working on a matlab project where I add effects to audio files (mp3, wav). Therefore, I load the files into arrays using the matlab function audioread(..).
Now, I want to export this to Android. I read that the best way is to use the Matlab Coder to export the matlab code to C/C++ (or Java) and then export it into android (more or less).
However, the function call audioplayer (and play) are Unsupported (that's what the code generation readiness issues says).
What can I do ? One idea was to play the sounds directly using c++ code (so after the code generation). But how to play sounds from arrays using c++ ?
Or if you guys have others ideas without touching c++ codes (so fixing the problem directly in matlab), I would be glad to hear it !
Thanks and have a good day !
Typically what I recommend in cases like this is to factor your code in two pieces:
The part that does the audio file I/O and audio playing (namely the OS-specific part)
The computational kernel for which you will generate code using MATLAB Coder. This piece usually takes numeric arrays representing the image or audio data as arguments.
I've used this approach to leverage MATLAB Coder generated code to do image filtering on Android.
To do part (1), as Navan says, you'll need to use Android APIs to read in audio files, write data back to files, and to play them as desired. Note, I haven't done extensive Android development, so doing these tasks may take some research or be difficult.
Once you have the data in a format suitable for the function(s) in (2), likely a numeric array, then you can call your generated code using JNI to add the desired effects. The generated code would return the data back to the Java code and you can then encode it, play it, or do as you please with it using the Android APIs.
Playing audio normally uses platform dependent libraries. In DSP System toolbox, there is an audio player object called dsp.AudioPlayer which supports C code generation. But I believe this uses platform dependent libraries in the generated code and it will not be straight forward to make it work in Android. You will be better off finding an audio player library for Android and hooking that in manually after generating code.
Requirement
Android open a .wav file in sd card, play it , add some effect (like echo, pitch shift etc), save the file with effect. Simple :(
What I know
I can open and play file using Soundpool or MediaPlayer.
I can give some effect while playing using both. ie for Media Player
I can set Environmental Reverb effect. Using SoundPool I can set
playing rate, which is kind of like pitch shift. I am successful in
implementing these right now.
But either of this classes doesn't have any method to save the
played file. So I can only play, I cannot save the music with
effect.
What I want to know
Is there any other classes of interest, other than MediaPlayer or
SoundPool. Never mind about saving, you just mention the class, I will do the
research about saving file with them.
Any 3rd party libraries where I can add effects and save? Happy if
it is open source and free. But mention them even if it is
proprietary.
Any other areas where I can look into. Does OpenAL support voice
filtering along with voice positioning? Will it work with Android?
Ready to do the dirty work. You please lend me the path..
EDIT: Did some more searching, and come across AudioTrack. But it also won't support saving to a file. So no luck there also..
EDIT Ok, what if I do it myself? Get raw bytes from a wav file, and work on that. I recorded a wav file using AudioRecord, got a wav file. Is there any resource describing low level audio processing (I mean at the bytes level).
EDIT Well bounty time is up, and I am giving bounty to the only answer that I got. After 7 days, what I understood is
We can't save what we play using MediaPlayer, AudioTrack etc.
There is no audio processing libraries available to use.
You can get raw wav files, and do the audio processing yourself. The
answer gave a good wrapper class for reading/writing wav files. A
good java code to read and change pitch of wav files is here.
The WavFile class http://www.labbookpages.co.uk/audio/javaWavFiles.html claims to read and write wav files and allow per-sample manipulation through arrays of sample values. It's certainly reasonably small, 23kbytes total source code.
I did struggle for a while to build an android app with the Wavfile Class included. This turned out to be because both WavFile and ReadExample (from the above link) were intended as standalone java programs, so include a method main(String [] args){}. Eclipse sees this and thinks the Class is a standalone runnable program, and, when I click the run button, tries to execute just the one Class with the java in the development machine, instead of launching the whole app to my phone. When I take care to run the whole app with the little drop-down menu on the run button, I don't have any trouble, and the WavFile Class and examples drop straight in, give zero warnings in the IDE, and work as advertised running on my phone.