I have to modify the Http Live Streaming implementation of Android Media Player.
The implementation is under the stagefright library
http://androidxref.com/4.0.4/xref/frameworks/base/media/libstagefright/httplive/LiveDataSource.cpp
I think these library will compile to a libstagefright.so which should be part of the Android system.
My question is if I make some changes to this library and compile a new libstagefright.so.
If I load this new libstagefright.so in my new application and call up the media player, will it use the code in my new libstagefright.so?
You will not be able to replace the original library, since when you try to loadLibrary it will load the library from within /system/lib. So unless you replace that (which is not possible on unrooted devices), you won't be able to load your custom code.
https://github.com/android/platform_system_core/blob/66ed50af6870210ce013a5588a688434a5d48ee9/rootdir/init.environ.rc.in sets the LD_LIBRARY_PATH by default. And loads it from these paths if available. If not, then your application's lib directory will be searched; but not the other way around.
I tried this with libwebkit.so in the past on various mainstream devices and haven't had any luck getting it to load instead of the one in /system/lib.
You can learn more by looking at:
doLoad from here https://android.googlesource.com/platform/libcore/+/41d00b744b7772f9302fdb94dddadb165b951220/luni/src/main/java/java/lang/Runtime.java
findLibrary here http://developer.android.com/reference/dalvik/system/BaseDexClassLoader.html#findLibrary(java.lang.String)
I'm pretty sure you can't replace the default class loader either for security reasons.
What you can do, though, is a straightforward fork the Media Player and have it load your modified libstagefright-modified.so. There could be other solutions, haven't looked at Media Player's code.
Knowing that all you want to do is parse the data before it gets to the MediaPlayer, I suggest not trying to alter the Android libraries. As soulseekah mentioned, it's not going to work without a rooted device. There are other options, although they both have drawbacks.
1) If you are only targeting recent versions (4.2 or later, I believe), you can take a look at new classes added to the android.media package, like MediaExtractor and MediaCodec. I'm not greatly familiar with those because they aren't available on the hardware with which I work, but they could be useful in getting to the raw data. Here is a decent sample of using them to play video. The drawback is those classes aren't available in earlier versions.
2) The other option is to put a local proxy on the device. Connect the MediaPlayer to the proxy and make the request to the media server yourself. See my answer here for a little more info on that. With a proxy, you will see all the data that comes through, giving you a chance to parse the ID3 tags. There is the drawback that you will have to parse the TS packets to put together an elementary stream (essentially doing the demuxer's job), but it will work with any version of Android. TS streams aren't difficult to disassemble, and ID3 tags aren't time consuming to parse, so I think this is a reasonable approach.
Related
What are the standard ways of reading and writing audio files on Android / Kotlin?
I am very confused. I've found plenty of posts that discuss this at some level, but they're all either giving a third party answer (someone's own implementation like https://medium.com/#rizveeredwan/working-with-wav-files-in-android-52e9500297e or https://stackoverflow.com/a/43569709/4959635 or https://gist.github.com/kmark/d8b1b01fb0d2febf5770) or using some Java class, of which I don't know how it's related to the Android SDK (https://stackoverflow.com/a/26598862/4959635, https://gist.github.com/niusounds/3e49013a8e942cdba3fbfe1c336b61fc, https://github.com/google/oboe/issues/548#issuecomment-502758633).
I cannot find a standard way from the Android documentation. Some answer said to use https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/read-bytes.html for reading, but I'm quite sure this doesn't parse the file header.
So what's the standard way of processing audio files on Android / Kotlin?
I'm already using dr_wav just fine on desktop, so I am actually thinking of just using that through NDK and maybe creating a wrapper to it.
Your use case is not clear from the question.
Assuming that you need to process raw audio data (PCM samples) - the standard way is to read the (compressed) input file using the MediaExtractor and decode the packets using the MediaCodec. Note that the documentation includes some example code.
The MediaCodec outputs ByteBuffers containing raw PCM samples. The binary format is described here.
Well, there is no strict standard.
In production, you usually choose stable third party library or your company's reusable internal solution for this kind of tasks. You still can implement it yourself, but it will cost you time, since most likely the implementation will consist of hundreds of lines of code and you probably will just create another variation of existing solution which is present on the internet.
I want to allow user to convert .mid files to .wav files in my Android app.
Actually, there is not plenty of information on the Net about any kind of midi to wav conversion, and so there is very litlle info about doing so in Android.
What should I do? Where to go?
TL;DR
Dependless on the way featured in this answer you pick, you'll have to include some c code in your project. And also you'll have to include lib lisenced under LGPL-2.1. You should really pay attention to both of these. Read this to understand it better. Really. Do read it. It IS important to understand.
Lazy option - use my lib
Better option - set FluidSynth up following either of the links (official github page guide and\or Medium article), include this snippet of code into your project and you're good to go. This code should be included into your project like it is explained in the Medium article listed above.
Very bad option - use Timidity++.
More detailed version
Not recommended option
There are two options when you convert mid files: FluidSynth and Timidity++. I wouldn't recommend you to use Timidity++ for purpose of converting .mid to raw audio. The lib is kind of old and it's not supported; docs and community are imposible to find. FluidSynth is a much better choice: it's newer, it's supported, it's gotta plenty of API docs, its community is kinda lil more alive than Timidity's is. I couldn't get Timidity working on Android.
Anyway, here are some links in case you'd like to use Timidity anyway.
somewhat usable timidity lib
example of converting mid to wav
a working app featuring timidity (I couldn't get anything going on in this code, but this app uses timidity and it's able to convert mid to wav, it's a fact)
It's not a complete list of timidity resources. Also, there are a few mysterious repos on the web that claim to be either timidity or some kind of timidity lib or even lib for android... But personally I never understood what was going on there, and so I don't recommend to go for timidity.
Lazy option
The laziest option is to use my own library. All the instructions are in the Github readme. Using my lib is not recommended way to do it, probably, since my implementation can have some big performance issues.
You can see examples of using this approach in the lib's readme.
Better option
The best option to convert .mid to .wav file is to use FluidSynth software synthesizer. This way you'll have to do some c codding. I told you.
The official github wiki got the instruction on how to set FluidSynth for Android, but I'd suggest you to read this Medium article about configuring this synthesizer instead, 'cause it's a lot more easier to follow and understand.
After you've set this thing up, you can do some neat .mid to .wav conversion. Here is the official docs for that. Gonna leave the code here in case the link goes down.
fluid_settings_t* settings;
fluid_synth_t* synth;
fluid_player_t* player;
fluid_file_renderer_t* renderer;
settings = new_fluid_settings();
// specify the file to store the audio to
// make sure you compiled fluidsynth with libsndfile to get a real wave file
// otherwise this file will only contain raw s16 stereo PCM
fluid_settings_setstr(settings, "audio.file.name", "/path/to/output.wav");
// use number of samples processed as timing source, rather than the system timer
fluid_settings_setstr(settings, "player.timing-source", "sample");
// since this is a non-realtime scenario, there is no need to pin the sample data
fluid_settings_setint(settings, "synth.lock-memory", 0);
synth = new_fluid_synth(settings);
// *** loading of a soundfont omitted ***
player = new_fluid_player(synth);
fluid_player_add(player, "/path/to/midifile.mid");
fluid_player_play(player);
renderer = new_fluid_file_renderer (synth);
while (fluid_player_get_status(player) == FLUID_PLAYER_PLAYING)
{
if (fluid_file_renderer_process_block(renderer) != FLUID_OK)
{
break;
}
}
// just for sure: stop the playback explicitly and wait until finished
fluid_player_stop(player);
fluid_player_join(player);
delete_fluid_file_renderer(renderer);
delete_fluid_player(player);
delete_fluid_synth(synth);
delete_fluid_settings(settings);
... and basically that is it. You can start converting .mid to .wav files right now.
Here is the example on how to integrate that code in your project.
Here is the example how to use this function in your android code.
I am new to android. I have two files of same length, One is audio file and one is video file with no audio. I want to make a video with audio by combining these two files. Help me to achieve this task.
I assume you have native Android app and familiarity with Java (or know porting the code in native C) and are willing to use other open-source classes in your project.
This is what you might give a head-start: Since this project is not actively maintained now, you might have to fork and use their logic into your code.
https://github.com/tqnst/MP4ParserMergeAudioVideo
Another alternative is using ffmpeg port for Android (however I am not sure how this works natively).
https://github.com/WritingMinds/ffmpeg-android-java
I am working on a matlab project where I add effects to audio files (mp3, wav). Therefore, I load the files into arrays using the matlab function audioread(..).
Now, I want to export this to Android. I read that the best way is to use the Matlab Coder to export the matlab code to C/C++ (or Java) and then export it into android (more or less).
However, the function call audioplayer (and play) are Unsupported (that's what the code generation readiness issues says).
What can I do ? One idea was to play the sounds directly using c++ code (so after the code generation). But how to play sounds from arrays using c++ ?
Or if you guys have others ideas without touching c++ codes (so fixing the problem directly in matlab), I would be glad to hear it !
Thanks and have a good day !
Typically what I recommend in cases like this is to factor your code in two pieces:
The part that does the audio file I/O and audio playing (namely the OS-specific part)
The computational kernel for which you will generate code using MATLAB Coder. This piece usually takes numeric arrays representing the image or audio data as arguments.
I've used this approach to leverage MATLAB Coder generated code to do image filtering on Android.
To do part (1), as Navan says, you'll need to use Android APIs to read in audio files, write data back to files, and to play them as desired. Note, I haven't done extensive Android development, so doing these tasks may take some research or be difficult.
Once you have the data in a format suitable for the function(s) in (2), likely a numeric array, then you can call your generated code using JNI to add the desired effects. The generated code would return the data back to the Java code and you can then encode it, play it, or do as you please with it using the Android APIs.
Playing audio normally uses platform dependent libraries. In DSP System toolbox, there is an audio player object called dsp.AudioPlayer which supports C code generation. But I believe this uses platform dependent libraries in the generated code and it will not be straight forward to make it work in Android. You will be better off finding an audio player library for Android and hooking that in manually after generating code.
I have a task which involves integration of a video decoder into Stagefright(Android's multimedia framework). I searched and found the following about creating a new plugin for Stagefright:
To add support for a new format, you need to:
Develop a new Extractor class, if the container is not supported yet.
Develop a new Decoder class, that implements the interface needed by the StageFright core to read the data.
Associate the mime-type of the files to read to your new Decoder in the OMXCodec.cpp file, in the kDecoderInfo array.
static const CodecInfo kDecoderInfo[] = {
{MEDIA_MIMETYPE_AUDIO_AAC, "OMX.TI.AAC.decode"},
{MEDIA_MIMETYPE_AUDIO_AAC, "AACDecoder"},
};
The above data is all i could find out on net. Right now i have a simple app that will take a file as an input and render it on the screen using native API's in android. Can anyone please tell me how to proceed further. And from where does all these OMXCodec.cpp and others come into picture and which directory of my project should i have them in. Please provide solutions regarding the same. Thanks in advance.
From your question, it appears that you are looking at a recommendation which is specific for Ice-Cream Sandwich or earlier versions of Android. The first thing you should be clear about is the version of the android i.e. Ice-Cream Sandwich or before or JellyBean and after. The integration of codecs is different across different releases of Android.
I have already commented on your other question which is specific for JellyBean and later (Reference: Android: How to integrate a decoder to multimedia framework)
If you would like to integrate your codec in Ice-Cream Sandwich or before, the steps are already available in your question. In addition to adding the decoder into kDecoderInfo list, you may like to setup certain quirks as shown here.
For the question on OMXCodec.cpp, you can find this file at
frameworks/base/media/libstagefright/ in case of Ice-Cream Sandwich and frameworks/av/media/libstagefright/ in case of JellyBean.
If you have followed all the steps to integrate the video decoder into the Stagefright framework, then the easiest test would be to perform the following:
Copy a media file into SD-Card
In OMXCodec.cpp, enable logs by removing the comment in this statement //#define LOG_NDEBUG 0 and run a mm in the directory. Copy the rebuilt libstagefright.so to /system/lib on your device.
Enable logcat and start capturing logs.
Goto gallery, select your file and allow the standard player to play your file.
Check your log file if the player has selected your OMX component by searching for your component name. If found, your integration of codec into Stagefright is successful. Else, you will have to debug and find out what is the problem.
Postscript:
Based on your queries, I presume you aren't familiar with Android sources. Please refer to androidxref site to become familiar with AOSP distributions.
Unless you are planning to support a new media file-format, you will not require to support Extractor class. MediaExtractor abstracts a file-format parser and helps to de-multiplex the different tracks in a media file.
I hope with this information, you should be able to get your codec integrated and functional in Android.