I'm working on an android app that plays video (using video view). the video is meant to have both music (left and right) and narration, but I want to selectively be able to turn off the narration track in the MediaPlayer.
Is the way to do this correctly to encode by mp4 video file with 3 audio tracks (right left and narration) and then turn off the naration audio track with deselectTrack()?
Not clear to me from the documentation that MediaPlayer can handle more than 2 audio tracks.
If the audio tracks are limited to 2, would it make sense to run two media player simultaneously (synching them up with seekTo())when I want the narration track to play?
Thanks.
Sorry to burst your bubble, but...
1) You have a misunderstanding about what a "track" denotes. A track can have multiple channels (e.g., a stereo track has left and right channels). As I understand it, stereo is the extent of the Android AudioTrack implementation at present. I haven't yet checked if the OpenSL implementation is more extensive than the Java API.
2) Only 1 audio track can be selected at a time, so you wouldn't be able to have background and narration simultaneously in the way you were thinking.
3) Audio tracks can only be selected in the prepared state (i.e., not after playback has started). The documentation mentions this limitation is not ideal, so it will probably change in the future. If not for this problem, your goal could be accomplished with two audio tracks encoded in the stream, one with both background & narration, the other just background.
You will probably find it difficult to synchronize two MediaPlayers, but I haven't tried. Maybe this approach would be acceptable for your situation, although be forewarned the seekTo method isn't accurate. It depends on the encoding of the files.
Something I would try if I were you is to have two complete encoded videos, one with narration, the other without. Use two MediaPlayers and keep them both prepared. When you want to switch use seekTo to put the correct one at (or near) the desired location. That way you don't have to worry about synchronization. If the video is large, this method could use significantly more resources, though.
Related
I have 5 audio files with each having length 5 seconds. I wanted to play each sound file one by one but the condition is if an audio file playing next file should play after 4 seconds ie adjacent audio file sounds should overlap for 1 second. How can I implement it? Which is the best audio player you can suggest?
The amount of specific work you can do with audio playback on the Java side is pretty limited on android.
It sounds like you will need to mix your sounds at some point during their playback to overlap.
The best way to do this in my head is through a C++ library called Oboe (I am currently working with this). This is a library created by Google for audio playback. Now hold on now, let me explain! I know implementing C++ (especially if your only on the Java stack right now) can add a bit of time to your project.
The reason this came to mind is because in this way of playing audio (through Oboe/C++), you physically move individual bits of the audio sample through a buffer stream. The C++ libraries also actually have a Mixer class that you can put 2 different audio samples (up to 100 actually) into to mix, and then eventually render through the buffer stream.
Using this methodology, you can add specific logic to manage when your audio starts playing (after 4 seconds if adjacent). At which point you can mix the first second of the next clip with the current playing clip.
Now the exciting bit, is you may be able to replicate this process in Java! I found this post which may be of help to you:
Android: How to mix 2 audio files and reproduce them with soundPool
Now I do warn you, rendering audio in this way (through buffer streams) is a complicated process, and some extra research may be needed to fully understand the process. I can't say I know all of the functionality of the Java audio libraries, but I'm willing to bet they don't have much support for mixing sound in the way that you need. So most likely you will have to mix it yourself, or your last resort might be to use the NDK (C++).
Hopefully this answer helps. The best wishes in your research! Hopefully you will find a simple way that works. (If you do, don't forget to share your findings on this question!)
I want to create an Android app that plays multiple mp3s simultaneously, with precise sync (less than 1/10 of a second off) and independent volume control. Size of each mp3 could be over 1MB, run time up to several minutes. My understanding is that MediaPlayer will not do the precise sync, and SoundPool can't handle files over 1MB or 5 seconds run time. I am experimenting with superpowered and may end up using that, but I'm wondering if there's anything simpler, given that I don't need any processing (reverb, flange, etc.), which is superpowered's focus.
Also ran across the YouTube video on Android high-performance audio, from Google I/O 2016. Wondering if anyone has any experience with this.
https://www.youtube.com/watch?v=F2ZDp-eNrh4
Superpowered was originally made for my DJ app (DJ Player in the App Store), where precisely syncing multiple tracks is a requirement.
Therefore, syncing multiple mp3s and independent volume control is definitely possible and core to Superpowered. All you need is the SuperpoweredAdvancedAudioPlayer class for this.
The CrossExample project in the SDK has two players playing in sync.
The built-in audio features in Android are highly device and/or build dependent. You can't get a consistent feature set with those. In general, the audio features of Android are not stable. That's why you need a specialized audio library which does everything "inside" your application (so is not a "wrapper" around Android's audio features).
When you are playing compressed files (AAC, MP3, etc) on Android in most situations they are decoded in hardware to save power, except when the output goes to a USB audio interface. The hardware codec accepts data in big chunks (again, to save power). Since it's not possible to issue a command to start playing multiple streams at once, what will often be happening is that one stream will already send a chunk of compressed audio to hardware codec, and it will start playing, while others haven't yet sent their data.
You really need to decode these files in your app and mix the output to produce a single audio stream. Then you will guarantee the desired synchronization. The built-in mixing facilities are mostly intended to allow multiple apps to use the same sound output, they are not designed for multitrack mixing.
How can I play background audio, in Android, without interrupting the MediaPlayer playback, by either using MediaPlayer (preferred) or OpenSL ES?
I know SoundPool is able to play sound effects without interrupting any MediaPlayer playback, but the size is limited to 1M per effect, which is way to less. Not requesting audio focus, via AudioManager doesn't seem to work either, audio doesn't play at all in this case.
And in the case of OpenSL ES, all audio generally stops when I start to play a longer asset file. It's similar to the behaviour of SoundPool described above.
Edit from the comments:
I don't want to interrupt other music players, it's the background
audio of a game, which shall play without interrupting the, for
example, music of a player. Games like Subway Surfer, Clash Royale and
such seem to have this achieved somehow, but I could not achieve it
via OpenSL ES, or MediaPlayer.
In order to play sound in background you can use SoundPool, AudioTracks and OpenSlES.
Soundpool: Use small files and make a sequence. In my last project i use 148 sound files (all small) in different scenarios and played using soundpool. Make a list and play one by one or in parallel. Also in games usually you have a small set of sound for particular scenario and it loops. Best is soundpool for that. You can also perform some effects like rate change. Also ogg files are way small, so use them.
AudioTrack: You can use RAW PCM data in audio track. If you want you can extract pcm data using MediaExtractor from almost all formats and use it. But it will be a little work for you in your case, but it is really good (supports huge data, even live streaming).
OpenSLES: Apparently android uses opensles in background for all its purpose. So using it will help you a lot. But it's not easy to get everything done on it. You need to learn more for lesser work.
I have been deeply working on OpenSlES for about 20 days and still i will say for small purpose use soundpool, medium to high level implementation use AudioTracks and for kickass implementation use OpenSLES.
PS: It will be bad effect on your user if you play game sound in background while they are playing their music or their call. Personal experience.
Has anyone tried using ExoPlayer to achieve this?
I tried looking online with no success.
When I say gapless playback, I am referring to the problem of using the media player to play local videos back to back. After the first video is done playing, there is a noticeable delay of 1 second before the second video starts.
Hoping this question helps in understanding this issue further.
For reference please look at the following question:
Android: MediaPlayer gapless or seamless Video Playing
ExoPlayer 2, which is now officially released, seems to support gapless playback using the ConcatenatingMediaSource class. From its developer guide:
Transitions between sources are seamless. There is no requirement that the sources being concatenated are of the same format (e.g. it’s fine to concatenate a video file containing 480p H264 with one that contains 720p VP9). The sources may even be of different types (e.g. it’s fine to concatenate a video with an audio only stream).
And the example code:
MediaSource firstSource = new ExtractorMediaSource(firstVideoUri, ...);
MediaSource secondSource = new ExtractorMediaSource(secondVideoUri, ...);
// Plays the first video, then the second video.
ConcatenatingMediaSource concatenatedSource =
new ConcatenatingMediaSource(firstSource, secondSource);
EDIT: ExoPlayer 2 supports gapless playback, but as of the time of writing is still unreleased as a stable version.
You will most likely never be able to achieve perfect gapless playback of multiple tracks with ExoPlayer or Android Media Player. Neither have been written to support starting multiple tracks and I imagine it will stay out of scope for both of them.
You can achieve gapless playback by using 2 different player instances, once you have started and played the first, you can load the second and start playback once the first finishes. Using this method you could have a gapless solution, as long as you prepare the second video during playback of the first.
To take it further, you can also use 2 different surface textures for rendering the multiple videos, once the first video reaches the end you could fade out the texture and fade in the new one. Resulting in a nice seamless video effect.
Because of the nature of playing multiple videos at once you will most likely want to create your own timer for incrementing the time and deciding when to switch to the next video, rather than trying to use the callbacks from ExoPlayer or Android Media. This will allow you to keep track of the time in a more accurate fashion, without needing to keep talking to multiple video codecs.
I know this is not the answer you've been looking for, but it's the only reasonable answer. The sole way to ensure no gaps in playback is to download the entire file first and begin playback when it's done. Otherwise, in the event that you lose connectivity before the file is finished downloading, pausing is inescapable.
I just tried switching to ExoPlayer from the standard MediaPlayer implementation and the gap is the same if not worse. However I have used a very simple method of restarting the player when the status changes to ended. I don't know if there's a better proper way to do it, perhaps with 2 different ExoPlayers.
I'm writing my first android app, trying to playback two 10min soundfiles synchronously (imagine an instrumental track and an acapella), to be able to change the volume of each track independently). I am using two MediaPlayers for this, since a SoundPool is targeted at shorter audio samples as far as I read.
Now my problem is that, when pausing and resuming the playback, sometimes the players are not synchronous anymore, even though I set their positions to the same value before resuming playback.
I know that this is kind of inevitable, because they cannot be started at exactly the same moment and they may require different amounts of time for starting playback, but: Is there maybe any other approach to meet my requirements?
You can take a look at JetPlayer, this may accomplish what you want as far as synchronization.
To use it you create audio channels (your instrument channel and vocal channel) as MIDI files in a track and the player can keep them synchronized while allowing you to mute or unmute the different channels as appropriate.
The user guide for creating JET resouces can be found here.