I have a simple Android media player that can play multiple videos simultaneously on a single screen. So basically a single media player screen is divided into 4 parts, with 4 mediaPlayer instance glued together, and each part plays a given video.
It works almost OK when my video files are stored locally on the device. There are synchronization problems, but minor. But when I input a URL for HTTP streaming, there is significant synchronization problems. What is the problem? Generally, how can I remove the synchronization problems?
The only thing I could do was first instantiate the mediaplayers and prepare() them, then call start() one after the other so at least the start times be close to eachother. It doesn't have much effect though.
Here I have a method that return each of the mediaplayer instances:
MediaPlayer mediaPreparation(String filename, boolean setMute) {
String url = "myURL"; // your URL here
// create mediaplayer instance
MediaPlayer mediaPlayer = new MediaPlayer();
if (setMute) {
mediaPlayer.setVolume(0, 0);
}
try {
mediaPlayer.setDataSource(url);
mediaPlayer.prepare();
} catch (IOException e) {
}
mediaPlayer.setLooping(true);
// mediaPlayer.start();
return mediaPlayer;
}
And then I start them one by one:
mp[0].start();
mp[1].start();
mp[2].start();
mp[3].start();
In streaming cases, there is always a risk of data being not continuously available, so players buffer quite a few frames before start playing. And in this case, multiple streams might take different time to get buffered for sufficient quantity. I see one way you can try, mediacodec. Refer this, https://developer.android.com/reference/android/media/MediaCodec.html.
Go through particularly, releaseOutputBuffer() and its variants. You have more control over rendering (alter the timestamp if required, though I won't advice as playback won't be smooth). You can keep track of whether all 4 instances got a particular timestamped frame decoded or not and then render them at once.
I'm not sure if any Android media player offers this functionality.
I suspect there may be device dependencies also, as different devices may have different capabilities in the HW to decode and play multiple videos, and if some of your videos have to use SW decoding etc they will be much slower.
It may not meet your needs, but a common way to provide a grid of videos like this on an end device is to merge the videos together on the server side and deliver it to the device as a single video stream.
Update
One other thing to be aware of if using MediaCodec and leveraging the HW codecs - if the videos have different video profiles this can cause different decoding latency also.
This is to do with how the videos are encoded - in simple terms if a particular frame refers to information from a frame that comes after it (a common compression approach) then the decoder needs to buffer the frame until it has the refereed to frame also. Simpler compression approaches, for using Baseline profile, do not use this technique so don't have to buffer and hence may have lower latency. This appears to be different for different HW vendors also - see this note from Intel, in particular the low latency section at the end:
https://software.intel.com/en-us/android/articles/android-hardware-codec-mediacodec
I suspect the best approach to this particular aspect is to aim for the lowest common dominator - either only use Baseline profile or else try to delay all video display by some factor longer than the maximum latency you can expect from any individual video.
Related
My android app plays videos in Exoplayer 2, and now I'd like to play a video backwards.
I searched around a lot and found only the idea to convert it to a gif and this from WeiChungChang.
Is there any more straight-forward solution? Another player or a library that implements this for me is probably too much to ask, but converting it to a reverse gif gave me a lot of memory problems and I don't know what to do with the WeiChungChang idea. Playing only mp4 in reverse would be enough tho.
Videos are frequently encoded such that the encoding for a given frame is dependent on one or more frames before it, and also sometimes dependent on one or more frames after it also.
In other words to create the frame correctly you may need to refer to one or more previous and one or more subsequent frames.
This allows a video encoder reduce file or transmission size by encoding fully the information for every reference frame, sometimes called I frames, but for the frames before and/or after the reference frames only storing the delta to the reference frames.
Playing a video backwards is not a common player function and the player would typically have to decode the video as usual (i.e. forwards) to get the frames and then play them in the reverse order.
You could extend ExoPlayer to do this yourself but it may be easier to manipulate the video on the server side if possible first - there exist tools which will reverse a video and then your players will be able to play it as normal, for example https://www.videoreverser.com, https://www.kapwing.com/tools/reverse-video etc
If you need to reverse it on the device for your use case, then you could use ffmpeg on the device to achieve this - see an example ffmpeg command to do this here:
https://video.stackexchange.com/a/17739
If you are using ffmpeg it is generally easiest to use via a wrapper on Android such as this one, which will also allow you test the command before you add it to your app:
https://github.com/WritingMinds/ffmpeg-android-java
Note that video manipulation is time and processor hungry so this may be slow and consume more battery than you want on your mobile device if the video is long.
Using Picasso I was able to download and display my images very quickly in my Android app. Now i want to stream my videos from my S3 server and play them through my app faster than my code here:
try {
MediaController VideoController = new MediaController(VideoPlayerActivity.this);//Creates a media controller to this activity.
VideoController.setAnchorView(AdVideoView);//Adds the media controller to the video view.
Uri video = Uri.parse(VideoURL);//Creates a Uri to hold the URL of the video.
AdVideoView.setMediaController(VideoController);//Add the media controller to the video view.
AdVideoView.setVideoURI(video);//Make the video view play from the Uri.
} catch(Exception e) {
Log.e("Video Stream Error", e.getMessage());//Sets the message for the log.
e.printStackTrace();//Displays the error in the stack trace.
e.notify();
}
Is there a faster way to display videos through a GitHub or better code?
Thanks in advance!
The things that usually slow down streamed video playback are server and network related rather than client side - unless you have a very slow or very busy device it is unlikely it won't be able to play the video back at the rate it is received over the network.
Taking this and assuming you are are seeing delays in your streamed videos, there are a couple of common things to look for.
First, mp4 videos in normal format have the metadata at the end of the video file which is not good for streaming. There is a technique called quickstart, which moves the metadata to the start which you definitely want to use. More info here:
http://multimedia.cx/eggs/improving-qt-faststart/
Secondly, network connections can obviously vary and slow networks make streaming high quality video files a problem. A technique called adaptive bit rate streaming (ABR) allows the client request lower quality video 'chunks' if the network quality is bad and then change to higher quality when it improves.
ABR also helps startup time as it allows you quickly start the video stream by using a lower quality level, and hence smaller size chunk, and then increase the quality as the video progresses. You can see this effect when you start up most online video services, such as Netflix, today (July 2016).
One thing to note is that video hosting and streaming is a specialist area so it is generally easier to leverage existing streaming technologies and services rather than to build them your self. Some good places to look to get a feel for open source solutions:
https://gstreamer.freedesktop.org
http://www.videolan.org/vlc/streaming.html
Has anyone tried using ExoPlayer to achieve this?
I tried looking online with no success.
When I say gapless playback, I am referring to the problem of using the media player to play local videos back to back. After the first video is done playing, there is a noticeable delay of 1 second before the second video starts.
Hoping this question helps in understanding this issue further.
For reference please look at the following question:
Android: MediaPlayer gapless or seamless Video Playing
ExoPlayer 2, which is now officially released, seems to support gapless playback using the ConcatenatingMediaSource class. From its developer guide:
Transitions between sources are seamless. There is no requirement that the sources being concatenated are of the same format (e.g. it’s fine to concatenate a video file containing 480p H264 with one that contains 720p VP9). The sources may even be of different types (e.g. it’s fine to concatenate a video with an audio only stream).
And the example code:
MediaSource firstSource = new ExtractorMediaSource(firstVideoUri, ...);
MediaSource secondSource = new ExtractorMediaSource(secondVideoUri, ...);
// Plays the first video, then the second video.
ConcatenatingMediaSource concatenatedSource =
new ConcatenatingMediaSource(firstSource, secondSource);
EDIT: ExoPlayer 2 supports gapless playback, but as of the time of writing is still unreleased as a stable version.
You will most likely never be able to achieve perfect gapless playback of multiple tracks with ExoPlayer or Android Media Player. Neither have been written to support starting multiple tracks and I imagine it will stay out of scope for both of them.
You can achieve gapless playback by using 2 different player instances, once you have started and played the first, you can load the second and start playback once the first finishes. Using this method you could have a gapless solution, as long as you prepare the second video during playback of the first.
To take it further, you can also use 2 different surface textures for rendering the multiple videos, once the first video reaches the end you could fade out the texture and fade in the new one. Resulting in a nice seamless video effect.
Because of the nature of playing multiple videos at once you will most likely want to create your own timer for incrementing the time and deciding when to switch to the next video, rather than trying to use the callbacks from ExoPlayer or Android Media. This will allow you to keep track of the time in a more accurate fashion, without needing to keep talking to multiple video codecs.
I know this is not the answer you've been looking for, but it's the only reasonable answer. The sole way to ensure no gaps in playback is to download the entire file first and begin playback when it's done. Otherwise, in the event that you lose connectivity before the file is finished downloading, pausing is inescapable.
I just tried switching to ExoPlayer from the standard MediaPlayer implementation and the gap is the same if not worse. However I have used a very simple method of restarting the player when the status changes to ended. I don't know if there's a better proper way to do it, perhaps with 2 different ExoPlayers.
I'm working on an android app that plays video (using video view). the video is meant to have both music (left and right) and narration, but I want to selectively be able to turn off the narration track in the MediaPlayer.
Is the way to do this correctly to encode by mp4 video file with 3 audio tracks (right left and narration) and then turn off the naration audio track with deselectTrack()?
Not clear to me from the documentation that MediaPlayer can handle more than 2 audio tracks.
If the audio tracks are limited to 2, would it make sense to run two media player simultaneously (synching them up with seekTo())when I want the narration track to play?
Thanks.
Sorry to burst your bubble, but...
1) You have a misunderstanding about what a "track" denotes. A track can have multiple channels (e.g., a stereo track has left and right channels). As I understand it, stereo is the extent of the Android AudioTrack implementation at present. I haven't yet checked if the OpenSL implementation is more extensive than the Java API.
2) Only 1 audio track can be selected at a time, so you wouldn't be able to have background and narration simultaneously in the way you were thinking.
3) Audio tracks can only be selected in the prepared state (i.e., not after playback has started). The documentation mentions this limitation is not ideal, so it will probably change in the future. If not for this problem, your goal could be accomplished with two audio tracks encoded in the stream, one with both background & narration, the other just background.
You will probably find it difficult to synchronize two MediaPlayers, but I haven't tried. Maybe this approach would be acceptable for your situation, although be forewarned the seekTo method isn't accurate. It depends on the encoding of the files.
Something I would try if I were you is to have two complete encoded videos, one with narration, the other without. Use two MediaPlayers and keep them both prepared. When you want to switch use seekTo to put the correct one at (or near) the desired location. That way you don't have to worry about synchronization. If the video is large, this method could use significantly more resources, though.
I need to be able to play two or more (let's say, up to 5) short ogg files simultaneously. And by simultaneously I mean in perfect synchrony. I am able to load them to SoundPool and play, but this sometimes creates a noticeable difference in playback start time, which I want to get rid of.
From my understanding this can be avoided if mixing PCMs into one buffer and playing. But OGG's are not PCMs and need to be somehow efficiently decoded before playing and latency must be very low, ideally as soon as user presses the button. So I figured I need a way to stream OGG into PCM and as I receive buffers I would mix them and feed to AudioTrack. My requirement is Android 2.3.3+, so I cannot use any new codecs provided in Jelly Bean.
Also although OGGs themselves are small, there is a lot of them. So keeping them all decoded in memory (SoundPool or some pre-decoding) may case problems too.
Can someone give me a tip where to dig? Can OpenSL ES do that for me? Or should I think about integrating ffmpeg? And is it even possible to stream simultaneus files with low latency?
Thanks
You can play sounds using AssetPlayers, but this sometimes creates a noticeable difference in playback start time, yeh...
So, i recomend to decode ogg using Ogg Vorbis (like here) and then using this PCM buffer for BufferPlayer.
Btw, check this OpenSL ES wrappers
https://github.com/Suvitruf/Android-ndk/tree/master/OpenSLES