Develop Reference

How to download and add subtitles to video using Exoplayer

I have a video player app which used the Exoplayer library to play videos but I want to be able to download and add subtitles dynamically using something like OpenSubtitles and then add them to the currently playing video.
Video players with similar features are MX Player and X Player.
How can I be able to achieve a similar functionality. Your answers are highly appreciated!

Yes, actually. OpenSubtitles.org has a nice API for searching and downloading.
Here you go
https://opensubtitles.stoplight.io/docs/opensubtitles-api/e3750fd63a100-getting-started

You can download subtitles separately or checking downloading media and add them using sideloading subtitles or you can use SingleSampleMediaSource like the following:
MediaSource[] mediaSources = new MediaSource[subtitlesCount + videoTrack];
mediaSources[0] = new ProgressiveMediaSource.Factory(new FileDataSource.Factory())
.createMediaSource(MediaItem.fromUri(videoPath));
//add subtitles tracks, and use correct format
SingleSampleMediaSource.Factory singleSampleSourceFactory = new SingleSampleMediaSource.Factory(new DefaultDataSourceFactory(context));
for (int i = 0; i < videoInfo.getSubtitlePaths().size(); i++) {
mediaSources[i + videoTrack] = singleSampleSourceFactory.createMediaSource(
new MediaItem.Subtitle(Uri.parse(subtitlePath), MimeTypes.TEXT_VTT, language), C.TIME_UNSET);
}
//use MergingMediaSource to combine the media sources
MergingMediaSource mergedSource = new MergingMediaSource(mediaSources);
....init player
mPlayer.addMediaSource(mergedSource);
mPlayer.prepare();

An AAC audio stream is playable in VLC for Android, but not in Exoplayer

I have an RTMP stream I want to play in my app using the Exoplayer library. My setup for that is as follows:
TrackSelector trackSelector = new DefaultTrackSelector();
RtmpDataSourceFactory rtmpDataSourceFactory = new RtmpDataSourceFactory(bandwidthMeter);
ExtractorsFactory extractorsFactory = new DefaultExtractorsFactory();
factory = new ExtractorMediaSource.Factory(rtmpDataSourceFactory);
factory.setExtractorsFactory(extractorsFactory);
createSource();
mPlayer = ExoPlayerFactory.newSimpleInstance(mActivity, trackSelector, new DefaultLoadControl(
new DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE),
1000, // min buffer
3000, // max buffer
1000, // playback
2000, //playback after rebuffer
DefaultLoadControl.DEFAULT_TARGET_BUFFER_BYTES,
true
));
vwExoPlayer.setPlayer(mPlayer);
mPlayer.addListener(mVideoStreamHandler);
mPlayer.addVideoListener(new VideoListener() {
#Override
public void onVideoSizeChanged(int width, int height, int unappliedRotationDegrees, float pixelWidthHeightRatio) {
Log.d("hasil", "onVideoSizeChanged: w:" + width + ", h:" + height);
String res = width + "x" + height;
resolution.setText(res);
}
#Override
public void onRenderedFirstFrame() {
}
});
Where createSource() is as follows:
private void createSource() {
mMediaSource180 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_180));
mMediaSource360 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_360));
mMediaSource720 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_720));
mMediaSourceAudio = factory.createMediaSource(Uri.parse(API.GAME_AUDIO_STREAM_URL));
}
My current problem is that only the first three ExtractorMediaSources work fine in Exoplayer. The mMediaSourceAudio refuses to play in Exoplayer, but works just fine in the VLC Media Player for Android.
Right now I have a suspicion that the format is AAC-LTP, or whatever AAC variant that requires a codec available in VLC but not in default Android. However, I do not have access to the encoding process so I don't know for sure.
If this isn't the case, what is it?
EDIT:
I've been debugging the BandwidthMeter and added a MediaSourceEventListener. When I use the normal Video sources, onDownstreamFormatChanged() gets called, but not when I use that Audio Stream source.
In addition, the BandwidthMeter works fine, with bytes always downloaded in all parts of the stream and more bytes when the video stream comes in, but only in the Audio only stream that, when I call mPlayer.getBufferedPosition(), the returned value is always 0. Also, when I use the Audio Stream source, no OMX code was called - no decoders were set up.
Am I seeing a malformed audio stream, or do I need to change my Exoplayer's settings?
EDIT 2:
Further debugging reveals that, in all the Video streams and Audio stream, the same FlvExtractor is used. Even though the Video streams have the avc video track encoding and mp4a-latm audio track encoding. Is this normal?

Turns out it's because the stream was recognized to have two tracks/sampleQueues. One Audio track, and one track with null format. That null track was supposed to be the video track, which was supposed to exist according to the stream's flvHeader flag.
For now, I get around this by creating a custom MediaSource using a custom MediaPeriod. Said custom MediaPeriod having code to separate the video and audio tracks of the SampleQueues, then using the audio-only SampleQueue[] instead of the source SampleQueue[] when I want to play the audio-only stream.
Though this gives me another point of concern: There's something one can do to alter the 'has audio track (flag & 0x04) and video track (flag & 0x01)' flag in the rtmp stream, right?

Thanks for the comments, I'm new to ExoPlayer. But your comments helped me in debugging and getting multiple workarounds to the issue.
I tried to use custom MediaSource and custom MediaPeriod to address this audio issue. I have observed video format data coming after audio data incase of video+audio wowza stream, so the function maybeFinishPrepare() will wait for getting both video and audio format tag data before invoking onPrepared, incase if video tagData is received first. Incase of audio data received first, it wont wait and will call onPrepare().
With the above changes, I was able to play audio alone and video_audio wowza streams, where rtmp tagHeader with tagTypes were coming in the order of video tagData and then followed by audio data.
I wasn't able to use the same patch with srs server to play both audio_only and video_audio streams with the same changes. srs server is giving tagData in the order of audio and then video tagData,
So, I debugged further in FlvExtractor. In readFlvHeader, I have overriden the hasAudio and hasVideo variables. These variables will be set based on the first few tagHeaders(5 or 6). I used peekFully on input for 6 times in a loop. In each loop after fetching tagType and tagDataSize, tagDataSize is used to input.advancePeekPosition(), and tagType is used to identify whether we have audio/video format data in tagData. After peeking for first 6 consecutive tagHeaders, I was able to get actual values of hasAudio and hasVideo, and ignored the flvHeaders.flags, which were used to set these variables.
Custom FlvExtractor workaround, looked cleaner than custom MediaSource/MediaPeriod, as we will create those many tracks as necessary, as we are setting proper hasVideo/hasAudio values.

Stream radio on Android

I'm developing internet radio using android MediaPlayer. I need to update UI when the song is over and new song begins. How to find, that song is over?
This is my current code. mMediaPlayer.start(); calls when user tap the button.
mMediaPlayer = new MediaPlayer();
mMediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
try {
mMediaPlayer.setDataSource(streamURL);
mMediaPlayer.prepareAsync();
} catch (IOException e) {
e.printStackTrace();
}

Radio streaming is endless so the streaming is never going to end, however if what you can do is track the metadata in the radio streaming for the currently playing song, and if you detect that something has changed, then do something.
Something more important that I want to point is that MediaPlayer is awful slow for radio streaming, I developed a radio streaming app myself so I strongly reccomend you to use this library for this, for two reasons, one is faster and has a bigger buffer than the MediaPlayer (MediaPlayer class has at some point a hardcoded byte[] buffer = new byte[4096]; buffer that'll bring you some issues) and second because this library lets you handle the metadata in a more confortable way.

Set AudioStreamType for an audio file played from Resource (raw folder)

[First App] I am creating a sort of Alarm app that allows user to select alarm sound from either sd-card or app-supplied sounds. Since, the app essentially plays alarms, I want the volume to be 'alarm volume' of the device. I am able to achieve this for sd card sounds. But, I am unable to setAudioStreamType for raw resource sounds.
I am using following code :
MediaPlayer m_player = new MediaPlayer();
m_player.setAudioStreamType(AudioManager.STREAM_ALARM);
switch (bin_name) { //bin_name = various user selectable music files
default:
m_player = MediaPlayer.create(context, R.raw.blu);
break;
}
m_player.setLooping(true);
m_player.start();
My blu.mp3 plays at media volume only. Upon checking the documentation for MediaPlayer.create(Context context, int resid), I found this :
Note that since prepare() is called automatically in this method, you cannot change the audio stream type (see setAudioStreamType(int)), audio session ID (see setAudioSessionId(int)) or audio attributes (see setAudioAttributes(AudioAttributes) of the new MediaPlayer.
I also tried finding code samples for above method but none of them showed how to set AudioStreamType to AudioManager.STEAM_ALARM. I will accept answers with alternative ways that simply play the sound with infinite loop. How to achieve this ?

As the documentation you are referring to says, you must create and prepare the MediaPlayer yourself. Haven't tried with the STREAM_ALARM but I'm using following snippet to play on STREAM_VOICE_CALL
Uri uri = Uri.parse("android.resource://com.example.app/" + R.raw.hdsweep);
MediaPlayer mMediaPlayer = new MediaPlayer();
mMediaPlayer.setAudioStreamType(AudioManager.STREAM_VOICE_CALL);
mMediaPlayer.setDataSource(context, uri);
mMediaPlayer.prepare();
mMediaPlayer.start()

Using AudioTrack in AudioTrack.MODE_STATIC?

I've found lots of tutorials and posts showing how to use AudioTrack to play wav files in AudioTrack.MODE_STREAM and I've successfully implemented this example.
However I'm having issues with performance when playing multiple audio tracks at once and thinking that I should first create the tracks using AudioTrack.MODE_STATIC then just call play each time.
I can't find any resources on how to implement this. How can I do this?
Thanks

The two main sticking points for me were realizing that .write() comes first and that the instantiated player must have the size of the entire clip as the buffer_size_in_bytes.
Assuming you have recorded a PCM file using AudioRecord, you can play it back with STATIC_MODE like so...
File file = new File(FILENAME);
int audioLength = (int)file.length();
byte filedata[] = new byte[audioLength];
try{
InputStream inputStream = new BufferedInputStream(new FileInputStream(FILENAME));
int lengthOfAudioClip = inputStream.read(filedata, 0, audioLength);
player = new AudioTrack(STREAM_TYPE, SAMPLE_RATE, CHANNEL_OUT_CONFIG, AUDIO_FORMAT,audioLength, AUDIO_MODE);
player.write(filedata, OFFSET, lengthOfAudioClip);
player.setPlaybackRate(playbackRate);
player.play();
}