Develop Reference

An AAC audio stream is playable in VLC for Android, but not in Exoplayer

I have an RTMP stream I want to play in my app using the Exoplayer library. My setup for that is as follows:
TrackSelector trackSelector = new DefaultTrackSelector();
RtmpDataSourceFactory rtmpDataSourceFactory = new RtmpDataSourceFactory(bandwidthMeter);
ExtractorsFactory extractorsFactory = new DefaultExtractorsFactory();
factory = new ExtractorMediaSource.Factory(rtmpDataSourceFactory);
factory.setExtractorsFactory(extractorsFactory);
createSource();
mPlayer = ExoPlayerFactory.newSimpleInstance(mActivity, trackSelector, new DefaultLoadControl(
new DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE),
1000, // min buffer
3000, // max buffer
1000, // playback
2000, //playback after rebuffer
DefaultLoadControl.DEFAULT_TARGET_BUFFER_BYTES,
true
));
vwExoPlayer.setPlayer(mPlayer);
mPlayer.addListener(mVideoStreamHandler);
mPlayer.addVideoListener(new VideoListener() {
#Override
public void onVideoSizeChanged(int width, int height, int unappliedRotationDegrees, float pixelWidthHeightRatio) {
Log.d("hasil", "onVideoSizeChanged: w:" + width + ", h:" + height);
String res = width + "x" + height;
resolution.setText(res);
}
#Override
public void onRenderedFirstFrame() {
}
});
Where createSource() is as follows:
private void createSource() {
mMediaSource180 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_180));
mMediaSource360 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_360));
mMediaSource720 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_720));
mMediaSourceAudio = factory.createMediaSource(Uri.parse(API.GAME_AUDIO_STREAM_URL));
}
My current problem is that only the first three ExtractorMediaSources work fine in Exoplayer. The mMediaSourceAudio refuses to play in Exoplayer, but works just fine in the VLC Media Player for Android.
Right now I have a suspicion that the format is AAC-LTP, or whatever AAC variant that requires a codec available in VLC but not in default Android. However, I do not have access to the encoding process so I don't know for sure.
If this isn't the case, what is it?
EDIT:
I've been debugging the BandwidthMeter and added a MediaSourceEventListener. When I use the normal Video sources, onDownstreamFormatChanged() gets called, but not when I use that Audio Stream source.
In addition, the BandwidthMeter works fine, with bytes always downloaded in all parts of the stream and more bytes when the video stream comes in, but only in the Audio only stream that, when I call mPlayer.getBufferedPosition(), the returned value is always 0. Also, when I use the Audio Stream source, no OMX code was called - no decoders were set up.
Am I seeing a malformed audio stream, or do I need to change my Exoplayer's settings?
EDIT 2:
Further debugging reveals that, in all the Video streams and Audio stream, the same FlvExtractor is used. Even though the Video streams have the avc video track encoding and mp4a-latm audio track encoding. Is this normal?

Turns out it's because the stream was recognized to have two tracks/sampleQueues. One Audio track, and one track with null format. That null track was supposed to be the video track, which was supposed to exist according to the stream's flvHeader flag.
For now, I get around this by creating a custom MediaSource using a custom MediaPeriod. Said custom MediaPeriod having code to separate the video and audio tracks of the SampleQueues, then using the audio-only SampleQueue[] instead of the source SampleQueue[] when I want to play the audio-only stream.
Though this gives me another point of concern: There's something one can do to alter the 'has audio track (flag & 0x04) and video track (flag & 0x01)' flag in the rtmp stream, right?

Thanks for the comments, I'm new to ExoPlayer. But your comments helped me in debugging and getting multiple workarounds to the issue.
I tried to use custom MediaSource and custom MediaPeriod to address this audio issue. I have observed video format data coming after audio data incase of video+audio wowza stream, so the function maybeFinishPrepare() will wait for getting both video and audio format tag data before invoking onPrepared, incase if video tagData is received first. Incase of audio data received first, it wont wait and will call onPrepare().
With the above changes, I was able to play audio alone and video_audio wowza streams, where rtmp tagHeader with tagTypes were coming in the order of video tagData and then followed by audio data.
I wasn't able to use the same patch with srs server to play both audio_only and video_audio streams with the same changes. srs server is giving tagData in the order of audio and then video tagData,
So, I debugged further in FlvExtractor. In readFlvHeader, I have overriden the hasAudio and hasVideo variables. These variables will be set based on the first few tagHeaders(5 or 6). I used peekFully on input for 6 times in a loop. In each loop after fetching tagType and tagDataSize, tagDataSize is used to input.advancePeekPosition(), and tagType is used to identify whether we have audio/video format data in tagData. After peeking for first 6 consecutive tagHeaders, I was able to get actual values of hasAudio and hasVideo, and ignored the flvHeaders.flags, which were used to set these variables.
Custom FlvExtractor workaround, looked cleaner than custom MediaSource/MediaPeriod, as we will create those many tracks as necessary, as we are setting proper hasVideo/hasAudio values.

Decrypting audio/video on-the-fly to MediaPlayer

I have been looking around for few days, but unable to get a clear answer on how to do it.
I have some encrypted audio/video files on my sdcard, which I would like to play which it will send to MediaPlayer, and start playing while decryption is working on the same time. It kinds of creates a buffer initially, and when it has enough initial data to start playing, MediaPlayer will start playing, and on background decrypting and keep sending the data over.
I looked around and most of the solution seem to point to having a localhttpserver, and then send the file to the server, and when it comes back, it will start playing the audio/video like a live stream. What I don't know is where does the decryption code comes in? In the server side? If so, in which part of it?
I have also seen this http://libeasy.alwaysdata.net/ solution, that possibly able to solve it, but I can't seem to understand how that works. I know that it creates a localserver, but on the Cipher part, how does my own decryption comes in play?
private void myPlay(String path) {
mServer = new LocalSingleHttpServer();
mServer.setCipher(myGetCipher());
mServer.start();
path = mServer.getURL(path);
mVideoView.setVideoPath(path);
mVideoView.start();
}
public void onCompletion(MediaPlayer mp) { // MediaPlayer.OnCompletionListener interface
mServer.stop();
}
As I am quite new to Android development, so do bear with me if I am unclear or anything.
Thank you.

The decryption is done by the http server, if you provide it a Cipher.
Code something like this (you will have to add the exception catching somewhere):
import javax.crypto.Cipher;
private Cipher myGetCipher() {
Cipher c = Cipher.getInstance("RC4");
c.init(Cipher.DECRYPT_MODE, new SecretKeySpec("myPassword".getBytes(), "RC4"));
return c
}

Using MediaRecorder to write to a buffer or FIFO

I am developing a low data rate VoIP kind of project . I need to capture audio at low data rates and store it in an internal buffer or FIFO (NOT in a file).
I would like to use low data rate .AMR encoders, which means AudioRecord is out. MediaRecorder looks like it does exactly what I want except that it seems to write to a file.
MediaRecorder takes a FileDescriptor... is there any way I can write a class that implements the FileDescriptor interface... acting as a sync for bytes... but instead of sending them to a file they are stored in a buffer? The documentation on FileDescriptor specifically says that Applications shouldn't write their own but why not and is it possible anyway?
http://docs.oracle.com/javase/1.4.2/docs/api/java/io/FileDescriptor.html
In short, I'd like to develop my own stream, and trick MediaRecorder to send data to it. Perhaps doing something tricky with opening both ends of a socket within the same APK and giving MediaRecorder the socket to write to? Using the socket as my FIFO? I'm somewhat new to this so any help/suggestions greatly appreciated.
I have a related question on the RX side. I'd like to have a buffer/fifo that feeds MediaPlayer. Can I trick MediaPlayer to accept data from a buffer fed by my own proprietary stream?

I know its a bit late to answer this question now...
...But if it helps here's the solution.
Android MediaRecorder's method setOutputFile() accepts FileDescriptor as a parameter.
As for your need a unix data pipe could be created and its FD could be passed as an argument in the following manner...
mediaRecorder.setOutputFile(getPipeFD());
FileDescriptor getPipeFD()
{
final String FUNCTION = "getPipeFD";
FileDescriptor outputPipe = null;
try
{
ParcelFileDescriptor[] pipe = ParcelFileDescriptor.createPipe();
outputPipe = pipe[1].getFileDescriptor();
}
catch(Exception e)
{
Log.e(TAG, FUNCTION + " : " + e.getMessage());
}
return outputPipe;
}
The ParcelFileDescriptor.createPipe() creates a Unix Data Pipe and returns an array of ParcelFileDescriptors. The first object refers to the read channel (Source Channel) and the second one refers to the write channel (Sink Channel) of the pipe. Use MediaRecorder object to write the recorded data to the write channel...
As far as MediaPlayer is concerned the same technique could be used by passing the FileDescriptor object related to the created pipe's read channel to the setDataSource() method...

Can one retrieve data from a MediaPlayer's stream?

If I were to stream some sort of media to a MediaPlayer, is there any way I could copy it before/as/after it is played? For instance, if I were to stream a YouTube clip, is it possible to save that clip as it is being played?
Edit:
(Ocelot's answer made me realise how localised this question is).
What I am looking to do is copy the stream of a MediaPlayer already in progress (be it youtube or music stream). I want to be able to be notified when a new stream starts and ends. So far the only thing I found (for the latter) that is even remotely close it the broadcast string ACTION_AUDIO_BECOMING_NOISY but that doesn't really do anything for what I need. I there any way to do this?

I haven't tested this, and it looks like quite a bit of work, but here is what I would try:
Create a subclass of Socket. In this class, you can handle all byte reads, and save the stream locally or do whatever you want with it
Create your own content provider, which you can use to pass URIs to your media player, in your own format. Example: mystream://youtube.com/watch?v=3Rhy37u
In your content provider, override the openFile method and in it, open your own socket, and create a ParcelFileDescriptor with it.
Now, simply passing the new format url to your mediaplayer should make all streams go through your Socket, where you can save your data.

one way is to first find out where the video is in the server for exmaple in youtube with simple regex like this :
Regex("(?<=&t=)[^&]*").Match(file).Value;
you could retrieve url to the video and then download it like
public static void Download(string videoID, string newFilePath)
{
WebClient wc = new WebClient();
string file = wc.DownloadString(string.Format("http://www.youtube.com/watch?v={0}", videoID));
string t = new Regex("(?<=&t=)[^&]*").Match(file).Value;
wc.DownloadFile(string.Format("http://www.youtube.com/get_video?t={0}=&video_id={1}",t,videoID), newFilePath);
}
it's c# code but you could easily convert it to java.

#zrgiu
I tried to go with this solution, but the MediaPlayer retrieves a FileDescriptor from the URI, so sadly no http URL can be passed like this.
I also found another solution, it suggests to create a local ProxyServer on your device to serve files from the internet, it should be possible to also save the files streamed via the proxy.

Feeding data from memory to MediaPlayer

Scenario:
Have encrypted mp3 files in my .apk. Need to decrypt and send to MediaPlayer object.
Problem:
After I read the files and decrypt them, how do I get MediaPlayer to play them ?
Now. MediaPlayer has 4 versions of setDataSource().
setDataSource(String path)
setDataSource(FileDescriptor fd)
setDataSource(FileDescriptor fd, long offset, long length)
setDataSource(Context context, Uri uri)
None of which are ideal for the situation. Guess ideal would be to give MediaPlayer an InputStream ?
Possible solutions:
Write decrypted data to file play
that file. A lot of IO overhead.
Create a dummy http server
(ServerSocket ?) and pass the url to
MediaPlayer. Again, messy. Am I
even allowed to create a socket.
Does anyone have a better solution ?

byte[] callData = ...;
String base64EncodedString = Base64.encodeToString(callData, Base64.DEFAULT);
try
{
String url = "data:audio/amr;base64,"+base64EncodedString;
MediaPlayer mediaPlayer = new MediaPlayer();
mediaPlayer.setDataSource(url);
mediaPlayer.prepare();
mediaPlayer.start();
}
catch(Exception ex){
...
}

If you don't need all the functionality in MediaPlayer, I recommend trying AudioTrack. It's meant for basically what you describe. Unfortunately, MediaPlayer doesn't take an AudioTrack in its constructor, so the best solution in that case is to include a dummy Http server that sends your data back from a URL (which is what the Android 1.0 release notes recommends).

I'm not a 100% sure, but I don't think you have any other option than to temporarily save the the decrypted file before playing it.
This question is kind of similar, but I don't think you use the easy solution suggested there since you have an encrypted file. There is also provided a link to a tutorial for Custom Audio Streaming with MediaPlayer, but it seems like their solution also use a temporary file.