I have a video stream source that sends bytes of H.264-encoded video. I'd like to build an application with Android's WebRTC classes, to send this video stream to a WebRTC peer. These built-in classes seem to only support raw video sources... not video already processed by a codec.
I simply need to create an offer with only one video codec/bitrate configuration. For my use case, I don't need to autoscale the bandwidth usage, nor offer any codecs other than the original H.264 stream of bytes.
Is there a way to utilize the built-in Android WebRTC classes for this? If not, is there another set of WebRTC classes? Or, must I re-implement something to create that SDP offer and do all the peer connectivity and what not?
Related
I am streaming live video from my camera on my android phone to my computer using the MediaRecorder class.
recorder.setCamera(mCamera);
recorder.setVideoSource(MediaRecorder.VideoSource.CAMERA);
recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
recorder.setOutputFile(uav_UDP_Client.pfd.getFileDescriptor());
recorder.setVideoEncoder(MediaRecorder.VideoEncoder.H264);
That's the basic idea. So I would like to show this stream in real time. My plan is to use FFMpeg to turn the latest frame into a .bmp and show the .bmp on my C# program every time there is a new frame.
The problem is there is no header until I stop the recording. So I can not use FFMpeg unless there is a header. I've looked at spydroid and using RTP but I do not want to use this method for various reasons.
Any ideas on how I can do this easily?
You can consider streaming a MPEG2 TS and playing it back on your screen or you can also stream H.264 data over RTP and use a client to decode and display the same.
In Android, there is a sample executable which performs RTP packetization of H.264 stream and streams it over the network. You can find more details about the MyTransmitter from this file, which could serve as a good reference to your solution.
Additional Information
In Android 4.2.0 release onwards, there is a similar feature supported by the framework called Miracast or Wi-Fi Display which is standardized by Wi-Fi forum, which is a slightly complex use-case.
My android app needs to play a live video from a remote RTMP server (Adobe Flash Media Server).
As android's android.media.MediaPlayer doesn't support rtmp protocol, I found a library which provides me with streaming functionality. I.e. I can connect and receive portions of video stream as byte array.
The question is how I can use this incoming video stream data to display it in a view?
It seems that current standard API doesn't allow me to do that. MediaPlayer accepts either file or url. For audio data there is an AudioTrack which allows to achieve similar goal to mine, but for audio. For video I don't see an option.
Any suggestions are appreciated, maybe there is a third party media library which may provide the functionality for Android.
I want to stream video recording from my android phone to network media server.
The first problem is that when setting MediaRecorder output to socket, the stream is missing some mdat size headers. This can be fixed by preprocessing that stream locally and adding missing data to stream in order to produce valid output stream.
The question is how to proceed from there.
How can I go about output that stream as an RTMP stream?
First, let's unwind your question. As you've surmised, RTMP isn't currently supported by Android. You can use a few side libraries to add support, but these may not be full implementations or have other undesirable side effects and bugs that cause them to fail to meet your needs.
The common alternative in this case is to use RTSP. It provides a comparable session format that has its own RFC, and its packet structure when combined with RTP is very similar (sans some details) to your desired protocol. You could perform the necessary fixups here to transmute RTP/RTSP into RTMP, but as mentioned, such effort is currently outside the development scope of your application.
So, let's assume you would like to use RTMP (invalidating this thread) and that the above-linked library does not meet your needs.
You could, for example, follow this tutorial for recording and playback using Livu, Wowza, and Adobe Flash Player, talking with the Livu developer(s) about licensing their client. Or, you could use this client library and its full Android recorder example to build your client.
To summarize:
RTSP
This thread, using Darwin Media Server, Windows Media Services, or VLC
RTMP
This library,
This thread and this tutorial, using Livu, Wowza, and Adobe Flash Player
This client library and this example recorder
Best of luck with your application. I admit that I have a less than comprehensive understanding of all of these libraries, but these appear to be the standard solutions in this space at the time of this writing.
Edit:
According to the OP, walking the RTMP library set:
This library: He couldn't make the library demos work. More importantly, RTMP functionality is incomplete.
This thread and this tutorial, using Livu, Wowza, and Adobe Flash Player: This has a long tutorial on how to consume video, but its tutorial on publication is potentially terse and insufficient.
This client library and this example recorder: The given example only covers audio publication. More work is needed to make this complete.
In short: more work is needed. Other answers, and improvements upon these examples, are what's needed here.
If you are using a web-browser on Android device, you can use WebRTC for video capturing and server-side recording, i.e with Web Call Server 4
Thus the full path would be:
Android Chrome [WebRTC] > WCS4 > recording
So you don't need RTMP protocol here.
If you are using a standalone RTMP app, you can use any RTMP server for video recording. As i know Wowza supports H.264+Speex recording.
I am receiving the MPEG-TS (MPEG transport stream) packets with the multiplexed H.264 video and AAC audio streams. I need to be able to show the audio and video on the Android phone. My assumption is that I need:
MPEG-TS de-multiplexer
AAC decoder
H.264 decoder
Synchronize the audio and video playback
Assuming that I am right then (in Android 2.x) MPEG-TS de-multiplexer is not part of the OS and must be ported, both AAC and H.264 decoder are part of the Android OS, but I am not sure if they have interface, which allows passing the data in buffers and if they allow mutual timing synchronization. In the worst case those components must be ported here as well.
Can you give me some advices where to start? I was thinking about the FFMPEG porting. Are there any other ways?
Regards,
STeN
Android 4.x has OpenMAX which can play TS with H264 and AAC. You don't even need to worry about synchronisation of audio and video.
Look at the nativemedia sample in the NDK.
If you want to support previous versions of Android, then ffmpeg might be a good choice, but it the maximum it can give you is just decoded video frames in RGB or any other format and decoded audio in PCM. Then you will have to implement renderer and audio playback yourself. I would recommend reading this tutorial - http://dranger.com/ffmpeg/. It is not android specific but it will give you idea how video play works.
You may refer to the android-ffmpeg project on github.
https://github.com/guardianproject/android-ffmpeg
In Gingerbread ( 2.3 ), actually there is a MPEG TS parser in the stagefright framework that you could use. Also, I believe it is well integrated with H264 and AAC decoders. MPEG TS parser is not advertised anywhere but the support is silently sitting there. I believe they have brought it to support Apple HTTP Live streaming in HC or later version but the code is sitting there in the Gingerbread ( 2.3 ) codebase as well. With a minor modification in the framework, you can playback http live streaming ( which actually sends TS packets). I guess the above information would be helpful for you.
Vibgyor
(DISCLAIMER: I'm personally involved in developing the free and open source program linked below)
A static version of FFMpeg (both library and commandline) is provided by ZShaolin http://dyne.org/software/zshaolin also contains other media conversion tools.
Its use can facilitate scripting experiments without having to compile FFMpeg from scratch.
I am going to make a PTT project on Android. Could you tell me how deep Android supports Voice and Multimedia API (such as RTP,RTSP,VoIP) for developers?
MediaPlayer supports playing rtsp://.. URLs. Audio and Video are supported. Check media format support to see which codecs are supported.
MediaPlayer internally automatically handles RTSP and RTP, so there is not much you need to handle. OTOH it does not give any low-level control over this process.
About VoIP: Android only consumes RTSP/RTP streams, but does not create/publish them. So this is one-way only.
Android 2.3 (Gingerbread) is said to have support for SIP-based VoIP.
Use AudioTrack\AudioRecord APIs for streaming audio recording and playback. They allow you to deal with raw audio frames.
RTP packets are pretty straightforward - it's just a packet that contains some minimal header data and a raw audio frame. It is easy to implement - check out SipDroid for a reference implementation of RTP packets.