I'm trying to build a system that live-streams video and audio captured by android phones. I want to use media recorder to encode the data and then send it over RTP but the problem is how can i get the encoded data in a buffer.
You can't. At least you can't without some hacks. Media recorder does not support writing to buffers.
The trick is to create pipe, extract pipe descriptor and pass it to setOutputFile(FileDescriptor fd) function. There are some issues with this approach, as MediaRecorder does not write media content in stream-oriented way. In other words, it relies on the fact that it can navigate back through file and write some package headers later on.
More details on this can be found here: Broadcasting video with Android - without writing to local files
I found two other options (I haven't tried either):
A FileDescriptor to a memory buffer: http://www.devdaily.com/java/jwarehouse/android/core/java/android/os/MemoryFile.java.shtml
Android 4.0 implements the OpenMAX multimedia API: http://developer.android.com/about/versions/android-4.0-highlights.html
The latter is probably your best bet.
Related
I have a video stream source that sends bytes of H.264-encoded video. I'd like to build an application with Android's WebRTC classes, to send this video stream to a WebRTC peer. These built-in classes seem to only support raw video sources... not video already processed by a codec.
I simply need to create an offer with only one video codec/bitrate configuration. For my use case, I don't need to autoscale the bandwidth usage, nor offer any codecs other than the original H.264 stream of bytes.
Is there a way to utilize the built-in Android WebRTC classes for this? If not, is there another set of WebRTC classes? Or, must I re-implement something to create that SDP offer and do all the peer connectivity and what not?
What is the best (performance wise) way to get and stream a video from an android device's camera to a PC?
I have seen this question asked here before and there exist a few open source programs that do just that, but there exist multiple ways from which I don't know which one is the best!
for example:
Should the android part be written in c++ or java (performance/api wise)?
Which api should I use to get the video from camera?
What is the best way to stream the video?
I don't intend to support old android versions (<4.x), so if the best way/api is relatively new it's fine by me.
I'm not familiar with Android development but I'll try to answer.
I suppose that the actual encoding of the raw image data is probably done on hardware chip (otherwise software encoding would probably kill your battery) and it looks like MediaCodec class is exactly what you need. I suppose you want to implement some kind of live streaming service and the latency is important. If so, then you should stick to UDP based transmission methods. Using RTP protocol or MPEG-TS container format would be the best choice for this purpose. Of course you can also use TCP based methods for streaming like HLS or DASH (both of them use HTTP).
You should also take a look at Table 1 Core media format and codec support:
It tells us for example that using H.264 AVC Encoder supports MPEG-TS container and that HLS version 3 is also supported for Android 4.0 and above.
I'm trying to make a H.264 sniffer for my android distribution.
Basically, what I want is to just dump any H.264 stream that is passed from a lambda android application by intercepting it on its way to the hardware video decoder.
I have an odroid-c1 board (AmLogic S805 SoC) and my android build setup is ready. Now, what I need to know is where is the code called when a new H.264 frame is ready to be sent to decoding. Surely there must be a common place ?
When searching the build tree for files referencing H.264 or OMX, I have various results including libstagefright/omx/, ffmpeg/libavcodec/, LibPlayer/amffmpeg/ (Amlogic's own fork of FFmpeg) [..].
If you have any idea or name of functions that are part of the video decoding path, I'll take them :). Thanks !
As far as I know the Amlogic CPU contains two DSPs for Audio and Video decoding. Data is delivered to them through the amports driver in kernel space.
The userspace part of this driver is the libamcodec which provides a thin layer to this driver.
I do not know what kind of layers in Android are invloded but most probably they use Amlogic ffmpeg which is using libamplayer as middle layer which calls in the end libamcodec....
Updating my previous question...
Currently I am working on a live media streaming application. I have been able to use the Media Recorder class to record medias and store it in a file. Currently I am trying to stream this media from android to Adobe Media streamer using RTP. What I have done us using the parcel file descriptor I have packaged the video contents which are captured from the device camera. Now I would want to use the RTPPackets class provided by Sipdroid to convert this packets into RTP packets.
I am having problems integrating this RTP Packets and using the same in to my application as I see there are multiple steps to be done
1) Creating RTP packets, even though I have got the code for RTP packetizer I am not exactly sure how to use the same and where to use this. Because I clearly not sure on what would be the default payload value, ssrc and ccrc values. May the first time these will carry the default values but the second time onward what would set the values for these parameters would it be the parcel file descriptor?
2) Creating a simple server kind of code on the mobile which continues to create the RTP packets and keep sending them to the Adobe media server?
Any help would be much appreciated.
Thanks,
K.Saravanan.
I am streaming live video from my camera on my android phone to my computer using the MediaRecorder class.
recorder.setCamera(mCamera);
recorder.setVideoSource(MediaRecorder.VideoSource.CAMERA);
recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
recorder.setOutputFile(uav_UDP_Client.pfd.getFileDescriptor());
recorder.setVideoEncoder(MediaRecorder.VideoEncoder.H264);
That's the basic idea. So I would like to show this stream in real time. My plan is to use FFMpeg to turn the latest frame into a .bmp and show the .bmp on my C# program every time there is a new frame.
The problem is there is no header until I stop the recording. So I can not use FFMpeg unless there is a header. I've looked at spydroid and using RTP but I do not want to use this method for various reasons.
Any ideas on how I can do this easily?
You can consider streaming a MPEG2 TS and playing it back on your screen or you can also stream H.264 data over RTP and use a client to decode and display the same.
In Android, there is a sample executable which performs RTP packetization of H.264 stream and streams it over the network. You can find more details about the MyTransmitter from this file, which could serve as a good reference to your solution.
Additional Information
In Android 4.2.0 release onwards, there is a similar feature supported by the framework called Miracast or Wi-Fi Display which is standardized by Wi-Fi forum, which is a slightly complex use-case.