I am receiving the MPEG-TS (MPEG transport stream) packets with the multiplexed H.264 video and AAC audio streams. I need to be able to show the audio and video on the Android phone. My assumption is that I need:
MPEG-TS de-multiplexer
AAC decoder
H.264 decoder
Synchronize the audio and video playback
Assuming that I am right then (in Android 2.x) MPEG-TS de-multiplexer is not part of the OS and must be ported, both AAC and H.264 decoder are part of the Android OS, but I am not sure if they have interface, which allows passing the data in buffers and if they allow mutual timing synchronization. In the worst case those components must be ported here as well.
Can you give me some advices where to start? I was thinking about the FFMPEG porting. Are there any other ways?
Regards,
STeN
Android 4.x has OpenMAX which can play TS with H264 and AAC. You don't even need to worry about synchronisation of audio and video.
Look at the nativemedia sample in the NDK.
If you want to support previous versions of Android, then ffmpeg might be a good choice, but it the maximum it can give you is just decoded video frames in RGB or any other format and decoded audio in PCM. Then you will have to implement renderer and audio playback yourself. I would recommend reading this tutorial - http://dranger.com/ffmpeg/. It is not android specific but it will give you idea how video play works.
You may refer to the android-ffmpeg project on github.
https://github.com/guardianproject/android-ffmpeg
In Gingerbread ( 2.3 ), actually there is a MPEG TS parser in the stagefright framework that you could use. Also, I believe it is well integrated with H264 and AAC decoders. MPEG TS parser is not advertised anywhere but the support is silently sitting there. I believe they have brought it to support Apple HTTP Live streaming in HC or later version but the code is sitting there in the Gingerbread ( 2.3 ) codebase as well. With a minor modification in the framework, you can playback http live streaming ( which actually sends TS packets). I guess the above information would be helpful for you.
Vibgyor
(DISCLAIMER: I'm personally involved in developing the free and open source program linked below)
A static version of FFMpeg (both library and commandline) is provided by ZShaolin http://dyne.org/software/zshaolin also contains other media conversion tools.
Its use can facilitate scripting experiments without having to compile FFMpeg from scratch.
Related
I am searching for a library which offer ability for streaming video from android device (5.1+) and recording it at the same time.
I tried MediaRecorder - the usual way to record videos on android - but with it I am not able to stream it over webrtc or rtsp because camera is busy.
Currently I am using libstreaming. With little modification done app can record and stream over rtsp concurrently. But this lib lacks support for hardware codec in MTK and SPRG chipsets.
I am wonder if you can recommend a solution or another lib which.
By the moment lib works only on nexus 4 with qcom chipset.
After several days of research, I came to the decision to use a combination of FFMpeg and MediaCodec.
It seems that the only way to get frames from camera at high rate is to use Android MediaCodec API. But MediaCodec supports only mp4 file formats, which is not an option for me (I need ts), while FFMpeg can process\create any kind of human known video formats.
Currently I am trying to make it work together (read ByteBuffer from MediaCodec and feed FFMpeg recorder with it).
Useful links:
Grafika project: https://github.com/google/grafika
ContinuousCapture and Show + record are the most interesting parts to check
javacpp (specifically FFMpeg wrapper): https://github.com/bytedeco/javacpp
Has example with recording and streaming.
kickflip sdk: https://github.com/Kickflip/kickflip-android-sdk
The library which makes two mentioned above tools works together and also is open sourced. Sadly it doesn't solve my problem fully. The feature I need is requested but not already implemented: https://github.com/bytedeco/javacv/issues/95
In my project I need to implement an HLS (HTTP live Streaming) for an android device and it stream to an iOS device to play where android device will record the video and send it to server and iOS device will play the stream from the server using an m3u8 file. In the link below
Click Here
They have mention "Currently, the supported delivery format is MPEG-2 Transport Streams for audio-video".
Now problem is that in android you can record only in mp4 by default (correct me if i am wrong). Now I need some third party API or library like ffmpeg, Gstreamer, Xuggler, Jcodec to transcode recorded mp4 to ts files.
ffmpeg, jffmpeg and Gstreamer have a learning curve and to setup time and also need NDK. So I need some help because I don't have enough time to try one of these please refer me if you know any library which is easy to use and does not have a complex learning and setup time. Like Jcodec which is pure java base and plug and play type library but I don't think it can do this for me as they have mention in there documentation they support h262 codec support yet but i need h264 and ACC for audio.
FYI:
JJPMEG
It is a Java binding to FFmpeg and it have an android verison too. Maybe you can give it a try.
https://code.google.com/p/jjmpeg/
Or:
Maybe you can just record the video with supporting encoding and transcode the video in the server side?
Is it possible to invoke(deploy) HTTP Live Streaming (HLS) on Android(4.x)?
https://developer.apple.com/streaming/
Obviously iOS devices can both capture/play, and I know android can at least play, but how about capturing? I wonder interoperability.
Thanks.
The best answer I found so far is
Creating a HLS video stream with FFmpeg
12 May 2013
http://walterebert.com/blog/creating-on-hls-video-stream-with-ffmpeg/
For video conversion I use FFmpeg. Creation of HLS is possible with FFmpeg, but not really well documented. So I had to figure out how to create the video streams. After a lot of research and experimentation I created my FFmpeg HLS reference implementation that is available on Bitbucket.
On iOS the created video plays without problems on new devices. Older iOS devices with a maximum resolution of 480×320 pixels seem to select the best quality stream available, even if they cannot play it. For Android you have to create a MP4 video and before converting it into a MPEG stream. Doing this in a single command creates a choppy stream on Android. Flash playback has still some issues if you change the bitrate. So I still have some work to do.
These are the writings of Walter Ebert on web development, web design and free, open source software
Yes. HLS is widely used on Android 4.x.
I want to stream video recording from my android phone to network media server.
The first problem is that when setting MediaRecorder output to socket, the stream is missing some mdat size headers. This can be fixed by preprocessing that stream locally and adding missing data to stream in order to produce valid output stream.
The question is how to proceed from there.
How can I go about output that stream as an RTMP stream?
First, let's unwind your question. As you've surmised, RTMP isn't currently supported by Android. You can use a few side libraries to add support, but these may not be full implementations or have other undesirable side effects and bugs that cause them to fail to meet your needs.
The common alternative in this case is to use RTSP. It provides a comparable session format that has its own RFC, and its packet structure when combined with RTP is very similar (sans some details) to your desired protocol. You could perform the necessary fixups here to transmute RTP/RTSP into RTMP, but as mentioned, such effort is currently outside the development scope of your application.
So, let's assume you would like to use RTMP (invalidating this thread) and that the above-linked library does not meet your needs.
You could, for example, follow this tutorial for recording and playback using Livu, Wowza, and Adobe Flash Player, talking with the Livu developer(s) about licensing their client. Or, you could use this client library and its full Android recorder example to build your client.
To summarize:
RTSP
This thread, using Darwin Media Server, Windows Media Services, or VLC
RTMP
This library,
This thread and this tutorial, using Livu, Wowza, and Adobe Flash Player
This client library and this example recorder
Best of luck with your application. I admit that I have a less than comprehensive understanding of all of these libraries, but these appear to be the standard solutions in this space at the time of this writing.
Edit:
According to the OP, walking the RTMP library set:
This library: He couldn't make the library demos work. More importantly, RTMP functionality is incomplete.
This thread and this tutorial, using Livu, Wowza, and Adobe Flash Player: This has a long tutorial on how to consume video, but its tutorial on publication is potentially terse and insufficient.
This client library and this example recorder: The given example only covers audio publication. More work is needed to make this complete.
In short: more work is needed. Other answers, and improvements upon these examples, are what's needed here.
If you are using a web-browser on Android device, you can use WebRTC for video capturing and server-side recording, i.e with Web Call Server 4
Thus the full path would be:
Android Chrome [WebRTC] > WCS4 > recording
So you don't need RTMP protocol here.
If you are using a standalone RTMP app, you can use any RTMP server for video recording. As i know Wowza supports H.264+Speex recording.
I have simplified my question and offered a bounty:
What options are there for compressing raw PCM audio data to a mp3 on a Android device.
My original post:
I'm creating a synthesiser on my Android phone, and I've been generating PCM data to send to the speakers. Now I'm wondering if I can encode this PCM data as a mp3 to save to the sdcard. The MediaRecorder object can encode audio coming from the microphone into various formats, but doesn't allow the encoding from programmatically generated audio data.
So my question is, is there a standard Android API for encoding audio? If not, what pure Java or NDK based solutions are there? And can you recommend any of them?
Failing this I'll just have to save my generated audio as a WAV file, which I can easily do.
Pure Java
Look into Tritonus's clean room implementation of javasound which offers an MP3 encoder plugin here: http://www.tritonus.org/plugins.html
Secondly, I would suggest looking into jzoom's libraries JLayer or JLayerME: http://www.javazoom.net/javalayer/javalayer.html (this may only be decode, not sure)
If those doesn't suit your need you can look at this article from 2000 about adding MP3 capabilities to J2SE (with source): http://www.javaworld.com/javaworld/jw-11-2000/jw-1103-mp3.html
Native route
If you want "native" performance I would look at an FFmpeg or Lame port for Android.
Lame: http://lame.sourceforge.net/
As far as i know you can't do this using only the tools in the SDK. According to the official developer guide there isn't an MP3 encoder in the platform (Android Supported Media Formats), so you have to port an encoder on your own using the NDK, then write some wrapper code to receive the audio samples through JNI.
I'm currently working on porting some audio decoders from the Rockbox project for my own music player, and it can record audio into MP3, so maybe you should try to look into it's source and find the encoder library. Most of the decoders have ARM optimalizations which speeds up things noticable, so i guess some of the encoders have also this addition.
Mp3 encoder is not available in android.you have to compile libav with mp3 lame lib you can find code from
http://libavandroid.wordpress.com