I am developing a hybrid mobile application (using Ionic 2) that allows users to overlay audio on a video. Essentially, the video and audio track are able to play at the same time. The challenge is getting this behavior with Google Cast functionality – I want the user to be able to Cast the video as well as the separate audio overlay to their Cast device.
I’ve decided to develop a custom Cordova plug-in (starting with Android) that implements the Google Cast API natively for the following reasons: Google Cast Web API depends on the Chrome browser so our hybrid app running in Cordova’s WebView won’t work, and the most up-to-date Cordova plugin that I have seen A) doesn’t implement this audio overlay functionality and B) appears to be abandoned since Sept 2015.
That’s the background, now the question.
How I understand it so far, generally the Cast API works by sending the Receiver Application a URI, and the Receiver Application takes care of fetching this resource either from a server or from the Sender app’s resources. This poses a problem for me: my application utilizes two resources simultaneously.
I saw here that MediaInfo can represent a grouping of MediaTracks, which could be audio, video, text…etc. Am I able to have multiple MediaTracks active at the same time? Should I be exploring custom implementations in a Custom Receiver App to enable multiple active MediaTracks?
Should I look into demuxing the mp4, mixing the audio streams and muxing into a separate, temporary mp4 file, and handing the URI of this temp mp4 off to the Cast Device?
I’m in the research phase right now, but will be implementing and testing various solutions over the next few weeks. What other creative solutions can anyone think of? Has anyone done this before? And lastly, can anyone say for certain that this cannot be done?
Any help/advice is appreciated.
Cast SDK on the receiver doesn't support more than one active media element, so even if you write a custom receiver and include two media elements, one for video and one for audio, only one can be active at a time, so that is not going to work. If you can mix them into one mp4, then that is going to be the best approach and can work with Default or Styled receivers as well (hence no need to write a custom one).
Related
I am planning on having a YouTube player in my android app and found two alternatives: IFRAME API or with YouTube Android Player API. So far I don't find any reference for comparing the two approaches.
I am new to both so I don't have a good background to compare both (yet). But so far this is what I know:
IFRAME
Pro: Don't need to get Developer/App Key to access the API.
Pro: Don't need to include YouTubePlayer API's jar (don't increase your APK size)
Con: Unnecessary webview and javascript glue code to hook up
YT Player API:
Pro: Native Java, no need of javascript glue code
Con: Need YouTube app on the device
Con: Need to get Developer/App Key and need to include the API jar to your APK.
Playing with both, I don't know yet any perf/memory usage between the two. I also don't know if we can have more detailed events from the API vs through IFRAME.
I am trying to assess these but would like to hear if any of you have opinions on these.
Thank you
Here you can find a few reason for not using the YouTube Player API.
Overall I'd say: if need to do basic stuff (like using YouTubeBaseActivity/YouTubeStandalonePlayer) you can safely use it. If you need to use the YouTubePlayerFragment a WebView based approach may be a better idea.
Why should you consider not using the official library from YouTube?
If you’re concerned about stability:
The YouTube Android Player API is not the best API ever designed. You are probably going to be fine if you need to use the
YouTubeBaseActivity/YouTubeStandalonePlayer, but you’re going to run into issues with the YouTubePlayerFragment.
The library has some very old bugs, this one is the most significant I have encountered. While developing my app I kept running into it, seeing my app randomly crash for apparently no reason. It made my app unstable and never ready for production.
The bug is still there, as far as I know. A new version of the library should be in the making, but it has yet to be released.
If you don’t want to be tied to Google Play and the YouTube app:
In order to run an app that utilizes the YouTube Android Player API a device needs to have both Google Play and the last version of the YouTube app installed.
This may not be a limitation in most cases, since you’re probably going to distribute your app through Google Play. But I have talked with people that had this problem, maybe you care about it as well.
If you want more control over the player looks and behavior:
The YouTube Android Player API is not open source, therefore the customization you can do goes as far as the API allows to. Maybe you want to change the UI of the player or write some custom behavior specific for your use cases. That is going to be hard to do with the official library.
If you don’t want to register your app in the Google Developers Console
I am new to mobile programming, and I am making an application which will play video files. I am trying Azure media service (AMS) with Xamarin Forms. I would like to know how to it works to have a file that will play with Widevine on android and Fairplay on IOS.
I need explanations!
Thanks in advance!
The implementation on the client side for those different DRM's is about as different as you can imagine. Xamarin.Forms does not out of the box provide a component that can handle the DRM specific methods. You will likely need to create platform specific plugins for that or find a ready-made component for that - although in my search i haven't found that. The closest is a cross platform video player such as https://github.com/adamfisher/Xamarin.Forms.VideoPlayer
Also, you are not likely to get a file to play, it will likely be a stream with different manifests for the two platforms. Azure Media Services however can do that on the fly from a set of MP4's. So it might look like MP4's but what is actually served to the client is a video manifest file.
There is no library/component in Xamarin that allows you to handle multiple DRM schemes. Your best option is to use platform libraries wrapped, to be able to consume DRM content. I would check out the Inside Secure DRM solution, that allows the playback of the DRM Schemes that you listed.
Is it possible to get the audio coming from a different application? I'd like to perform app specific-modifications to the audio coming from other running applications. After a lot of searching, I still can't seem to find any examples or proof that you may be able to.
Is it possible to intercept audio data using google+ hangout api? I writing an app using g+ hangout for android and I would like to process the audio. To be precessive, I want to denoise speech and use speech-to-text (e.g. google search, sphinx) to make basic voice commands.
Because I have full control of the android app it doesn't matter for me if I will have a callback with audio data from hangout or I can record audio using android AudioRecorder and then somehow forward those data to google hangout (Though the latter solution would be better because we can denoise on the android device). Actually I would be happy with any feasible workaround that may work at this stage of the API.
The Hangouts API is not going to help you develop this feature.
What you need is a platform agnostic API for accessing hangouts data. The API is instead intended to solve a different problem. It allows you to write HTML/JavaScript applications that run inside the canvas of hangouts running on desktop web browsers.
One possible "workaround" that I'm currently investigating, myself—
publish the hangout "on air"
get the youtube live id (available as of 2012-08-22, roughly... since Hangout API 1.2) ~ gapi.hangout.onair.getYouTubeLiveId() https://developers.google.com/+/hangouts/api/gapi.hangout.onair#gapi.hangout.onair.getYouTubeLiveId (note that this can only be grabbed by the host?)
grab http://www.youtube.com/watch?v=${LIVEID} // suggestion: look at youtube-dl: http://rg3.github.com/youtube-dl/documentation.html
and then use ffmpeg to process the flv
Information for this answer was primarily grabbed from Downloading videos in flv format from youtube. and http://grokbase.com/t/gg/google-plus-developers/128fbteedb/google-hangout-api-url-to-youtube-stream-screenshot-and-hangout-topic
For my question, time is more important than perfection. I need any advice you can quickly give me about embedding a video in a mobile website which is targeted for Android devices. Since I have only very little time for research (for the first research, only today, but of course the implementation will follow later), I need any advice you can give to me, no matter how vague or imperfect it may be.
Here is the scenario:
A video must be embedded in a mobile website, using HTML.
Only Android clients are relevant, because iPhones and non-Android devices will be directed to another URL.
Preferably, the video should be integrated into the rest of the web page. In case an external player needs to be opened, it would be preferable if that player has a "return to web page" button of some kind that will return the user to the web page.
The customer is aware that this will not work perfectly on all devices, but we want to find a solution that works well at least on the more modern devices.
If you have any experience regarding this field, please let me know of it. Here are some areas that might be helpful:
HTML code that you have found to be working
Encoding specifics regarding the video (e.g. which codecs to use or not to use)
Other StackOverflow questions regarding this topic which I have overseen in my haste (sorry about that, time is sparse for me right now)
Since I can only choose one of the replies to be the answer, my choice for that has to be very arbitrary.
So far, I have found these two other StackOverflow questions, which cover parts of this topic:
What video format will play on all Android devices?
Embed video and play on android
What about embedding flash? Not all android devices have it but the ones have it should be able to play properly encoded mp4 video
For video encoding check http://developer.android.com/guide/appendix/media-formats.html