Develop Reference

getUserMedia green screen on Android

I'm trying to use getUserMedia to show the cam live stream on a web page. It works (if I save the stream to a file, it is ok) but the video tag show only a static green image as preview. Is anyone facing the same problem?
My setup:
Samsung S8 (SM-G950F)
Android 9
Chrome 107.0.5304.91
HTML
<video autoplay="true" width="100%" id="video-test"></video>
JS
var video = document.querySelector("#video-test");
navigator.mediaDevices.getUserMedia({
video: {
mandatory: { minFrameRate: 10, minWidth: 100, minHeigth: 100 },
}
})
.then(function (stream) {
video.srcObject = stream;
})
.catch(function (err0r) {
console.log("Something went wrong!", err0r);
});
Result
Thank you

Ah, the green screen of death. Why is it green? Because it's trying to display an all-zero YUV image.
Your constraints in your call to .getUserMedia() are incorrect. mandatory is no longer a thing. You need something like this:
{
video: {
width: { min: 100, ideal: 320 },
height: { min: 100, ideal: 240 },
frameRate: { min: 10, ideal: 10 }
}
}
It's an inconvenient truth that you have to muck around to get constraints values that work on many different platforms. And, you have to accept what gUM gives you. If you demand exact constraints using something like frameRate: { exact: 10 } you may get an OverconstrainedError exception, or some other exception.

Can not change camera in mobile Chrome or Safari during WebRTC call

I use navigator.mediaDevices.enumerateDevices to retrieve list of all video devices (element.kind === 'videoinput') and then call navigator.mediaDevices.getUserMedia(constraints) call to rotate video devices (using deviceId as constraint). Everything works fine on Windows Chrome / Firefox, but on android phone (tried Samsung, Asus, Huawei with Android 8/9) this call fails for back camera with NotReadableError / Could not start video source (for Chrome) or AbortError / Starting video failed (for Firefox).
Strangely same code works ok in iOS / Safari.
Also this only happens when WebRTC call is present in browser. If there is no call I can select any video device.
Also if I select back camera first and try to establish the call, it does not work, I get similar error.
I know it's far-fetched but maybe someone had same/similar issue?
All browser versions are up-to-date.
[UPDATE - code snippet and log]
switchCamera() {
try {
if (this.localStream) {
const tracks = this.localStream.getTracks();
console.log('switchCamera stopping this.localStream tracks', tracks);
tracks.forEach((track: MediaStreamTrack) => {
console.log('switchCamera stopping track', track);
track.stop();
});
console.log('switchCamera stop stream');
}
const constraints = {
audio: true,
video: { facingMode: this.faceCamera ? 'environment' : 'face' }
};
this.faceCamera = !this.faceCamera;
console.log('switchCamera constraints: ', constraints);
navigator.mediaDevices.getUserMedia(constraints)
.then(stream => {
console.log('getUserMedia:', stream);
this.logText('got stream');
this.localVideo.srcObject = stream;
const videoTracks = stream.getVideoTracks();
const audioTracks = stream.getAudioTracks();
console.log('videoTracks', videoTracks);
if (videoTracks.length > 0) {
console.log(`Using video device: ${videoTracks[0].label}`);
}
const videoTrack = videoTracks[0];
const audioTrack = audioTracks[0];
console.log('Replacing track for pc', videoTrack, audioTrack);
const pc = this.session.sessionDescriptionHandler.peerConnection;
const videoSender = pc.getSenders().find(s => {
return s.track && s.track.kind === videoTrack.kind;
});
const audioSender = pc.getSenders().find(s => {
return s.track && s.track.kind === audioTrack.kind;
});
if (videoSender) {
console.log('videoSender.replaceTrack', videoTrack);
videoSender.replaceTrack(videoTrack);
}
if (audioSender) {
console.log('audioSender.replaceTrack', audioTrack);
audioSender.replaceTrack(audioTrack);
}
})
.catch(e => {
console.log('getUserMedia error:', e.name, e.code, e.message);
});
} catch (e) {
window.alert(e);
}
}
this is the log from chrome remote device debug:
The error is "NotReadableError", "Could not start video source" which means that the underlying device handle could not be obtained by chrome.
Again, safari/ios works ok.

For mobile devices, there is a dedicated way of how to select between front & back camera.
VideoFacingMode - https://www.w3.org/TR/mediacapture-streams/#dom-videofacingmodeenum
TL;DR
window.navigator.mediaDevices.enumerateDevices().then(devices => {
if (devices.filter(device => device.kind === 'videoinput').length > 1) {
navigator.mediaDevices.getUserMedia({video: {facingMode: 'user' /*'environment'*/}}).then(console.log.bind(this))
}
})
It works for mobile Safari, Chrome and FF.
NOTE
Remember, to stop the previous video track before calling the
getUserMedia with video again, otherwise, you will get an
exception.

Ok, so I narrowed it down to calling navigator.mediaDevices.getUserMedia() in ngInit() (this is Angular app).
Even if I remove all code in .then() handler function, the effect is the same.
Only removing this call solves the issue.
Not sure at this time why such behavior, will investigate it more thoroughly and update.

To switch between front and back cameras on mobile, you need to stop the previous stream before opening a new stream.
if (videoIn.srcObject) {
videoIn.srcObject.getTracks().forEach((track) => {
track.stop();
});

Titanium - Send picture to server : file size bug

I'd like to send pictures, from my app (iOS and Android) to my server. My code works with small pictures, but if the size is too big, when I send the data, nothing happens and the application slows down.
Could you explain me the problems in my code and how to resolve it ? Thanks a lot :)
Here is my code :
var attached_media = [];
var file_btn = Ti.UI.createButton({ title: L('select') });
file_btn.addEventListener('click',function(e){
Titanium.Media.showCamera({
success:function(e) {
if(e.mediaType == Ti.Media.MEDIA_TYPE_PHOTO) {
attached_media.push(Ti.Utils.base64encode(e.media).text);
}
},
saveToPhotoGallery:true,
allowEditing: false,
mediaTypes: [Ti.Media.MEDIA_TYPE_PHOTO]
});
});
var send_button = Titanium.UI.createButton({
title: 'Send',
});
send_button.addEventListener('click',function(e){
var req = ......
req.send({ 'medias':JSON.stringify(attached_media), 'user_id':Ti.App.Properties.getInt('user_id')});
});
I removed the unnecessary code, because it was too long ! :)

What I was able to understand from the provided info is that you are having problems while uploading large size pics, like the one from camera which turns out to be more than 2-3MB.
The only solution at present I can suggest you is to compress the image using this iOS-Android module Ti-ImageFactory before saving or sending it to server.
I recommend to compress the image right after you captured it in Camera's success callback like this:
file_btn.addEventListener('click',function(e){
Titanium.Media.showCamera({
success:function(e) {
if(e.mediaType == Ti.Media.MEDIA_TYPE_PHOTO) {
Ti.API.info("Initial pic bytes = " + e.media.length);
// if bytes length of pic is larger than 3MB or 3145728 bytes, set compression to 0.5,
// else keep it to default which is 0.7
var imf = require('ti.imagefactory');
var compressedPic = (e.media.length > 3145728) ? imf.compress(0.5) : imf.compress();
attached_media.push(Ti.Utils.base64encode(compressedPic).text);
Ti.API.info("Compressed pic bytes = " + compressedPic.length);
compressedPic = null;
}
},
saveToPhotoGallery:true,
allowEditing: false,
mediaTypes: [Ti.Media.MEDIA_TYPE_PHOTO]
});
});
Added Code - If captured picture size is more than 3MB, then compress it by 0.5 level, else compress it using default level 0.7. Also checking the initial pic size & compressed pic size to match better results as per app's requirements for faster uploading.
You can also pass a compression level in compress() method. See docs for more.

Youtube iframe player won't show videos in android 4.x WebView

I'm trying to use the youtube iframe player inside an android WebView but it fails on android 4.x while on 5.x everything works great.
When I try it in 4.x the player loads the video, when I start it I see nothing but hear the video sound.
The logcat gets filled with chromium error messages over and over again:
E/chromium(20362): [ERROR:gles2_cmd_decoder.cc(5942)] [.Compositor-Onscreen-0x5db83bb8]GL ERROR :GL_INVALID_OPERATION : glUseProgram: program not linked
E/chromium(20362): [ERROR:gles2_cmd_decoder.cc(5718)] [.Compositor-Onscreen-0x5db83bb8]GL ERROR :GL_INVALID_OPERATION : glUniformMatrix4fv: wrong uniform function for type
E/chromium(20362): [ERROR:gles2_cmd_decoder.cc(5718)] [.Compositor-Onscreen-0x5db83bb8]GL ERROR :GL_INVALID_OPERATION : glUniform1iv: wrong uniform function for type
Here's how I set the WebView:
this.webview = (WebView) this.findViewById(R.id.webview);
WebSettings settings = this.webview.getSettings();
settings.setJavaScriptEnabled(true);
settings.setAppCacheEnabled(false);
this.webview.setWebChromeClient(new WebChromeClient());
...
this.webview.loadUrl(ADDRESS_OF_PAGE);
Also, I enable hardware acceleration in the manifest:
<application
...
android:hardwareAccelerated="true">
Here's the js part:
var player;
var VIDEO_ID = "EIsauUFIguE";
window.onYouTubeIframeAPIReady = function() {
console.log("youtube api ready, loading player");
player = new YT.Player("playerIframe", {
width: "100%",
height: "100%",
videoId: null,
events: {
onReady: onPlayerReady,
onStateChange: onPlayerStateChange,
onPlaybackQualityChange: onPlayerPlaybackQualityChange,
onError: onPlayerError
},
playerVars: {
modestbranding: 1,
playsinline: 1,
fs: 0,
showinfo: 0
}
});
}
function onPlayerReady(event) {
console.log("player is ready: ", event);
playVideo();
}
function onPlayerStateChange(event) {
console.log("player state change: ", event);
}
function onPlayerPlaybackQualityChange(event) {
console.log("player playback quality change: ", event);
}
function onPlayerError(event) {
console.log("player error: ", event);
}
function playVideo() {
console.log("playing video: " + VIDEO_ID);
player.loadVideoById(VIDEO_ID);
player.setPlaybackQuality("medium");
}
function loadPlayer() {
console.log("loading youtube api");
var tag = document.createElement("script");
tag.src = "https://www.youtube.com/iframe_api";
var firstScriptTag = document.getElementsByTagName("script")[0];
firstScriptTag.parentNode.insertBefore(tag, firstScriptTag);
}
loadPlayer();
Any idea why it behaves this way and only in android 4.x?
Thanks.

I had the same problem, but after fighting with this long time, i didn't find a solution. I tested all the configurations with webview and did many different html changes to put the iframe.
After my researching i only can add:
In Android 4 versions, can't autoplay. The user has to press play button into the iframe to start play the video. Particularly:
In Android 4.1 it reproduces the audio but not the image.
In Android 4.3 i tested in devices which can reproduce audio but no video and devices that can't reproduce anything.
In Android 4.4 don't reproduce anything
From Android 5 and + everything works fine.
Good luck and I plus vote this question to know if anyone can put light into this question :)

Big delay executing sound on iPad and Android

I want to play a sound when the user click on a image. I'm using SoudManager plugin.
Here you can check what I'm doing:
soundManager.setup({
url: 'swf',
onready: function() {
soundCarro = soundManager.createSound({
id: 'soundCarro',
url: 'sound/carro_camona.mp3'
});
soundMoto = soundManager.createSound({
id: 'soundMoto',
url: 'sound/moto_camona.mp3'
});
soundNautico = soundManager.createSound({
id: 'soundNautico',
url: 'sound/nautico_camona.mp3'
});
}
});
As you can see, I create 3 sound objects (soundCarro, soundMoto and soundNautico).
Here the action, where when the user clicks on it, the scrollTo function is called:
<img id="ca" class="ca logo" onclick="scrollTo('secao-carros',true);" src="images/svg/CAhome.svg">
Here you can see the scrollTo function:
function scrollTo(target,sound){
if(sound){
var delay = 0;
if(target == 'secao-carros'){
soundCarro.play();
delay = 700;
duration = 750;
}
if(target == 'secao-motos'){
soundMoto.play();
delay = 900;
duration = 750;
}
if(target == 'secao-nauticos'){
soundNautico.play();
delay = 850;
duration = 2000;
}
$('html, body').delay(delay).animate({ scrollTop: $('#'+target).offset().top }, {duration: duration});
}else{
$('html, body').animate({ scrollTop: $('#'+target).offset().top }, {duration: 750});
}
}
As you can see, in this function I execute the sound object created (ex: soundNautico.play();).
The issue is that on iPad and Android devices this sound gets a big delay to execute, but in desktop browsers it works perfect!
How can I prevent this delay?

That's because desktop browsers will preload the audio so when you call .play() the browser will begin playing it immediately because it has already buffered some (if not all) of the audio.
I know that iOS and most other mobile browsers will A) only allow you to play a single audio file at a time, B) not allow you to cache audio at all, and C) only allow you initialize/play an audio object if the user physically initiated the action on the same call stack. So you're basically out of luck if you don't want the delay.
Edit: You could add an event listener to the audio and only trigger the scroll after the audio had buffered. However, you cannot load more than one sound at a time.
audio.addEventListener("canplay", function(e) {
doScroll();
audio.play();
}, false);
```