I'm trying to use getUserMedia to show the cam live stream on a web page. It works (if I save the stream to a file, it is ok) but the video tag show only a static green image as preview. Is anyone facing the same problem?
My setup:
Samsung S8 (SM-G950F)
Android 9
Chrome 107.0.5304.91
HTML
<video autoplay="true" width="100%" id="video-test"></video>
JS
var video = document.querySelector("#video-test");
navigator.mediaDevices.getUserMedia({
video: {
mandatory: { minFrameRate: 10, minWidth: 100, minHeigth: 100 },
}
})
.then(function (stream) {
video.srcObject = stream;
})
.catch(function (err0r) {
console.log("Something went wrong!", err0r);
});
Result
Thank you
Ah, the green screen of death. Why is it green? Because it's trying to display an all-zero YUV image.
Your constraints in your call to .getUserMedia() are incorrect. mandatory is no longer a thing. You need something like this:
{
video: {
width: { min: 100, ideal: 320 },
height: { min: 100, ideal: 240 },
frameRate: { min: 10, ideal: 10 }
}
}
It's an inconvenient truth that you have to muck around to get constraints values that work on many different platforms. And, you have to accept what gUM gives you. If you demand exact constraints using something like frameRate: { exact: 10 } you may get an OverconstrainedError exception, or some other exception.
Related
I am trying to save images i take with the camera on my phonegap app to a more permanent destination. I run the app on iOS and Android so i need this to work on ios and android.
So far i can get the image file name and temporary storage path.
I can also get the base file system path for the device i am running (currently an Android Galaxy S7 edge).
However, i am unable to move/copy the file from the temporary storage to the new one as it fails.
Ive got the following code and ive got a try/catch around the code which moves the images. in the catch i have an alert which triggers always as there is something wrong with my code.
Ive tried changing the moveTo to CopyTo and changing the code from output.moveTo to things like cordova.file.moveTo without success.
If anyone could help i would be greatful.
Thanks Kind regards
const tempFilename = imageURI.substr(imageURI.lastIndexOf('/') + 1);
const tempBaseFilesystemPath = imageURI.substr(0, imageURI.lastIndexOf('/') + 1);
const newBaseFilesystemPath = cordova.file.dataDirectory;
var output=tempBaseFilesystemPath+tempFilename;
var saveOutput=newBaseFilesystemPath+tempFilename;
alert(tempFilename+","+tempBaseFilesystemPath+","+newBaseFilesystemPath+","+tempFilename);
try{
output.moveTo(tempBaseFilesystemPath, tempFilename, newBaseFilesystemPath,tempFilename)
.then(function (success) {
alert("done");
}, function (error) {
alert("not done");
});
}
catch{
alert("didnt run");
}
const storedPhoto = output;
localStorage.setItem('imageLoc',storedPhoto);
localStorage.setItem('goodPic','not');
},
function( message ) {
},
{
quality: 30,
destinationType: Camera.DestinationType.FILE_URI,
});
}
I use navigator.mediaDevices.enumerateDevices to retrieve list of all video devices (element.kind === 'videoinput') and then call navigator.mediaDevices.getUserMedia(constraints) call to rotate video devices (using deviceId as constraint). Everything works fine on Windows Chrome / Firefox, but on android phone (tried Samsung, Asus, Huawei with Android 8/9) this call fails for back camera with NotReadableError / Could not start video source (for Chrome) or AbortError / Starting video failed (for Firefox).
Strangely same code works ok in iOS / Safari.
Also this only happens when WebRTC call is present in browser. If there is no call I can select any video device.
Also if I select back camera first and try to establish the call, it does not work, I get similar error.
I know it's far-fetched but maybe someone had same/similar issue?
All browser versions are up-to-date.
[UPDATE - code snippet and log]
switchCamera() {
try {
if (this.localStream) {
const tracks = this.localStream.getTracks();
console.log('switchCamera stopping this.localStream tracks', tracks);
tracks.forEach((track: MediaStreamTrack) => {
console.log('switchCamera stopping track', track);
track.stop();
});
console.log('switchCamera stop stream');
}
const constraints = {
audio: true,
video: { facingMode: this.faceCamera ? 'environment' : 'face' }
};
this.faceCamera = !this.faceCamera;
console.log('switchCamera constraints: ', constraints);
navigator.mediaDevices.getUserMedia(constraints)
.then(stream => {
console.log('getUserMedia:', stream);
this.logText('got stream');
this.localVideo.srcObject = stream;
const videoTracks = stream.getVideoTracks();
const audioTracks = stream.getAudioTracks();
console.log('videoTracks', videoTracks);
if (videoTracks.length > 0) {
console.log(`Using video device: ${videoTracks[0].label}`);
}
const videoTrack = videoTracks[0];
const audioTrack = audioTracks[0];
console.log('Replacing track for pc', videoTrack, audioTrack);
const pc = this.session.sessionDescriptionHandler.peerConnection;
const videoSender = pc.getSenders().find(s => {
return s.track && s.track.kind === videoTrack.kind;
});
const audioSender = pc.getSenders().find(s => {
return s.track && s.track.kind === audioTrack.kind;
});
if (videoSender) {
console.log('videoSender.replaceTrack', videoTrack);
videoSender.replaceTrack(videoTrack);
}
if (audioSender) {
console.log('audioSender.replaceTrack', audioTrack);
audioSender.replaceTrack(audioTrack);
}
})
.catch(e => {
console.log('getUserMedia error:', e.name, e.code, e.message);
});
} catch (e) {
window.alert(e);
}
}
this is the log from chrome remote device debug:
The error is "NotReadableError", "Could not start video source" which means that the underlying device handle could not be obtained by chrome.
Again, safari/ios works ok.
For mobile devices, there is a dedicated way of how to select between front & back camera.
VideoFacingMode - https://www.w3.org/TR/mediacapture-streams/#dom-videofacingmodeenum
TL;DR
window.navigator.mediaDevices.enumerateDevices().then(devices => {
if (devices.filter(device => device.kind === 'videoinput').length > 1) {
navigator.mediaDevices.getUserMedia({video: {facingMode: 'user' /*'environment'*/}}).then(console.log.bind(this))
}
})
It works for mobile Safari, Chrome and FF.
NOTE
Remember, to stop the previous video track before calling the
getUserMedia with video again, otherwise, you will get an
exception.
Ok, so I narrowed it down to calling navigator.mediaDevices.getUserMedia() in ngInit() (this is Angular app).
Even if I remove all code in .then() handler function, the effect is the same.
Only removing this call solves the issue.
Not sure at this time why such behavior, will investigate it more thoroughly and update.
To switch between front and back cameras on mobile, you need to stop the previous stream before opening a new stream.
if (videoIn.srcObject) {
videoIn.srcObject.getTracks().forEach((track) => {
track.stop();
});
I'd like to send pictures, from my app (iOS and Android) to my server. My code works with small pictures, but if the size is too big, when I send the data, nothing happens and the application slows down.
Could you explain me the problems in my code and how to resolve it ? Thanks a lot :)
Here is my code :
var attached_media = [];
var file_btn = Ti.UI.createButton({ title: L('select') });
file_btn.addEventListener('click',function(e){
Titanium.Media.showCamera({
success:function(e) {
if(e.mediaType == Ti.Media.MEDIA_TYPE_PHOTO) {
attached_media.push(Ti.Utils.base64encode(e.media).text);
}
},
saveToPhotoGallery:true,
allowEditing: false,
mediaTypes: [Ti.Media.MEDIA_TYPE_PHOTO]
});
});
var send_button = Titanium.UI.createButton({
title: 'Send',
});
send_button.addEventListener('click',function(e){
var req = ......
req.send({ 'medias':JSON.stringify(attached_media), 'user_id':Ti.App.Properties.getInt('user_id')});
});
I removed the unnecessary code, because it was too long ! :)
What I was able to understand from the provided info is that you are having problems while uploading large size pics, like the one from camera which turns out to be more than 2-3MB.
The only solution at present I can suggest you is to compress the image using this iOS-Android module Ti-ImageFactory before saving or sending it to server.
I recommend to compress the image right after you captured it in Camera's success callback like this:
file_btn.addEventListener('click',function(e){
Titanium.Media.showCamera({
success:function(e) {
if(e.mediaType == Ti.Media.MEDIA_TYPE_PHOTO) {
Ti.API.info("Initial pic bytes = " + e.media.length);
// if bytes length of pic is larger than 3MB or 3145728 bytes, set compression to 0.5,
// else keep it to default which is 0.7
var imf = require('ti.imagefactory');
var compressedPic = (e.media.length > 3145728) ? imf.compress(0.5) : imf.compress();
attached_media.push(Ti.Utils.base64encode(compressedPic).text);
Ti.API.info("Compressed pic bytes = " + compressedPic.length);
compressedPic = null;
}
},
saveToPhotoGallery:true,
allowEditing: false,
mediaTypes: [Ti.Media.MEDIA_TYPE_PHOTO]
});
});
Added Code - If captured picture size is more than 3MB, then compress it by 0.5 level, else compress it using default level 0.7. Also checking the initial pic size & compressed pic size to match better results as per app's requirements for faster uploading.
You can also pass a compression level in compress() method. See docs for more.
I have an android application that sends the camera stream through a webview through peerjs (webrtc) the web application on the browser receives the video and streams it.
Things are working but the video on the web is too slow and the image freezes for some time before getting the second image...
Is there a way to make the resolution lower ? or buffer the video on the web application ? or can it be something wrong with my implementation ?
Android Webview code:
initVideo = function(videoSourceValue) {
var video = document.querySelector('video');
navigator.getUserMedia({video: {optional: [{
sourceId: videoSourceValue
}]
}
},function(stream) {
video.src = window.URL.createObjectURL(stream);
$('#peerId').text("calling : " + SERVER_PEER_ID);
var mediaConnection = peer.call(SERVER_PEER_ID, stream);
mediaConnection.on('stream', function(remoteStream) {
// Show stream in some video/canvas element.
});
},function(e){
console.log('failed',e);
});
}
Web part:
function getVideoStream() {
PEER.on('call', function(call) {
var mediaConnection = navigator.getUserMedia({video: true}, function(stream) {
call.answer(stream); // Answer the call with an A/V stream.
call.on('stream', onReceiveStream);
}, function(err) {
console.log('Failed to get local stream' ,err);
});
});
}
function onReceiveStream(stream){
console.log('received stream');
$('video').prop('src',window.URL.createObjectURL(stream));
}
Thanks
Update 1
I tried to add {reliable : true}, still having the same issue.
I'm also sending location data to the server, and it seems that the video streams and location data are sent together periodically (the chart on the web showing speed and the video move at the same time) but the frame rate is too slow.
When you establish the video/audio stream you can specify some constraints...
var videoOptions = (isCordova) ? {audio: true, video: true} :
{ audio: true,
video: {
mandatory: {
maxWidth: 640,
maxHeight: 360,
// maxAspectRatio:4/3,
// maxFrameRate:1
},
quality: 7,
width: { ideal: 320 },
height: { ideal: 240 }
}
};
navigator.getUserMedia(videoOptions, function (stream) {
In the above code, if you are on a device (android/ios) you don't get to choose, but you can control it on the browser. A quality of 5 is the level that the video driver author deemed as an acceptable trade off between quality and bandwidth. Limiting the dimensions of the picture helps too.
See this link for mode details: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia
My issue was completely unrelated to bandwidth, I simply didn't put autoplay on the video tag , so the video was only refreshing when a redraw was happening.
Thanks a lot for your answers they really give insight on how things work in webrtc.
I'm integrating a modified SDK 3.0 sample into a big app in which I'm working.
When I detect whatever Trackable2DObject and show an ImageDrawable (for example, the surf table), this ImageDrawable appears from the z-index very slowly.
Is there any way to disable this animation?
Thanks in advance!
EDIT: My JS code only contains this:
var World = {
loaded: false,
init: function initFn() {
this.createOverlays();
},
createOverlays: function createOverlaysFn() {
// Initialize Tracker
this.tracker = new AR.Tracker("assets/wul4bus.wtc", {
onLoaded: this.worldLoaded
});
// Create overlay for page one
var imgOne = new AR.ImageResource("assets/redsys_marker_orange.png");
var overlayOne = new AR.ImageDrawable(imgOne, 0.5, {
zorder: 0,
offsetX: -0.15,
offsetY: 0,
onClick: this.createClickTrigger
});
var titleLabel = new AR.Label("Bus", 0.07, {
zOrder: 1,
offsetX: -0.15,
offsetY: 0.04,
style: {
textColor: '#FFFFFF',
fontStyle: AR.CONST.FONT_STYLE.BOLD
}
});
var pageOne = new AR.Trackable2DObject(this.tracker, "002", {
drawables: {
cam: [overlayOne, titleLabel]
}
});
},
createClickTrigger: function createClickTriggerFn() {
document.location = "architectsdk://product?id=1236";
},
worldLoaded: function worldLoadedFn() {
document.body.removeChild(document.getElementById('loadingMessage'));
}
};
World.init();
As you see, there is no AR.PropertyAnimation class along this code. Anyway, a small animation succedeed sometimes when the target is discovered, and trackable object appears a bit slowly, slower than SDK 2 IMHO. It happens with some Trackable2DObject more than others.
Please have a look at the API Reference of the Wikitude SDK.
You will find details around Animations there.
If you need a Sample collection and platform specific stuff, have a look at the Wikitude Documentation page.
Kind regards,
Andreas