I am referring and also going through source code of AppRTCDemo which is a demo application for WebRTC.
What i am trying is:
Build my own WebRTC application which will do AV calls on a Android Device.
Replace existing https://apprtc.appspot.com/ server and related functionality.
For archiving above points, I want to understand basic flow of WebRTC function calls and steps to make/receive calls (functions that i need to calls and there flow).
I have gone through the source code and understood few things,
but as code is pretty complicated to understand, and without any documentation.
It will be great help if some one provides any examples or documents explaining the steps for making/receiving AV calls (how we get/set SDP, how to render local/remote video etc.).
I have seen these posts and are very helpful:
WebRTC java server trouble
https://www.webrtc-experiment.com/docs/WebRTC-PeerConnection.html
I am able to build and run AppRTCDemo App.
Any help on this will be great help!
There is no timeline, it's asynchronous but i will try to explain but there is two main flow, the flow of offer and answer with SDP and the flow of icecandidate.
Flow 1 : SDP
Step 1 - Offer peer :
On the offer side, create a RTCPeerconnection (with stun, trun servers as parameters).
var STUN = {
url:'stun:stun.l.google.com:19302'
};
var TURN = {
url: 'turn:homeo#turn.bistri.com:80',
credential: 'homeo'
};
var iceServers = {
iceServers: [STUN, TURN]
};
var peer = new RTCPeerConnection(iceServers);
Step 2 - Offer peer :
Call getUserMedia with your constraints. In the success callback, add the stream to the RTCPeerconnection using the addStream method. Then you can create the offer with calling createOffer on the Peerconnection Object.
navigator.webkitGetUserMedia(
{
audio: false,
video: {
mandatory: {
maxWidth: screen.width,
maxHeight: screen.height,
minFrameRate: 1,
maxFrameRate: 25
}
}
},
gotStream, function(e){console.log("getUserMedia error: ", e);});
function gotStream(stream){
//If you want too see your own camera
vid.src = webkitURL.createObjectURL(stream);
peer.addStream(stream);
peer.createOffer(onSdpSuccess, onSdpError);
}
Step 3 - Offer peer :
In the callback method of the createOffer, set the parameter (the sdp offer) as the localDescription of the RTCPeerConnection (who will start gathering the ICE candidate). Then send the offer to the other peer using the signaling server. (I will not describe signaling server, it's just passing data to one from another).
function onSdpSuccess(sdp) {
console.log(sdp);
peer.setLocalDescription(sdp);
//I use socket.io for my signaling server
socket.emit('offer',sdp);
}
Step 5 - Answer peer :
The answer peer, each time it receives an offer, create a RTCPeerconnection with TURN, STUN server, then getUserMedia, then in the callback, add the stream to the RTCPeerConnection. With the SDP offer use setRemoteDescription with the sdpOffer. Then Trigger the createAnswer.
In the success callback of the createAnswer, use setLocalDescription with the parameter and then send the answer sdp to the offer peer using signaling server.
//Receive by a socket.io socket
//The callbacks are useless unless for tracking
socket.on('offer', function (sdp) {
peer.setRemoteDescription(new RTCSessionDescription(sdp), onSdpSuccess, onSdpError);
peer.createAnswer(function (sdp) {
peer.setLocalDescription(sdp);
socket.emit('answer',sdp);
}, onSdpError);
});
Step 7 : Offer peer
Receive the sdp answer, setRemoteDescription on the RTCPeerConnection.
socket.on('answer', function (sdp) {
peer.setRemoteDescription(new RTCSessionDescription(sdp), function(){console.log("Remote Description Success")}, function(){console.log("Remote Description Error")});
});
Flow 2 : ICECandidate
Both side :
Each time the RTCPeerConnection fire onicecandidate, send the candidate to the other peer through signalingserver.
When a icecandidate is received, coming from signaling server, just add it to the RTCPeerConnection using the addIceCandidate(New RTCIceCandidate(obj))
peer.onicecandidate = function (event) {
console.log("New Candidate");
console.log(event.candidate);
socket.emit('candidate',event.candidate);
};
socket.on('candidate', function (candidate) {
console.log("New Remote Candidate");
console.log(candidate);
peer.addIceCandidate(new RTCIceCandidate({
sdpMLineIndex: candidate.sdpMLineIndex,
candidate: candidate.candidate
}));
});
Finally :
If two flow above works well use the onaddstream event on each RTCPeerConnection. When ICE Candidates will pair each other and find the best way for peer-to-peer, they will add the stream negociated with the SDP and that is going through the peer to peer connection. So in this event, you juste need to add your stream then to a video tag for example and it's good.
peer.onaddstream = function (event) {
vid.src = webkitURL.createObjectURL(event.stream);
console.log("New Stream");
console.log(event.stream);
};
I will edit tommorow with some code i think to help understand what i'm saying. If have question go for it.
Here is my signaling server :
var app = require('express')();
var server = require('http').Server(app);
var io = require('socket.io')(server);
server.listen(3000);
app.get('/', function (req, res) {
res.send('The cake is a lie');
});
io.on('connection', function (socket) {
console.log('NEW CONNECTION');
socket.on('offer', function (data) {
console.log(data);
socket.broadcast.emit("offer",data);
});
socket.on('answer', function (data) {
console.log(data);
socket.broadcast.emit("answer",data);
});
socket.on('candidate', function (data) {
console.log(data);
socket.broadcast.emit("candidate",data);
});
});
Related
I'm trying to connect an Android app using Native Socket.IO to an Express server. The server has a very simple job (currently) of taking incoming traffic on 4200 and sending it back out to other clients so they can display it - although right now I'm just debugging the server and this Android implementation, hoping for the beautiful text "client connected"
This is the server API
// Socket handling
io.on( 'connection', (socket) => {
console.log('Client Connected');
socket.on('temperature', (data) => {
console.log('Engine Data:' + data);
// For now, just spit data back to all other connected clients
socket.broadcast.emit('temperature', {num: data});
});
socket.on('disconnect', ()=> {
console.log('Client disconnected');
});
});
Currently this works with a basic Node.JS client sending random numbers, so I know the server works:
<script>
var socket = io('http://localhost:4200');
socket.on('connection', (socket) => {
console.log('Connection Success');
});
let temperature = 0;
//Send stuff every 100ms
setInterval(() => {
let lastTemp = temperature < 90 ? temperature : 0;
temperature = Math.floor(Math.random() * 5) + lastTemp;
socket.emit('temperature', temperature);
}, 50);
</script>
However my Android app is having issues - the server doesn't log a connection like it does the browser client, and while it isn't throwing an exception on connection, the socket ID is just NULL and the state is "not connected" when debugging. I've been following different tutorials and trying solutions posted on StackOverflow, so I currently have a main activity utilizing a seek bar for dummy values that looks like this:
//! SocketIO stuff
import com.github.nkzawa.socketio.client.IO
import com.github.nkzawa.socketio.client.Socket
class MainActivity : AppCompatActivity() {
//! Instantiate socket for other functions that will utilize it - this will likely be part of its own class
lateinit var mSocket: Socket
//! Main Activity creation
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
setSupportActionBar(findViewById(R.id.toolbar))
//! SeekBar for setting dummy input by user
val seek: SeekBar = findViewById<SeekBar>(R.id.seekBar)
seek.max = 100
seek.progress = 50
//! CurrentVal is our current dummy sensor value
var currentVal: Int;
//! Connect to web server
try {
//! Configure library options
val options = IO.Options()
options.reconnection = true
options.forceNew = true
//This address is the way you can connect to localhost with AVD(Android Virtual Device)
mSocket = IO.socket("http://10.0.2.2:4200/", options) // emulator loopback address
} catch (e: Exception) {
e.printStackTrace()
Log.d("Failure", "Failed to connect to server")
}
Log.d("Connected", "socket ID:${mSocket.id()}")
mSocket.connect()
seek.setOnSeekBarChangeListener(object : SeekBar.OnSeekBarChangeListener {
...
override fun onStopTrackingTouch(seek: SeekBar) {
//Progress stopped
//get value from seek bar
currentVal = seek.progress
//Send the value via our socket
mSocket.emit("temperature", currentVal)
}
})
}
My Manifest includes the following based on feedback from other StackOverflow suggestions
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<application
...
android:usesCleartextTraffic="true"
...
However the server doesn't reflect that a client is connected. When I look at Wireshark, I see that there is indeed a port 55735 that's trying to connect to the server running on 4200, however the server is responding with "400 Bad Request" and the JSON string value is "Unsupported protocol version"
2899 23.395504 127.0.0.1 127.0.0.1 HTTP/JSON 287 HTTP/1.1 400 Bad Request , JavaScript Object Notation (application/json)
This leads me to believe that there's some compatibility issue with the Socket.IO library I'm using and Socket.IO on the server. The tutorial I'm using is on the Socket.IO website... but it's also from 2015. Should I be following a different way to use Socket.IO on Android?
I seem to have figured out the issue - the socket.io library for Android was not compatible with the latest 3.1.1 in NPM
This was mainly shown by watching the data in Wireshark - I had communication between the ports, however the response - mainly the JSON in the response - indicated an incorrect protocol version
In short the dependency in my build.gradle file for the app, shown in the original tutorial from Socket.IO from 2015 is WAY out of date
implementation 'com.github.nkzawa:socket.io-client:0.6.0'
//OUT OF DATE
This implementation from Joyce Hong was closer...
implementation ('io.socket:socket.io-client:1.0.0') {
exclude group: 'org.json', module: 'json'
}
//Still nope, but that was a clue
However, the implementation that's compatible with my 3.1.1 Socket.IO library in NPM is
implementation ('io.socket:socket.io-client:2.0.0') {
exclude group: 'org.json', module: 'json'
}
//YES
If anyone else is running into this issue where there's traffic between ports, but neither is registering a connection, check your versions
I have implemented android twilio call with this tutorial,
https://github.com/twilio/voice-quickstart-android
Everything works perfectly as they have mentioned. The call rings I can attend the call and listen to the VoiceResponse message I saved in server. My requirement is I need to talk to the one android twilio application to other android with same twilio application instead of receiving VoiceResponse message. If I make phone calls to actual phone numbers then I can talk and listen without any problem, but from application to application speaking does not work.
I am using node js as server code, the first calling person code is given below.
client.api.calls.create({
url: url,
to: phoneNumber,
from: callerId,
}, function(err, call) {
if (err) { console.error('There was a problem starting the call: ', err); }
console.log('Call with sid: ${call.sid} was started');
});
xml response for url is
router.post('/callSecond', function(request, response) {
const voiceResponse = new VoiceResponse();
const dial = voiceResponse.dial({ callerId: 'client:al' });
voiceResponse.say("Congratulations! You have received your first inbound call! Good bye. Welcome to Twilio! Welcome to Twilio!!!! Welcome to Twilio");
dial.client("leo");
console.log('Response :' + voiceResponse.toString());
response.send(voiceResponse.toString());
});
Can anyone please help me to find a solution for this, speaking to each other using twilio mobile application.
Thank you in advance
I have done the DeepLink by referring this(i.e https://www.npmjs.com/package/node-deeplink)and it is working good and fine.But I have to implement same with http server end point so for that I have created one http server and listening on port 2000.My deeplink is http://host:2000/deep?url=nofer
Now My server is handling request as
var http = require('http');
var deeplink = require('node-deeplink');
http.createServer(handleRequest).listen(2000);
function handleRequest(request, response){
try {
//log the request on console
console.log(request.url);
if(request.url=="/deep"){
deeplink({
fallback: 'http://na.nofer.com',
android_package_name: 'com.na.nofer'
});
}
} catch(err) {
console.log(err);
}
}
app is not installed in my android device .once i have request with http://host:2000/deep?url=nofer in android browser ,request is landing but it is not directing to neighter fallback nor app store.
Can some one suggest on same.
I'm using WebSocket with React Native and have some issues on it. When I refresh the app, I found that previous WebSocket connection(used before refresh) was still alive, and not closed properly. Every time I was reloaded the App, it makes new connection. Then I shutdown the app, it releases all connections together.
When I'm testing almost same code with browser, when I refresh the page, socket is closed automatically and create new Websocket on page loads.
If this problem is still existing on production environment, it could be very critical.
This is server side code(I used express-ws):
const sockets = {};
app.ws('/', (socket, req) => {
// Generate unique id to socket and save to socket list
const uid = uuid.v4();
socket.uid = uid;
console.log(`Socket ${uid} connected.`);
sockets[uid] = socket;
socket.on('message', (data) => {
broadcast(data);
});
socket.on('close', () => {
console.log(`Socket ${uid} disconnected.`);
delete sockets[uid];
broadcast({
type: 'plain',
message: `User ${socket.username} leaved the channel.`
});
socket = null; // make sure to remove socket
});
});
Server just saves received WebSocket on connected, and remove it when closed. You can see that I logged Socket id that I made when got connection for identify each websocket handy, it looks like this(used node-uuid):
Socket d0b62d3a-2ed9-4258-9d2d-e83a3eb99e6c disconnected.
Ok, when I tested this server with Web browsers, it worked well. Every refresh successfully were closed socket and created new one.
But with React Native, socket never closed on refresh the app. After several refreshes, and finally exit the app, suddenly these messages were appear on the console at once:
Socket d0b62d3a-2ed9-4258-9d2d-e83a3eb99e6c disconnected.
Socket 2e1a2bba-ac64-4368-aeca-a02721f28ce5 disconnected.
Socket 6673c9a3-9667-425a-8923-efbc40196226 disconnected.
Socket 08fee145-e9bf-4af0-a245-dda2fe6a8e56 disconnected.
Socket 55ce1926-d7fa-488b-92d9-ff3dea874496 disconnected.
Socket 9c4c166d-d6d6-4a11-b400-a3eac51ab91f disconnected.
Socket 9f6db512-649c-440e-8b88-9a77d20e1943 disconnected.
Socket a9e6c0bd-419c-40af-865a-2573eca26a0f disconnected.
Socket 70835c7a-3230-4e20-b133-b6c2942dff22 disconnected.
Socket 5c83d81c-c0f1-4b9a-b5fb-7f09430a2f09 disconnected.
Socket 4f11aea4-d0ad-4e2b-9613-9e994657ecaf disconnected.
Below code is WebSocket handling part of my React Native app.
import store from './store';
import * as ChatActions from './actions/Chat';
import * as NetworkActions from './actions/Network';
const socket = null;
export function init() {
socket = new WebSocket('ws://mywebsite.com:3300');
socket.onopen = () => {
store.dispatch({
type: NetworkActions.NETWORK_SOCK_CONNECTED,
data: {}
});
};
socket.onmessage = (e) => {
var data = e.data;
try {
data = JSON.parse(e.data);
}
catch(err) {}
store.dispatch({
type: ChatActions.CHAT_RECV,
data: data.message
});
};
socket.onclose = (e) => {
store.dispatch({
type: NetworkActions.NETWORK_SOCK_CLOSED,
data: {}
});
};
}
export function send(data) {
data = typeof data === 'object' ? JSON.stringify(data) : data;
socket.send(data);
}
If I should close socket before app ends manually, and anyone know about this, please gimme some advice. I'm not much know about React Native, and not found related posts on google, so it will be very appreciate it gimme some advice. Thanks!
P.S. Oh, and I used Android for testing.
Your code looks good. By refresh do you mean refreshing the javascript while running in debug mode? That should never happen in production, the javascript is stored on the device.
If you need to explicitly close the connection when the app is in the background, check out:
http://facebook.github.io/react-native/docs/appstate.html
Then you can call ws.close() when the app is in the background :)
I work to develop a chat app with android on node.js server.I made a simple chat app.But everyone gets involved in the same room.I want to make a chat between only two people.How can I do that? What information should I send to the server?such as sender_name,receiver_name,message as a json format.
-How can I to do list online-users?
-And How can I start a chat between only two people?
What are the requirements for it?
This might help:
I'd recommend setting up namespaces for each room you have your chat in. I did something similar in my own code. Note: Rooms and namespaces are a little different from each other in socket.io itself (socket.io has both: http://socket.io/docs/rooms-and-namespaces/).
In the Server code:
I have a method under socket.on('connection') that is similar to
socket.on('groupConnect', function(group){
var groupNsp = io.of('/' + group);
}
This essentially makes sure that a namespace is exists under the name of the desired one. It doesn't mess it up or reset the namespace when it is called again.
Then for receiving the messages:
socket.on('message', function(data){
var msg = data.msg;
var nsp = data.nsp;
io.of(nsp).emit('message', msg);
}
You could also add the nsp to the data you have already and then just send the data again to the clients.
Then, in the client code:
var socketOut = io.connect('http://yourdomain:1337/');
var someGroupOrMethodToGetGroup;
socketOut.emit('groupConnect', someGroupOrMethodToGetGroup);
var nsp;
setTimeout(function(){
socket = io.connect('http://yourdomain:1377/' + someGroupOrMethodToGetGroup);
socket.on('message', function(msg){
displayMessage(msg);
}
nsp = '/' + someGroupOrMethodToGetGroup;
}, 1500);
Then in my displayMessage code I have:
socketOut.emit('message', { msg: desiredMessage, nsp: nsp });
From another answer of mine (https://stackoverflow.com/a/24805494/3842050).