Develop Reference

How do you solve high latency with UDP on mobile devices

I am having an issue getting the network latency to be low for my multiplayer shooter game made with Unity. I am using UDP to send player positions from the game client to my amazon server and back to another game client. My game clients are sending 60 byte UDP packets to the amazon server at a rate of 8 packets per second.
When I play the game on two separate iOS devices (iPhone 7 and iPad mini) the network latency is very low and the players are able to see each other move instantly. However, if I run the game on my iPhone 7 and play against another player who is using a samsung galaxy s4 running android, which is a lower powered device, I experience 5 second latency on the android device when receiving player positions from the iOS device. The odd thing is that the iOS device can receive player positions from the android device instantly. The same issue happens when playing on the iPhone 7 against a client on mac, except the iPhone 7 experiences the 5 second latency when receiving player positions from the mac client. The issue also happens when playing on the Samsung Galaxy S4 against a client on mac, where the galaxy s4 experiences the 5 second latency.
I have tried to increase the udp receive buffer size to 16 MB but it didn't change anything.
Here is a sample of my game client code where I send player positions to the amazon server:
void Update(){
// manually constrain rotation because rigidbody constraints don't work
this.transform.rotation = Quaternion.Euler(new Vector3(0, this.transform.rotation.eulerAngles.y, 0));
this.transform.position = new Vector3(this.transform.position.x, boatHeightConstant, this.transform.position.z);
// Speed Boost Handling
if(isSpeedBoosting == true){
tiltFactor = tiltModifier * (VelocityRatio() + 1.0f);
speedBoostTimer += Time.deltaTime;
}
else{
tiltFactor = VelocityRatio() + 1.0f;
}
if(speedBoostTimer >= speedBoostDuration){
isSpeedBoosting = false;
speedBoostTimer = 0f;
endSpeedBoost = true;
}
if(endSpeedBoost == true){
GetComponentInChildren<MainWeapon>().EndFireRateBoost();
endSpeedBoost = false;
}
// Attack Boost Handling
if(isAttackBoosting == true){
attackBoostTimer += Time.deltaTime;
}
if(attackBoostTimer >= attackBoostDuration){
isAttackBoosting = false;
attackBoostTimer = 0f;
endAttackBoost = true;
}
if(endAttackBoost == true){
GetComponentInChildren<MainWeapon>().ResetDamage();
GetComponentInChildren<MainWeapon>().ResetHeatUpRate();
endAttackBoost = false;
}
if (GetComponent<InputManager>().GetInputType() == 0 || GetComponent<InputManager>().GetInputType() == 1 )
{
if (isSpeedBoosting == true)
{
Move(speedModifier);
}
else
{
Move(1f);
}
if (syncTimer <= 0f) {
syncTimer = networkRefreshRate;
SyncTransform ();
} else {
syncTimer -= Time.deltaTime;
}
}
else
{
NetworkMove();
}
}
/// <summary>
/// This function is constantly called to upload the local player's position to the server so the server is
/// aware of this player's movement and can share this local players current position with other players in the game.
/// </summary>
public void SyncTransform() {
if (isLocalPlayer == true && client != null && client.IsConnected()) {
client.SendPlayerTransform(SharedData.storage["userId"], transform.position, transform.rotation, currentLife);
}
}
Here is a sample of the UDP sender class in my game client:
public void SendPlayerTransform(string playerId, Vector3 position, Quaternion rotation, int currentLife) {
Message.Writer writer = new Message.Writer(Message.MessageType.PlayerTransform, udpClient, remoteEndPoint);
// Write the timestamp of this message
writer.WriteLong(DateTime.UtcNow.Ticks);
// write the player id
writer.WriteString(SharedData.storage["userId"]);
// write the position vector
writer.WriteFloatArray(CommonGameFunctions.ConvertVectorToFloatArray(position));
// write the rotation vector
writer.WriteFloatArray(CommonGameFunctions.ConvertQuaternionToFloatArray(rotation));
writer.WriteInt (currentLife);
// Now send the message
writer.Send();
}
Here is a sample of where I receive the UDP messages on the game client:
public int HandleUdpMessages() {
if (udpTimerStarted == false) {
lastUdpMessageReceivedTime = DateTime.Now;
udpTimerStarted = true;
} else if (udpTimerStarted == true && udpClient.Available == 0){
TimeSpan t = DateTime.Now - lastUdpMessageReceivedTime;
if (t.Seconds >= 10f) {
// no message received for last 10 seconds then throw IO exception
//throw new SocketException();
}
}
if (udpClient.Available > 0) {
var messageReader = new Message.Reader (udpClient);
messageReader.BlockingRead (ref localEndPoint, UdpReceiveTimeout);
var messageType = messageReader.ReadMessageTypeUdp ();
lastUdpMessageReceivedTime = DateTime.Now;
Debug.Log ("Received udp message: " + messageType);
switch (messageType) {
// Player position update message
case Message.MessageType.PlayerTransform:
HandlePlayerTransform (messageReader);
break;
case Message.MessageType.PlayerScore:
HandlePlayerScore (messageReader);
break;
case Message.MessageType.RockHealth:
HandleRockHealth (messageReader);
break;
case Message.MessageType.PlayerHealth:
HandlePlayerHealth (messageReader);
break;
case Message.MessageType.ShieldHealth:
HandleShieldHealth (messageReader);
break;
default:
Debug.LogError ("Unhandled message " + messageType);
break;
}
}
return 0;
}
public void HandlePlayerTransform(Message.Reader reader)
{
long timeStamp = reader.ReadLong ();
string playerId = reader.ReadString();
if (playerMessageTimeStamps [playerId].latestPlayerTransform > timeStamp)
return;
Vector3 position = new Vector3();
Quaternion rotation = new Quaternion();
// read position
position = CommonGameFunctions.ConvertFloatArrayToVector3(reader.ReadFloatArray(3));
rotation = CommonGameFunctions.ConvertFloatArrayToQuaternion(reader.ReadFloatArray(4));
// Now update the transform of the right player
Player player = Player.playerTable[playerId];
if (player == null) {
return;
}
player.SetNetworkPostion(position);
player.SetNetworkRotation(rotation);
}
On my server this is the main game loop which runs on its own dedicated thread.
// Now all the players are connected to the server
// We can start the main game loop
while (gameRunning == true) {
HandlePlayersWhoDroppedOut ();
if (PlayersLeftGame () == true) {
DisconnectAllPlayers ();
Debug.LogError ("Player's left match, returning from thread");
return;
} else {
foreach (NetworkPlayer p in participants) {
try {
p.HandleTcpMessages ();
p.HandleUdpMessages ();
} catch (IOException e) {
droppedPlayers.Add (p);
}
}
try {
RespawnRocksIfDestroyed ();
} catch (System.IO.IOException e) {
DisconnectAllPlayers ();
return;
Debug.LogError ("Failed to spawn rocks");
}
}
}

The problem with my code was that I was reading exactly one UDP message in each iteration of the UDP message handler function. I changed my function to read ALL the available UDP messages in the buffer and the lag reduced by 80%. UDP messages queue up in the buffer quicker than the message handler function repeats so this is why the problem was happening.

Using data received from Arduino via Bluetooth

I am working on an Android app that will constantly receive data from an Arduino HC-05 and store it in a database. For now I am focusing on receiving the data properly (so just displaying it on the screen is my next step).
I have followed this guide (PDF) to setup a basic app that will receive data from the Arduino. However, it doesn't include a section about using the received data. I've tried adding two lines of code to the handleMessage function in order to display the received data on screen, but I don't see any difference in the textview (it stays "Hello World!").
I know that my Arduino is sending the data just fine, since on another app called "Serial Bluetooth Terminal" it is displayed properly. Here is the code for the Handler class:
Handler mHandler = new Handler() {
#Override
public void handleMessage(Message msg) {
byte[] writeBuf = (byte[]) msg.obj;
int begin = (int)msg.arg1;
int end = (int)msg.arg2;
switch(msg.what) {
case 1:
String writeMessage = new String(writeBuf);
writeMessage = writeMessage.substring(begin, end);
// 2 lines of code I've added here:
TextView tempTextView = (TextView) findViewById(R.id.text_view);
tempTextView.setText(writeMessage);
break;
}
}
};
Aside from the 2 lines, the code is basically the same as the final stage of the guide. And here is the code for the Arduino (which currently just sends "1 2.00 3.00" over and over):
#include <SoftwareSerial.h>
SoftwareSerial BTSerial(0, 1); //RX|TX
int Temp= 1;
float Ph = 2;
float Ec = 3;
void setup() {
Serial.begin(9600);
BTSerial.begin(9600);
}
void loop() {
Serial.print(Temp);
Serial.print(" ");
delay(100);
Serial.print(Ph);
Serial.print(" ");
delay(100);
Serial.print(Ec);
Serial.print(" ");
delay(100);
Serial.print('\n');
if(Serial.available())
BTSerial.write(Serial.read());
}

How do you synchronize android clients with a python server?

I have a python server and about 10 android clients, using sockets. It is really important that when the server sends a message, all clients receive it at the same time (say 1/10th of a second of difference).
But the connection is over Wifi, and some devices get the message later than others, which gives a very messy result. I don't want to get the latency of every device because this is a very unreliable approach. I want something as accurate as possible.
For example, in FPS games, it is common to have a countdown at the start of the round, and every player can start playing at the same time. What kind of logic lies behind this?
As for what my code currently looks like:
I use a BufferedReader in android to read every line sent by the server. The server is a console application in which you can type a message, and when you press enter, every listed client receives it with a new thread for every client.
java method receiving messages:
private void readMessage() throws IOException {
String data;
while ((data = mBufferedReader.readLine()) != null) {
data = data.toUpperCase();
if (data.startsWith("POSITION")) {
String[] splitData = data.split("/");
Log.d(Constants.TAG, splitData[1]);
mMainActivity.setDevicePosition(Integer.parseInt(splitData[1]));
} else {
String message = data.substring(data.indexOf('/') + 1, data.length());
int devices = Integer.parseInt(data.substring(0, data.indexOf('/')));
if (message.length() >= devices) {
message += " ";
} else {
int difference = devices - message.length();
for (int i = 0; i < difference; i++) {
message += " ";
}
}
mMainActivity.printMessage(message);
}
}
}
python line :
for cl in clients_list:
start_new_thread(send_message_thread, (cl, message,))

Identify Messages using Message API Android

I have probably a simple question but has me stumped. For my Android Wear application, I have two sensors working (step counter and heartrate).The wear app then sends these values back to the mobile application. I am sending them using the Message API. My stepcount sendMessage() and heartrate sendMessage() method look the same. Here is my heartrate sendMessage method.
private void sendMessageToHandheld(final String message) {
if (mGoogleApiClient == null)
return;
Log.d(LOG_TAG,"sending a message to handheld: "+message);
// use the api client to send the heartbeat value to our handheld
final PendingResult<NodeApi.GetConnectedNodesResult> nodes = Wearable.NodeApi.getConnectedNodes(mGoogleApiClient);
nodes.setResultCallback(new ResultCallback<NodeApi.GetConnectedNodesResult>() {
#Override
public void onResult(NodeApi.GetConnectedNodesResult result) {
final List<Node> nodes = result.getNodes();
if (nodes != null) {
for (int i=0; i<nodes.size(); i++) {
final Node node = nodes.get(i);
Wearable.MessageApi.sendMessage(mGoogleApiClient, node.getId(), message, bytes);
}
}
}
});
}
Problem is I am only using one messageReceived method on the mobile. So I cant differentiate from the step value coming in or the heartrate value coming in.
#Override
public void onMessageReceived(MessageEvent messageEvent) {
super.onMessageReceived(messageEvent);
Log.d(LOG_TAG, "received a message from wear: " + messageEvent.getPath());
if (messageEvent.getPath().contains("HEARTBEAT")){
// save the new heartbeat value
currentValue = Integer.parseInt(messageEvent.getPath());
if(handler!=null) {
// if a handler is registered, send the value as new message
Log.d(LOG_TAG, "received a heartbeat message from wear: " + messageEvent.getPath());
handler.sendEmptyMessage(currentValue);
}
}
else {
// save the new steps value
currentStepValue = Integer.parseInt(messageEvent.getPath());
if (handler != null) {
// if a handler is registered, send the value as new message
Log.d(LOG_TAG, "received a step message from wear: " + messageEvent.getPath());
handler.sendEmptyMessage(currentStepValue);
}
}
I tried passing in a byte array into the Heartrate sendMessage() and other strings as flags so that I could tell the values apart but no luck. Anyone know what the best way to go about this would be?
Any help would be appreciated.
Cheers

It seems you are sending the data inside the path attribute. This is not the correct use of this parameter.
Let's take a look at the MessageApi.sendMessage(GoogleApiClient client, String nodeId, String path, byte[] data method.
What you want to do is use String path to provide identifier for your message, for example in your case it would be step_counter and heartbeat. This way you can identify it on the other side, when you receive the message.
The sensor data should go into data field. You can put it raw there, but a better way is to create a DataMap and then serialize it into byte[]. This way you can enrich the data later easily.

Losing data with RealTimeSocket

I'm porting a game to use Google Play Game Services with multiplayer support. I'm using RealTimeSocket instead of realtime message because the game already has socket support.
To get the socket I call GamesClient.getRealTimeSocketForParticipant, and then I could get input and output streams as use it as a usual socket.
My problem is that if a device receives data before the call to getRealTimeSocketForParticipant, I will not be able to read this data. For instance:
Device A calls getRealTimeSocketForParticipant.
Device A sends "Hello".
Device B calls getRealTimeSocketForParticipant.
Device B receives nothing.
Device A sends "World".
Device B receives "World".
I have modified one of the example projects (ButtonClicker) and replicated the problem here. I have modified the code to use realtime socket, and modified the startGame method to this:
String mReceivedData = "";
byte mNextByteToSend = 0;
void startGame(boolean multiplayer)
{
mMultiplayer = multiplayer;
updateScoreDisplay();
switchToScreen(R.id.screen_game);
findViewById(R.id.button_click_me).setVisibility(View.VISIBLE);
GamesClient client = getGamesClient();
String myid = mActiveRoom.getParticipantId(client.getCurrentPlayerId());
ArrayList<String> ids = mActiveRoom.getParticipantIds();
String remoteId = null;
for(int i=0; i<ids.size(); i++)
{
String test = ids.get(i);
if( !test.equals(myid) )
{
remoteId = test;
break;
}
}
//One of devices should sleep in 5 seconds before start
if( myid.compareTo(remoteId) > 0 )
{
try
{
//The device that sleeps will loose the first bytes.
Log.d(TAG, "Sleeping in 5 seconds...");
Thread.sleep(5*1000);
}
catch(Exception e)
{
}
}
else
{
Log.d(TAG, "No sleep, getting socket now.");
}
try
{
final RealTimeSocket rts = client.getRealTimeSocketForParticipant(mRoomId, remoteId);
final InputStream inStream = rts.getInputStream();
final OutputStream outStream = rts.getOutputStream();
final TextView textView =((TextView) findViewById(R.id.score0));
//Thread.sleep(5*1000); Having a sleep here instead minimizes the risk to get the problem.
final Handler h = new Handler();
h.postDelayed(new Runnable()
{
#Override
public void run()
{
try
{
int byteToRead = inStream.available();
for(int i=0; i<byteToRead; i++)
{
mReceivedData += " " + inStream.read();
}
if( byteToRead > 0 )
{
Log.d(TAG, "Received data: " + mReceivedData);
textView.setText(mReceivedData);
}
Log.d(TAG, "Sending: " + mNextByteToSend);
outStream.write(mNextByteToSend);
mNextByteToSend++;
h.postDelayed(this, 1000);
}
catch(Exception e)
{
}
}
}, 1000);
}
catch(Exception e)
{
Log.e(TAG, "Some error: " + e.getMessage(), e);
}
}
The code ensures that one of the two devices sleeps 5 seconds before the call to getRealTimeSocketForParticipant. For the device that doesn't sleep the output will be something like:
No sleep, getting socket now.
Sending: 0
Sending: 1
Sending: 2
Sending: 3
Sending: 4
Received data: 0
Sending: 5
Received data: 0 1
Sending: 6
Received data: 0 1 2
That's expected, no data lost. But for the other device I get this:
Sleeping in 5 seconds...
Received data: 4
Sending: 0
Received data: 4 5
Sending: 1
Received data: 4 5 6
Sending: 2
Received data: 4 5 6 7
Sending: 3
The first bytes are lost. Is there anyway to avoid this?

If i'm understanding the API correctly, the messages exchanged through a real time socket are unrealiable, so you can't always have assurance that all players received all messages you sent. I couldn't find info about the network protocol used by RealTimeSocket, but I suspect it's UDP.
If that's really the case, I'm afraid there's little you can do short of implementing some sort of handshake yourself. Choose one device (ex.: the one with the lowest ID) to be the "synchronizer", and have it create a set with every other device. Send a message ("SYN") such as "where are you? x y z" (not literally, of course) every second, until the others respond "I'm here! (y)" ("ACK"). Remove from the set the devices that sent a response, until the set is empty. At this point, send everyone a "game's starting!" and go on.
Note that any of these messages can be lost: if the "ACK" is lost, next time the "SYN" is sent the device should answer again. If the "game's starting" message is lost, tough luck, the device will keep waiting until it receives a different message, at such point it should consider itself free to start (though delayed).
One last note: even if the underlying protocol is TCP, it's still not 100% reliable, no protocol is. See this question for more info, if you don't know this fact already.