Develop Reference

Packet loss while receiving UDP broadcast in android device

For receiving UDP broadcast packets from the server to an android device, i used a service class and listen for packets in a thread. It receives the packet successfully. The problem is that if multiple packets are being sent from the server in the same time then packet loss will be the result.
I even tried with a queue and processing the received packets in separate thread then also i am not getting the packet. I am completely new to network programming any help would be widely appreciated
void startListenForUdpBroadcast() {
UDPBroadcastThread = new Thread(new Runnable() {
public void run() {
try {
InetAddress broadcastIP = InetAddress.getByName(UdpConstants.IP_ADDRESS);
Integer port = UdpConstants.RECEIVER_PORT;
while (shouldRestartSocketListen) {
listenAndWaitAndThrowIntent(broadcastIP, port);
}
} catch (Exception e) {
Log.i("UDP", "no longer listening for UDP broadcasts cause of error " + e.getMessage());
}
}
});
UDPBroadcastThread.setPriority(Thread.MAX_PRIORITY); //Setting The Listener thread to MAX PRIORITY to minimize packet loss.
UDPBroadcastThread.start();
}
This code listens for new packets and pushes to queue
private void listenAndWaitAndThrowIntent(InetAddress broadcastIP, Integer port) throws Exception {
byte[] recvBuf = new byte[64000];
if (socket == null || socket.isClosed()) {
socket = new DatagramSocket(port, broadcastIP);
socket.setBroadcast(true);
}
//socket.setSoTimeout(1000);
DatagramPacket packet = new DatagramPacket(recvBuf, recvBuf.length);
socket.receive(packet);
messQueue.add(packet);
}
This checks the queue for new messages and process it
/**
* #purpose Checking queue and If anything is added to the queue then broadcast it to UI
*/
private void checkQueue() {
queueThread = new Thread(new Runnable() {
public void run() {
try {
while (shouldRestartSocketListen) {
if (!messQueue.isEmpty()) {
broadcastIntent(messQueue.poll());
}
}
} catch (Exception e) {
}
}
});
queueThread.start();
}

The problem with UDP is that your sender (your server) does not know you (your android device) missed some. It's not lost because you can't read it fast enough, sometimes just over the air interference/congestion or a busy socket.
The receiver would only know if:
you get an error while processing data since you're missing data
OR your UDP packets are numbered sequentially in its header and you detect a missing number (eg. 1,2,4 - missing 3)
Once the packet is lost, it's lost. You got two options:
implement a resend request: upon detection of a missing packet, the receiver would need to notify the sender to resend that missing packet until it does get it, and your packet processing might be halted until it does
OR be able to ignore it, "hey, we can do it without him", and fill-in with blank data (eg. a bitmap would have some blank pixels, like a broken image)
throttle your sending speed down so the packets wont jam up and get lost
the smaller your packets, the more likely they'll live
(option 1: all this resend requesting is just pseudo-TCP, so you might just consider abandoning UDP and go TCP)

I think your problem is mainly that you use Udp Broadcast over wifi.
Their are two very well documented answers why this is a very slow way to operate and why there are much more packet losts:
answer number one.
answer number two.
The thing I did to solve the extremely slow bandwidth was some kind of multi-unicast protocol:
Manage the list of clients you have connected.
Send each packet you have in your server to all of your clients separately with send call.
This is the code in java:
DatagramPacket packet = new DatagramPacket(buffer, size);
packet.setPort(PORT);
for (byte[] clientAddress : ClientsAddressList) {
packet.setAddress(InetAddress.getByAddress(clientAddress));
transceiverSocket.send(packet);
}

If you receive multiple datagrams in a short burst, your receiver loop may have trouble keeping up, and the OS-level RCVBUF in the socket may overflow (causing the OS to drop a packet it indeed did receive).
You might get better handling of short bursts if you increase the RCVBUF size. Prior to doing this, get an idea of how big it is already via socket.getReceiveBufferSize. Also bear in mind that the number of bytes in the receive buffer must accommodate not just the payload but also the headers and the sk_buff structure that stores packets in the kernel (see, e.g. lxr.free-electrons.com/source/include/linux/…).
You can adjust the recieve buffer size via socket.setReceiveBufferSize - but bear in mind that this message is just a hint, and may be overridden by settings in the kernel (if you request a size bigger than the max size allowable by the current kernel network settings, then you'll only get the max size).
After requesting a bigger receive buffer, you should double check what the kernel has allowed by calling socket.getReceiveBufferSize.
If you have the right permissions, you should be able to tweak the max buffer size the kernel will allow - see e.g. https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html
[ Note that, in general, this will accommodate for short bursts where datagrams arrive faster than your client loop can read them - but if the client loop is chronically slower than datagram delivery, you'll still get occasional drops due to overflow. In this case, you need to speed up your client loop, or slow down the sender.
Also, as otherwise noted in other answers, packets may actually be dropped by your network - and mobile networks especially may be prone to this - so if you absolutely need guaranteed delivery you should use TCP. However, if this were your primary problem, you would expect to see dropped packets even when your server sends them slowly, rather than in a burst.]

I suppose that you are capturing only a single packet by saying
socket.receive(packet);
This is a Blocking I/O call which will wait infinitely until it receives a packet so once first packet arrives it is done waiting and next command executes i.e
messQueue.add(packet);
However when multiple packets are been received you need to continue receiving packets. in your case you just stopped receiving packages after arrival of first package
Note: UDP being a un-reliable protocol doesn't guarantee packet delivery so there might be a case a packet is lost , However this can't be a problem on every run of your program , However a nice way to check whether the packet is hitting your machine and problem is within your application (application is not able to handle the packets recieved) use tcpdump (it's a command-line utility for linux-based OS or MAC) use the following command
sudo tcpdump -i <interface name(one that handles traffic) eth0, eth1 .. en0, en1 (for MAC)> host <ipaddress or hostname of UDP Broadcast server>
Example:
sudo tcpdump -i en1 host 187.156.13.5
(if tcpdump command not found then go forward and install it)
By using this command you will see packets pumping in from destination ump server on terminal if you see more then one packets arriving then you would be sure that packets are arriving at machine , However application falls short to address the packet
it might help

With reference to above explanation by me following changes you can make to make code behave according to requirement
I suppose you can make following changes to make your problem code work instead of creating socket into listenAndWaitAndThrowIntent(InetAddress broadcastIP, Integer port ) method create it in startListenForUdpBroadcast() as follows
socket = new DatagramSocket(port, broadcastIP);
socket.setBroadcast(true);
while (shouldRestartSocketListen) {
listenAndWaitAndThrowIntent(broadcastIP, port, socket);
}
Now you also need to change implementation of listenAndWaitAndThrowIntent method as follows
private void listenAndWaitAndThrowIntent(InetAddress broadcastIP,
Integer port, DatagramSocket socket) throws Exception {
byte[] recvBuf = new byte[64000];
//socket.setSoTimeout(1000);
//change value of condition in for loop to desired number of packets you would like to receive in below example it is 2 so it captures two packets
for (int i=0 ; i <= 2 ; i++ ){
DatagramPacket packet = new DatagramPacket(recvBuf, recvBuf.length);
socket.receive(packet);
messQueue.add(packet);
}
}
Try this it should work !!
it might help

Get the whole received bytes by a single call of "recv()" function call of WinSock2 Api

I wrote a client app in android device using bluetooth android api and server app in windows pc using winSock2 api. Now my problem is recv() can not receive more than 1011 bytes in a single call.
Here is the code of send and receive method-
Android data sending code is like as:
byte[] testByte = new byte[1024];
outputStream.write(testByte );
Windows Receiving code is
char response[4096];
//acceptedSocket: client socket accepted by server
//protocol- BTHPROTO_RFCOMM
//SocketType - SOCK_STREAM;
receivedBytes = recv(acceptedSocket, response, sizeof(response), 0);
here receiveBytes = 1011. So i have to call recv() again to receive next 13 bytes.
Now if i change the recv() function call with flags MSG_WAITALL and response array to be of size 1024, then it returns receivedBytes = 1024 bytes, I can receive whole 1024 bytes at a single call.
In that case, i have to send size of the data first from the
client app and then have to send the data. So I need to transmit two times
from Android client app.
But I want to receive whole 1024 bytes at a single call of recv() and transmitting once from Android client PP;
How can i achieve this?
Thanks in advance.

You can't.
recv() isn't specified to transfer more than one byte at a time.
You have to loop.

Implementing VOIP over udp, what is the approach to take when the player cannot cope up with received packet's speed

I am implementing p2p voice over IP on android, using UDP, openSL and some audio compression.
The voice chat works good when 2 peers are connected, and the lag is steadily about 100ms.
When a third peer connects, one or more peer progressively accumulate lag, reaching 15+ seconds in some cases.
My guess is that for some reason processing speed can't cope up with the receiving speed (packets gets accumulated in the audio buffer).
What is the correct approach to take in this case?
Should I discard some packets so that the lag goes back to a small value?
I don't think I have a problem with the application implementation, I have one thread for UDP receiving (no sleep, only I/O) and enqueuing audio for play, one for voice recording and UDP transmission(no sleep, only I/O) and some other threads that do unrelated things.
Is this a common situation in VOIP, or is there a problem with this design?
EDIT:
I was previously mixing the sound with OpenSL, but now I switched to the simpler android API which seems to alleviate the problem but does not solve it.
This gives the image of the relevant steps I'm taking:
// setup
private AudioTrack track[] = new AudioTrack[MAX_CHANNELS];
for( int i=0; i<MAX_CHANNELS; i++ ) {
track[i] = new AudioTrack(AudioManager.STREAM_VOICE_CALL, SAMPLING_RATE, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, outBufferSize, AudioTrack.MODE_STREAM);
track[i].play();
}
// [...]
// -- RECEIVE THREAD --
DatagramPacket datagram = socket.receiveMessage();
// ... determine channel by ip
track[channel].write(datagram.getData(), datagram.getOffset(), datagram.getLength());

Since AudioTrack.write is blocking, you should make sure you have one thread for every channel you are writing to, so that write calls don't get stuck waiting for other channels to complete.

Android BluetoothSocket OutputStream write blocks infinitely

I need to programmatically write data of say 1 to 100 MB in chunks of 1024 bytes to the remote Bluetooth device. Both are android devices. Here is a sample code snippet in my client program to transfer data –
bTSocket.connect(); //connect to remote BT device
DataOutputStream outStream = new DataOutputStream(bTSocket.getOutputStream());
byte[] buffer = new byte[1024];
int bytesToTransfer = 1000000;
while (bytesToTransfer > 0) {
outStream.write(buffer);
outStream.flush();
bytesToTransfer -= 1024;
}
outStream.close();
While running this piece of code on Android 2.2(Froyo), it works fine. However in case of Android 2.3.4 and 4.0.4, outStream.write(buffer) blocks infinitely after transfer of some data (say of 100 KB). Worth mentioning, the remote device is not configured for listening data. Is there any limitation on the amount of data that can be written?

The Bluetooth socket operates in blocking mode for both reads and writes.
If you fill up the send buffer, then the only thing that .write() can do to stop you trying to send any more data is to block. The alternative to it blocking would be to return an "operation would block!" error code, just like TCP sockets can do when placed in non-blocking mode. But the Bluetooth socket doesn't provide any such non-blocking mode.
You state that the remote Bluetooth device is not reading from its socket. With this being the case, the local sending buffer and remote receive buffer, with each only being of a certain finite size, will eventually fill up. At this point, your .write() operation is going to block until the remote end reads something from its socket. You can't just keep pumping in megabytes of data and expect it to just buffer it all somewhere.
The differences you experience between different Android platforms are probably down to the different amounts of buffer space available in the related Bluetooth stacks.

Much time taken in data read from socket

I'm using sockets for data transfer from one android phone and PC, I am using DataInputStream.
But it takes a long time in data transfer, about 10 minutes to transfer a 4 MB file.
Could anyone suggest any better way to do that?
I did some changes in my code and now it is taking 15 seconds to read about 1 Mb of data. I want to improve its performance.
My Code is:
InputStream is= socket.getInputStream();
DataInputStream inChannel= new DataInputStream(new BufferedInputStream(in));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int oneByte;
while ((oneByte = inChannel.read()) != -1) {
if (oneByte == 0) {
break;
}
baos.write(oneByte);
byteCount++;
}
byte[] inData = baos.toByteArray();
baos.close();

Are you sure you are not sending empty packets in between there or packets doing something else? If you are using TCP if a packet doesn't reach its destination (a buffer is overfilled with other packets somewhere in between your connection on the router or on one of the devices) the packet will have to be resent. This might be because of your setup. Given the little information you have given us, I can give the the following advice:
Look into more advanced typologies, I am going to assume you are not using any of these:
A send a receive thread; the send thread pushes out packets which you put into a queue:
|P| - packet held inside a Que part of your program
send out thread <-| P | P | P | <-App/program
reveive -> | P | P | P| P -> App/Program deques and analyses data
because your send and receive queuing system works along side each other they are constantly buffering up the received and sent packets. What seems to be happening with your setup (as far as I assume) is that you have a loop which grabs the latest packets and analyses them. This means that the hardware (part of your network hardware) buffer is overfilling with packets which your program is doing other things and by the time your program comes around to collect them, some of them are missing. This causes one side to go "I didn't get those packets, send them again", in other words, your resending the packets you should of sucked up into a Que which are waiting to be dequeued and analysed. The queue can grow regardless of how quickly the program is back to grab that data and do stuff with it (of course you are confined to the RAM). That way you ensure that you have possession of packets that you are supposed to receive rather than rely on your network card/receiver to hold on to them for you, possibly overflowing its buffer.
Another approach is to do a handshake system were one of the programs waits until the last packet is trasmitted, recieved and the other side goes "Cool, send me the next one". This slows down your download/upload speed but is a little bit quicker than packets falling off the end of the end of one of the buffers (each node, such as your router buffers your packets in case more come in than it can handle at a cycle) in the network.
You should utilize a state machine if you can on one or both ends. When your app is downloading the file, lock it into a receive state so its not trying to send/receive other stuff at the same time. Once the download is complete, switch to any other state (say, open file state). If you don't know much about state machines, I recommend you look at the wikipedia article:
http://en.wikipedia.org/wiki/Finite-state_machine