For receiving UDP broadcast packets from the server to an android device, i used a service class and listen for packets in a thread. It receives the packet successfully. The problem is that if multiple packets are being sent from the server in the same time then packet loss will be the result.
I even tried with a queue and processing the received packets in separate thread then also i am not getting the packet. I am completely new to network programming any help would be widely appreciated
void startListenForUdpBroadcast() {
UDPBroadcastThread = new Thread(new Runnable() {
public void run() {
try {
InetAddress broadcastIP = InetAddress.getByName(UdpConstants.IP_ADDRESS);
Integer port = UdpConstants.RECEIVER_PORT;
while (shouldRestartSocketListen) {
listenAndWaitAndThrowIntent(broadcastIP, port);
}
} catch (Exception e) {
Log.i("UDP", "no longer listening for UDP broadcasts cause of error " + e.getMessage());
}
}
});
UDPBroadcastThread.setPriority(Thread.MAX_PRIORITY); //Setting The Listener thread to MAX PRIORITY to minimize packet loss.
UDPBroadcastThread.start();
}
This code listens for new packets and pushes to queue
private void listenAndWaitAndThrowIntent(InetAddress broadcastIP, Integer port) throws Exception {
byte[] recvBuf = new byte[64000];
if (socket == null || socket.isClosed()) {
socket = new DatagramSocket(port, broadcastIP);
socket.setBroadcast(true);
}
//socket.setSoTimeout(1000);
DatagramPacket packet = new DatagramPacket(recvBuf, recvBuf.length);
socket.receive(packet);
messQueue.add(packet);
}
This checks the queue for new messages and process it
/**
* #purpose Checking queue and If anything is added to the queue then broadcast it to UI
*/
private void checkQueue() {
queueThread = new Thread(new Runnable() {
public void run() {
try {
while (shouldRestartSocketListen) {
if (!messQueue.isEmpty()) {
broadcastIntent(messQueue.poll());
}
}
} catch (Exception e) {
}
}
});
queueThread.start();
}
The problem with UDP is that your sender (your server) does not know you (your android device) missed some. It's not lost because you can't read it fast enough, sometimes just over the air interference/congestion or a busy socket.
The receiver would only know if:
you get an error while processing data since you're missing data
OR your UDP packets are numbered sequentially in its header and you detect a missing number (eg. 1,2,4 - missing 3)
Once the packet is lost, it's lost. You got two options:
implement a resend request: upon detection of a missing packet, the receiver would need to notify the sender to resend that missing packet until it does get it, and your packet processing might be halted until it does
OR be able to ignore it, "hey, we can do it without him", and fill-in with blank data (eg. a bitmap would have some blank pixels, like a broken image)
throttle your sending speed down so the packets wont jam up and get lost
the smaller your packets, the more likely they'll live
(option 1: all this resend requesting is just pseudo-TCP, so you might just consider abandoning UDP and go TCP)
I think your problem is mainly that you use Udp Broadcast over wifi.
Their are two very well documented answers why this is a very slow way to operate and why there are much more packet losts:
answer number one.
answer number two.
The thing I did to solve the extremely slow bandwidth was some kind of multi-unicast protocol:
Manage the list of clients you have connected.
Send each packet you have in your server to all of your clients separately with send call.
This is the code in java:
DatagramPacket packet = new DatagramPacket(buffer, size);
packet.setPort(PORT);
for (byte[] clientAddress : ClientsAddressList) {
packet.setAddress(InetAddress.getByAddress(clientAddress));
transceiverSocket.send(packet);
}
If you receive multiple datagrams in a short burst, your receiver loop may have trouble keeping up, and the OS-level RCVBUF in the socket may overflow (causing the OS to drop a packet it indeed did receive).
You might get better handling of short bursts if you increase the RCVBUF size. Prior to doing this, get an idea of how big it is already via socket.getReceiveBufferSize. Also bear in mind that the number of bytes in the receive buffer must accommodate not just the payload but also the headers and the sk_buff structure that stores packets in the kernel (see, e.g. lxr.free-electrons.com/source/include/linux/…).
You can adjust the recieve buffer size via socket.setReceiveBufferSize - but bear in mind that this message is just a hint, and may be overridden by settings in the kernel (if you request a size bigger than the max size allowable by the current kernel network settings, then you'll only get the max size).
After requesting a bigger receive buffer, you should double check what the kernel has allowed by calling socket.getReceiveBufferSize.
If you have the right permissions, you should be able to tweak the max buffer size the kernel will allow - see e.g. https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html
[ Note that, in general, this will accommodate for short bursts where datagrams arrive faster than your client loop can read them - but if the client loop is chronically slower than datagram delivery, you'll still get occasional drops due to overflow. In this case, you need to speed up your client loop, or slow down the sender.
Also, as otherwise noted in other answers, packets may actually be dropped by your network - and mobile networks especially may be prone to this - so if you absolutely need guaranteed delivery you should use TCP. However, if this were your primary problem, you would expect to see dropped packets even when your server sends them slowly, rather than in a burst.]
I suppose that you are capturing only a single packet by saying
socket.receive(packet);
This is a Blocking I/O call which will wait infinitely until it receives a packet so once first packet arrives it is done waiting and next command executes i.e
messQueue.add(packet);
However when multiple packets are been received you need to continue receiving packets. in your case you just stopped receiving packages after arrival of first package
Note: UDP being a un-reliable protocol doesn't guarantee packet delivery so there might be a case a packet is lost , However this can't be a problem on every run of your program , However a nice way to check whether the packet is hitting your machine and problem is within your application (application is not able to handle the packets recieved) use tcpdump (it's a command-line utility for linux-based OS or MAC) use the following command
sudo tcpdump -i <interface name(one that handles traffic) eth0, eth1 .. en0, en1 (for MAC)> host <ipaddress or hostname of UDP Broadcast server>
Example:
sudo tcpdump -i en1 host 187.156.13.5
(if tcpdump command not found then go forward and install it)
By using this command you will see packets pumping in from destination ump server on terminal if you see more then one packets arriving then you would be sure that packets are arriving at machine , However application falls short to address the packet
it might help
With reference to above explanation by me following changes you can make to make code behave according to requirement
I suppose you can make following changes to make your problem code work instead of creating socket into listenAndWaitAndThrowIntent(InetAddress broadcastIP, Integer port ) method create it in startListenForUdpBroadcast() as follows
socket = new DatagramSocket(port, broadcastIP);
socket.setBroadcast(true);
while (shouldRestartSocketListen) {
listenAndWaitAndThrowIntent(broadcastIP, port, socket);
}
Now you also need to change implementation of listenAndWaitAndThrowIntent method as follows
private void listenAndWaitAndThrowIntent(InetAddress broadcastIP,
Integer port, DatagramSocket socket) throws Exception {
byte[] recvBuf = new byte[64000];
//socket.setSoTimeout(1000);
//change value of condition in for loop to desired number of packets you would like to receive in below example it is 2 so it captures two packets
for (int i=0 ; i <= 2 ; i++ ){
DatagramPacket packet = new DatagramPacket(recvBuf, recvBuf.length);
socket.receive(packet);
messQueue.add(packet);
}
}
Try this it should work !!
it might help
Related
I am facing a weird problem. My TCP server has a print line that prints out
the buffer value whenever I receive data from the TCP client.
My TCP Client will be able to send in the buffer, in double values through bytes. On Server receive, the bytes will be received and I output the value on the print line.
For example:
TCP Client send: "0.25"
TCP Server receive: "0.25"
However, because my client is able to control the speed at which data is being sent out, sometimes the server receive values concatenated together when I send out values too fast.
For example:
TCP Client send: "0.27"
TCP Client send: "0.53"
TCP Client send: "0.91"
TCP Server receive: "0.270.530.90"
But what I would expect is this:
TCP Server receive: "0.27"
TCP Server receive: "0.53"
TCP Server receive: "0.90"
Is there a way to solve this issue?
Here is my sending of data on the client side on my android
LoginActivity.getSocket().getOutputStream().write(String.valueOf(multiplier).getBytes());
And here is my receiving of data on the server side.
while(1){
nread = sl_Recv(clientSocket, buffer, sizeof(buffer)-1, 0);
buffer[nread] = '\0';
if(nread==0){
UART_PRINT("Connection Lost!\n\r");
UART_PRINT("Listening again...\n\r");
clientSocket = sl_Accept(welcomeSocket, ( struct SlSockAddr_t *)&serverStorage, (SlSocklen_t*)&addr_size);
if(clientSocket<0){
sl_Close(welcomeSocket);
sl_Close(clientSocket);
ASSERT_ON_ERROR(ACCEPT_ERROR);
}
UART_PRINT("Successfully Connected!!!\n\r");
}
else if(nread==-1){
ASSERT_ON_ERROR(RECV_ERROR);
}
else{
if(atof(buffer)>=0 && atof(buffer)<=1){
UART_PRINT("Data Received: %s\n\r",buffer);
}
}
}
What you have described is not possible unless you have coding bugs, but your expectations are misplaced. TCP goes to a lot of trouble to optimize network usage, and it provides a byte stream to the application, not a message service. Any expectation you may have that it will treat one send as one message and preserve that message boundary when receiving is completely without foundation.
EDIT Problems with your code:
You are ignoring end of stream and errors. You need to check nread for both 0 and -1.
You also need to ensure that adding the trailing null doesn't overrun the buffer, which would be best done by using sizeof buffer -1 as the length in the recv() call.
You aren't separating the numbers when you send them, so if they get coalesced, as they will, you get wrong values.
I am sending newlines
No you aren't:
LoginActivity.getSocket().getOutputStream().write(String.valueOf(multiplier).getBytes());
There are no newlines there, only the bytes of a String value of (apparently) a double.
Thanks for the help everyone. I was pretty sure either the socket on client side was having delays in sending data, or the server side having delays in receiving data.
I added this line of code to my client side on my android.
socket.setTcpNoDelay(true);
And my program works as what I have expected. Not sure if there will be any negative effects for using that, but I didn't seem to face any problems after that.
I am running into some issues with the Java socket API. I am trying to display the number of players currently connected to my game. It is easy to determine when a player has connected. However, it seems unnecessarily difficult to determine when a player has disconnected using the socket API.
Calling isConnected() on a socket that has been disconnected remotely always seems to return true. Similarly, calling isClosed() on a socket that has been closed remotely always seems to return false. I have read that to actually determine whether or not a socket has been closed, data must be written to the output stream and an exception must be caught. This seems like a really unclean way to handle this situation. We would just constantly have to spam a garbage message over the network to ever know when a socket had closed.
Is there any other solution?
There is no TCP API that will tell you the current state of the connection. isConnected() and isClosed() tell you the current state of your socket. Not the same thing.
isConnected() tells you whether you have connected this socket. You have, so it returns true.
isClosed() tells you whether you have closed this socket. Until you have, it returns false.
If the peer has closed the connection in an orderly way
read() returns -1
readLine() returns null
readXXX() throws EOFException for any other XXX.
A write will throw an IOException: 'connection reset by peer', eventually, subject to buffering delays.
If the connection has dropped for any other reason, a write will throw an IOException, eventually, as above, and a read may do the same thing.
If the peer is still connected but not using the connection, a read timeout can be used.
Contrary to what you may read elsewhere, ClosedChannelException doesn't tell you this. [Neither does SocketException: socket closed.] It only tells you that you closed the channel, and then continued to use it. In other words, a programming error on your part. It does not indicate a closed connection.
As a result of some experiments with Java 7 on Windows XP it also appears that if:
you're selecting on OP_READ
select() returns a value of greater than zero
the associated SelectionKey is already invalid (key.isValid() == false)
it means the peer has reset the connection. However this may be peculiar to either the JRE version or platform.
It is general practice in various messaging protocols to keep heartbeating each other (keep sending ping packets) the packet does not need to be very large. The probing mechanism will allow you to detect the disconnected client even before TCP figures it out in general (TCP timeout is far higher) Send a probe and wait for say 5 seconds for a reply, if you do not see reply for say 2-3 subsequent probes, your player is disconnected.
Also, related question
I see the other answer just posted, but I think you are interactive with clients playing your game, so I may pose another approach (while BufferedReader is definitely valid in some cases).
If you wanted to... you could delegate the "registration" responsibility to the client. I.e. you would have a collection of connected users with a timestamp on the last message received from each... if a client times out, you would force a re-registration of the client, but that leads to the quote and idea below.
I have read that to actually determine whether or not a socket has
been closed data must be written to the output stream and an exception
must be caught. This seems like a really unclean way to handle this
situation.
If your Java code did not close/disconnect the Socket, then how else would you be notified that the remote host closed your connection? Ultimately, your try/catch is doing roughly the same thing that a poller listening for events on the ACTUAL socket would be doing. Consider the following:
your local system could close your socket without notifying you... that is just the implementation of Socket (i.e. it doesn't poll the hardware/driver/firmware/whatever for state change).
new Socket(Proxy p)... there are multiple parties (6 endpoints really) that could be closing the connection on you...
I think one of the features of the abstracted languages is that you are abstracted from the minutia. Think of the using keyword in C# (try/finally) for SqlConnection s or whatever... it's just the cost of doing business... I think that try/catch/finally is the accepted and necesary pattern for Socket use.
I faced similar problem. In my case client must send data periodically. I hope you have same requirement. Then I set SO_TIMEOUT socket.setSoTimeout(1000 * 60 * 5); which is throw java.net.SocketTimeoutException when specified time is expired. Then I can detect dead client easily.
I think this is nature of tcp connections, in that standards it takes about 6 minutes of silence in transmission before we conclude that out connection is gone!
So I don`t think you can find an exact solution for this problem. Maybe the better way is to write some handy code to guess when server should suppose a user connection is closed.
As #user207421 say there is no way to know the current state of the connection because of the TCP/IP Protocol Architecture Model. So the server has to notice you before closing the connection or you check it by yourself.
This is a simple example that shows how to know the socket is closed by the server:
sockAdr = new InetSocketAddress(SERVER_HOSTNAME, SERVER_PORT);
socket = new Socket();
timeout = 5000;
socket.connect(sockAdr, timeout);
reader = new BufferedReader(new InputStreamReader(socket.getInputStream());
while ((data = reader.readLine())!=null)
log.e(TAG, "received -> " + data);
log.e(TAG, "Socket closed !");
Here you are another general solution for any data type.
int offset = 0;
byte[] buffer = new byte[8192];
try {
do {
int b = inputStream.read();
if (b == -1)
break;
buffer[offset++] = (byte) b;
//check offset with buffer length and reallocate array if needed
} while (inputStream.available() > 0);
} catch (SocketException e) {
//connection was lost
}
//process buffer
Thats how I handle it
while(true) {
if((receiveMessage = receiveRead.readLine()) != null ) {
System.out.println("first message same :"+receiveMessage);
System.out.println(receiveMessage);
}
else if(receiveRead.readLine()==null)
{
System.out.println("Client has disconected: "+sock.isClosed());
System.exit(1);
} }
if the result.code == null
On Linux when write()ing into a socket which the other side, unknown to you, closed will provoke a SIGPIPE signal/exception however you want to call it. However if you don't want to be caught out by the SIGPIPE you can use send() with the flag MSG_NOSIGNAL. The send() call will return with -1 and in this case you can check errno which will tell you that you tried to write a broken pipe (in this case a socket) with the value EPIPE which according to errno.h is equivalent to 32. As a reaction to the EPIPE you could double back and try to reopen the socket and try to send your information again.
I'm trying to get a Xamarin app to receive around 10 kbps of SPP data through bluetooth from our custom PCB. The BC127 module on the PCB can send a maximum packet size of 255 bytes, so the app needs to receive a packet every ~25ms. I'm using the packets to separate the data, so they need to be received one at a time.
The test app is constantly trying to receive data, however sometimes when it arrives back at the start of the receive method, there are 2+ packets waiting in the receive stream. My question is, what is causing the huge delays in my bluetooth receive method? I've attached the method below. After connecting to the bluetooth module, the app does nothing but run this method in a while(true) loop.
public byte[] ReadBluetoothPacket()
{
int bytesAvailable = 0;
while (bytesAvailable == 0) // Wait for packet to arrive
{
bytesAvailable = stream.BaseInputStream.Available(); // Find out how big the packet is
}
byte[] result = new byte[bytesAvailable];
_socket.InputStream.Read(result, 0, bytesAvailable); // Add the bytes to the buffer
return result;
}
I believe the issue was due to the Android OS doing background tasks which were holding up my receive thread. As it turned out to only be an intermittent problem, rewriting the data receiver to handle multiple packets at once solved it.
I'm using sockets for data transfer from one android phone and PC, I am using DataInputStream.
But it takes a long time in data transfer, about 10 minutes to transfer a 4 MB file.
Could anyone suggest any better way to do that?
I did some changes in my code and now it is taking 15 seconds to read about 1 Mb of data. I want to improve its performance.
My Code is:
InputStream is= socket.getInputStream();
DataInputStream inChannel= new DataInputStream(new BufferedInputStream(in));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int oneByte;
while ((oneByte = inChannel.read()) != -1) {
if (oneByte == 0) {
break;
}
baos.write(oneByte);
byteCount++;
}
byte[] inData = baos.toByteArray();
baos.close();
Are you sure you are not sending empty packets in between there or packets doing something else? If you are using TCP if a packet doesn't reach its destination (a buffer is overfilled with other packets somewhere in between your connection on the router or on one of the devices) the packet will have to be resent. This might be because of your setup. Given the little information you have given us, I can give the the following advice:
Look into more advanced typologies, I am going to assume you are not using any of these:
A send a receive thread; the send thread pushes out packets which you put into a queue:
|P| - packet held inside a Que part of your program
send out thread <-| P | P | P | <-App/program
reveive -> | P | P | P| P -> App/Program deques and analyses data
because your send and receive queuing system works along side each other they are constantly buffering up the received and sent packets. What seems to be happening with your setup (as far as I assume) is that you have a loop which grabs the latest packets and analyses them. This means that the hardware (part of your network hardware) buffer is overfilling with packets which your program is doing other things and by the time your program comes around to collect them, some of them are missing. This causes one side to go "I didn't get those packets, send them again", in other words, your resending the packets you should of sucked up into a Que which are waiting to be dequeued and analysed. The queue can grow regardless of how quickly the program is back to grab that data and do stuff with it (of course you are confined to the RAM). That way you ensure that you have possession of packets that you are supposed to receive rather than rely on your network card/receiver to hold on to them for you, possibly overflowing its buffer.
Another approach is to do a handshake system were one of the programs waits until the last packet is trasmitted, recieved and the other side goes "Cool, send me the next one". This slows down your download/upload speed but is a little bit quicker than packets falling off the end of the end of one of the buffers (each node, such as your router buffers your packets in case more come in than it can handle at a cycle) in the network.
You should utilize a state machine if you can on one or both ends. When your app is downloading the file, lock it into a receive state so its not trying to send/receive other stuff at the same time. Once the download is complete, switch to any other state (say, open file state). If you don't know much about state machines, I recommend you look at the wikipedia article:
http://en.wikipedia.org/wiki/Finite-state_machine
I am making a game for Android, in which two devices connect and then trade a small amount of data, then disconnect and use that data. I started with the Bluetooth Chat example. At this point, I have it mostly working, but some (most) times, one of the devices never receives any data. This is the same app on both devices, so the send and receive code for both is the same.
I am connecting like this:
public BluetoothClientThread(BluetoothDevice get_device,boolean secure){
BluetoothSocket temp=null;
device=get_device;
socket_type=secure?"Secure":"Insecure";
try{
if(secure){
temp=device.createRfcommSocketToServiceRecord(UUID_SECURE);
}
else{
temp=device.createRfcommSocketToServiceRecord(UUID_INSECURE);
}
}
catch(IOException e){
StorageManager.error_log_add(context,TAG,"create() on "+socket_type+" client socket failed.",e);
}
socket=temp;
}
public void run(){
bluetooth_adapter.cancelDiscovery();
try{
socket.connect();
}
catch(IOException e){
StorageManager.error_log_add(context,TAG,"Connection failed.",e);
try{
socket.close();
}
catch(IOException ioe){
StorageManager.error_log_add(context,TAG,"close() on "+socket_type+" client socket failed during connection failure.",ioe);
}
connection_failed();
return;
}
synchronized(BluetoothService.this){
bluetooth_client_thread=null;
}
connected(socket,device,socket_type,false);
}
Once connected, I grab the socket's getInputStream() and getOutputStream().
I then use the following code to read input on the socket:
public void run(){
String packet="";
while(true){
try{
int get_byte=input.read();
char get_char=(char)get_byte;
if(get_char!='\u0003'){
StorageManager.error_log_add(context,"BluetoothService","Adding packet raw data: \""+get_char+"\"",null);
packet+=get_char;
}
else{
StorageManager.error_log_add(context,"BluetoothService","Packet received:\n"+packet,null);
handler.obtainMessage(Activity_Battle_Menu.HANDLER_READ,packet).sendToTarget();
packet="";
}
if(done && they_done){
handler.obtainMessage(Activity_Battle_Menu.HANDLER_READY).sendToTarget();
}
}
catch(IOException e){
StorageManager.error_log_add(context,TAG,"Connection lost.",e);
connection_lost();
break;
}
}
}
It reads data one byte at a time, appending the bytes to a packet String. When the ETX character is reached (which I stick on the end of everything when I send it), a complete packet has been received, and is passed to the activity via handler, where it is acted upon.
Here is my writing code:
public void write(byte[] buffer){
try{
StorageManager.error_log_add(context,"BluetoothService","Writing data:\n"+new String(buffer),null);
output.write(buffer);
}
catch(IOException e){
StorageManager.error_log_add(context,TAG,"write() on "+socket_type+" connected socket failed.",e);
}
}
As you can see, I write to a log when a data "packet" is sent, when a byte is received, and when a whole "packet" has been received.
I'm not sure if any of this is relevant, but:
I am testing this with a Nook Color with Cyanogenmod 7.1 (Android 2.3.7) and Android x86 running in VirtualBox (Android 2.3.5). My app's minSdkVersion is 8 (Android 2.2). For the Android running on my computer, I'm using a Rocketfish USB Bluetooth adapter.
There are supposed to be two packets of data sent by each instance.
First, a data packet holding all of the needed game data is sent.
When the other instance's game data is received, the app sets its own boolean done to true.
It also sends a "done" packet to the other instance.
Thus, once both instances are done, and know the other is done, we know all data has been sent and received, and they both disconnect/stop their Bluetooth threads and act on the data.
Since I have it logging on both devices when data is sent and received, the logs tell me a story of what is going on.
-First, I start both games and go to the Bluetooth activity (obviously).
-Next, I connect to the Nook instance from the emulated instance.
-The emulated instance sends its game data packet.
-The Nook instance receives the emulated instance's game data, and sends its own (sometimes these two are switched in order, just because they happen asynchronously).
-Since it received a game data packet, the Nook instance now sends its "done" packet.
-That is it. The emulated instance never receives either packet. It never even receives a single byte, and the input.read() just blocks, waiting for input that never arrives (like it should).
So that is my problem. It seems that one of two things is happening:
The data never gets sent by output.write() on the Nook instance, even though it does not throw any exception, and the log verifies that it gets called.
The data never gets received by input.read() on the emulated instance. Maybe it gets lost in transmission?
As far as I can tell, my code is rock solid. The problem seems to be Bluetooth-related. Maybe one or both Android instance has a screwy Bluetooth setup?
I also want to add that every rare once in a while (maybe 1/15 times), all of the packets are sent and received properly, and the game proceeds as it should. So whatever is wrong only USUALLY happens, but not always.
I got access to another Android device with Bluetooth. I've since tested my game, and it connects and the devices trade data correctly. I've found several other bugs with my program that occur after the Bluetooth stuff, but that is neither here nor there!
The Nook Color from before and an HTC Status are what worked together. I'm officially concluding, then, that I have a faulty Bluetooth adapter on my computer. The code posted above works nicely enough.