Develop Reference

Android TCP Socket InputStrem Intermittent Read or too Slow

I need to implement a TCP comunication between an IoT device(custom) and an Android App.
For the Wifi device we have a Server Socket, while in Android i have an AsyncTask as a Client Socket. Both the device and the smarthone are connected to the same network.
Here is the Android Client Socket code for the initialization/socket-read and socket-write:
Variables:
static public Socket nsocket; //Network Socket
static public DataInputStream nis; //Network Input Stream
static private OutputStream nos; //Network Output Stream
AsyncTask method doInBackgroud:
#Override
protected Boolean doInBackground(Void... params) { //This runs on a different thread
boolean result = false;
try {
//Init/Create Socket
SocketInit(IP, PORT);
// Socket Manager
SocketUpdate();
} catch (IOException e) {
e.printStackTrace();
Log.i("AsyncTask", "doInBackground: IOException");
clearCmdInStack();
MainActivity.SocketDisconnectAndNetworkTaskRestart();
result = true;
} catch (Exception e) {
e.printStackTrace();
Log.i("AsyncTask", "doInBackground: Exception");
result = true;
} finally {
try {
SocketDisconnect();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
Log.i("AsyncTask", "doInBackground: Finished");
}
return result;
}
Socket Initializzation:
public void SocketInit(String ip, int port) throws IOException {
InetAddress addr = InetAddress.getByName(ip);
SocketAddress sockaddr = new InetSocketAddress(addr, port);
nsocket = new Socket();
nsocket.setReuseAddress(false);
nsocket.setTcpNoDelay(true);
nsocket.setKeepAlive(true);
nsocket.setSoTimeout(0);
nsocket.connect(sockaddr, 0);
StartInputStream();
StartOutputStream();
}
Read from Socket:
private void SocketUpdate() throws IOException, ClassNotFoundException {
int read = 0;
// If connected Start read
if (socketSingleton.isSocketConnected()) {
// Print "Connected!" to UI
setPublishType(Publish.CONNECTED);
publishProgress();
if(mConnectingProgressDialog != null)
mConnectingProgressDialog.dismiss(); //End Connecting Progress Dialog Bar
//Set Communications Up
setCommunicationsUp(true);
Log.i("AsyncTask", "doInBackground: Socket created, streams assigned");
Log.i("AsyncTask", "doInBackground: Waiting for inital data...");
byte[] buffer = new byte[3];
do{
nis.readFully(buffer, 0, 3);
setPublishType(Publish.READ);
publishProgress(buffer);
}while(!isCancelled());
SocketDisconnect();
}
}
Streams init:
public void StartInputStream() throws IOException{
nis = new DataInputStream(nsocket.getInputStream());
}
public void StartOutputStream() throws IOException{
nos = nsocket.getOutputStream();
}
Read and Write methods:
public int Read(byte[] b, int off, int len) throws IOException{
return nis.read(b, off, len); //This is blocking
}
public void Write(byte b[]) throws IOException {
nos.write(b);
nos.flush();
}
public boolean sendDataToNetwork(final String cmd)
{
if (isSocketConnected())
{
Log.i("AsyncTask", "SendDataToNetwork: Writing message to socket");
new Thread(new Runnable()
{
public void run()
{
try
{
Write(cmd.getBytes());
}
catch (Exception e)
{
e.printStackTrace();
Log.i("AsyncTask", "SendDataToNetwork: Message send failed. Caught an exception");
}
}
}).start();
return true;
}
Log.i("AsyncTask", "SendDataToNetwork: Cannot send message. Socket is closed");
return false;
}
The application is very simple, the android app sends a command(via sendDataToNetwork method) to the IoT device and the latter sends back an "ACK" Command string.
The problem
The problem is that while the IoT device always receives the command, the smartphone rarely gets the ACK back. Sometimes i get something like "ACKACKACKACK". By debugging the IoT device i'm sure that it successfully sends back the ACK, so the problem lies in the InputStream read() method which doesn't retrieve the string right away.
Is there a way to empty the InputStream buffer right away, so that i get an "ACK" string back from the IoT device every time i send a command?
Update
I've updated the socket config so that there are no more buffer limitations and i've replaced read() method with readFully. It greatly improved, but still make some mistakes. For istance one out of 2-3 times no ack is received and i get 2 ack the next turn. Is this perhaps the computational limit of the IoT device? Or is there still margin for a better approach?

the problem lies in the InputStream read() method which doesn't empty the buffer right away.
I don't know what 'empty the buffer' means here, but InputStream.read() is specified to return as soon as even one byte has been transferred.
Is there a way to empty the InputStream buffer right away, so that i get an "ACK" string back from the IoT device every time i send a command?
The actual problem is that you could be reading more than one ACK at a time. And there are others.
If you're trying to read exactly three bytes, you should be using DataInputStream.readFully() with a byte array of three bytes.
This will also get rid of the need for the following array copy.
You should not mess with the socket buffer sizes except to increase them. 20 and 700 are both ridiculously small values, and will not be the actual values used, as the platform can adjust the value supplied. Your claim that this improved things isn't credible.
You should not spin-loop while available() is zero. This is literally a waste of time. Your comment says you are blocked in the following read call. You aren't, although you should be. You are spinning here. Remove this.

Android Bluetooth SPP - How to stop transmitting the internal send buffer ?

I am trying to control/operate a motor from an android phone in "as close as possible" realtime using the Android SPP Bluetooth socket interface. The motor ought to run in a so called 'dead man' operation mode. So the motor will only turn if a button on the android APP is touched and ought to stop immediately if the touch is released.
I implemented this by continuously sending 'keep turning' telegrams of 20 Bytes about every 20ms to keep the motor turning and to have the motor stop immediately as soon as no more telegrams are received or if a STOP telegram is received.
This seem to work acceptable well on some phone but others continue sending 'keep turning' telegrams even after the MotionEvent.ACTION_UP event has been processed and no more data are being send.
I assume that this is caused by some internal buffers that cache the transmit data and continue sending until the buffer is empty.
Simple questions:
Is there a way to purge the BT stream transmit buffer to stop all data transfer immediately?
Or can I get the fill level of the transmit buffer in which case I would not put anything more than about 2 telegrams into it?
Or is there a way to specify the buffer size when opening the stream?
Searching the net, I was not able to find anything that talks about BT stream buffer size of buffer management.
And Yes, I have implemented read and write functions as threads and I do not have any problems in reading all telegrams, and I do not need to deliver telegrams in real time but I should be able to stop sending 'keep turning' telegrams within about 50 to 100ms.
Any hints are very welcome.

I am sorry that I did not add the code, I thought it may not be necessary as it is straight forward as:
#Override
public boolean onTouch(final View v,MotionEvent event) {
int eventAction = event.getAction();
switch (eventAction) {
case MotionEvent.ACTION_DOWN:
if (v == btnUp || v == btnDown) {
// Start a thread that sends the goUP or DOWN command every 10 ms until
// btnUp released
tvCounter.setText("----");
action_touched = true;
new Thread(new Runnable() {
#Override
public void run() {
int counter = 1;
// Disable heart beat
ServiceRequest.send(EnRequest.REQ_SET_HEARTBEAT,0);
// Send GoUp command plus a wrapping counter byte every nn ms
// until the button is released
while (action_touched) {
try {
setDeadmanMove(v==btnUp,counter);
Thread.sleep(20);
++counter;
}
catch (InterruptedException ex) {
action_touched = false;
}
catch (Exception ex) {
action_touched = false;
}
}
// Send a STOP command
setDeadmanStop();
// Enable heart beat again
ServiceRequest.send(EnRequest.REQ_SET_HEARTBEAT,1);
// We are done
}
}).start();
}
break;
case MotionEvent.ACTION_UP:
// Stop Thread
action_touched = false;
break;
}
return true;
}
The snipped below is part of the communication class that manages the Bluetooth serial communication.
public void btWrite(DeviceRecord message) {
if (runBTreceiver) {
if (message.isValidRecord()) {
try {
lock.lock();
++lockCounter;
mmBufferedOut.write(message.getFullRecord());
mmBufferedOut.flush();
}
catch (IOException e) {
if (GlobalData.isDebugger) Log.i(TAG, "Failed sending " + message + " " + e.getMessage());
ServiceResponse.send(EnEvent.EVT_BT_RECEIVER_ERROR, "Error data send: " + e.getMessage());
resetConnection();
runBTreceiver=false;
}
finally {
--lockCounter;
lock.unlock();
}
}
}
}
The code snipped that allocates and opens the Bluetooth connection
try {
// Set up a pointer to the remote node using it's address.
BluetoothDevice device = myBluetoothAdapter.getRemoteDevice(myBluetoothMacId);
if (device != null)
{
// Two things are needed to make a connection:
// A MAC address, which we got above.
// A Service ID or UUID. In this case we are using the
// UUID for SPP.
try {
myBluetoothSocket = device.createRfcommSocketToServiceRecord(GlobalData.MY_UUID);
}
catch (IOException e) {
sendEventStatus(EnEvent.EVT_BTADAPTER_FAIL,
String.format(GlobalData.rString(R.string.srv_failcrt),BTERROR_CREATE,e.getMessage()));
}
// Establish the connection. This will block until it connects or
// timeout?
try {
if (! myBluetoothSocket.isConnected()) {
myBluetoothSocket.connect();
}
}
catch (IOException e) {
try {
Log.e("","trying fallback...");
myBluetoothSocket =(BluetoothSocket) device.getClass().getMethod("createRfcommSocket", new Class[] {int.class}).invoke(device,1);
myBluetoothSocket.connect();
}
catch (IOException e2) {
sendEventStatus(EnEvent.EVT_BTADAPTER_FAIL,e2.getMessage());
}
}
}
else {
sendEventStatus(EnEvent.EVT_BTADAPTER_FAIL,
String.format(GlobalData.rString(R.string.srv_failcrt),BTERROR_DEVICE,"getRemoteDevice failed"));
}
}
catch (Exception e) {
sendEventStatus(EnEvent.EVT_BTADAPTER_FAIL, e.getMessage());
return;
}
InputStream tmpIn = null;
OutputStream tmpOut = null;
mmSocket = socket;
// Get the input and output streams, using temp objects because
// member streams are final
try {
tmpIn = socket.getInputStream();
tmpOut = socket.getOutputStream();
}
catch (IOException e) {
ServiceResponse.send(EnEvent.EVT_ERROR, GlobalData.rString(R.string.srv_failcst) + e.getMessage());
resetConnection();
runBTreceiver=false;
}
mmInStream = tmpIn;
// mmOutStream = tmpOut;
mmBufferedOut = new BufferedOutputStream(tmpOut,80);
// Initial request
btWrite(new DeviceRecord(0, 4));
I have never discovered any problems sending and receiving data via this code. All records are sent and received properly. Only problem was that I am unable to purge the transmit buffer at the moment the operate button was released.
To overcome this problem, I have changed the protocol in such a way, that only a single 'keep turning' telegram is send at a time, the next telegram will be send after a response from the other end (sort of handshaking), the program then continue to run this ping/pong until the button is released.
This method works quite well as the transmit buffer will never hold more than one telegram at a time.
the mentioned problem is solved though but I still have no clue of whether it would be possible to purge a transmit buffer

Android: HttpURLConnection doesn't disconnect

by running netstat on the server:
UNTIL A FEW DAYS AGO: I could see the connection being ESTABLISHED only for about a second, and then it would disappear from the list
NOW: it stays as ESTABLISHED for about 10 seconds, then it goes into FIN_WAIT1 and FIN_WAIT2
the Android code is the same, the server is still the same
is it possible that some kind of Android update might have changed things?
I can't really explain it.
I report the code below. The urlConnection.disconnect() gets executed, but the connection remains established on the server.
HttpURLConnection urlConnection = null;
System.setProperty("http.keepAlive", "false");
try {
URL url = new URL(stringUrl);
urlConnection = (HttpURLConnection) url.openConnection();
InputStream instream = new BufferedInputStream(urlConnection.getInputStream());
...
instream.close();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (urlConnection!=null) {
urlConnection.disconnect();
}
}

The moment all data on input stream is consumed, the connection is released automatically and added to the connection pool. The underlying socket connection is not released, assuming the connection will be reused in near future. It is a good practice to call disconnect in finally block, as it takes care of connection release in case of exceptions.
Here is the implementation of read method of FixedLengthInputStream:
#Override public int read(byte[] buffer, int offset, int count) throws IOException {
Arrays.checkOffsetAndCount(buffer.length, offset, count);
checkNotClosed();
if (bytesRemaining == 0) {
return -1;
}
int read = in.read(buffer, offset, Math.min(count, bytesRemaining));
if (read == -1) {
unexpectedEndOfInput(); // the server didn't supply the promised content length
throw new IOException("unexpected end of stream");
}
bytesRemaining -= read;
cacheWrite(buffer, offset, read);
if (bytesRemaining == 0) {
endOfInput(true);
}
return read;
}
When bytesRemaining variable becomes 0, endOfInput is called which will futher call release method with true parameter, which will ensures the connection is pooled.
protected final void endOfInput(boolean reuseSocket) throws IOException {
if (cacheRequest != null) {
cacheBody.close();
}
httpEngine.release(reuseSocket);
}
Here is the release method implementation. The last if check ensures whether connection need to be closed down or added to the connection pool for reuse.
public final void release(boolean reusable) {
// If the response body comes from the cache, close it.
if (responseBodyIn == cachedResponseBody) {
IoUtils.closeQuietly(responseBodyIn);
}
if (!connectionReleased && connection != null) {
connectionReleased = true;
// We cannot reuse sockets that have incomplete output.
if (requestBodyOut != null && !requestBodyOut.closed) {
reusable = false;
}
// If the headers specify that the connection shouldn't be reused, don't reuse it.
if (hasConnectionCloseHeader()) {
reusable = false;
}
if (responseBodyIn instanceof UnknownLengthHttpInputStream) {
reusable = false;
}
if (reusable && responseBodyIn != null) {
// We must discard the response body before the connection can be reused.
try {
Streams.skipAll(responseBodyIn);
} catch (IOException e) {
reusable = false;
}
}
if (!reusable) {
connection.closeSocketAndStreams();
connection = null;
} else if (automaticallyReleaseConnectionToPool) {
HttpConnectionPool.INSTANCE.recycle(connection);
connection = null;
}
}
}
Note: I had previously answered couple of SO questions related to HttpURLConnection, which can help you in understanding the underlying implementation. Here are the links : Link1 and Link2.

As per how the TCP protocol works, when you close a connection, it doesn't automatically disappear from within your socket list.
When you send the termination signal to the other part, there starts a protocol (a procedure, morelike), where the first step is precisely your intention of closing the connection. You send a signal to the other node and that would involve the FIN_WAIT1 status.
When the user has received that signal, the next step is to acknowledge it from the remote side. This means that the opposite server sends you another signal symbolizing that the node is ready to close the connection too. That would be the FIN_WAIT2 status.
Between these two steps, it might happen that the remote node hasn't responded yet (so you're not been acknowledged that you want to close the connection). In that time, you would be in an intermediate state called CLOSE_WAIT (resuming: once you've sent the FIN signal to the remote server and they haven't responded yet).
The TIME_WAIT state would mean that you are giving some graceful time to the server before definitely closing it to receive some packets. You do this because connection anomalies might happen, and the remote server could have not received the 'disconnection' message and send you some packet. So when that happens, instead of creating a new socket between both nodes, you associate it to the one you have in the TIME_WAIT state and simply discard that packet because probably the sequence number will not be ordered.
There are some other states you might see, but according to the way you describe it, it seems pretty normal to me, unless when you call the .disconnect() method, the ESTABLISHED status would last. In that case something's not working as expected (it might be related to some kind of overloading or non-optimized code which might make your execution very slow).

You have to disconnect your open UrlConnection
HttpURLConnection urlConnection = null;
System.setProperty("http.keepAlive", "false");
try {
URL url = new URL(stringUrl);
urlConnection = (HttpURLConnection) url.openConnection();
InputStream instream = new BufferedInputStream(urlConnection.getInputStream());
...
instream.close();
urlConnection.disconnect(); //HERE
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (urlConnection!=null) {
urlConnection.disconnect();
}
}

Do we need to consume HttpURLConnection's error stream when IOException thrown

According to technical guide from Oracle Java, we should consume HttpURLConnection's error stream when IOException thrown
http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html
What can you do to help with Keep-Alive? Do not abandon a connection
by ignoring the response body. Doing so may results in idle TCP
connections. That needs to be garbage collected when they are no
longer referenced.
If getInputStream() successfully returns, read the entire response
body.
When calling getInputStream() from HttpURLConnection, if an
IOException occurs, catch the exception and call getErrorStream() to
get the response body (if there is any).
Reading the response body cleans up the connection even if you are not
interested in the response content itself. But if the response body is
long and you are not interested in the rest of it after seeing the
beginning, you can close the InputStream. But you need to be aware
that more data could be on its way. Thus the connection may not be
cleared for reuse.
Here's a code example that complies to the above recommendation:
Here's the code example
try {
URL a = new URL(args[0]);
URLConnection urlc = a.openConnection();
is = conn.getInputStream();
int ret = 0;
while ((ret = is.read(buf)) > 0) {
processBuf(buf);
}
// close the inputstream
is.close();
} catch (IOException e) {
try {
respCode = ((HttpURLConnection)conn).getResponseCode();
es = ((HttpURLConnection)conn).getErrorStream();
int ret = 0;
// read the response body
while ((ret = es.read(buf)) > 0) {
processBuf(buf);
}
// close the errorstream
es.close();
} catch(IOException ex) {
// deal with the exception
}
}
Does this applicable to Android platform? As I don't see such technique in most of the Android code example.

If you are not interested in displaying the error message to the user, close the InputStream or invoke disconnect on HttpURLConnection in finally block without reading the error message. This is what you see in most of the examples.
I came across following comment in one of the source code, while browsing the implementation of HttpURLConnection. That could be the reason why connections are closed without reading all data.
This should be invoked when the connection is closed unexpectedly to
invalidate the cache entry and to prevent the HTTP connection from
being reused. HTTP messages are sent in serial so whenever a message
cannot be read to completion, subsequent messages cannot be read
either and the connection must be discarded.
According to Android's implementation of HttpURLConnection, in case of exception:
If error is not read and the InputStream is closed, the connection will be considered as not reusable and closed down.
If you read the error and then close the InputStream, connection is considered as reusable and is added to the connection pool.
You can see in the below image, variable connection & connectionReleased are set to null and true respectively, as soon as all data is read. Note that getErrorStream returns the InputStream, so it is valid in exception scenario also.
Code analysis : Let's look at the FixedLengthInputStream one of the specialized InputStream implementation. Here is the close method implementation:
#Override public void close() throws IOException {
if (closed) {
return;
}
closed = true;
if (bytesRemaining != 0) {
unexpectedEndOfInput();
}
}
Instance variable bytesRemaining contains byte count still available on the InputStream to be read. Here is the unexpectedEndOfInput method implementation:
protected final void unexpectedEndOfInput() {
if (cacheRequest != null) {
cacheRequest.abort();
}
httpEngine.release(false);
}
Here is the release method implementation. Calling disconnect on HttpURLConnection instance leads the call to this release method with false as parameter.
The last if check ensures whether connection need to be closed down or added to the connection pool for reuse.
public final void release(boolean reusable) {
// If the response body comes from the cache, close it.
if (responseBodyIn == cachedResponseBody) {
IoUtils.closeQuietly(responseBodyIn);
}
if (!connectionReleased && connection != null) {
connectionReleased = true;
// We cannot reuse sockets that have incomplete output.
if (requestBodyOut != null && !requestBodyOut.closed) {
reusable = false;
}
// If the headers specify that the connection shouldn't be reused, don't reuse it.
if (hasConnectionCloseHeader()) {
reusable = false;
}
if (responseBodyIn instanceof UnknownLengthHttpInputStream) {
reusable = false;
}
if (reusable && responseBodyIn != null) {
// We must discard the response body before the connection can be reused.
try {
Streams.skipAll(responseBodyIn);
} catch (IOException e) {
reusable = false;
}
}
if (!reusable) {
connection.closeSocketAndStreams();
connection = null;
} else if (automaticallyReleaseConnectionToPool) {
HttpConnectionPool.INSTANCE.recycle(connection);
connection = null;
}
}
}
The code shared by you, in which the IOException is handled, error stream is read and then closed, ensures the Connection is reusable and is added to the connection pool. The moment all data is read from InputStream the Connection is added to the connection pool. Here is the read method implementation of FixedLengthInputStream :
#Override public int read(byte[] buffer, int offset, int count) throws IOException {
Arrays.checkOffsetAndCount(buffer.length, offset, count);
checkNotClosed();
if (bytesRemaining == 0) {
return -1;
}
int read = in.read(buffer, offset, Math.min(count, bytesRemaining));
if (read == -1) {
unexpectedEndOfInput(); // the server didn't supply the promised content length
throw new IOException("unexpected end of stream");
}
bytesRemaining -= read;
cacheWrite(buffer, offset, read);
if (bytesRemaining == 0) {
endOfInput(true);
}
return read;
}
When bytesRemaining variable becomes 0, endOfInput is called which will futher call release method with true parameter, which will ensures the connection is pooled.
protected final void endOfInput(boolean reuseSocket) throws IOException {
if (cacheRequest != null) {
cacheBody.close();
}
httpEngine.release(reuseSocket);
}

If it's documented for Java it's binding for the Android platform.

Reconnecting to an Android Socket in order to wait for Server

So I have TCP server in Windows that is programmed in C++ and a client in JAVA, Android 4.0.4.
In Android, I connect like this:
public boolean sendConnectRequest()
{
while (isSocketConnected == false)
{
try {
if(comSocket == null)
comSocket = new Socket("192.168.0.1",1531);
isSocketConnected = comSocket.isConnected();
if (isSocketConnected) {
out = comSocket.getOutputStream();
in = comSocket.getInputStream();
}
else
comSocket.close();
}
catch (IOException ex)
{
Log.e("TCP Error", ex.getLocalizedMessage());
}
}
return true;
}
I typically have no problems with this code on the first connection to the server.
When i disconnect from the server, I call this:
public void closeConnection() {
if (comSocket != null)
{
try {
comSocket.close();
isSocketConnected = false;
if (out != null)
out.close();
if (in != null)
out.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
So here is the problem... I hit the home button on the smartphone, which places the program in pause. I start the program again and it calls the resume function in activity, which in turn starts the process toward reconnection. The connection is attempted and i get no errors. However, my Windows server records no connection. In Windows, I know that I am still blocked at:
SOCKET connectionSocket = accept(tcpNetworkData->socket, (struct sockaddr*)&from, &fromlen);
I believe this is normal. When I am in debug mode on the Android side, I notice that it returns immediately from the line: comSocket = new Socket("192.168.0.1",1531); This should indicate to me that a connection is made.
If you follow me so far... I should also say that if I shut the server down, the client resets by closing the connection and opening a new one. This time the comSocket = new Socket("192.168.0.1",1531) does not block as it should and the execution keeps going. This is obviously wrong. I think it is a resource release problem but why? With Winsock you can solve this problem with this line of code:
int so_reuseaddr = TRUE;
setsockopt(networkData->socket, SOL_SOCKET, SO_REUSEADDR, (const char*)&so_reuseaddr,sizeof(so_reuseaddr));
Can you do something similar with Android or do you have to? Thank you for your help!

According to the javadoc the connection is established once you call the constructor.
Socket(InetAddress address, int port)
Creates a stream socket and connects it to the specified port number at the specified IP address.
When you press the home button, your app goes in background but it does not get killed immediately, so your comSocket might be not null when you get back to your application. In that case you are not calling the constructor again, thus you are not reconnecting to the server. What you should do is
if(comSocket == null){
comSocket = new Socket("192.168.0.1",1531);
}else{
comSocket.connect(new InetSocketAddress("192.168.0.1",1531));
}
(and please please place the curly brackets :-) )

Something to keep in mind is that the isConnected() method isn't very reliable for detecting when the remote side has closed the connection, (here is an example).
You have to figure this out by reading or writing on the associated Input/Output Streams.
Try using PrintWriter.checkError(), which will return true as soon as the client can no longer connect to the server.