Develop Reference

TCP client for Android: text is not received in full

I am converting a Java desktop project to Android. Part of it includes a TCP connection to a server and parsing a long text from the server to the client (the Android application). This is the code that I have for the desktop project that I also try to use in Android:
// Method is called when a button is tapped
public void tapButton() {
// Create a message to the server that requests for the Departure navdata
String messageToServer = someMethodToMakeHandshakeMessage();
// Connect to the server
if (!messageToServer.equals("")) {
String finalMessageToServer = messageToServer;
new Thread(() -> {
String navdata = connectClient(finalMessageToServer);
getActivity().runOnUiThread(() -> messageReceived(navdata));
// I am also using messageReceived(navdata) without runOnUiThread with the same result
}).start();
}
}
public String connectClient(String messageOut) {
Socket socket = null;
DataInputStream input = null;
DataOutputStream output = null;
BufferedReader br = null;
// Final message from the server
String data = "";
// Message from the server that should terminate TCP connection
String terminator = "END_DATA";
try {
// Create socket and streams
socket = new Socket(someIPAddress, somePort);
input = new DataInputStream(socket.getInputStream());
output = new DataOutputStream(socket.getOutputStream());
//Send message to the server
output.writeBytes(messageOut);
//Read Response
br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
StringBuilder sb = new StringBuilder();
String s = "";
int value = 0;
// Process the message from the server and add to the StringBuilder
while((value = br.read()) != -1) {
// converts int to character
char c = (char)value;
sb.append(c);
if(sb.toString().contains(terminator)) {
break;
}
}
// Create the final string
data = sb.toString();
}
catch (UnknownHostException e) {
// Dealing with exception
}
catch (EOFException e) {
// Dealing with exception
}
catch (IOException e) {
// Dealing with exception
}
finally {
try {
if(socket!=null) { socket.close();}
if(input != null) { input.close();}
if(output != null) { output.close();}
if(br != null) { br.close();}
}
catch (IOException ex) {
// Dealing with exception
}
socket = null;
input = null;
output = null;
br = null;
}
return data;
}
public void messageReceived(String message) {
// Method to deal with received data
}
Whereas the code works fine in the desktop Java application, I have problems with Android (using an emulator). The text is not sent in full length and is cut somewhere in the middle (only 20-50% received by the client; the number of parsed characters differs all the time). Besides, I have noticed that it is taking too long to connect to the server, but, I guess, this is due to working with an emulator.
Should a TCP client receiving long texts from the server be implemented in Android somewhat differently?
EDIT: Implemented the following code using a suggestion by #blackapps:
String line = br.readLine();
while (line != null) {
sb.append(line);
line = br.readLine();
if (line.trim().isEmpty()) {
Log.i("EMPTY LINE>>>>>>>>>>>>>>>>>",line);
}
if(line.equals(terminator)) {
break;
}
}
// Create the final string
data = sb.toString();
}
Two issues. I would like to keep the empty lines in the received text. The terminator is not detected. I think, it is separated from the main text with two empty lines. However, after the first empty line, it goes to indefinite loop and connection never terminated.
EDIT #2.
After having spent several hours trying to figure out what is going on, making changes to the server, and comparing the number of bytes sent and received, I have noticed that this is not the problem with the code. It appears that the client receives the full text. The problem is with how the text is written in the console using the Log.i(String, String) method. I have added the good old System.out.println() in the code, and the whole text was shown in the console. However, the text from Log.i() was cut off in the middle. As this is my first experience with Android Studio, what the heck is going on?
Thanks a lot!

Let talk about TCP socket first.
When talking about TCP socket, it's a stream of data.
TCP views data as an unstructured, but ordered, stream of bytes. It's different from the kinds of socket.io.
From time to time, TCP will grab chunks of data from the send buffer and pass the data to the network layer. The maximum amount of data that can be grabbed and placed in a segment is limited by the maximum segment size (MSS). The MSS is typically set by first determining the length of the largest link-layer frame.
So it depends on the device.
For example, you have two messages, each of them has 1000 bytes data, and you call:
-------------- client side ----------------
client.send(theFirstMessage) // 1000 bytes
client.send(theSecondMessage) // 1000 bytes
-------------- server side -----------------
socket.onReceived(data => {
// process(data)
})
With above pseudocode you should note that:
The data which received and called on onReceived block couldn't be 1000 bytes of theFirstMessage.
It could be first 400 bytes, then on other event you receive 400 bytes, then more 400 bytes (200 of the first one and 200 of the second one).
It could be 1200 bytes (1000 of the first one and 200 of the second one).
TCP views data as an unstructured, but ordered, stream of bytes. Socket.io is a wrapper, when it uses TCP socket, it collect and combine/split the data for you, so that you received the events with exactly the data was sent from other side.
When you work with TCP, you have to do it your self, you have to define the application protocol to do it.
There're two common ways to send/receive TCP requests:
Splitter, you choose a splitter. For example, we choose 32 bits AABBCCDD as the splitter (same as you choose END_DATA string), but keep in mind it's binary data. Then you have to ensure that the data in request doesn't contains the splitter. To do that, you have to encode the request. For example we can encode request as base64, then use the character which isn't included in base64 table as the splitter.
Prefix length, the above method has its overhead as we have to encode request data. The prefix length method is a better choice.
We can prefix the length of request before.
The pseudocode:
// use Int32, 4 bytes to indicate the length of message after it
-------------- client side ----------------
client.send(theFirstMessage.length) // Int32
client.send(theFirstMessage) // 1000 bytes
client.send(theSecondMessage.length)
client.send(theSecondMessage) // 1000 bytes
-------------- server side -----------------
var buffer = Buffer()
socket.onReceived(data => {
buffer.append(data)
let length = Int32(buffer[0...3])
if (buffer.length >= length + 4) {
let theRequest = buffer[4 ... 4 + length - 1]
process(theRequest)
buffer = buffer.dropFirst(4 + length)
}
})
One more thing, when working with TCP socket, it's just stream of bytes, so the endianness is important https://en.wikipedia.org/wiki/Endianness
For example, an android device is little endian and server side (or other android device) is big endian. Then 4 bytes of Int32 from the android device, when received on server side, it will be decoded wrongly if you don't care about it.
So, the prefix length should be encoded by specific endianness.

Arduino/Android Bluetooth delay

We are developping an app that uses Bluetooth library to communicate with an Arduino in bluetooth via an HC-05 module. We made a dummy configuration to test the delay without any computation from eather the Arduino or the app and we have a huge delay of about 1 second between a request and an answer...
Protocol looks easy : Android send byte -2 and if byte received is -2, Arduino send -6, -9 and Android answer again and again.
Android Code :
h = new Handler() {
public void handleMessage(android.os.Message msg) {
switch (msg.what) {
case RECIEVE_MESSAGE: // if receive massage
byte[] readBuf = (byte[]) msg.obj;
for(int i=0;i < readBuf.length;i++)
{
if((int) readBuf[i] != 0) {
txtArduino.append(String.valueOf((int) readBuf[i]) + ", ");
}
}
byte[] msg = {-2};
mConnectedThread.writeByte(msg);
break;
}
};
};
Arduino Code :
const int receveidBuffLen = 8*4;
void setup() {
Serial.begin(115200);
}
void loop() {
if (Serial.available() > 0)
{
byte buff[receveidBuffLen];
Serial.readBytes(buff, receveidBuffLen);
for(int i=0; i < receveidBuffLen;i++)
{
if(buff[i] == (byte) -2) // 254
{
byte message[2] = {(byte) -6, (byte) -9};
Serial.write(message, 2);
Serial.flush();
}
}
}
delay(3);
}
Does anyone know where the delay comes from?
We changed the HC05 baudrate (from 9600 to 115 200) : nothing happened. We changed HC05 with another : nothing happened. We used the Blue2Serial library (Bluetooth as SPP) before and delay was the same... We used another controler (ESP8266) and delay still was 1 second...

Looks like this string is an issue:
Serial.readBytes(buff, receveidBuffLen);
Where receveidBuffLen is 32.
Although you get single byte at a time, you're trying to read 32 of them. Of course, if there are no more bytes, the code will be stuck until timeout.
Furthermore, after bytes is read, you never check how many bytes were actually read, but do scan whole the array from bottom to top:
for(int i=0; i < receveidBuffLen;i++)
instead, you have to do something like this:
int bytesAvailable = Serial.available();
if (bytesAvailable > 0)
{
byte buff[receveidBuffLen];
int bytesToRead = (bytesAvailable < receveidBuffLen) ? bytesAvailable : receveidBuffLen;
// Read no more than the buffer size, but not more than available
int bytesActuallyRead = Serial.readBytes(buff, bytesToRead);
for(int i=0; i < bytesActuallyRead;i++)
...

There are a couple problems with the code that might cause delays:
delay function at end of loop - This will slow down the processing that the Ardunio can keep up with
Calling Serial.flush() - This will block the processing loop() until the internal TX serial buffer is empty. That means the Arduino is blocked and new RX data can pile up, slowing the response time.
Calling Serial.readBytes() - You should focus on the smallest unit of data and process that each loop() iteration. If you are trying to deal with multiple message per loop, that will slow now the loop time causing a delay.
You can try to implement a SerialEvent pattern on the Arduino. We will only read one byte at a time from the serial buffer, keeping the processing that the loop() function has todo to a bare minimum. If we receive the -2 byte we will mark a flag. If the flag is marked the loop() function will call the Serial.write() function but will not block for the data to transmit. Here is a quick example.
bool sendMessage = false;
byte message[2] = {(byte) -6, (byte) -9};
void loop()
{
if (sendMessage == true)
{
Serial.write(message, 2);
sendMessage = false;
}
}
/*
SerialEvent occurs whenever a new data comes in the hardware serial RX. This
routine is run between each time loop() runs, so using delay inside loop can
delay response. Multiple bytes of data may be available.
*/
void serialEvent()
{
while (Serial.available())
{
// get the new byte:
byte inChar = ((byte) Serial.read());
if (inChar == ((byte) -2))
{
sendMessage = true;
}
}
}

We just find some solutions by ourselves and want to share them :
Initial situation : 1050 ms for an answer. Alls solutions are independent and done with the initial situation.
Remove Serial.flush() : 1022 ms.
Add a simple Serial.setTimeout(100) in Arduino Code : 135 ms. (Oh man!)
Add a simple timeout to inputStream of 100ms in Android : 95 ms.
Which solution is the best, we can't say but it works now...

Android USB host : interrupt do not respond immedietly

I have a usb device which have a button.
And I want to an android app to catch a signal of the button.
I found inferface and endpoint number of the button.
It had seemed to perform ordinarily at galaxy S3 and galaxy note.
But later, I found that it has delay at other phones.
I was able to receive instant responses about 10% of the time; usually there was a 2-second delay, with some cases where the whole response was lost.
Although I couldn't figure out the exact reason, I realized that the phones that had response delays were those with kernel version 3.4 or later.
Here is the code that I used initially.
if(mConnection != null){
mConnection.claimInterface(mInterfaces.get(0), true);
final UsbEndpoint endpoint = mInterfaces.get(0).getEndpoint(0);
Thread getSignalThread = new Thread(new Runnable() {
#Override
public synchronized void run() {
byte[] buffer = new byte[8];
final ByteBuffer byteBuffer = ByteBuffer.wrap(buffer);
while(mConnection!=null){
int len = mConnection.bulkTransfer(endpoint, buffer, buffer.length, 0);
if( len>=0 ){
// do my own code
}
}
}
});
getSignalThread.setPriority(Thread.MAX_PRIORITY);
getSignalThread.start();
}
edit timeout
when the timeout was set to 50ms, I wasn't able to receive responses most of the time. When the timeout was 500ms, I was able to initially get some delayed-responses; however, I lost all responses after several tries with this setting.
Using UsbRequest
In addition to using the bulktransfer method, I also tried using UsbRequest and below is the code that I used.
#Override
public synchronized void run() {
byte[] buffer = new byte[8];
final ByteBuffer byteBuffer = ByteBuffer.wrap(buffer);
UsbRequest inRequest = new UsbRequest();
inRequest.initialize(mConnection, endpoint);
while(mConnection!=null){
inRequest.queue( byteBuffer , buffer.length);
if( mConnection.requestWait() == inRequest ){
// do my own code
}
}
}
However, the same kind of delay happened even after using UsbRequest.
Using libusb
I also tried using libusb_interrupt_transfer from an open source library called libusb.
However this also produced the same type of delay that I had when using UsbDeviceConnection.
unsigned char data_bt[8] = { 0, };
uint32_t out[2];
int transfered = 0;
while (devh_usb != NULL) {
libusb_interrupt_transfer(devh_usb, 0x83, data_bt, 8, &transfered, 0);
memcpy(out, data_bt, 8);
if (out[0] == PUSH) {
LOGI("button pushed!!!");
memset(data_bt, 0, 8);
//(env)->CallVoidMethod( thiz, mid);
}
}
After looking into the part where libusb_interrupt_transfer is processed libusb, I was able to figure out the general steps of interrupt_transfer:
1. make a transfer object of type interrupt
2. make a urb object that points to the transfer object
3. submit the urb object to the device's fd
4. detect any changes in the fd object via urb object
5. read urb through ioctl
steps 3, 4, 5 are the steps regarding file i/o.
I was able to find out that at step 4 the program waits for the button press before moving onto the next step.
Therefore I tried changing poll to epoll in order to check if the poll function was causing the delay; unfortunately nothing changed.
I also tried setting the timeout of the poll function to 500ms and making it always get values of the fd through ioctl but only found out that the value changed 2~3 seconds after pressing the button.
So in conclusion I feel that there is a delay in the process of updating the value of the fd after pressing the button. If there is anyone who could help me with this issue, please let me know. Thank you.
Thanks for reading

Google Multiplayer RealTime socket streams

I wonder if someone used RealTime sockets with Google Multiplayer (rather than messages).
I have a code that works OK with streams derived from "native"(IP) socket, so I expected it to work with RealTime socket streams too. Unfortunately this is not the case.
The following code works fine with RealTime sockets
Sending end:
int s1, s2;
os.write(new byte[] {(byte)s1, (byte)s2};
os.flush(); // May be redundant, according to Google docs
Receiving end:
byte[] buffer = new byte[2];
is.read(buffer);
int r1=buffer[0] & 0xff;
int r2=buffer[1] & 0xff;
However, since the length of chuck is unknown in advance, I prefer to spit the chunk into two pieces: length and the data, read one after another. Consider therefore a different code:
Sending end:
byte s1, s2;
os.write(s1);
os.write(s2);
os.flush();
Receiving end:
int r1=is.read();
int r2=is.read();
In this case, only first byte is read, while the second byte never comes!
Since Android docs don't recommend flush, I tried to make a wrapper for caching several writes into one on flush:
public class OutputStreamWrapper extends OutputStream {
private OutputStream innerOs;
private ByteArrayOutputStream baos;
public OutputStreamWrapper(OutputStream innerOs) {
this.innerOs = innerOs;
baos = new ByteArrayOutputStream();
}
#Override
public void write(int oneByte) throws IOException {
baos.write(oneByte);
}
#Override
public void flush() throws IOException {
if (baos.size() > 0)
innerOs.write(baos.toByteArray());
baos.reset();
}
#Override
public void write(byte[] buffer, int offset, int count)
throws IOException {
baos.write(buffer, offset, count);
}
#Override
public void close() throws IOException {
flush();
innerOs.close();
baos.close();
}
}
The problem persists!
Am I doing something wrong?
If I can't find the solution, I will have to write custom streams as wrappers for Real-Time Messages, but it is really a shame to avoid "ready to use" streams.

No answer for two months... Have to answer it somehow :)
I tried several approaches, but couldn't get it working. Either the implementation of RealTime sockets is broken (BTW, I haven't come across any example of using those), or I still misunderstand something.
As a result, I found nothing better than making my own RealTime sockets by sending messages (reliable or unreliable, depending on request) under the hood.
The code is so weird that I am ashamed to publish it. But this is the idea:
outputStream has a BiteArrayOuputStream of size equals to max allowed size of RealTimeMessage.
It fires a message either on flush() or when the array is full. There is an indicator of a split packet.
I keep a queue of received messages. The input streams polls the queue and after the whole packet has been collected it returns the bytes.
Works great for me!

Interrupt HttpURLConnection request Android

I am creating simple class which sends requests using HttpURLConnection to server and receive responses. I want to add interrupt() method which interrupt current request (imagine request is running in AsyncTask and interrupt() is called from main thread). There are 2 processes which takes a lot of time and I don't know how to interrupt them:
writing to output stream
reading from input stream
So let's say that I want to for example interrupt reading from input stream which I read like this:
downloadThread = new Thread(new Runnable() {
#Override
public void run() {
try {
buffer = readFully(connection.getInputStream());
} catch( Exception e ) {
e.printStackTrace();
}
}
});
downloadThread.start();
And readFully() method is:
public byte[] readFully(InputStream input) throws IOException {
byte[] buffer = new byte[8192];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
while ((bytesRead = input.read(buffer)) != -1) {
output.write(buffer, 0, bytesRead);
}
return output.toByteArray();
}
How can I stop reading from input stream (in other words, downloading) ?
stop() method of Thread is deprecated and cannot be used. There are written everywhere that I should regularly check if thread should be stopped and stop it manually. But how can I do that when whole time takes this line (If I understand well):
connection.getInputStream()
I think this is the same kind of question but it is unanswered (see comments of solution):
How to stop HttpURLConnection connect on Android
Please, don't refer me to some REST library. I would really like to know how to handle this issue. Thanks ;).

connection.getInputStream() will return as soon as a connection is established with the server and the client is ready to start streaming the response.
So it depends on the connection: if you're e.g. downloading a large file, then most (nearly all) of the time should be spent inside the readFully() method while it is streamed, while if the response is extremely short of empty then readFully() will do very little.
In the first case, the easiest way to "interrupt" is to set a boolean flag and check it inside the loop.
while ((bytesRead = input.read(buffer)) != -1)
{
if (cancelled)
return null;
output.write(buffer, 0, bytesRead);
}
This "boolean flag" can be the isCancelled() method if you're implementing this inside an AsyncTask's doInBackground() (recommended).
In the second case, there's not much you can do, short of killing the thread outright (not recommended at all).

I think it would be better to implement it using Android AsyncTask class.
http://developer.android.com/reference/android/os/AsyncTask.html#cancel(boolean)
There is a method cancel() there and also isCancelled() which you could use to check if the task was cancelled and exit the InputStream reading loop.