Develop Reference

TCP client for Android: text is not received in full

I am converting a Java desktop project to Android. Part of it includes a TCP connection to a server and parsing a long text from the server to the client (the Android application). This is the code that I have for the desktop project that I also try to use in Android:
// Method is called when a button is tapped
public void tapButton() {
// Create a message to the server that requests for the Departure navdata
String messageToServer = someMethodToMakeHandshakeMessage();
// Connect to the server
if (!messageToServer.equals("")) {
String finalMessageToServer = messageToServer;
new Thread(() -> {
String navdata = connectClient(finalMessageToServer);
getActivity().runOnUiThread(() -> messageReceived(navdata));
// I am also using messageReceived(navdata) without runOnUiThread with the same result
}).start();
}
}
public String connectClient(String messageOut) {
Socket socket = null;
DataInputStream input = null;
DataOutputStream output = null;
BufferedReader br = null;
// Final message from the server
String data = "";
// Message from the server that should terminate TCP connection
String terminator = "END_DATA";
try {
// Create socket and streams
socket = new Socket(someIPAddress, somePort);
input = new DataInputStream(socket.getInputStream());
output = new DataOutputStream(socket.getOutputStream());
//Send message to the server
output.writeBytes(messageOut);
//Read Response
br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
StringBuilder sb = new StringBuilder();
String s = "";
int value = 0;
// Process the message from the server and add to the StringBuilder
while((value = br.read()) != -1) {
// converts int to character
char c = (char)value;
sb.append(c);
if(sb.toString().contains(terminator)) {
break;
}
}
// Create the final string
data = sb.toString();
}
catch (UnknownHostException e) {
// Dealing with exception
}
catch (EOFException e) {
// Dealing with exception
}
catch (IOException e) {
// Dealing with exception
}
finally {
try {
if(socket!=null) { socket.close();}
if(input != null) { input.close();}
if(output != null) { output.close();}
if(br != null) { br.close();}
}
catch (IOException ex) {
// Dealing with exception
}
socket = null;
input = null;
output = null;
br = null;
}
return data;
}
public void messageReceived(String message) {
// Method to deal with received data
}
Whereas the code works fine in the desktop Java application, I have problems with Android (using an emulator). The text is not sent in full length and is cut somewhere in the middle (only 20-50% received by the client; the number of parsed characters differs all the time). Besides, I have noticed that it is taking too long to connect to the server, but, I guess, this is due to working with an emulator.
Should a TCP client receiving long texts from the server be implemented in Android somewhat differently?
EDIT: Implemented the following code using a suggestion by #blackapps:
String line = br.readLine();
while (line != null) {
sb.append(line);
line = br.readLine();
if (line.trim().isEmpty()) {
Log.i("EMPTY LINE>>>>>>>>>>>>>>>>>",line);
}
if(line.equals(terminator)) {
break;
}
}
// Create the final string
data = sb.toString();
}
Two issues. I would like to keep the empty lines in the received text. The terminator is not detected. I think, it is separated from the main text with two empty lines. However, after the first empty line, it goes to indefinite loop and connection never terminated.
EDIT #2.
After having spent several hours trying to figure out what is going on, making changes to the server, and comparing the number of bytes sent and received, I have noticed that this is not the problem with the code. It appears that the client receives the full text. The problem is with how the text is written in the console using the Log.i(String, String) method. I have added the good old System.out.println() in the code, and the whole text was shown in the console. However, the text from Log.i() was cut off in the middle. As this is my first experience with Android Studio, what the heck is going on?
Thanks a lot!

Let talk about TCP socket first.
When talking about TCP socket, it's a stream of data.
TCP views data as an unstructured, but ordered, stream of bytes. It's different from the kinds of socket.io.
From time to time, TCP will grab chunks of data from the send buffer and pass the data to the network layer. The maximum amount of data that can be grabbed and placed in a segment is limited by the maximum segment size (MSS). The MSS is typically set by first determining the length of the largest link-layer frame.
So it depends on the device.
For example, you have two messages, each of them has 1000 bytes data, and you call:
-------------- client side ----------------
client.send(theFirstMessage) // 1000 bytes
client.send(theSecondMessage) // 1000 bytes
-------------- server side -----------------
socket.onReceived(data => {
// process(data)
})
With above pseudocode you should note that:
The data which received and called on onReceived block couldn't be 1000 bytes of theFirstMessage.
It could be first 400 bytes, then on other event you receive 400 bytes, then more 400 bytes (200 of the first one and 200 of the second one).
It could be 1200 bytes (1000 of the first one and 200 of the second one).
TCP views data as an unstructured, but ordered, stream of bytes. Socket.io is a wrapper, when it uses TCP socket, it collect and combine/split the data for you, so that you received the events with exactly the data was sent from other side.
When you work with TCP, you have to do it your self, you have to define the application protocol to do it.
There're two common ways to send/receive TCP requests:
Splitter, you choose a splitter. For example, we choose 32 bits AABBCCDD as the splitter (same as you choose END_DATA string), but keep in mind it's binary data. Then you have to ensure that the data in request doesn't contains the splitter. To do that, you have to encode the request. For example we can encode request as base64, then use the character which isn't included in base64 table as the splitter.
Prefix length, the above method has its overhead as we have to encode request data. The prefix length method is a better choice.
We can prefix the length of request before.
The pseudocode:
// use Int32, 4 bytes to indicate the length of message after it
-------------- client side ----------------
client.send(theFirstMessage.length) // Int32
client.send(theFirstMessage) // 1000 bytes
client.send(theSecondMessage.length)
client.send(theSecondMessage) // 1000 bytes
-------------- server side -----------------
var buffer = Buffer()
socket.onReceived(data => {
buffer.append(data)
let length = Int32(buffer[0...3])
if (buffer.length >= length + 4) {
let theRequest = buffer[4 ... 4 + length - 1]
process(theRequest)
buffer = buffer.dropFirst(4 + length)
}
})
One more thing, when working with TCP socket, it's just stream of bytes, so the endianness is important https://en.wikipedia.org/wiki/Endianness
For example, an android device is little endian and server side (or other android device) is big endian. Then 4 bytes of Int32 from the android device, when received on server side, it will be decoded wrongly if you don't care about it.
So, the prefix length should be encoded by specific endianness.

Stream video frame Android-to-android

I currently work on an app where I use the phone camera and open CV to process the frames. Now I thought it would be cool to be able to send the frames to another Android client. I thought frame by frame with steamer could work, but don't know how to setup the host and if it's not efficient. Any suggestions?

If you just want to send each frame as a raw set of data you can use sockets.
This code below is old now but it worked fine when last tested - it sends an entire video but you can use the same to send whatever file you want:
//Send the video file to helper over a Socket connection so he helper can compress the video file
Socket helperSocket = null;
try {
Log.d("VideoChunkDistributeTask doInBackground","connecting to: " + helperIPAddress + ":" + helperPort);
helperSocket = new Socket(helperIPAddress, helperPort);
BufferedOutputStream helperSocketBOS = new BufferedOutputStream(helperSocket.getOutputStream());
byte[] buffer = new byte[4096];
//Write the video chunk to the output stream
//Open the file
File videoChunkFile = new File(videoChunkFileName);
BufferedInputStream chunkFileIS = new BufferedInputStream(new FileInputStream(videoChunkFile));
//First send a long with the file length - wrap the BufferedOutputStream in a DataOuputStream to
//allow us send a long directly
DataOutputStream helperSocketDOS = new DataOutputStream(
new BufferedOutputStream(helperSocket.getOutputStream()));
long chunkLength = videoChunkFile.length();
helperSocketDOS.writeLong(chunkLength);
Log.d("VideoChunkDistributeTask doInBackground","chunkLength: " + chunkLength);
//Now loop through the video chunk file sending it to the helper via the socket - note this will simply
//do nothing if the file is empty
int readCount = 0;
int totalReadCount = 0;
while(totalReadCount < chunkLength) {
//write the buffer to the output stream of the socket
readCount = chunkFileIS.read(buffer);
helperSocketDOS.write(buffer, 0, readCount);
totalReadCount += readCount;
}
Log.d("VideoChunkDistributeTask doInBackground","file sent");
chunkFileIS.close();
helperSocketDOS.flush();
} catch (UnknownHostException e) {
Log.d("VideoChunkDistributeTask doInBackground","unknown host");
e.printStackTrace();
return null;
} catch (IOException e) {
Log.d("VideoChunkDistributeTask doInBackground","IO exceptiont");
e.printStackTrace();
return null;
}
The full source code is at: https://github.com/mickod/ColabAndroid/tree/master/src/com/amodtech/colabandroid
You may also find there are more up to date socket libraries available which might be better for you to use, but the general principles should be similar.
If you want to stream your video so that the other app can play it like a regular video it streams from the web, then you would want to set up a web server on the 'sending' device. At this point it might be easier to send it to a server and stream from there instead.

How to send image from matlab to android over bluetooth?

Problem:
I want to send an image from matlab to android over bluetooth.
Matlab and android are connected to each other and I can send strings without a problem.
fprintf(tabletObj, 'sleep');
I have a really huge byteArray containing the image I want to send to android. Here you can see just the first bytes:
planString = [-119,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,72,0,0,0,72,8,6,0,0,0,85,-19,-77,71,0,0,0,4,115,66,73,84,8,8,8,8,124,8,100,-120,0,0,29,2,73,68,65,84,120,-100,-75,-100,121,-68,37, ... ]
After that, I set in matlab the OutputBufferSize to the size of the image and send it to the tablet.
s = whos('planString');
obj1.OutputBufferSize = s.bytes;
% Send it to tablet
fwrite(tabletObj, planString, 'int8');
In android you can see following incoming bytes.
Why are there just the first 6 bytes and not more?
The next incoming bytes are more then just 6 bytes, why?
I set the buffersize in android to the same size like matlab.
private void listen() {
byte[] buffer = new byte[picSize]; // buffer store for the stream
Log.i(TAG, "buffer length" + buffer.length);
while (true) {
try {
inputStream.read(buffer);
newMessageReceived(new String(buffer, "UTF-8")); // Send the obtained bytes to the UI activity
} catch (IOException e) {
break;
}
}
}
Edit #1:
I used following code to get only the "right" bytes and put that into an ArrayList with bytes. Now, it seems like that I have just the needed bytes. But it's too slow! You need to wait for more than 1 min. to get all bytes from matlab. Is there a better solution? Why are the incoming bytes split sometimes in 3, sometimes in 15, ...? (see picture below code)
ArrayList<byte[]> bytes = new ArrayList<byte[]>();
...
int nread = inputStream.read(buffer);
byte[] newOne = new byte[nread];
System.arraycopy(buffer, 0, newOne, 0, nread);
bytes.add(newOne);

private void listen() {
byte[] buffer = new byte[10000];
int nbytes = 0;
while (true) {
try {
int nread = inputStream.read(buffer, nbytes, buffer.length - nbytes);
nbytes += nread;
... // after getting all bytes
newMessageReceived(buffer, nbytes); // Send bytes to the UI activity
} catch (IOException e) {
break;
}
}
}
#greenapps thanks for the solution.

Converting int to byte[] in android

I am using the write() method in order to write in a file of the external storage. This method only accepts byte[] as an input. I have tried passing a String and I get an error message ("The method write(int) in the type FileOutputStream is not applicable for the arguments String"). If I pass an int, I don't get error but in the file nothing is written. The value I get from calling getNumSentPackets() is an int and I need to convert it to byte[]. I have been looking at other questions already answered here and I have tried the ByteBuffer option but the result I get in the file is not what I want, this means, I don't get the number of sent packets. Can anybody help me, please?
This is my code:
public void createFile(String name) {
try {
String filename = name;
File myFile = new File(Environment.getExternalStorageDirectory(), filename);
if (!myFile.exists())
myFile.createNewFile();
String title = "FLOODING RESULTS FILE\n\n";
String sent = "Number of sent packets\n";
FileOutputStream fos;
byte[] data = title.getBytes();
byte[] intSent = sent.getBytes();
int numSent = mSender.getNumSentPackets();
byte[] numSentBytes = ByteBuffer.allocate(10).putInt(numSent).array();
try{
fos = new FileOutputStream(myFile);
fos.write(data);
fos.write(intSent);
fos.write(numSentBytes);
fos.flush();
fos.close();
} catch (Exception e) {
e.printStackTrace();
}
} catch (IOException e) {
e.printStackTrace();
}
}
public static int getNumSentPackets() {
return nSentPackets;
}
The expected output file would be as follows:
FLOODING RESULTS FILE
Number of sent packets 200
200 is only an example, meaning with this that I would like to see there a number which would correspond to the total number of sent packets.
Thank you in advance.

As I am a lazy developer, I like to use the existing facilities in my languages of choice, for example, for java, a PrintWriter.
public void createFile(String filename) {
try {
File myFile = new File(Environment.getExternalStorageDirectory(), filename);
PrintWriter out = new PrintWriter(myFile); // this will create the file if necessary
out.println("FLOODING RESULTS FILE");
out.println();
out.print("Number of sent packets ");
out.println(mSender.getNumSentPackets());
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
This is much easier to read and maintain than your current approach, and looks more idiomatic.

ByteBuffer.allocate(capacity).putInt(yourInt).array();

The text representation of "200" requires you to write 3 characters. All files are just a bunch of bytes in the end so there needs to be a mapping from character to some byte value. Assuming ASCII(*) the data to write into the file would be
// '2','0','0'
byte[] textVersion = { 50, 48, 48 }
int on the other hand is a 32bit numeric value, i.e. has 4 bytes and 200 is equivalent to
byte[] intVersion = { 0, 0, 0, 200 }
When using a ByteBuffer, you'll get this. If you write that into a file and a text viewer tries to display that it would display something like ◻◻◻Č if you're lucky. A 0 is actually a non printable control character and therefore often either skipped when printing or replaced with strange looking character like boxes. The 200 would be equivalent to Č in Windows-CP1250. It has no meaning on it's own when interpreted as UTF8 - it's the start of a 2 byte sequence and so the next 2 byte are required to determine which character to display.
You could have used
String.valueOf(200).getBytes( /* you should specify which encoding to use here */ );
which will create the "200" string first, then return you the required bytes for those 3 characters.
You should however use Java's character based IO facility: The numerous (and confusing) Reader & Writer implementations. They all(*^2) wrap an InputStream or OutputStream in the end and do the text to byte conversion for you.
PrintWriter is probably the most convenient to use but not without flaw: https://stackoverflow.com/a/15803472/995891
FileWriter should be avoided because you can't specify the encoding
The longer alternative route would be
BufferedWriter writer = new BufferedWriter(
new OutputStreamWriter(
new FileOutputStream(file), encoding));
writer.write("Hello ");
writer.write(String.valueOf(200));
writer.newLine();
(*) most encodings are ASCII compatible for the first 127 characters which basically covers normal english text.
(*^2) nothing forces a Writer to output the characters into a stream, e.g. StringWriter. But they are used mostly that way.

Transferring large amounts of data over bluetooth on Android Gingerbread

I'm trying to transfer about a megabyte of arbitrary data at a time from one android phone to another. Currently, I write the size, a command code and the data to a DataOutputStream around a BufferedOutputStream, around the OutputStream returned from bluetoothSocketInstance.getOutputStream().
The receiving phone reads the size and command code and then reads from the input stream until it has gotten all the data it is expecting. This works for short strings, but for larger files not all the data is transferred. Running the app in the debugger shows that the write returns without any exceptions and the read reads a fraction of the bytes expected and then blocks indefinitely. It also does not throw any exceptions.
Is there a buffer somewhere that is filling up? Is there something else I need to do to ensure that all the data gets transferred?
My code for the sender and receiver are below:
Sender:
try {
DataOutputStream d = new DataOutputStream(new BufferedOutputStream(mmOutStream,buffer.length+8));
//int b= buffer.length;
d.writeInt(buffer.length);
d.writeInt(command);
d.write(buffer);
d.flush();
} catch (IOException e) {
Log.e(TAG, "Exception during write", e);
}
}
Receiver:
try {
// Read from the InputStream
int messageSize= inStream.readInt();
int messageCode = inStream.readInt();
bytes=0;
buffer =new byte[messageSize];
while(bytes < messageSize)
{
bytes += inStream.read(buffer,bytes,messageSize - bytes);
}
message = bytes;
} catch (IOException e) {
Log.e(TAG, "disconnected", e);
connectionLost();
break;
}

After some more testing on my end, I changed my sending code to look like this:
for(int i=0; i<buffer.length;i+=BIG_NUM)
{
int b = ((i+BIG_NUM) < buffer.length) ? BIG_NUM: buffer.length - i;
d.write(buffer,i,b);
d.flush();
}
The files now get sent. Does anyone have an idea why? Does the call to flush() block until the data has actually been transferred? Is there any documentation about the size of the send and receive buffers that would help me to decide how large I can safely make BIG_NUM?

I have similar problem, when sending file there are some parts missing. I try BufferedOutputStream but problem still exist.
Finally i find simple solution:
You don't need to send buffer length, just split sending buffer to byte array (for example [8192]) and on receive side make sure that this buffer is much bigger about 4 or 8 times than sending buffer. This worked for me and file is sent completed.