I am developing an Android application where I must send and receive data over a serial link, an UART to be precise.
My UART handling class is centered around this function:
fun receive(expectedLength: Int, timeout: Int): ByteArray {
// println("Should receive $expectedLength bytes within $timeout ms")
var read_bytes = 0
var ret: Int
var readBuffer: ByteArray = byteArrayOf()
var start_millis: Long
start_millis = System.currentTimeMillis()
while (read_bytes < expectedLength && (System.currentTimeMillis() - start_millis < timeout) ) {
if (uart.inputStream.available() == 0) {
ret = -1
Thread.sleep(10)
} else {
ret = uart.inputStream.read()
}
if (ret >= 0 ) {
readBuffer = readBuffer.plus(ret.toByte())
read_bytes++
start_millis = System.currentTimeMillis()
}
}
return readBuffer
} // receive expected length with timeout
In real life there are several issues with this function, like for instance a "message" can be shorter or longer than the expectedLength (perhaps I'm waiting for an acknowledgement/confirmation worth of 10 bytes but receive an error of 6 bytes, or even worse, the other way around) or may take slightly longer to arrive.
All this translates into a cascade of errors throughout the application where various services can only chalk this up under communication/hardware errors. Not even retrying is safe.
Is there an alternative approach?
Related
I'm unable to fetch all data from inputstream while connected via bluetooth hc-05 on my android app. Here is my code:
val knownUUIDForDevice = UUID.fromString("00001101-0000-1000-8000-00805f9b34fb")
val remoteDevice: BluetoothDevice = BluetoothAdapter.getDefaultAdapter().getRemoteDevice(macAddress)
val btSocket: BluetoothSocket = remoteDevice.createRfcommSocketToServiceRecord(knownUUIDForDevice)
btSocket.connect()
connectedThread = ConnectedThread(btSocket, rawDataMessageCallback)
connectedThread.start()
My connectedThread code:
class ConnectedThread(private val btSocket: BluetoothSocket)
: Thread() {
private var inStream: InputStream? = null
internal var outStream: OutputStream? = null
init {
inStream = btSocket.inputStream
outStream = btSocket.outputStream
}
override fun run() {
val buffer = ByteArray(512)
inStream = btSocket.inputStream
outStream = btSocket.outputStream
while (true) {
try {
inStream!!.read(buffer)
} catch (e: IOException) {
println(e)
break
}
}
}
}
When I tested connection between my device & realterm app, I was able to see all data. Application above is capable of receiving only about 80% of data. I'm sure it is not fault of device, because I'm able to see all data in realterm app.
What I've tried:
1) Add sleep before/after read method
2) wrap inputStream into buffered stream
3) used inStream!!.readBytes() instead of inStream!!.read(buffer) -> thread freezed up on this method (inside there was call to read method())
4) checking available data with available method(), but it gave me incorrect results - i.e. when stream was going to end, it shows 50 bytes were available, but actually I saw it was 200 bytes available in buffer... Anyway, according to documentation, I should never rely on this method.
5) Used IOUtils from apache commons & Bytestreams from guava -> result the same as in 3rd point, thread freeze
6) Changed size of buffer: tried 1, 10, 100, 256, 512, 1024, 10000 7
Question is: is it possible of loss of data in inputStream?
I am debugging a RTC video stutters issues on Android and I tried serval different devices. To simplify the question, I just keep sending udp packets with an interval about 10ms from a MAC and receiving them on Android with a good wifi. I can see big jitters (bigger than 200ms) almost every minute, can be bigger than 600ms sometimes. Especially when I open and close a task manager. Not duplicated with localhost testing. Can this be fixed?
while(1) {
int s = recvfrom(socket_fd, buffer, sizeof(buffer), 0, (struct sockaddr *)&recv_addr, (socklen_t *)&addr_len);
if (s > 0) {
struct timeval tv_ioctl;
tv_ioctl.tv_sec = 0;
tv_ioctl.tv_usec = 0;
int error = ioctl(socket_fd, SIOCGSTAMP, &tv_ioctl);
if (error == 0) {
int64_t ms = tv_ioctl.tv_sec * 1000LL + tv_ioctl.tv_usec/1000;
if (pre_rev_ms == 0) {
pre_rev_ms = ms;
}
if (ms - pre_rev_ms > 200) {
LOGV("Udp glitches\n");
}
pre_rev_ms = ms;
}
}
}
We are developping an app that uses Bluetooth library to communicate with an Arduino in bluetooth via an HC-05 module. We made a dummy configuration to test the delay without any computation from eather the Arduino or the app and we have a huge delay of about 1 second between a request and an answer...
Protocol looks easy : Android send byte -2 and if byte received is -2, Arduino send -6, -9 and Android answer again and again.
Android Code :
h = new Handler() {
public void handleMessage(android.os.Message msg) {
switch (msg.what) {
case RECIEVE_MESSAGE: // if receive massage
byte[] readBuf = (byte[]) msg.obj;
for(int i=0;i < readBuf.length;i++)
{
if((int) readBuf[i] != 0) {
txtArduino.append(String.valueOf((int) readBuf[i]) + ", ");
}
}
byte[] msg = {-2};
mConnectedThread.writeByte(msg);
break;
}
};
};
Arduino Code :
const int receveidBuffLen = 8*4;
void setup() {
Serial.begin(115200);
}
void loop() {
if (Serial.available() > 0)
{
byte buff[receveidBuffLen];
Serial.readBytes(buff, receveidBuffLen);
for(int i=0; i < receveidBuffLen;i++)
{
if(buff[i] == (byte) -2) // 254
{
byte message[2] = {(byte) -6, (byte) -9};
Serial.write(message, 2);
Serial.flush();
}
}
}
delay(3);
}
Does anyone know where the delay comes from?
We changed the HC05 baudrate (from 9600 to 115 200) : nothing happened. We changed HC05 with another : nothing happened. We used the Blue2Serial library (Bluetooth as SPP) before and delay was the same... We used another controler (ESP8266) and delay still was 1 second...
Looks like this string is an issue:
Serial.readBytes(buff, receveidBuffLen);
Where receveidBuffLen is 32.
Although you get single byte at a time, you're trying to read 32 of them. Of course, if there are no more bytes, the code will be stuck until timeout.
Furthermore, after bytes is read, you never check how many bytes were actually read, but do scan whole the array from bottom to top:
for(int i=0; i < receveidBuffLen;i++)
instead, you have to do something like this:
int bytesAvailable = Serial.available();
if (bytesAvailable > 0)
{
byte buff[receveidBuffLen];
int bytesToRead = (bytesAvailable < receveidBuffLen) ? bytesAvailable : receveidBuffLen;
// Read no more than the buffer size, but not more than available
int bytesActuallyRead = Serial.readBytes(buff, bytesToRead);
for(int i=0; i < bytesActuallyRead;i++)
...
There are a couple problems with the code that might cause delays:
delay function at end of loop - This will slow down the processing that the Ardunio can keep up with
Calling Serial.flush() - This will block the processing loop() until the internal TX serial buffer is empty. That means the Arduino is blocked and new RX data can pile up, slowing the response time.
Calling Serial.readBytes() - You should focus on the smallest unit of data and process that each loop() iteration. If you are trying to deal with multiple message per loop, that will slow now the loop time causing a delay.
You can try to implement a SerialEvent pattern on the Arduino. We will only read one byte at a time from the serial buffer, keeping the processing that the loop() function has todo to a bare minimum. If we receive the -2 byte we will mark a flag. If the flag is marked the loop() function will call the Serial.write() function but will not block for the data to transmit. Here is a quick example.
bool sendMessage = false;
byte message[2] = {(byte) -6, (byte) -9};
void loop()
{
if (sendMessage == true)
{
Serial.write(message, 2);
sendMessage = false;
}
}
/*
SerialEvent occurs whenever a new data comes in the hardware serial RX. This
routine is run between each time loop() runs, so using delay inside loop can
delay response. Multiple bytes of data may be available.
*/
void serialEvent()
{
while (Serial.available())
{
// get the new byte:
byte inChar = ((byte) Serial.read());
if (inChar == ((byte) -2))
{
sendMessage = true;
}
}
}
We just find some solutions by ourselves and want to share them :
Initial situation : 1050 ms for an answer. Alls solutions are independent and done with the initial situation.
Remove Serial.flush() : 1022 ms.
Add a simple Serial.setTimeout(100) in Arduino Code : 135 ms. (Oh man!)
Add a simple timeout to inputStream of 100ms in Android : 95 ms.
Which solution is the best, we can't say but it works now...
Since my problem is close to this one, I haven been looing at feedbacks from this possible solution : Reading on a NetworkStream = 100% CPU usage but I fail to find the solution I need.
Much like in this other question, I want to use something else than an infinite while loop.
More precisely, I am using Xamarin to build Android application in Visual Studio. Since I need a Bluetooth service I am using a Stream to read and send data.
Reading data from Stream.InputStrem is where I have a problem : is there some sort of a blocking call to wait for data to be available without using a while (true) loop ?
I tried :
Begin/End Read
Task.Run and await
Here is a code sample:
public byte[] RetrieveDataFromStream()
{
List<byte> packet = new List<byte>();
int readBytes = 0;
while (_inputStream.CanRead && _inputStream.IsDataAvailable() && readBytes < 1024 && _state == STATE_CONNECTED)
{
try
{
byte[] buffer = new byte[1];
readBytes = _inputStream.Read(buffer, 0, buffer.Length);
packet.Add(buffer[0]);
}
catch (Java.IO.IOException e)
{
return null;
}
}
return packet.ToArray();
}
I call this method from a while loop.
This loop will check until this method returns something else than NULL in which case I will process the data accordingly.
As soon as there is data to be processed, the CPU usage gets low, way lower than if there was no data to process.
I know why my CPU usage is high : the loop will check as often as possible if there is something to read. On the plus side, there is close to no delay when recieving data, but no, that's not a viable solution.
Any ideas to change this ?
# UPDATE 1
As per Marc Gravell's idea, here is what I would like to understand and try :
byte buffer = new byte[4096];
while (_inputStream.CanRead
&& (readBytes = _inputStream.Read(buffer, 0, buffer.Length)) > 0
&& _state == STATE_CONNECTED)
{
for(int i = 0 ; i < readBytes; i++)
packet.Add(buffer[i]);
// or better: some kind of packet.AddRange(buffer, 0, readBytes)
}
How do you call this code snippet ?
Two questions :
If there is nothing to read, then the while condition will be
dismissed : what to do next ?
Once you're done reading, what do you do next ? What do you do to catch any new incoming packets ?
Here are some explanations that should help :
The android device is connected, via bluetooth, to another device that sends data. It will always send a pre-designed packet with a specified size (1024)
That device can stream the data continuously for some time but can also stop at any time for a long period too. How to deal with such behavior ?
An immediate fix would be:
don't read one byte at a time
don't create a new buffer per-byte
don't sit in a hot loop when there is no data available
For example:
byte buffer = new byte[4096];
while (_inputStream.CanRead
&& (readBytes = _inputStream.Read(buffer, 0, buffer.Length)) > 0
&& _state == STATE_CONNECTED)
{
for(int i = 0 ; i < readBytes; i++)
packet.Add(buffer[i]);
// or better: some kind of packet.AddRange(buffer, 0, readBytes)
}
Note that the use of readBytes in the original while check looked somewhat... confused; I've replaced it with a "while we don't get an EOF" check; feel free to add your own logic.
I don't know why but half the sites that go through ssl get a buffer_underflow during my read.
When I have my program chained to call different ssl sites consecutively it doesn't work on half the links, but if I call one by one individually, they work. For example, I used Chrome's developer tools to call https://www.facebook.com on my nexus 7 tablet. When I see the requests, The links called are:
https://www.facebook.com/
https://fbstatic-a.akamaihd.net/rsrc.php/v2/y7/r/Zj_YpNlIRKt.css
https://fbstatic-a.akamaihd.net/rsrc.php/v2/yI/r/5EMLHs-7t29.css etc
... (about 26 links).
If I chain them together (to simulate a call to https://www.facebook.com from the browser), I get half the links getting buffer underflows until eventually I have to close their connections (Reading 0 bytes). However, if I cal them one by one individually, they are always fine. Here is my code for reading:
public int readSSLFrom(SelectionKey key, SecureIO session) throws IOException{
int result = 0;
String TAG = "readSSLFrom";
Log.i(TAG,"Hanshake status: "+session.sslEngine.getHandshakeStatus().toString());
synchronized (buffer){
ByteBuffer sslIn = ByteBuffer.allocate(session.getApplicationSizeBuffer());
ByteBuffer tmp = ByteBuffer.allocate(session.getApplicationSizeBuffer());
ReadableByteChannel channel = (ReadableByteChannel) key.channel();
if (buffer.remaining() < session.getPacketBufferSize()){
increaseSize(session.getPacketBufferSize());
}
int read = 0;
while (((read = channel.read(sslIn)) > 0) &&
buffer.remaining() >= session.getApplicationSizeBuffer()){
if (read < 0){
session.sslEngine.closeInbound();
return -1;
}
inner: while (sslIn.position() > 0){
sslIn.flip();
tmp.clear();
SSLEngineResult res = session.sslEngine.unwrap(sslIn, tmp);
result = result + res.bytesProduced();
sslIn.compact();
tmp.flip();
if (tmp.hasRemaining()){
buffer.put(tmp);
}
switch (res.getStatus()){
case BUFFER_OVERFLOW:
Log.i(TAG,"Buffer overflow");
throw new Error();
case BUFFER_UNDERFLOW:
Log.i(TAG,"Buffer underflow");
if (session.getPacketBufferSize() > tmp.capacity()){
Log.i(TAG,"increasing capacity");
ByteBuffer b = ByteBuffer.allocate(session.getPacketBufferSize());
sslIn.flip();
b.put(sslIn);
sslIn = b;
}
break inner;
case CLOSED:
Log.i(TAG,"Closed");
if (sslIn.position() == 0){
break inner;
} else{
return -1;
}
case OK:
Log.i(TAG,"OK");
session.checkHandshake(key);
break;
default:
break;
}
}
}
if (read < 0){
//session.sslEngine.closeInbound();
return -1;
}
}
dataEnd = buffer.position();
return result;
}
Thank you.
Buffer underflows are acceptable during unwrap and occur often. This happens when you have a partial TLS record (<16KB). This can happen in two cases, 1) When you have less than 16KB, so you get no result when you unwrap - just cache all this data and wait for the remainder to arrive to unwrap it. 2) When you have more than 16KB but the last TLS packet isn't complete, for example 20KB or 36KB. In this case the first 16KB/32KB will give you a result during an unwrap and you need to cache the remaining 4KB and just wait for the rest of the 12KB that completes this TLS packet - before you can unwrap.
I hope this helps, try out my code and see if it works for you.
Sorry this didn't fit in the comment so I responded with an answer instead.