Develop Reference

Bluetooth Low Energy (between ESP32 and Android smartphone): Data transmission quite slow

This is my first question ever asked on this board
The project explained short:
5 sensors, connected with an esp32 board are transmitting 1000 samples/second, each sample has 16 bit. Those values should be transmitted via BLE (With the BLE Arduino library and an ESP32). The connected device (Smartphone) should read those values and do something with them (Also via BLE, with the following library: https://github.com/RobotPajamas/Blueteeth). The ESP32 is the Server! Java is used in Android Studio!
The problem:
While testing the BLE connection a simple "hello world" was transmitted as the value for a characteristic. Every time i received the "hello world" on the android-device-side, a variable was incremented: The problem is, the variable only got incremented 4 times in one second. This means (assuming 1 char in a string equals 1 byte) 11byte*4(1/s)=44byte/s are being transmitted. -> This clearly is not enough (should not BLE transmit ~2MBit/s (minus the protocol-data))
Code Fragments
ESP32: BLE-Server that transmits value
#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEServer.h>
#define SERVICE_UUID "4fafc201-1fb5-459e-8fcc-c5c9c331914b"
#define CHARACTERISTIC_UUID "beb5483e-36e1-4688-b7f5-ea07361b26a8"
class MyCallbacks: public BLECharacteristicCallbacks {
void onWrite(BLECharacteristic *pCharacteristic) {
std::string value = pCharacteristic->getValue();
if (value.length() > 0) {
Serial.println("*********");
Serial.print("New value: ");
for (int i = 0; i < value.length(); i++)
Serial.print(value[i]);
Serial.println();
Serial.println("*********");
}
}
};
void setup() {
Serial.begin(115200);
BLEDevice::init("MyESP32");
BLEServer *pServer = BLEDevice::createServer();
BLEService *pService = pServer->createService(SERVICE_UUID);
BLECharacteristic *pCharacteristic = pService->createCharacteristic(
CHARACTERISTIC_UUID,
BLECharacteristic::PROPERTY_READ |
BLECharacteristic::PROPERTY_WRITE
);
pCharacteristic->setCallbacks(new MyCallbacks());
pCharacteristic->setValue("Hello World");
pService->start();
BLEAdvertising *pAdvertising = pServer->getAdvertising();
pAdvertising->start();
}
void loop() {
// put your main code here, to run repeatedly:
delay(2000);
}
Android Studio Code (Snippet of the receiving source):
try
{
while(sampleBluetoothData)
{
this.selectedDevice.readCharacteristic(MainActivity.characteristicUUID, MainActivity.serviceUUID, (response, data) ->
{
if (response != BlueteethResponse.NO_ERROR) {
return;
}
Log.d("AUSGANG", new String(data) + "times: "+ i);
i++;
});
}
}
catch (Exception e)
{
e.printStackTrace();
}
The write on the ESP32 side is a blank example code of the Arduino IDE, the read on the Android-side is made by the BLE-Library publisher. Yes the Log.d effects the performance, but it does not drop it that much.
The variable "data" of the Android code is the received char-array. The bluetooth-reading runs on a background thread.
Question I ask myself now:
Is the Android-Studio library the problem or the Arduino library
Is this a normal behaviour, that if a value of a characteristic does not change, it is being transmitted quite slowly.
How fast can you update a value of a characteristic
Thank you in advance!

BLE can definitely transfer much more than 4 portions of 11 bytes per second.
Approach of reading:
Generally, continuos reading all the time is NOT the expected BLE way - it's better to subscribe to data changes, so ESP32 will notify only when needed (e.g. do selectedDevice.subscribeToCharacteristic once, instead of reading in a loop, but then ESP32 code should be changed accordingly)
I guess selectedDevice.readCharacteristic requests asynchronous BLE read, and when you call it in while(sampleBluetoothData), your Bluetooth library is adding more and more read requests. Maybe it would be wise to request new read only after the previous read is done - in read callback add if(sampleBluetoothData) { this.readAgain(); }
Consider making a testing prototype from this kickstart example: BLEProof on github - Android & ESP32, read, write, notify (but it uses just system API without Bluetooth library, you approach is better, it's easier and safer to use the library).
What else to check:
Android side: are you sure that your code doesn't go inside of if (response != BlueteethResponse.NO_ERROR) ?
Android side: to ensure Bluetooth library is not overloaded with read requests, try adding a delay 50 milliseconds in the reading loop (just to check, it's not a solution)
Android side: are you sure that you don't have other BLE read/writes while you read those data?
ESP32 side: use shorter BLE connection interval (BLE throughput article) - add pAdvertising->setMinPreferred(0x06); and pAdvertising->setMaxPreferred(0x20); before pAdvertising->start(); (but that sets only "preferred" interval, Android may ignore that)

Using read requests, you are mainly limited by the connection interval for transfer speeds - that is 2 intervals for request + response.
If for example your client has a connection interval of 50ms, you should expect to read a characteristic of up to 20 bytes 10 times per second.
If another client has a connection interval of 30ms, this rate improves to 16.6 reads per second.
The fastest negotiatiable connection interval is 7.5ms for a maximum of 66.6 reads per second (10.7kbps with 20 byte reads).

I was reading BLE wikipedia page and the minimum data rate is 125Kbit/s , so I think that in your case is viable, because you only will transmit 16Kbit/s. Take a look in BLE wikipedia.

Cannot send large APDU commands with libnfc using nfc_initiator_transceive_bytes()

I'm trying to build a C++ wrapper around libnfc to make a communication between my Android and the PN532 RFID module.
This helped me a lot: http://nfc-tools.org/index.php/Libnfc:APDU_example
This code is meant to send an APDU command where the body is contained in message (I'm not sending any header bytes etc.) and read the response into response.
Problem: If message exceeds 262 characters then I get a buffer overflow detected error. Otherwise it works perfectly well. I don't even think the error is thrown by the NFC library.
bool send(const std::string &message, std::string &response){
std::vector<uint8_t> apduCmd(message.begin(), message.end());
uint8_t *capdu = &apduCmd[0];
size_t capdulen = apduCmd.size();
uint8_t rapdu[10];
size_t rapdulen = 10;
// BUFFER OVERFLOW HERE
int res = nfc_initiator_transceive_bytes(m_nfcDevice, capdu, capdulen, rapdu, rapdulen, 500);
if (res<0) {
return false;
}
if(res<2 || rapdu[res-2] != 0x90 || rapdu[res-1] != 0x00){
return false;
}
// byteArrayToString omitting the last two bytes
response = byteArrayToString(rapdu, 0, res-2);
return true;
}

The limit of 262 bytes is a hard limit imposed the PN532 NFC chip. This is the maximum size of the raw data that can be sent (and received) in one InDataExchange command. libnfc explicitly enforces this limit for the method nfc_initiator_transceive_bytes() (see the definition of abtCmd in pn53x_initiator_transceive_bytes() and the definition of PN53x_EXTENDED_FRAME__DATA_MAX_LEN).
What you could do to overcome this limit, is to compose your own ISO/IEC 14443-4 blocks (using InCommunicateThru, i.e. nfc_initiator_transceive_bytes() with m_nfcDevice->bEasyFraming turned off. While each frame would still be limited to 263 bytes (the PN532 actually allows 264 bytes for InCommunicateThru but libnfc seems to limit this to 263 bytes), you can then pack your extended length APDUs into multiple ISO/IEC 14443-4 I-blocks. However, you would need to handle the whole ISO/IEC 14443-4 framing on your own (which means that you also have to take care of receive acknowledgements, etc.)
Finally, since the other communication endpoint is an Android device: Many Android devices do not support extended length APDUs. Consequently, even if you send longer APDUs, you might not be able to receive and process them on the Android side. Also, be aware that you should send proper APDUs conforming to the structures defined in ISO/IEC 7816-4 (i.e. APDUs with valid header and lengths fields), otherwise you might run into issues when talking to some devices.

Android nfcA.connect(), nfcA.transceive(), nfcA.setTimeout() and nfcA.getMaxTransceiveLength()

I have a a number of newbie NfcA questions. There seems to be little guidance on this in the docs and elsewhere on the web, so I hope no-one minds me stringing a few basic questions together here...
I am using nfcA.transceive() to write data to my NTAG213 tag like this:
byte[] result = nfcA.transceive(new byte[] {
(byte)0xA2, // WRITE
(byte)(pageNum & 0x0ff),
myData[0], myData[1], myData[2], myData[3]
});
1. The result array is a single byte of value 10. What does this mean and what other values should I look out for?
I am also using the same method to read data from my NTAG213 tags:
byte[] result = nfcA.transceive(new byte[] {
(byte)0x30, // READ
(byte)(pageNum & 0x0ff)
});
2. I expected this to return 4 bytes of user data (i.e., the 4 bytes that correspond to my pageNum), but it returned 16 bytes. Why is that the case?
3. Is it good practise to check nfcA.isConnected() before calling nfcA.connect() and, if so, is there likely to be any sigificant performance penalty in doing so? (I ask as I have seen code examples from reputable sources of both.)
4. Is it better to call nfcA.setTimeout() before or after nfcA.connect()?
5. For my NTAG213 tags nfcA.getMaxTransceiveLength() returns 253. Does that really mean I can write up to 251 bytes of user data (plus the 2 other bytes) in one go and, if so, is that advisable or is it better to write each page (4 bytes) with separate nfcA.transceive() calls?

1. The result array for a WRITE command is a single byte of value 10. What does this mean and what other values should I look out for?
The value 10 (Ah in hexadecimal or 1010b in binary representation) is an explicit ACK, an acknowledgement returned when a command that returns no data succeeds.
The possible values are actual data, ACK, passive ACK, or NACK. These are defined by the NFC Forum Digital Protocol specification and by the NFC Forum Type 2 Tag Operation specification.
If the command is expected to return actual data on success, the data is returned instead of an explicit ACK value.
ACK is defined as a 4-bit short frame (see NFC Forum Digital Protocol specification and ISO/IEC 14443-3 for further details) with the value 1010b (Ah).
A passive ACK is defined as the tag not sending a response at all within a certain timeout.
NACK is defined as a 4-bit short frame with the value 0x0xb (where x is either 0 or 1).
The NTAG213/215/216 product data sheet is a bit more specific on possible NACK values:
0000b (0h) indicates an invalid command argument.
0001b (1h) indicates a parity or CRC error.
0100b (4h) indicates an invalid authentication counter overflow.
0101b (5h) indicates an EEPROM write error.
In addition to the above, the NFC stack implementations on some devices do not properly propagate NACK responses to the app. Instead they either throw a TagLostException or return null. Similarly, you might(?) get a TagLostException indicating a passive ACK.
Thus, you would typically check the result of the transceive method for the following (unless you send a command that is expected to result in a passive ACK):
try {
response = nfca.transceive(command);
if (response == null) {
// either communication to the tag was lost or a NACK was received
} else if ((response.length == 1) && ((response[0] & 0x00A) != 0x00A)) {
// NACK response according to Digital Protocol/T2TOP
} else {
// success: response contains ACK or actual data
}
} catch (TagLostException e) {
// either communication to the tag was lost or a NACK was received
}
2. I expected the READ method to to return 4 bytes of user data (i.e. the 4 bytes that correspond to my pageNum), but it returned 16 bytes. Why is that the case?
The READ command is defined to return 4 blocks of data starting with the specified block number (in the NFC Forum Type 2 Tag Operation specification). Thus, if you send a READ command for block 4, you get the data of blocks 4, 5, 6, and 7.
3. Is it good practise to check nfcA.isConnected() before calling nfcA.connect() and, if so, is there likely to be any sigificant performance penalty in doing so?
If you receive the Tag handle directly from the NFC system service (through an NFC intent) the tag won't be connected. So unless you use the Tag handle before calling nfca.connect(), I don't see why you would want to call nfca.isConnected() before. However, calling that method before connecting has barely any performance overhead since calling isConnected() on a closed tag technology object will be handled by the famework API without calling into the NFC system service. Hence, it's not much more overhead than a simple if over a boolean member variable of the NfcA object.
4. Is it better to call nfcA.setTimeout() before or after nfcA.connect()?
I'm not sure about that one. However, the transceive timeout is typically reset on disconnecting the tag technology.
5. For my NTAG213 tags nfcA.getMaxTransceiveLength() returns 253. Does that really mean I can write up to 251 bytes of user data (plus the 2 other bytes) in one go and, if so, is that advisable or is it better to write each page (4 bytes) with separate nfcA.transceive() calls?
No, you can only write one block at a time. This is limited by the WRITE command of the NTAG213, which only supports one block as data input.
However, a transceive buffer size of 253 allows you to use the FAST_READ command to read multiple blocks (up to 62, so up to 45 for the NTAG213) at a time:
int firstBlockNum = 0;
int lastBlockNum = 42;
byte[] result = nfcA.transceive(new byte[] {
(byte)0x3A, // FAST_READ
(byte)(firstBlockNum & 0x0ff),
(byte)(lastBlockNum & 0x0ff),
});

What's meaning of range start=range end in http response

When I try to analyse CDN download, some logs looks like below:
GET http://1234.apk?track=mmmmmmm range:bytes-sent=[500-500], content-length:1500 ...
In my understanding, range:bytes-sent represents continue download after break and it should have different number in bytes-sent, the followings are reasonable:
bytes-sent=[500-600]
bytes-sent=[500-]
bytes-sent=[-500]
but what meaning of range start = range end like [500-500]? It seems no data should be downloaded but generates http response.
Thanks first~

The bytes mentioned in the range are also sent. Hence if the server wants to only send the 500th byte, the server would send [500-500] as the bytes range. Have a look at https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-p5-range-26 which has an example of how the first and last byte are sent :
o The first and last bytes only (bytes 0 and 9999):
bytes=0-0,-1

How can I determine the length of received bytes of UsbRequest.queue(..) method?

I have troubles with UsbRequest class in Android 3.1.
This is my code:
ByteBuffer buffer = ByteBuffer.allocate(4096);
buffer.order(ByteOrder.LITTLE_ENDIAN);
UsbRequest request = new UsbRequest();
request.initialize(mConnection, mEndpointIn);
request.queue(buffer, 4096);
if (mConnection.requestWait() == request) {
byte[] data = buffer.array();
}
The size of array data is 4096, but the length of really received bytes is much more smaller.
How can i determine the size of really received bytes?
Thanks.

This was a bug in Android. On afflicted versions, there's no workaround, because the implementation simply doesn't pass the length up.
It was fixed in JB-MR1 (API level 17 onwards).

It seems to me that the current asynchronous USB API has no way to return the read size. 2 "workarounds" use synchronous transfers as there you receive the number of bytes read/written or maybe the protocol you are implementing sends you the number of bytes you'll receive. E.g. i'm currently implementing something where every higher-level packet i receive has the number of bytes in the first 4 bytes of the packet. Based on this number i know if i have to do multiple reads.

You can use request.queue(buffer, bufferLength);. This should solve your problem. Now, you should refer android documentation, it's well documented and helpful.

I believe buffer.limit() should return the number of received bytes. Does that work?