I'm in the process of developing an application which reads data from a DAQ that streams its data over Bluetooth. The packet sizes can change, as can the sampling rate (1Hz - 512Hz), and I'm able to loop through and read the data off the device using a buffer.
My question is, how do I correctly process the data when there is such a variable of packet size and sampling rate? How do I determine the buffer size?
Currently I'm simply opening a socket, opening an input stream, and then using a while loop (while the socket is open == true) to read from the stream, and process the data (simple decoding, not an extraneous task).
As an example, there are 23 bytes in a packet, and I have the sampling rate very low at the moment at 1Hz. I have a buffer of 256 bytes, which means that it wont accommodate a full packet at the end of the buffer, and I've written code for it to run over onto the next buffer. Once the data is read grab one packet from the buffer, decode and store it, do the next one, etc.
How should I be streaming, and manipulating the data correctly? Eventually I'll be grabbing something in the region of 44 packets at 512Hz, right at Bluetooths transfer limit, and I want to be able to process it as effective as possible, and display errors when a packet is dropped in the process, etc.
TL;DR: how do I correctly stream data using buffers and/or interrupts.
Related
If I want to transfer a lot of data (e.g. 1 MB file) over BLE, what's the best way to do it?
I control both sides of the connection, but the client side is iOS/Android so only has access to GATT. I can't do anything with L2CAP.
I also can't wait for Bluetooth 4.1, 6LoWPAN, Connection-Oriented-Channels or anything like that.
I would assume the answer is to have one "request" characteristic that you write a data request to ("Give me 3000 bytes starting at byte 0"), and a "data out" characteristic that sends lots of 20 byte notifications (the maximum characteristic size) containing the data.
Is there a better way?
Yes we are using the approach you have mentioned.
Request data with the last index number(First time the index is 0)
The server send you with data with index no.Store the index no for subsequent format
continue Step 1 and 2 till the time server sends end of data-probably with index -1 or something.
Make sure you transfer the data you required in the most space efficient format.See if you can zip the files and transfer it.
You can update connection interval to small value with smallest 6*1.25 ms in remote BLE device.
Actually, BLE is designed for Low energy, small packet, low data rate.
L2cap data will be transmitted in different data channel with frequency hop. Packets TX/RX happen within each connection interval and max number of packets TX/RX in an event is restricted by specification, finally implemented by manufacture. So we can change connection interval as small as possible to increase data rate.
Refer BT 4.0 spec Vol 2, 7.8.18 LE Connection Update Command.
Try to negotiate a larger MTU than the default.
Then each notification can be larger. Even though it will be fragmented by the L2CAP layer, you will get a slightly larger throughput since the packet header will be smaller.
I just try to use the recieve() method of the socket.
When I send the data of a short String,as "send data",for 100
times,the recieve() function performs well.
While I send the data of a long String,as "send data to the
client,send data to the client,send data to the client,send data to
the client,send data to the client,send data to the client,send data
to the client,send data to the client",for 100 times,the recieve()
function cannot performs well.
In android project:
It just recieved about 30 packets, that is, other 70 packets are lost when I use another computer to send pakcets. But I checked the recieve buffer size, it is adequate to contain 100 packets above.
It recieves all the 100 packets when I use the localhost address to test. And I used the capturing tool named wireshark to capture the packets and the capturing tool can capture all the 100 packets everytime.
So I can eliminate the possibility of that another computer did not send all the packets.So I included that the problem exists in the emulator.
The above two cases both happened in android project,so the recieve buffer size is the same,
why in the former it will lose packets??
In java project:It recieves all the 100 packets too.
If you need all the packets then you should use TCP/IP protocol. In UDP protocol you can lose data because it is doesnot support reliable connection like Tcp/ip. UDP is designed to be faster at transferring data at the cost of unreliable connection. Based on network routing the packets you received could also arrive in different order.
Emulator TCP/IP stack's udp buffer size is smaller than the computer. So if the UDP buffer fills up, the emulator TCP/IP stack starts dropping packet. The PC TCP/IP stack pushes data at faster rate which the emulator can't handle. You can try reducing the rate at sender end to avoid filling the emulator buffer or increase the receive socket buffer size of android app to higher value by using the SO_RCVBUF socket option.
I'm working on an Android Bluetooth project to send and receive data from a custom made hardware. I used Bluetooth Chat Sample in SDK as basis.
I am sending data from one device to another (LG Nexus 4). All is ok until I reach a length of 1004 bytes (it is the audio data). At that point it splits it into 2 messages of 990 and 14 bytes in most of cases. but is strange sometimes its sending 1004 without splitting (approx. 4 times in 100).
I am sending this packet of 1004 bytes, in which there is 4 bytes is my header and rest of 1000 bytes is actual data which I want to use as per command in header, now if packets are splitting as per above mentioned way than I cannot handle the flow.
So, please let me know why packets are splitting in such way and how can I stop this splitting or, if I cannot do this, than please suggest me any alternative way to do this.
Thanks.
Data sent via Bluetooth socket is abstracted as a stream. Here the transport layer is broken into packets, where packet has a maximum size of almost 1KB(1000 bytes). So you can devise a mechanism in which you can send the message length info in the header, then on receiving side you will have to make subsequent read() calls; each returning data for one packet.
I'm using sockets for data transfer from one android phone and PC, I am using DataInputStream.
But it takes a long time in data transfer, about 10 minutes to transfer a 4 MB file.
Could anyone suggest any better way to do that?
I did some changes in my code and now it is taking 15 seconds to read about 1 Mb of data. I want to improve its performance.
My Code is:
InputStream is= socket.getInputStream();
DataInputStream inChannel= new DataInputStream(new BufferedInputStream(in));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int oneByte;
while ((oneByte = inChannel.read()) != -1) {
if (oneByte == 0) {
break;
}
baos.write(oneByte);
byteCount++;
}
byte[] inData = baos.toByteArray();
baos.close();
Are you sure you are not sending empty packets in between there or packets doing something else? If you are using TCP if a packet doesn't reach its destination (a buffer is overfilled with other packets somewhere in between your connection on the router or on one of the devices) the packet will have to be resent. This might be because of your setup. Given the little information you have given us, I can give the the following advice:
Look into more advanced typologies, I am going to assume you are not using any of these:
A send a receive thread; the send thread pushes out packets which you put into a queue:
|P| - packet held inside a Que part of your program
send out thread <-| P | P | P | <-App/program
reveive -> | P | P | P| P -> App/Program deques and analyses data
because your send and receive queuing system works along side each other they are constantly buffering up the received and sent packets. What seems to be happening with your setup (as far as I assume) is that you have a loop which grabs the latest packets and analyses them. This means that the hardware (part of your network hardware) buffer is overfilling with packets which your program is doing other things and by the time your program comes around to collect them, some of them are missing. This causes one side to go "I didn't get those packets, send them again", in other words, your resending the packets you should of sucked up into a Que which are waiting to be dequeued and analysed. The queue can grow regardless of how quickly the program is back to grab that data and do stuff with it (of course you are confined to the RAM). That way you ensure that you have possession of packets that you are supposed to receive rather than rely on your network card/receiver to hold on to them for you, possibly overflowing its buffer.
Another approach is to do a handshake system were one of the programs waits until the last packet is trasmitted, recieved and the other side goes "Cool, send me the next one". This slows down your download/upload speed but is a little bit quicker than packets falling off the end of the end of one of the buffers (each node, such as your router buffers your packets in case more come in than it can handle at a cycle) in the network.
You should utilize a state machine if you can on one or both ends. When your app is downloading the file, lock it into a receive state so its not trying to send/receive other stuff at the same time. Once the download is complete, switch to any other state (say, open file state). If you don't know much about state machines, I recommend you look at the wikipedia article:
http://en.wikipedia.org/wiki/Finite-state_machine
To read data from an opened USB connection one uses UsbDeviceConnection bulkTransfer method.
I have the situation that after sending a request the routine should read all incoming data, which can be just 8 bytes or even 300+ bytes.
The problem is that bulkTransfer does not read all the bytes for some reason - it just returns with some bytes while there could be more to come.
Is it better to write a loop around bulkTransfer until there is no more data, or increase the timeout, or possibly there is another approach.
What would be the best way to handle this?
I tried the loop approach and I am surprised that it only reads about 10-20 bytes each time with a timeout of even 1second. I am sure there is more data already available, don't know why it does not read more at a time.
Does it matter which buffersize one uses - many examples have 4096 but how does changing this influence the reading of data?
Many thanks
If the device sends 10 bytes at a time and does so repeatedly to finish the 300+ bytes, you still only get these 10-byte chunks each time you read with bulk transfer.
In other words, the method does not wait to fill up your buffer. It returns as soon as there is something. So if you really want to collect the entire 300+ byte response, you do need to loop.