I am writing an android app that recieves data over bluetooth. The bytes comming in can be of any size example: 00023>024935928598235>9284>
As you can see each set is seperated by ">". The data comes in extremely fast. I would like some ideas for an implementation. See my problem is that I need to read the data into a byte array that can and then convert it to a string and split them according to the delimeter of ">".
so in the above example:
00023
024935928598235
9284
If i set byte[] data = new byte[8] then when reading the incomming data it might get 00023>02 which is not what i want. I'm not sure how to implement something like this. Any ideas?
Here's one approach. You'll have to implement the readDataFromBluetooth() and somehow set dataAvailable, but this should get you on the right track.
byte[] data = new byte[1024];
List<String> chunks = new LinkedList<String>();
StringBuilder chunk = new StringBuilder();
while (dataAvailable) {
data = readDataFromBluetooth();
for (byte b : data) {
if (b == '<') {
chunks.add(chunk.toString());
chunk.setLength(0);
} else {
chunk.append(b);
}
}
}
if (chunk.length() > 0)
chunks.add(chunk.toString());
I would recommend using a buffered stream, but maybe a bit bigger that 8 bytes, as you suggest, and the read one and one character from the beginning of the stream, accumulating the string. When you encounter a ">", send the value you have accumulated off to a queue for a background thread processing. Use standard producer/consumer implementation techniques (e.g. the Monitor pattern) to communicate via the queue.
Related
I'm working with android sensors and have a method inside a listener that keeps appending data on a string builder with really high frequency. After some data is collected I compress the string with gzip and write it on a file to avoid out of memory exceptions. This keeps repeating forever. This is all in the same thread so as the file gets bigger it starts to block the thread and the data appending on the string. I do create new files if they get too large but i think i need to implement a threading and lock mechanism for the compression and file writing to avoid any blocking but at the same time not have any problems with leakage of data. Can anyone help me with that? Im not sure if im wording my question correctly.
// on rotation method of gyroscope
#Override
public void onRotation(long timestamp,float rx, float ry, float rz) {
try {
//get string of new lines of the write data for the sensor
str.append("gyroTest,userTag=testUser,deviceTag="+deviceName+" rx="+rx+",ry="+ry+",rz="+rz+" "+timestamp+"\n");
if(count >=2000){
b = GZIPCompression.compress(str);
Log.i(FILE_TAG, "Write gyroscope file");
FileHandling.testWrite( GYROSCOPE,b);
str.setLength(0);
count=0;
}
count++;
} catch (Exception e) {
e.printStackTrace();
}
}
You're on the right track in that you need to separate reading from the sensor, processing the data, and writing it all back to disk.
To pass the data from the sensor reads, you may consider using something like a LinkedBlockingQueue with your Strings.
private LinkedBlockingQueue<String> queue = new LinkedBlockingQueue<String>();
#Override
public void onRotation(long timestamp, float rx, float ry, float rz) {
queue.add(
"gyroTest,userTag=testUser,deviceTag="+deviceName+" rx="+rx+",ry="+ry+",rz="+rz+" "+timestamp+"\n"
);
}
And then in another Thread, looping until canceled, you could drain the queue, process, and write without blocking the reading (main) Thread.
private boolean canceled = false;
private void startProcessingQueue() {
Runnable processQueueRunnable = new Runnable() {
#Override
public void run() {
while (!canceled) {
drainQueueAndWriteLog();
Thread.sleep(250);
}
}
};
new Thread(processQueueRunnable)
.start();
}
private void drainQueueAndWriteLog() {
List<String> dequeuedRotations = new ArrayList<String>();
queue.drainTo(dequeuedRotations);
if (0 < dequeuedRotations.size()) {
// Write each line, or all lines together
}
}
Note: take care to ensure the runnable is canceled when your Activity is paused.
As mentioned in your question, the more data you're writing, the slower it's going to be. Since you're writing data from a sensor, it's inevitably going to grow. For this, you could partition your files into smaller segments, by using something like a date-based naming convention for your log files.
For instance, a log name pattern of yyyyMMddHHmm would create minute-spaced log files, which you could then later aggregate and sort.
private SimpleDateFormat logFileDateFormat = new SimpleDateFormat("yyyyMMddHHmm");
private String getCurrentLogFileName() {
return String.format(
"rotations-%s.log",
logFileDateFormat.format(new Date())
);
}
Just keep in mind that since you're not writing in the same thread you're reading from, your timestamps may not match up perfectly with your log file names. This shouldn't be a problem, though, as you're already including the timestamps in the persisted data.
Further down the line, if you're still finding you're not quite hitting the level of write-throughput that your project requires, you may also want to consider condensing the amount of information you're actually storing by encoding common byte usages, or even reducing the length of each key to their most-unique values. For example, consider this 1 line output:
"gyroTest,userTag=testUser,deviceTag=some-device-name rx=12345,ry=4567,rz=87901872166251542545144\n"
And now reducing the keys:
"gyroTest,u=testUser,d=some-device-name x=12345,y=4567,z=87901872166251542545144\n"
Removes 18 characters from every line that needs to be written, without sacrificing any information.
Also worth noting: you either need a space (or better a comma) before the timestamp in your data line, else you won't be able to nicely pick out rz from it. And your deviceName should be escaped with quotation marks if it can contain spaces, else it will conflict with pulling out rx.
I'm trying to develop a small app which receives some data via sockets and based on the data it receives,it prints a toast message for the user.I' getting the data,but the data apparently cannot be read properly.Here is the relavent portion for the same.
int red = -1;
byte[] buffer = new byte[1024]; // a read buffer of 5KiB
byte[] redData;
while ((red = cs.getInputStream().read(buffer)) > -1) {
String redDataTextappend;
redData = new byte[red];
redDataTextappend = new String(redData);
Log.w("Data got",redDataTextappend);
if (redDataTextappend == "hi")
{
//Display Toast message using runonUIThread(new Runnable);
}
else
{//Display Message Using runonUITHread(new Runnable);
}
This code runs on a separate thread as android does not allow networking operations on a separate thread.
The 4 Diamonds is the data displayed by the android studio and cs is the name of the socket which accepts connections.
Thanks.
You are simply printing the String that encodes to zero bytes since you never copy the data that is read in.
It would make more sense to convert the byte array to a hex string if the array contains arbitrary data: see the answers to this question for options for that.
If the array contains the encoding of a String in some charset, for example UTF-8, then do the following:
byte[] redData = Arrays.copyOf(buffer, red);
String redDataTextappend = new String(redData, Charset.forName("UTF-8"));
Log.w("Data got",redDataTextappend);
I want to append a line of text to an existing Azure cloud block blob from an Android device.
In VB.Net I would AcquireLease, getBlockBlobReference, DownloadToFile, add the line on the local files system, UploadToFile, ReleaseLease . Simple and secure, if a bit long-winded.
In Android, it looks a little more tricky. At the moment, my best solution is this:
CloudBlockBlob blob1=container.getBlockBlobReference(chosenOne+".txt");
String proposedLeaseId1 = UUID.randomUUID().toString();
OperationContext operationContext1 = new OperationContext();
blob1.acquireLease(15, proposedLeaseId1, null /*access condition*/,null/* BlobRequestOptions */, operationContext1);
AccessCondition condition = new AccessCondition();
condition.setLeaseID(proposedLeaseId1);
BlobInputStream blobIn = blob1.openInputStream();
blob1.downloadAttributes();
long blobLengthToUse = blob1.getProperties().getLength();
byte[] result = new byte[(int) blobLengthToUse];
blob1.downloadToByteArray(result,0);
blobIn.close();
CloudBlockBlob blob1 = container.getBlockBlobReference(chosenOne+".txt");
String proposedLeaseId1 = UUID.randomUUID().toString();
OperationContext operationContext1 = new OperationContext();
blob1.acquireLease(15, proposedLeaseId1, null /*access condition*/,null/* BlobRequestOptions */, operationContext1);
AccessCondition condition = new AccessCondition();
condition.setLeaseID(proposedLeaseId1);
BlobInputStream blobIn = blob1.openInputStream();
blob1.downloadAttributes();
long blobLengthToUse = blob1.getProperties().getLength();
byte[] result = new byte[(int) blobLengthToUse];
blob1.downloadToByteArray(result,0);
blobIn.close();
blob1.deleteIfExists(DeleteSnapshotsOption.NONE,condition, null, operationContext1);
BlobOutputStream blobOut = blob1.openOutputStream();
//this is a byte by byte write ...
//which is fine ... but no use if you want to replace ...
/*int next = blobIn.read();
while (next != -1) {
blobOut.write(next);
next = blobIn.read();
}*/
blobOut.write(result);
String strTemp="This is just a test string";
blobOut.write(strTemp.getBytes());
blobOut.close();
Apart from being extremely long-winded, I am concerned that as soon as I delete the blob, the lease will go and that I may hit integrity issues. I would appreciate any help in making this code simpler and more secure. I know that Microsoft are planning to introduce append blobs in 3Q 2015, but I want to implement this now.
You can call PutBlock to upload the appended content (the maximum size of each block is 4MB, so please split the appended content into blocks if required), and then call PutBlockList on this blob by passing in the previously committed blocks plus and newly appended blocks.
I have developped 2 Android applications. The first one, to write into an NFC tag, and the second to read the contents I have written .
This is what i did in the first application (WriteNFC)
private NdefRecord createRecord1(String data)
{
byte[] payload = data.getBytes(Charset.forName("UTF-8"));
byte[] empty = new byte[] {};
return new NdefRecord(NdefRecord.TNF_ABSOLUTE_URI, empty, empty, payload);
}
private NdefRecord createRecord2(String data)
{
byte[] payload = data.getBytes(Charset.forName("UTF-8"));
byte[] empty = new byte[] {};
return new NdefRecord(NdefRecord.TNF_ABSOLUTE_URI, payload, empty, empty);
}
And in the second application (ReadNFC)
NdefRecord cardRecord = msg.getRecords()[1];//Extract the second Record
String url_data = new String(cardRecord.getType());//Read data type
When I read with my own application (ReadNFC), of course I had on screen only the payload of the second Record, which I stored through "Record Type". But with a third-party application, especially that natively installed ("tag") -shown in photo-, It display correctly the first record, and for the second it's an empty field. How can I hide this field. Otherwise, how can I force the other third-party applications to not read the second record?
You simply cannot do that. Android will read the complete NDEF messages (i.e. all records) and pass it on in the Intent to an app.
Upps, it is no wonder this is happening, look at your code. First
return new NdefRecord(NdefRecord.TNF_ABSOLUTE_URI, empty, empty, payload);
then
return new NdefRecord(NdefRecord.TNF_ABSOLUTE_URI, payload, empty, empty);
So the 3rd party apps are really showing the correct data, you have a bug in your record creation.
The native NDEF support in android is somewhat crude (byte-array based), so I've written a library which helps in creating records - which you might find interesting. The above issue might be simple to resolve, but there are many other much more complex record types, so some help might come in handy ;-)
Edit: So if this is the preferred result, rather create an Unknown Record and put your 'secret' data as the payload - the will not be any good way for any 3rd party app to display that data - whereas the ID of the absolute URI Record certainly can be displayed by any NDEF-reading app (like mine?)
This third party app was bothersome to me, so I had to use foregroundDispatch to read the tag contents manually, there you have the freedom to read or not read anything you want.
This snippet is from the OnResume(). `
mNfcAdapter.enableForegroundDispatch(this, pendingIntent,
intentFiltersArray, techListsArray);
Toast.makeText(getApplicationContext(), "TAG disscovered",
Toast.LENGTH_LONG).show();
Parcelable[] rawMsgs = getIntent()
.getParcelableArrayExtra(NfcAdapter.EXTRA_NDEF_MESSAGES);
if (rawMsgs != null) {
NdefMessage[] msgs = new NdefMessage[rawMsgs.length];
for (int i = 0; i < rawMsgs.length; i++) {
msgs[i] = (NdefMessage) rawMsgs[i];
setText=new String(msgs[i].getRecords()[0].getPayload());
}
mInfoText.setText(setText);
}
}`
Here I get payload of the 1st record.
We usually get data from server response in android development.
/*
* get server response inputStream
*/
InputStream responseInputStream;
Solution1: get response string by multiple read.
/*
* get server response string
*/
StringBuffer responseString = new StringBuffer();
responseInputStream = new InputStreamReader(conn.getInputStream(),"UTF-8");
char[] charBuffer = new char[bufferSize];
int _postion = 0;
while ((_postion=responseInputStream.read(charBuffer)) > -1) {
responseString.append(charBuffer,0,_postion);
}
responseInputStream.close();
Solution2: get response only one read.
String responseString = null;
int content_length=1024;
// we can get content length from response header, here assign 1024 for simple.
responseInputStream = new InputStreamReader(conn.getInputStream(),"UTF-8");
char[] charBuffer = new char[content_length];
int _postion = 0;
int position = responseInputStream.read(charBuffer)
if(position>-1){
responseString = new String(charBuffer,0,position );
}
responseInputStream.close();
Which solution has better performance? why?
Notes: server response json format data that less than 1M bytes.
Why you're reinventing a wheel? ;)
If you're using HttpClient then just use EntityUtils.toString(...).
I guess you're using HttpURLConnection. Then look at EntityUtils.toString(...) from Apache HttpClient - source code. Your first approach is similar to it.
BTW, the second code is worse because:
new String(charBuffer,0,position ) runs garbage collector
In both and even in EntityUtils:
int content_length = 1024; in most cases 8192 is default for socket buffer, so your code might run while loop 8 times more often than it could.
I would recommend the second method IF you do not want to display the amount of data downloaded/transferred . As the object is read as a whole and since the size of your JSON string is comparable to 1M, it will take some time to download. At that time you can, atmost, put up a text for the user saying downloading... You cannot notify the user the amount downloaded.
But if you want to display the amount of data downloaded, use the first method that you gave. Where the you read the data from the server in parts. You can update the UI, with the amount downloaded. For eg 25 % downloaded...
char[] charBuffer = new char[bufferSize];
int _postion = 0;
int i=0;
while ((_postion=responseInputStream.read(charBuffer)) > -1) {
//((i*buffer_size)/content_length) * 100 % completed..
i++;
}
So, I would say the seconds method is better.
BTW Did you consider this?
ObjectInputStream in = new InputStreamReader(conn.getInputStream(),"UTF-8");
if(resposeCode==200)
{
String from_server=(String) in.readObject();
}
Reading the input String as an object. Any object whoe class implements serializable can be passed using ObjectOutputStream and received using ObjectInputStream()
I Think First one is good
Because, in First that will reading your response in char to char method .
Where , Second that will try to read whole response object or as key Filed of Object.
So ,As i think and as per my knowledge First is Better to camper with second.If anyone want to edit then it will truly appreciated.