currently trying to create the front-end for my Convolutional neural network image classifier.
what i'm trying to build is an app which sends a continuous stream of images to my server for classification, the total stream classification will be averaged and viewed for the user.
the reason i want this specific trait is because classifying many images of the same 3D object could improve accuracy.
building the app on camera2 demo from github, i wasn't able to google a solution.
i have tried creating a Asynctask which will rerun the takePhoto() method at onPostExecute but canceling the task is close to impossible, without talking about rerunning the task which doesnt allow me to cancel because i'm not saving the task as a variable.
also, each time i take a picture the device stutters a bit, is this unavoidable using camera2?
i have thought about moving to the older camera api, camera2 is a complete pain in the a**.
any new idea would be lovely at this point.
would apreciate if anyone who was able to stream images easily using camera2 or the older ap could post an example or help me with the issue.
this is my asynctask if this actualy could work and im just doing it wrong:
public class AsyncTakePicture extends AsyncTask<Void, Void, Void> {
private boolean running = true;
#Override
protected void onCancelled() {
running = false;
}
#Override
protected Void doInBackground(Void... params) {
while (running) {
takePicture();
try {
// no need for alot of images, so one image per 2 seconds is good
Thread.sleep(2000);
} catch (Exception e) {
running = false;
}
}
return null;
}
protected void onPostExecute(Void result) {
if (running){
myAsyncTask.execute();
} else {
Log.i("AsyncTask:"," done");
}
}
}
the idea is that once the task is called, it will rerun and take more photos until cancelled. Async is called because i don't want the UI to lag. thing is, to repeat the task i cannot initiate it as a variable, and if i cannot initiate then i cannot use cancel(boolean).
I need to perform Dynamic Time Wrapping(DTW) algorithm with some previously stored data and Accelerometer data. But I am unable to handle the huge amount of data coming from accelerometer. I need to determine a step. In order to do that I have stored a number of data previously and trying to match with the current data.
ArrayList<Double> test=new ArrayList<Double>();
public void onSensorChanged(SensorEvent event) {
double x=event.value[0];
double y=event.value[1];
double z=event.value[2];
double a=Math.sqrt(x*x+y*y+z*z);
test.add(a);
new ProgressRunner().executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR,x);
private class ProgressRunner extends AsyncTask<Double, Integer, Boolean>
{
#Override
protected Boolean doInBackground(Double... params) {
new Thread(new Runnable() {
#Override
public void run() {
double r=DTWDistance(StepSamples.sample1, standTest);
if(r<700) /* "Step Detected */
}
}).start();
return null;
}
But I am getting huge data coming from the sensor. My question is how do I handle this data and match with a sample data continuously. Also I have tested with a Log.d, I'm sure the problem is related to how I handle this Live data but I can't find a way out. Any suggestion would be helpful.
I have also tried this by doing it an interval of 2 sec but an OutOfMemeoryBoundsException throws
if((System.currentTimeMillis()-sTime)/1000>2){
new ProgressRunner().executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR,x);
}
One of the solution to address this problem is keeping a threshold for the change in the sensor values. If the change in deltaX, deltaY, deltaZ are greater then you can start the AsyncTask else dont.
Since u are not keeping any threshold values the number of threads u create is very huge in number.
EDIT 1:
Here is a link to a small tutorial that will help you to kick start... in this link he has used a time interval and a threshold for shaking the device... i bet this is a perfect solution for your requirement
Tutorial Link
Do you need to process every single value or are you fine processing every x (e.g. 0.1) seconds? I call a method that checks "now - lastProcessedTime". If that value is > 100ms, I will handle the data and update lastProcessedTime = now. If not, I will throw away the sensor value.
I am trying to calibrate an accelerometer, but I can't obtain the 6 sample values at 6 different acceleration readings required for the calibration. PreliminaryW is a double[6][3] array made to fill those sample values. It is 6 by 3 because each acceleration reading has an x, y, and a z component.
I am planning to sample them by pressing a button at the 6 different acceleration readings. This button makes "calibrate" true.
I ofcourse, first make "calibrating" true to start this thread.
For some unfathomable reason, preliminaryW[i] = currentAcc seems to be filling up from 0 to i with the same value instead of just i. I made sure that the currentAcc is different every time I press the "calibrate" button.
What is wrong with my code?
public synchronized void run() {
Log.d(TAG, "+ in Calibrator thread +");
int i = -1;
while (calibrating) {
if (calibrate) {
i = i + 1;
calibrate = false;
preliminaryW[i] = currentAcc;
if (i == 5) {
calibrating = false;
}
}
}
}
I'm not very familiar with the inner-workings of the accelerometer, and it's hard to decide why it's not working without seeing more of the code. For example, can you be sure that there's only one instance of the Thread, or are you creating multiple instances?
Why does this need to be in a Thread?
Looping like that is not good practice either, you should use wait/notify if you absolutely need a Thread. (more info at http://docs.oracle.com/javase/tutorial/essential/concurrency/guardmeth.html)
I've drafted a class that does approximately what you want, but doesn't use threading. You could create a Calibrator and then call performCalibration() with each new value:
class Calibrator{
int count = 0;
double[][] preliminaryW = new double[6][3];
public void performCalibration(double[] currentAcc){
preliminaryW[count] = currentAcc;
count++;
}
}
without the Thread and "busy loop", you might be able to omit those flags for 'calibrate' and 'calibrating' which would certainly help debugging as well.
Good Luck!
I think this is not an easy question.
I'll be brief and give a little example of what is happening.
Let's say we have a source of data in file Byron.txt:
SHE walks in beauty, like the night
Of cloudless climes and starry skies;
And all that 's best of dark and bright
Meet in her aspect and her eyes:
Thus mellow'd to that tender light
Which heaven to gaudy day denies.
And this code execute inside an AsyncTask:
final ArrayList<Record> poem = new ArrayList<Record>();
final Object objectLock = new Object();
private Record rec = new Record();
#Override
protected Void doInBackground(Void... args) {
String line = null;
int i;
int last;
try {
process = Runtime.getRuntime().exec("cat Byron.txt");
bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()), 8192);
synchronized (objectLock) {
poem.clear();
last = i = poem.size() - 1;
}
while(line = bufferedReader.readLine()) != null) {
rec.setString(line);
synchronized (objectLock) {
last++;
poem.add(last, rec);
}
while(!bPause && i < last) {
i++;
publishProgress(poem.get(i));
}
}
}
catch (Exception e) {
e.printStackTrace();
}
return null;
}
#Override
protected synchronized void onProgressUpdate(Record... m) {
if(m.length > 0) {
mContext.mTable.appendRow(m[0]);
}
}
where there is a TableLayout in the UI and each time we get a new line we add a new TableRow to it.
And this is the output we see in the UI:
SHE walks in beauty, like the night
Of cloudless climes and starry skies;
Of cloudless climes and starry skies;
Of cloudless climes and starry skies;
Thus mellow'd to that tender light
Thus mellow'd to that tender light
And we go into the debugger and we see why it happens.
Sometimes the synchronized (objectLock) is skipped and the loop continues.
There is no publishing because i already catched last.
Later the block is executed as many times as it was skipped,
, but the original line is lost and the current line is added instead to poem several times
Then, all the new lines are published until i catches last again.
So you see that I followed the code and I can explain what's happening, the question here is: Why the block is skipped?, Why?
I expected the synchronized block to stall until it can be executed.
At least this how I understood the function of synchronized (objectLock)
even without using wait() and notify()
I don't pretend to open a discussion here (although if you want we can open one in the chat area)
If you see some fault in the code, then, answer the question to let me know.
NOTES:
synchronized is needed because somewhere else in the app, the user may want to email the lines he got so far.
The user may pause the publishing (bpause); that's the while loop and i follows last only when bPause is false.
I decided to publish the answer. Even though I feel very embarrassed by its simplicity.
I discovered it only after I had already dug deep into AsyncTask class and message handling and whatnot.
I publish it in hope it will help people to check the basic things before jumping to
conclusions, and that someone out there will save himself half a day debugging because of
this post.
The Record rec was the same one each time. The poem ArrayList had the same element id for each entry. And the content changed on all of them at once, since they were all the same.
When the progress was published immediately it printed the right string, the last one. But if some delay cause the progress to publish later, then retrieving the poem.get(i) records retrieved a different entry but with the same pointer, thus, the same content.
The solution was to create a new Record each loop.
Do the synchronization for last object.
synchronized (last) {
last++;
poem.add(last, line);
}
I've been asking a series of evolving questions regarding my Android project that continually plots Bluetooth data in real-time. And I haven't done a very good job of asking questions.
So what I need to do is edit this question, clean it up, add important detail, and most importantly I need to add code fragments of relevant sections of code, especially sections I've hacked on quite a bit, and provide explanation about these sections of code. That way maybe I might get an answer to my question/concerns which are: Is my current solution an OK one? Is it going to hold up as I add new features?
Basically what I've already done is create a first version of my app by cobbling together some open source code Blueterm and OrientationSensor.
It's been suggested that I add a thread, a handler, a Service, or use Async Task, or AIDL, etc. But I have decided I don't want to modify or replace my existing solution unless I really should. Mainly I want to know if it's good enough to move forward and extend it to add other features.
By the way what I have previously referred to as BluetoothData is just bluetooth data: it's 16 bit data received from a remote Bluetooth device at the rate of 2 to 10 samples/second. My app is basically a data acquisition system that acquires/receives bluetooth data and plots it.
Here's a description of the Blueterm open source code I started with (see link above). Blueterm is basically a terminal emulator program that communicates over Bluetooth. It consists of several activities with Blueterm being the most important. It discovers, pairs, and connects with a remote Bluetooth device that supports SPP/RfComm. When connected I can use Blueterm to configure the remote device by sending it commands to turn on sampling, change the number of channels to sample (to one channel), change to format of the incoming data (I like comma separated data), etc
Here's a description of the OrientationSensorExample open source code I started with (see link above). It's basically an example application of the AnroidPlot library. The OrientationSensor activity implements SensorEventListener. This includes overriding onSenorChanged() which is called whenever new orientation sensor data is taken, and it redraws the graph.
Having cobbled together these two open source projects (Blueterm and OrientationSensorExample) into one application (Blueterm) here's a description of how the overall application (Blueterm) works. When I start Blueterm the whole screen emulates a nice blue terminal. From the Options Menu I discover, pair with, connect to, and configure a remote bluetooth device as described above. Once I have configured the remote device, I go again to the Options Menu and select "Plot data" which launches the Plot activity. The terminal emulator goes away, and a nice scrolling real-time plot from the Plot activity shows up.
Here's how I did this. In onOptionsItemSelected() I added a case to launch the Plot activity as follows:
#Override
public boolean onOptionsItemSelected(MenuItem item) {
switch (item.getItemId()) {
case R.id.connect:
if (getConnectionState() == BluetoothSerialService.STATE_NONE) {
// Launch the DeviceListActivity to see devices and do scan
Intent serverIntent = new Intent(this, DeviceListActivity.class);
startActivityForResult(serverIntent, REQUEST_CONNECT_DEVICE);
}
else
if (getConnectionState() == BluetoothSerialService.STATE_CONNECTED) {
mSerialService.stop();
mSerialService.start();
}
return true;
case R.id.preferences:
doPreferences();
return true;
case R.id.menu_special_keys:
doDocumentKeys();
return true;
case R.id.plot_data:
doPlotData();
return true;
}
return false;
}
private void doPlotData() {
Intent plot_data = new Intent(this, com.vtrandal.bluesentry.Plot.class);
startActivity(plot_data);
}
Then in the bluetooth background thread I added a call to update() to call plotData() as follows:
/**
* Look for new input from the ptty, send it to the terminal emulator.
*/
private void update() {
int bytesAvailable = mByteQueue.getBytesAvailable();
int bytesToRead = Math.min(bytesAvailable, mReceiveBuffer.length);
try {
int bytesRead = mByteQueue.read(mReceiveBuffer, 0, bytesToRead);
append(mReceiveBuffer, 0, bytesRead);
//VTR use existing handler that calls update() to get data into plotting activity
//OrientationSensor orientationSensor = new OrientationSensor();
Plot.plotData(mReceiveBuffer, 0, bytesRead);
} catch (InterruptedException e) {
//VTR OMG their swallowing this exception
}
}
Then in the Plot activity I basically cleaned house, removed "implements SensorEventListener" and some related methods and variables, and wrote plotData() to be called as shown above. Here's what plotData() and it's helper methods splitData() and nowPlotData() currently look like:
private static StringBuffer strData = new StringBuffer("");
public static void plotData(byte[] buffer, int base, int length) {
Log.i("Entering: ", "plotData()");
/*
byte[] buffer = (byte[]) msg.obj;
int base = msg.arg1;
int length = msg.arg2;
*/
for (int i = 0; i < length; i++) {
byte b = buffer[base + i];
try {
if (true) {
char printableB = (char) b;
if (b < 32 || b > 126) {
printableB = ' ';
}
Log.w("Log_plotData", "'" + Character.toString(printableB)
+ "' (" + Integer.toString(b) + ")");
strData.append(Character.toString(printableB));
if (b == 10)
{
Log.i("End of line: ", "processBlueData()");
Log.i("strData", strData.toString());
splitData(strData);
strData = new StringBuffer("");
}
}
} catch (Exception e) {
Log.e("Log_plotData_exception", "Exception while processing character "
+ Integer.toString(i) + " code "
+ Integer.toString(b), e);
}
}
Log.i("Leaving: ", "plotData()");
}
private static void splitData(StringBuffer strBuf) {
String strDash = strBuf.toString().trim();
String[] strDashSplit = strDash.split("-");
for (int ndx = 0; ndx < strDashSplit.length; ndx++)
{
if (strDashSplit[ndx].length() > 0)
Log.i("strDashSplit", ndx + ":" + strDashSplit[ndx]);
String strComma = strDashSplit[ndx].trim();
String[] strCommaSplit = strComma.split(",");
for (int mdx = 0; mdx < strCommaSplit.length; mdx++)
{
if (strCommaSplit[mdx].length() > 0)
Log.i("strCommaSplit", mdx + ":" + strCommaSplit[mdx]);
if (mdx == 1)
{
int raw = Integer.parseInt(strCommaSplit[1],16);
Log.i("raw", Integer.toString(raw));
float rawFloat = raw;
Log.i("rawFloat", Float.toString(rawFloat));
float ratio = (float) (rawFloat/65535.0);
Log.i("ratio", Float.toString(ratio));
float voltage = (float) (5.0*ratio);
Log.i("voltage", Float.toString(voltage));
nowPlotData(voltage);
}
}
}
}
public static void nowPlotData(float data) {
// get rid the oldest sample in history:
if (plotHistory.size() > HISTORY_SIZE) {
plotHistory.removeFirst();
}
// add the latest history sample:
plotHistory.addLast(data);
// update the plot with the updated history Lists:
plotHistorySeries.setModel(plotHistory, SimpleXYSeries.ArrayFormat.Y_VALS_ONLY);
//VTR null pointer exception?
if (plotHistoryPlot == null)
Log.i("aprHistoryPlot", "null pointer exception");
// redraw the Plots:
plotHistoryPlot.redraw();
}
Time for a summary: I basically found the update() method in the background thread that was created by the Blueterm activity. The update() method essentially appends newly received bluetooth data to the screen buffer using the append() method. So, the background thread's update() method looked like a good place to call plotPlot(). So I designed plotData() to plot the data being passed to append(). This works as long plotData() is a static method. I would appreciate an explanation as to why plotData() seemingly must be static in order to work.
And again my overall question/concerns: Is my current solution an OK one? Is it going to hold up as I add new features?
I found the method in the background thread that writes BluetoothData to the Logcat. So I am leveraging this method to call a static method, plotData(BluetoothData), in the Plotting Activity. It works nicely plotting the incoming BluetoothData in real-time.
This story does not add up, or BluetoothData is misnamed.
In Android, to plot to the screen, you need an Activity instance, with whatever widget(s) you are plotting on. A plotData() method that does the plotting can be static, but somehow it needs the Activity instance. So, one of the following must be true:
BluetoothData contains an Activity instance (and hence is misnamed), or
plotData() takes more than the one parameter you have indicated, or
you are holding onto an Activity instance in a static data member (BAD BAD BAD BAD BAD), or
plotData() is not a static method, so you are actually calling it on an Activity instance
But, since you repeatedly decline to provide the source code, despite having asked several questions about the code, it is impossible for us to say which of these is really your problem.
Is it not thread safe? Do I have a deadlock issue? Is my solution a fragile one? Am I circumventing the Android OS? Am I lucky that it's working at all? Or is there a proper way to extend the existing design?
The first five of these have different answers depending upon which of the four bullets above reflects reality, and I really do not feel like writing out a 20-cell grid of answers. Your last question assumes that you have actually explained your "design", which you have not.
UPDATE
Some comments based upon the substantial revision to the question:
I would appreciate an explanation as to why plotData() seemingly must be static in order to work.
It probably doesn't have to be static.
Is my current solution an OK one?
Probably not.
Static methods are rarely a problem on their own. Static data is frequently a problem, due to lack of thread safety, memory leaks, and the like. Hence, your objective is to minimize or eliminate all static data members.
There are at least four static data members at play here, perhaps more. One is strData. This isn't too bad, in that you reset the static data member to a fresh StringBuffer on each plotData() call, so your memory leak is modest. However, if plotData() somehow were to be called on multiple threads simultaneously -- and, since I don't know your threading model, that's at least possible -- you will have problems, since you have no synchronization.
However, the far bigger problem is represented by the plotHistory, plotHistorySeries, and plotHistoryPlot static data members are. I have no idea what these objects are. However, by their name and your overall objective, it would appear that redraw() actually draws to the screen, which means that plotHistoryPlot is or holds some subclass of View that is the thing being plotted upon. This means you violated a cardinal rule of Android development:
Never put something that references a transient Context in a static data member
Here, an Activity represents a transient Context, "transient" because activities do go away, Context because it inherits from Context. Your statically-referenced View holds a reference back to its Activity. Hence, this Activity can never be garbage collected, which is bad for business, let alone any possible thread-safety issues.
Again, this is an educated guess, since I don't know what those static data members really are.