We want to implement some kind of indoor position determination using iBeacons.
This article seems really interesting, in which the author implemented the Non-linear Least Squares Triangulation, using the Eigen C++ library and the Levenberg Marquardt algorithm. Since Eigen is written in C++, I tried to use JNI and Android NDK in order to use it but it throws a lot of errors that I have no idea how to solve and I couldn´t find anything online. I also tried to use Jeigen, but it does not have all the functions that we need.
So, my questions are:
Has someone ever implemented some kind of Trilateration using
beacons in Android?
Do you think that using Eigen+JNI+NDK is a good solution? If yes,
have you ever implemented Levenberg Marquardt using that
combination?
Are there better options than the Levenberg Marquardt algorithm for calculating the trilateration in a Android application?
Take a look at this library: https://github.com/lemmingapex/Trilateration
uses Levenberg-Marquardt algorithm from Apache Commons Math.
For example..
into the TrilaterationTest.java
you can see:
double[][] positions = new double[][] { { 1.0, 1.0 }, { 2.0, 1.0 } };
double[] distances = new double[] { 0.0, 1.0 };
TrilaterationFunction trilaterationFunction = new TrilaterationFunction(positions, distances);
NonLinearLeastSquaresSolver solver = new NonLinearLeastSquaresSolver(trilaterationFunction, new LevenbergMarquardtOptimizer());
double[] expectedPosition = new double[] { 1.0, 1.0 };
Optimum optimum = solver.solve();
testResults(expectedPosition, 0.0001, optimum);
but if you see the objectivec example https://github.com/RGADigital/indoor_navigation_iBeacons/blob/show-work/ios/Group5iBeacons/Debug/Managers/Location/NonLinear/NonLinear.mm you can note that the accuracy is used as an assessment parameter and not the distance.
Related
it's the first time for me that I ask help here. I will try to be as precise as possible in my question.
I am trying to develop a shape detection app for Android.
I first identified the algorithm which works for my case playing with Python. Basically for each frame I do this:
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, lower_color, upper_color)
contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
#here I filter my results
by this algorithm I am able to run the analysis realtime on videos having a frame rate of 120fps.
So I tryied to implement the same algorithm on Android Studio, doing the following for each Frame:
Imgproc.cvtColor(frameInput, tempFrame, Imgproc.COLOR_BGR2HSV);
Core.inRange(tempFrame,lowColorRoi,highColorRoi,tempFrame);
List<MatOfPoint> contours1 = new ArrayList<MatOfPoint>();
Imgproc.findContours(tempFrame /*.clone()*/, contours1, new Mat(), Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
for(MatOfPoint c : contours1){
//here I filter my results
}
and I see that only the findContour function takes 5-600ms to be performed at each iteration (I noticed that it takes also more using tempFrame.clone()), allowing more or less to run the analysis with only 2fps.
This speed is not acceptable at all of course. Do you have any suggestion about how to improve this speed? 30-40fps would be already a good target for me.
I will really appreciate any help from you all. Many thanks in advance.
I would suggest trying to do your shape analysis on a lower resolution version of the image, if that is acceptable. I often see directly proportional timing with number of pixels of the image and the number of channels of the image - so if you can halve the width and height it could be a 4 times performance improvement. If that works, likely the first thing to do is a resize, then all subsequent calls have a smaller burden.
Next, be careful using OpenCV in Java/Kotlin because there is a definite cost to marshalling over the JNI interface. You could write the majority of your code in native C++, and then make just a single call across JNI to a C++ function that handles all of the shape analysis at once.
hi I'm making a app which detects face landmarks ( 68 point )
I'm in trouble optimizing system. I'm using HOG method to detect faces.
In, detector(cv_grayscale, face_detections, -0.2); type "dlib::frontal_face_detector& detector"
There are so many computations in there. So, android cpu cannot cover them.
So, anybody who solved this problem or relevant issues ?
bool DetectFacesHOG(vector<cv::Rect_<double> >& o_regions, const cv::Mat_<uchar>& intensity, dlib::frontal_face_detector& detector, std::vector<double>& o_confidences)
{
double scaling = 1.3;
cv::Mat_<uchar> upsampled_intensity;
cv::resize(intensity, upsampled_intensity, cv::Size((int)(intensity.cols*scaling), (int)(intensity.rows*scaling)));
dlib::cv_image<uchar> cv_grayscale(upsampled_intensity);
std::vector<dlib::full_detection> face_detections;
// millions of computation !!!!!!!!!!!!!!!!!!!!!!!!
detector(cv_grayscale, face_detections, -0.2);
....
}
Download latest opencv android SDK from here.
it contains a lot of debugged samples. One of them is face detection and it detects faces with 22 frames per second speed on my Xperia-Z5 Phone. Finally, if opencv errors cause of rotation of camera, use this code. The code is very Clear and finds best frame resolution for your Camera View. İf you also want face recognition you can download C++ modules but you must use NDK(c++). Because Android SDK won't have face.h or other modules. You can combine detecting a face from java and recognize them from c++. Don't worry about speed opencv optimizes that. Face detecting lpcascade classificer xmls works high performance. But if you want more detect use haarcascade.
After some weeks of waiting I finally have my Project Tango. My idea is to create an app that generates a point cloud of my room and exports this to .xyz data. I'll then use the .xyz file to show the point cloud in a browser! I started off by compiling and adjusting the point cloud example that's on Google's github.
Right now I use the onXyzIjAvailable(TangoXyzIjData tangoXyzIjData) to get a frame of x y and z values; the points. I then save these frames in a PCLManager in the form of Vector3. After I'm done scanning my room, I simple write all the Vector3 from the PCLManager to a .xyz file using:
OutputStream os = new FileOutputStream(file);
size = pointCloud.size();
for (int i = 0; i < size; i++) {
String row = String.valueOf(pointCloud.get(i).x) + " "
+ String.valueOf(pointCloud.get(i).y) + " "
+ String.valueOf(pointCloud.get(i).z) + "\r\n";
os.write(row.getBytes());
}
os.close();
Everything works fine, not compilation errors or crashes. The only thing that seems to be going wrong is the rotation or translation of the points in the cloud. When I view the point cloud everything is messed up; the area I scanned is not recognizable, though the amount of points is the same as recorded.
Could this have to do something with the fact that I don't use PoseData together with the XyzIjData? I'm kind of new to this subject and have a hard time understanding what the PoseData exactly does. Could someone explain it to me and help me fix my point cloud?
Yes, you have to use TangoPoseData.
I guess you are using TangoXyzIjData correctly; but the data you get this way is relative to where the device is and how the device is tilted when you take the shot.
Here's how i solved this:
I started from java_point_to_point_example. In this example they get the coords of 2 different points with 2 different coordinate system and then write those coordinates wrt the base Coordinate frame pair.
First of all you have to setup your exstrinsics, so you'll be able to perform all the transformations you'll need. To do that I call mExstrinsics = setupExtrinsics(mTango) function at the end of my setTangoListener() function. Here's the code (that you can find also in the example I linked above).
private DeviceExtrinsics setupExtrinsics(Tango mTango) {
//camera to IMU tranform
TangoCoordinateFramePair framePair = new TangoCoordinateFramePair();
framePair.baseFrame = TangoPoseData.COORDINATE_FRAME_IMU;
framePair.targetFrame = TangoPoseData.COORDINATE_FRAME_CAMERA_COLOR;
TangoPoseData imu_T_rgb = mTango.getPoseAtTime(0.0,framePair);
//IMU to device transform
framePair.targetFrame = TangoPoseData.COORDINATE_FRAME_DEVICE;
TangoPoseData imu_T_device = mTango.getPoseAtTime(0.0,framePair);
//IMU to depth transform
framePair.targetFrame = TangoPoseData.COORDINATE_FRAME_CAMERA_DEPTH;
TangoPoseData imu_T_depth = mTango.getPoseAtTime(0.0,framePair);
return new DeviceExtrinsics(imu_T_device,imu_T_rgb,imu_T_depth);
}
Then when you get the point Cloud you have to "normalize" it. Using your exstrinsics is pretty simple:
public ArrayList<Vector3> normalize(TangoXyzIjData cloud, TangoPoseData cameraPose, DeviceExtrinsics extrinsics) {
ArrayList<Vector3> normalizedCloud = new ArrayList<>();
TangoPoseData camera_T_imu = ScenePoseCalculator.matrixToTangoPose(extrinsics.getDeviceTDepthCamera());
while (cloud.xyz.hasRemaining()) {
Vector3 rotatedV = ScenePoseCalculator.getPointInEngineFrame(
new Vector3(cloud.xyz.get(),cloud.xyz.get(),cloud.xyz.get()),
camera_T_imu,
cameraPose
);
normalizedCloud.add(rotatedV);
}
return normalizedCloud;
}
This should be enough, now you have a point cloud wrt you base frame of reference.
If you overimpose two or more of this "normalized" cloud you can get the 3D representation of your room.
There is another way to do this with rotation matrix, explained here.
My solution is pretty slow (it takes around 700ms to the dev kit to normalize a cloud of ~3000 points), so it is not suitable for a real time application for 3D reconstruction.
Atm i'm trying to use Tango 3D Reconstruction Library in C using NDK and JNI. The library is well documented but it is very painful to set up your environment and start using JNI. (I'm stuck at the moment in fact).
Drifting
There still is a problem when I turn around with the device. It seems that the point cloud spreads out a lot.
I guess you are experiencing some drifting.
Drifting happens when you use Motion Tracking alone: it consist of a lot of very small error in estimating your Pose that all together cause a big error in your pose relative to the world. For instance if you take your tango device and you walk in a circle tracking your TangoPoseData and then you draw you trajectory in a spreadsheet or whatever you want you'll notice that the Tablet will never return at his starting point because he is drifting away.
Solution to that is using Area Learning.
If you have no clear ideas about this topic i'll suggest watching this talk from Google I/O 2016. It will cover lots of point and give you a nice introduction.
Using area learning is quite simple.
You have just to change your base frame of reference in TangoPoseData.COORDINATE_FRAME_AREA_DESCRIPTION. In this way you tell your Tango to estimate his pose not wrt on where it was when you launched the app but wrt some fixed point in the area.
Here's my code:
private static final ArrayList<TangoCoordinateFramePair> FRAME_PAIRS =
new ArrayList<TangoCoordinateFramePair>();
{
FRAME_PAIRS.add(new TangoCoordinateFramePair(
TangoPoseData.COORDINATE_FRAME_AREA_DESCRIPTION,
TangoPoseData.COORDINATE_FRAME_DEVICE
));
}
Now you can use this FRAME_PAIRS as usual.
Then you have to modify your TangoConfig in order to issue Tango to use Area Learning using the key TangoConfig.KEY_BOOLEAN_DRIFT_CORRECTION. Remember that when using TangoConfig.KEY_BOOLEAN_DRIFT_CORRECTION you CAN'T use learningmode and load ADF (area description file).
So you cant use:
TangoConfig.KEY_BOOLEAN_LEARNINGMODE
TangoConfig.KEY_STRING_AREADESCRIPTION
Here's how I initialize TangoConfig in my app:
TangoConfig config = tango.getConfig(TangoConfig.CONFIG_TYPE_DEFAULT);
//Turning depth sensor on.
config.putBoolean(TangoConfig.KEY_BOOLEAN_DEPTH, true);
//Turning motiontracking on.
config.putBoolean(TangoConfig.KEY_BOOLEAN_MOTIONTRACKING,true);
//If tango gets stuck he tries to autorecover himself.
config.putBoolean(TangoConfig.KEY_BOOLEAN_AUTORECOVERY,true);
//Tango tries to store and remember places and rooms,
//this is used to reduce drifting.
config.putBoolean(TangoConfig.KEY_BOOLEAN_DRIFT_CORRECTION,true);
//Turns the color camera on.
config.putBoolean(TangoConfig.KEY_BOOLEAN_COLORCAMERA, true);
Using this technique you'll get rid of those spreads.
PS
In the Talk i linked above, at around 22:35 they show you how to port your application to Area Learning. In their example they use TangoConfig.KEY_BOOLEAN_ENABLE_DRIFT_CORRECTION. This key does not exist anymore (at least in Java API). Use TangoConfig.KEY_BOOLEAN_DRIFT_CORRECTION instead.
Hello I'm tring to recognize a car using cascade classifier, android and opencv library. My problem is that my phone is marking almoust everything as a car.
I've created my code based on:
https://www.youtube.com/watch?v=WEzm7L5zoZE
and face detection sample. My app behave very strange cause marking looks like random. I even don't know if marking car is correct or maybe it is just some random behaviour. At the moment it is even marking my keyboard as a car. I'm not sure what can I improve. I don't see any progress between training it up to 5 or 14 stages
I've trained my file up to 14 stages
my code looks like this:
#Override
public Mat onCameraFrame(Mat aInputFrame) {
// return FrameAnalyzer.analyzeFrame(aInputFrame);
// Create a grayscale image
Imgproc.cvtColor(aInputFrame, grayscaleImage, Imgproc.COLOR_RGBA2RGB);
MatOfRect objects = new MatOfRect();
// Use the classifier to detect faces
if (cascadeClassifier != null) {
cascadeClassifier.detectMultiScale(grayscaleImage, objects, 1.1, 1,
2, new Size(absoluteObjectSize, absoluteObjectSize),
new Size());
}
Rect[] dataArray = objects.toArray();
for (int i = 0; i < dataArray.length; i++) {
Core.rectangle(aInputFrame, dataArray[i].tl(), dataArray[i].br(),
new Scalar(0, 255, 0, 255), 3);
}
return aInputFrame;
}
Try changing the below.
Using COLOR_RGBA2RGB with cvtColor as in sample code will not give a gray scale image. Try RGBA2GRAY
Increase the number of neighbors in detectMultiScale. Now it's 2. More neighbors means more confidence in result.
Hope there are enough samples to train with. A quick search and reading through books, gives an impression like thousands of images are needed for training. For e.g. around 10000 images are used for OCR haar training. For face training, 3000 to 5000 samples are used.
More importantly, decide if you really want to go with haar training for identifying a car. There could be better methods of vehicle identification. For e.g. for a moving vehicle we could use optical flow based techniques.
I am developing an android game with box2d and use a fixed timestep system for advancing the physics.
However as I use this system it requires the box2d positions to be interpolates. I read this article
and have implemented an interpolation method very much like the one in the article.
The method seems to work nicely on the computer but on my phone the positions of objects are very jumpy. There is of course a big frame rate difference between PC and phone, but I think this algorithm should not mind that.
Here is the just of the code if you don't feel like looking at the article :
void PhysicsSystem::smoothStates_ ()
{
const float oneMinusRatio = 1.f - fixedTimestepAccumulatorRatio_;
for (b2Body * b = world_->GetBodyList (); b != NULL; b = b->GetNext ())
{
if (b->GetType () == b2_staticBody)
{
continue;
}
PhysicsComponent & c = PhysicsComponent::b2BodyToPhysicsComponent (* b);
c.smoothedPosition_ =
fixedTimestepAccumulatorRatio_ * b->GetPosition () +
oneMinusRatio * c.previousPosition_;
c.smoothedAngle_ =
fixedTimestepAccumulatorRatio_ * b->GetAngle () +
oneMinusRatio * c.previousAngle_;
}
}
Does anyone know why my game is acting like this?
Thanks for the help
That code in and of itself doesn't appear to have any issues as compared to the article. You might want to try posting this on https://gamedev.stackexchange.com/ and see if they have any recommendations.
Alternatively, here is a very well written article about having a semi-fixed time step, and decoupling physics and frame rate, which I imagine could be the problem. It isn't for Box2D, but reading over it might help you pinpoint the issue with your physics.