Develop Reference

Generate and export point cloud from Project Tango

After some weeks of waiting I finally have my Project Tango. My idea is to create an app that generates a point cloud of my room and exports this to .xyz data. I'll then use the .xyz file to show the point cloud in a browser! I started off by compiling and adjusting the point cloud example that's on Google's github.
Right now I use the onXyzIjAvailable(TangoXyzIjData tangoXyzIjData) to get a frame of x y and z values; the points. I then save these frames in a PCLManager in the form of Vector3. After I'm done scanning my room, I simple write all the Vector3 from the PCLManager to a .xyz file using:
OutputStream os = new FileOutputStream(file);
size = pointCloud.size();
for (int i = 0; i < size; i++) {
String row = String.valueOf(pointCloud.get(i).x) + " "
+ String.valueOf(pointCloud.get(i).y) + " "
+ String.valueOf(pointCloud.get(i).z) + "\r\n";
os.write(row.getBytes());
}
os.close();
Everything works fine, not compilation errors or crashes. The only thing that seems to be going wrong is the rotation or translation of the points in the cloud. When I view the point cloud everything is messed up; the area I scanned is not recognizable, though the amount of points is the same as recorded.
Could this have to do something with the fact that I don't use PoseData together with the XyzIjData? I'm kind of new to this subject and have a hard time understanding what the PoseData exactly does. Could someone explain it to me and help me fix my point cloud?

Yes, you have to use TangoPoseData.
I guess you are using TangoXyzIjData correctly; but the data you get this way is relative to where the device is and how the device is tilted when you take the shot.
Here's how i solved this:
I started from java_point_to_point_example. In this example they get the coords of 2 different points with 2 different coordinate system and then write those coordinates wrt the base Coordinate frame pair.
First of all you have to setup your exstrinsics, so you'll be able to perform all the transformations you'll need. To do that I call mExstrinsics = setupExtrinsics(mTango) function at the end of my setTangoListener() function. Here's the code (that you can find also in the example I linked above).
private DeviceExtrinsics setupExtrinsics(Tango mTango) {
//camera to IMU tranform
TangoCoordinateFramePair framePair = new TangoCoordinateFramePair();
framePair.baseFrame = TangoPoseData.COORDINATE_FRAME_IMU;
framePair.targetFrame = TangoPoseData.COORDINATE_FRAME_CAMERA_COLOR;
TangoPoseData imu_T_rgb = mTango.getPoseAtTime(0.0,framePair);
//IMU to device transform
framePair.targetFrame = TangoPoseData.COORDINATE_FRAME_DEVICE;
TangoPoseData imu_T_device = mTango.getPoseAtTime(0.0,framePair);
//IMU to depth transform
framePair.targetFrame = TangoPoseData.COORDINATE_FRAME_CAMERA_DEPTH;
TangoPoseData imu_T_depth = mTango.getPoseAtTime(0.0,framePair);
return new DeviceExtrinsics(imu_T_device,imu_T_rgb,imu_T_depth);
}
Then when you get the point Cloud you have to "normalize" it. Using your exstrinsics is pretty simple:
public ArrayList<Vector3> normalize(TangoXyzIjData cloud, TangoPoseData cameraPose, DeviceExtrinsics extrinsics) {
ArrayList<Vector3> normalizedCloud = new ArrayList<>();
TangoPoseData camera_T_imu = ScenePoseCalculator.matrixToTangoPose(extrinsics.getDeviceTDepthCamera());
while (cloud.xyz.hasRemaining()) {
Vector3 rotatedV = ScenePoseCalculator.getPointInEngineFrame(
new Vector3(cloud.xyz.get(),cloud.xyz.get(),cloud.xyz.get()),
camera_T_imu,
cameraPose
);
normalizedCloud.add(rotatedV);
}
return normalizedCloud;
}
This should be enough, now you have a point cloud wrt you base frame of reference.
If you overimpose two or more of this "normalized" cloud you can get the 3D representation of your room.
There is another way to do this with rotation matrix, explained here.
My solution is pretty slow (it takes around 700ms to the dev kit to normalize a cloud of ~3000 points), so it is not suitable for a real time application for 3D reconstruction.
Atm i'm trying to use Tango 3D Reconstruction Library in C using NDK and JNI. The library is well documented but it is very painful to set up your environment and start using JNI. (I'm stuck at the moment in fact).
Drifting
There still is a problem when I turn around with the device. It seems that the point cloud spreads out a lot.
I guess you are experiencing some drifting.
Drifting happens when you use Motion Tracking alone: it consist of a lot of very small error in estimating your Pose that all together cause a big error in your pose relative to the world. For instance if you take your tango device and you walk in a circle tracking your TangoPoseData and then you draw you trajectory in a spreadsheet or whatever you want you'll notice that the Tablet will never return at his starting point because he is drifting away.
Solution to that is using Area Learning.
If you have no clear ideas about this topic i'll suggest watching this talk from Google I/O 2016. It will cover lots of point and give you a nice introduction.
Using area learning is quite simple.
You have just to change your base frame of reference in TangoPoseData.COORDINATE_FRAME_AREA_DESCRIPTION. In this way you tell your Tango to estimate his pose not wrt on where it was when you launched the app but wrt some fixed point in the area.
Here's my code:
private static final ArrayList<TangoCoordinateFramePair> FRAME_PAIRS =
new ArrayList<TangoCoordinateFramePair>();
{
FRAME_PAIRS.add(new TangoCoordinateFramePair(
TangoPoseData.COORDINATE_FRAME_AREA_DESCRIPTION,
TangoPoseData.COORDINATE_FRAME_DEVICE
));
}
Now you can use this FRAME_PAIRS as usual.
Then you have to modify your TangoConfig in order to issue Tango to use Area Learning using the key TangoConfig.KEY_BOOLEAN_DRIFT_CORRECTION. Remember that when using TangoConfig.KEY_BOOLEAN_DRIFT_CORRECTION you CAN'T use learningmode and load ADF (area description file).
So you cant use:
TangoConfig.KEY_BOOLEAN_LEARNINGMODE
TangoConfig.KEY_STRING_AREADESCRIPTION
Here's how I initialize TangoConfig in my app:
TangoConfig config = tango.getConfig(TangoConfig.CONFIG_TYPE_DEFAULT);
//Turning depth sensor on.
config.putBoolean(TangoConfig.KEY_BOOLEAN_DEPTH, true);
//Turning motiontracking on.
config.putBoolean(TangoConfig.KEY_BOOLEAN_MOTIONTRACKING,true);
//If tango gets stuck he tries to autorecover himself.
config.putBoolean(TangoConfig.KEY_BOOLEAN_AUTORECOVERY,true);
//Tango tries to store and remember places and rooms,
//this is used to reduce drifting.
config.putBoolean(TangoConfig.KEY_BOOLEAN_DRIFT_CORRECTION,true);
//Turns the color camera on.
config.putBoolean(TangoConfig.KEY_BOOLEAN_COLORCAMERA, true);
Using this technique you'll get rid of those spreads.
PS
In the Talk i linked above, at around 22:35 they show you how to port your application to Area Learning. In their example they use TangoConfig.KEY_BOOLEAN_ENABLE_DRIFT_CORRECTION. This key does not exist anymore (at least in Java API). Use TangoConfig.KEY_BOOLEAN_DRIFT_CORRECTION instead.

Custom markers showing with white color on some devices (Google Maps v3 Android SDK)

I have Google Maps SDK running with custom markers on an android app. Everything works fine on our test phones -- an A7000, an Samsung and other. However, when I run the app on Nexus 5 or LG devices, one type of custom marker displaying white on the map.
While the markers are showing white,will perform desired behavior.
We're super confused about this, especially because another custom marker (that is very similar) works fine. Other than hardware, the only other difference between our test phones and the Nexus 5 is that the Nexus is running Android 6.0, vs 5.x and 4.x for our test phones.
Code using to add marker
for(i=0;i<latLngs.size;i++)
{
LatLng latLng=latLngs.get(i);
Marker m = mMap.addMarker(new MarkerOptions().position(latLng).title("Title").icon(BitmapDescriptorFactory.fromResource(R.drawable.marker)));
}
for change the marker icon using
m.setIcon(BitmapDescriptorFactory.fromResource(R.drawable.marker1));
On nexus device google map appear like below, some off the markers become white ,i am using loop to add markers.

EDIT: With version 9.2.56 of the Google Play Services app released on June 13th, the bug has been fixed.
I found a workaround to your problem, however it is quite ugly and it's probably not advised to use it unless you know you're going to use a little number of markers.
Instead of this:
//Getting a reference to your activity's resources
final Resources resources = myActivity.getResources();
//Defining your drawable res id
final int resId = R.drawable.my_drawable_res_id;
marker.setIcon(BitmapDescriptorFactory.fromResource(resId));
Do this:
//Getting a reference to your activity's resources
final Resources resources = myActivity.getResources();
//Defining your drawable res id
final int resId = R.drawable.my_drawable_res_id;
marker.setIcon(
BitmapDescriptorFactory.fromBitmap(
BitmapFactory.decodeResource(resources, resId)));
Workaround found after reading this link (thanks #antonio in your comments):
https://code.google.com/p/gmaps-api-issues/issues/detail?id=9765
The bug is caused on certain devices with a recent version of the Google Play Services library (probably 8.7+ or 9+).
It happens if you share a BitmapDescriptor with multiple markers, so the workaround is to re-create one every time. I think BitmapDescriptorFactory.fromResource might cache read resources somehow, that's why you need to decode it as a bitmap.
Here is the quote of the #10 reply of transbao at the link antonio gave:
We can repro this bug which causes certain marker icons to render as
white patches. Your app may be affected if an icon bitmap is shared
among multiple markers, although the issue only manifests in specific
usage scenarios.
In the short term, we'd recommend the workaround in #8 -- use a unique
Bitmap object for each marker:
marker.setIcon(BitmapDescriptorFactory.fromBitmap(
BitmapFactory.decodeResource(getResources(),
R.drawable.drawableid)));
and
new MarkerOptions().icon(BitmapDescriptorFactory.fromBitmap(
BitmapFactory.decodeResource(getResources(),
R.drawable.drawableid)));
Creating a BitmapDescriptor once and reusing it won’t be sufficient.
E.g. if you’re doing:
BitmapDescriptor bd = ...; marker1.setIcon(bd); marker2.setIcon(bd);
...then the workaround would be:
marker1.setIcon(BitmapDescriptorFactory.fromBitmap(
BitmapFactory.decodeResource(getResources(),
R.drawable.drawableid)));
marker2.setIcon(BitmapDescriptorFactory.fromBitmap(
BitmapFactory.decodeResource(getResources(),
R.drawable.drawableid)));
Please note that, if your app uses a lot of markers, this workaround
could possibly result in higher memory consumption. Also,
unfortunately the workaround doesn’t apply when the default icon is
used via BitmapDescriptorFactory.defaultMarker().

Skobbler Annotation disappears from map when zooming out on Android

Currently, I am adding a list of annotations to a mapview with code similar to the following:
// Add to map view
SKAnnotation annotation = new SKAnnotation(i++);
annotation.getLocation().setLongitude(result.longitude);
annotation.getLocation().setLatitude(result.latitude);
annotation.setMininumZoomLevel(1);
annotation.setAnnotationType(SKAnnotation.SK_ANNOTATION_TYPE_PURPLE);
mapView.addAnnotation(annotation, SKAnimationSettings.ANIMATION_POP_OUT);
Yet whenever I view the annotations on the map, they disappear after I zoom out to anything under zoom level 4.0. Looking at the docs for the Annotation class (as well as confirming in the code), I see that the default zoom level is set to 4, yet it seems that my call to .setMinimumZoomLevel is ignored.
Does anyone have any insight into what is happening or if this might be a known bug within the SDK?
I'm using Skobbler 2.5 on Android.
Thanks for any help in the matter!

Based off Ando's comment on the original question and referencing the documentation here, I updated the code to use the workaround he described to allow annotations to show up down to zoom level 2.
Original code:
SKAnnotation annotation = new SKAnnotation(i++);
annotation.getLocation().setLongitude(result.longitude);
annotation.getLocation().setLatitude(result.latitude);
annotation.setMininumZoomLevel(1); // Note: this does not work
annotation.setAnnotationType(SKAnnotation.SK_ANNOTATION_TYPE_PURPLE);
mapView.addAnnotation(annotation, SKAnimationSettings.ANIMATION_POP_OUT);
Updated code:
SKAnnotation annotation = new SKAnnotation(i++);
annotation.getLocation().setLongitude(result.longitude);
annotation.getLocation().setLatitude(result.latitude);
annotation.setMininumZoomLevel(2);
DisplayMetrics metrics = new DisplayMetrics();
getWindowManager().getDefaultDisplay().getMetrics(metrics);
if (metrics.densityDpi < DisplayMetrics.DENSITY_HIGH) {
annotation.setImagePath(SKMaps.getInstance().getMapInitSettings().
getMapResourcesPath() + "/.Common/icon_greypin#2x.png");
// set the size of the image in pixel
annotation.setImageSize(128);
} else {
annotation.setImagePath(SKMaps.getInstance().getMapInitSettings().
getMapResourcesPath()+ "/.Common/icon_greypin#3x.png");
// set the size of the image in pixels
annotation.setImageSize(256);
}
mapView.addAnnotation(annotation, SKAnimationSettings.ANIMATION_POP_OUT);
A couple things to note:
.setImagePath() and .setImageSize() are both deprecated methods in the latest SDK even though they're still referenced in the documentation above. Not sure if that means there is another alternative to displaying images via an absolute path approach, or if they're simply phasing this functionality out.
In my particular example, we're using the purple pin to display annotations, but the absolute path file name for that pin is actually called icon_greypin. It looks like the other pin image file name are named appropriately however.
Anyways, this served as a solution for my particular problem until the SDK is updated, so I'm marking it as the answer and I hope it helps someone else! Thanks to Ando for the step in the right direction!

Graph/Plots on ANDROID

I am new to android and trying to learn creation or plotting of graphs in android. I've come across 2 libraries:
GraphView
AndroidPlot.
My intent would be to receive some sound file and plot it on a graph. So, for this purpose which library would be better. Also I wanna know, where I can see the complete implementation or definitions of these libraries i.e. the structure and code for the API's used in the above libraries.
Also I have tried some sample codes available on net. But I'm looking for a more sophiticated code which I could develop on eclipse ADT and hence can learn something out of it.

My intent would be to receive some sound file and plot it on a graph
Neither library does this by default. The libraries are used to plot a graph given a set of data points. Getting the data points from the sound file is up to you.
So, for this purpose which library would be better.
Either library should be fine once you get the data points.
Also I wanna know, where I can see the complete implementation or definitions of these libraries i.e. the structure and code for the API's used in the above libraries.
Check out the sources for GraphView and AndroidPlot.

I have used Achartengine some times and it works great. I modified it without difficulties.

If You are drawing simple Line Graph using canvas to draw the graph.

Use AndroidPlot. This code draw the content of Vector(in this case filled of zeros). You only have to pass the info of the wav file to the vector. And you can check this thread for the reading issue.
Android: Reading wav file and displaying its values
plot = (XYPlot) findViewById(R.id.Grafica);
plot.setDomainStep(XYStepMode.INCREMENT_BY_VAL, 0.5);
plot.setRangeStep(XYStepMode.INCREMENT_BY_VAL, 1);
plot.getGraphWidget().getGridBackgroundPaint().setColor(Color.rgb(255, 255, 255));
plot.getGraphWidget().getDomainGridLinePaint().setColor(Color.rgb(255, 255, 255));
plot.getGraphWidget().getRangeGridLinePaint().setColor(Color.rgb(255, 255, 255));
plot.getGraphWidget().setDomainLabelPaint(null);
plot.getGraphWidget().setRangeLabelPaint(null);
plot.getGraphWidget().setDomainOriginLabelPaint(null);
plot.getGraphWidget().setRangeOriginLabelPaint(null);
plot.setBorderStyle(BorderStyle.NONE, null, null);
plot.getLayoutManager().remove(plot.getLegendWidget());
plot.getLayoutManager().remove(plot.getDomainLabelWidget());
plot.getLayoutManager().remove(plot.getRangeLabelWidget());
plot.getLayoutManager().remove(plot.getTitleWidget());
//plot.getBackgroundPaint().setColor(Color.WHITE);
//plot.getGraphWidget().getBackgroundPaint().setColor(Color.WHITE);
plot.setRangeBoundaries(-25, 25, BoundaryMode.FIXED);// check that, these //boundaries wasn't for audio files
InicializarLasVariables();
for(int i=0; i<min/11;i++){
DatoY=0;
Vector.add(DatoY);
}
XYSeries series = new SimpleXYSeries(Vector, SimpleXYSeries.ArrayFormat.Y_VALS_ONLY,"");
LineAndPointFormatter seriesFormat = new LineAndPointFormatter(Color.rgb(0, 0, 0), 0x000000, 0x000000, null);
plot.clear();
plot.addSeries(series, seriesFormat);

Optimal path between source and destination - android

I'm new to android development and I've done some applications involving Google maps. What I'm trying to develop now is an Optimal Path Finder App. I know that there are many application already available for this purpose, but I plan to do this on my own. I'm planning to use Dijkstra's algorithm for this purpose. The initial input is the source and destination, and the output must be the optimal path drawn on a Google map between the source and destination. I've an idea about the steps needed to do this application.
Step 1 : Extract the intermediate places between the source and destination.
Step 2 : Add these nodes to a graph data structure.
Step 3 : Apply Dijkstra's Algorithm and find the path.
Step 4 : Draw the path on a Google map.
Am I thinking in the right way?

Yes you're thinking in the right way :) ! Also consider openstreetmaps for the data source and mapsforge as an offline map tile provider.

Try this:
1.put ruledijkstra.html onto folder asset
2.load using this code:
private void getLocation(){
WebSettings webSettings = mWebView.getSettings();
mWebView.loadUrl("file:///android_asset/maps.html");
String path="javascript:loadAtStart(-6.184505,106.797685,11)";
mWebView.loadUrl(path);
}
3.code for asset:
ruledijkstra.html or you can google:etc
http://www.zlink.com.br/optimap/index.php?&loc0=-7.1209077392192785,-34.9075984954834&loc1=-7.121418750873643,-34.903221130371094&loc2=-7.128828355854018,-34.93025779724121