I am developing an app where on image detection I am playing a video as anchor node. Everything is working perfect except video stoping on camera focus out. I can hear audio from video playing even if I move camera away from detected image. I have tried augmented image tracking state stop but it did not help.
Is there any callback or observer where I can check if camera is not focusing on augmented image?
AFAIK there is no call-back available at this time - this type of thing has been discussed on the ARCore issues list and it was noted that it is outside the scope of ARCore: https://github.com/google-ar/arcore-android-sdk/issues/78
You can, however, check yourself by mapping your nodes world points to screen point and then checking if it is within the 'arSceneView.scene.camera' view bounds - see below for values for x and y which will indicate if it is outside the view, from the ARCore documentation: https://developers.google.com/ar/reference/java/sceneform/reference/com/google/ar/sceneform/Camera#worldToScreenPoint(com.google.ar.sceneform.math.Vector3)
public Vector3 worldToScreenPoint (Vector3 point)
Convert a point from world space into screen space.
The X value is negative when the point is left of the viewport, between 0 and the width of the SceneView when the point is within the viewport, and greater than the width when the point is to the right of the viewport.
The Y value is negative when the point is below the viewport, between 0 and the height of the SceneView when the point is within the viewport, and greater than the height when the point is above the viewport.
The Z value is always 0 since the return value is a 2D coordinate.
Related
I am successfully converting (latitude, longitude) coordinates to (East, North) coordinates and am trying to figure out a way to accurately place the (East, North) coordinates in the AR world.
Example of my current issue:
Device coordinate system:
A conversion from a (latitude, longitude) coordinate gives me an (East, North) coordinate of,
(+x, 0, +z) where x=East, z=North
Now, if I am facing Northwards the EN coordinate will be placed behind me, as the forward facing axis is -z . If I am facing Southwards, the EN coordinate will be placed behind me once again because it is dependent on my device's orientation.
My question:
In ARCore is it possible to fix a device's coordinate system no matter what orientation the device is in? Or is there an algorithm that takes into account device orientation and allows static placement of Anchors?
EDIT:
I posted this same question on the ARCore-Sceneform GitHub and these are the answers I received:
At first, let's see what Google ARCore engineers say about World coordinates:
World Coordinate Space
As ARCore's understanding of the environment changes, it adjusts its model of the world to keep things consistent. When this happens, the numerical location (coordinates) of the camera and anchors can change significantly to maintain appropriate relative positions of the physical locations they represent.
These changes mean that every frame should be considered to be in a completely unique world coordinate space. The numerical coordinates of anchors and the camera should never be used outside the rendering frame during which they were retrieved. If a position needs to be considered beyond the scope of a single rendering frame, either an anchor should be created or a position relative to a nearby existing anchor should be used.
But!
I firmly believe that a smartphone with its camera and sensors begin their way (in the ArSession) from relative center of coordinate space (it's relative for us, for ARCore it's absolute, of course):
// X, Y, Z coordinates must be counted from zero, mustn't they?
(x: 0.0, y: 0.0, z: 0.0)
Then, under certain circumstances, the world coordinate space might be changed.
We are creating an app which is used for calculating measurements of any a window/door using Project tango Device. For that we need to follow below mentioned steps:
1. Capture Image (normal 2D image). Store this image.
2. Also capture point cloud while capturing the Image. Store the point cloud in a PCD file.
3. Indicate the position of the window/door in the image displayed on a canvas by drawing a rectangle on the image. See Image.
4. As the rectangle is drawn automatically calculate width and height of the window/door using the stored PointCloud data.
We have managed to do 1, 2 and 3.
For 4 we have two issues:
A. Determine the Points in the PointCloud corresponding to the drawn rectangle i.e. the window/door. We believe that this involves determining the plane in which the window/door is located e.g. Assuming that the axis along the depth (i.e. from the camera to the object) is Z-axis then we need to determine the value(s) of Z that correspond to the plane in which the window/door is located in PointCloud. How can this be done? Please can someone suggest a feasible and an efficient way of doing so?
B. Once we determine the sub-PointCloud corresponding to the drawn rectangle find the distance between the minimum and maximum points along the X & Y axis to determine the width and height respectively. How can this be done?
Any help with demo code or app reference is appreciated.enter image description here
enter image description here
find contour of the point cloud. Use iterative ransac to fit lines to the contour. Intersect the lines and get the corner points
for 3d, compute the surface normals. Then compute the curvature which is a differential of the surface normal. These are corner points.
PCL (Point Cloud Library) has all of these functions.
Currently I'm developing an Android App for blind people that uses the device's camera.
The app shows in a SurfaceView the preview of the camera's captured frames. When the screen is touched, the app gets the coordinates from the screen, with event.getX() and event.getY(). However, what I really want to do is to obtain the coordinates of the touched point but taking as a reference the coordinates of the frame.
With view.getLocationonScreen() or view.getLocationInWindow() I can get the location of referred to the top left position where SurfaceView begins, but the frame don't occupies all the surface of the SurfaceView.
How can I transform the coordinates of the screen to relative coordinates to the frame? Or how can I know the coordinates where the frame is located inside the Surface View? When I use getMarginLeft() or similar I obtain the value 0.
I want to show the movement of a car in a road . I have a textfile containing the positions and I built the movement by updating the position of a car every second .lets say the plain is (200,200) . now what should I do for positions that are outside this screen ? how could I follow my car there ?
should I set up a camera or something?
by the way my app is 2D.
From my experience, there is no actual concept of setting up a camera in 2D programming, but I could be wrong. You'll have to do this yourself, create a camera class etc.....
What I think will end up happening is that the car will stay centered on the screen and everything under it will be moving instead. Depends on what you're trying to achieve.
So if your car is moving northeast at 20 km/h, don't actually move the car, make everything under the car move southwest at 20km/h (or how many pixels per frame this comes out to)
This is if you want to follow the car. If you want to center the "camera" on the car whenever it goes out of bounds you'll probably have to move the landscape and the car towards the center of the screen.
EDIT: I'm assuming that the car will be the main focus?? So it should always be at the center of the screen.
All objects in the game should have a velocity and a position. The position tells you where the object currently is and the velocity tells you how many x's and how many y's it should be moving per frame. So every frame you would say position = position + velocity.
The non-car objects can move off the screen as they wish without having the camera follow them, so let them go. Keep the car centered and adjust all the other objects' velocities based on the car's.
Ex:
Car's velocity (3, 0) ---> means it's moving right in the straight line at 3 pixels per frame
Object 1 velocity (4, 0) ---> means it's also moving right in a straight line but 4 pixels per frame
The velocity of object 1 will have to adjust itself according the the car's velocity. So say:
object1.position = object1.position + (object1.velocity - car.velocity)
Object 1's new velocity is (1, 0), so it's moving faster than the car by one.
If the car gains speed to let's say (5, 0) then object one will appear to be moving backwards by 1.
I'm having difficulties understanding about the OpenGL perspective view. I've read tons of information however it didn't help me trying to achieve what I'm after. Which is making sure my 3d scene is filling the entire screen on every Android device.
To test this, I will be drawing a quad in 3d space which in the end should touch every corner, filling up the whole device's screen. I could then use this quad, or actually its coordinates to specify a bounding box at a certain Z distance which I could use to put my geometry and making sure those fill up my screen. When the screen resizes, or I am to run it on another screen resolution, I would recalculate this bounding box and geometry. I'm not talking about static geometry, but for instance say I want to fill the screen with balls and it doesn't matter how big or how many balls there are, the only important thing is the screen is filled and there are no redundant balls outside the visible frustum.
As far as I understand when specifying the viewport you actually bind pixel values to the frustum's boundaries. I know that you can actually set an orthographic view in a way your window pixels match 3d geometry position but I'm not sure how this works in perspective view.
Here I'm assuming the viewport width and height to be mapped to the nearZ. So when a ball is at Z=1f it has it's original size
When moving the ball into the screen so into the direction of farZ, the ball would be scaled down in order for the perspective to work. So a ball at Z=51f for instance, would appear smaller on my screen and I would need more balls to fill up the entire screen.
Now in order to do so, I'm looking for the purple boundaries
Actually I need these boundaries to fill the entire screen on different frustum sizes (width x height) while the frustum angle and Z distance for the balls is always the same
I guess I could use trig to calculate these purple boundaries (see blue triangle note)
Am I correctly marking the frustum angle, it being the vertical angle of the frustum?
Could someone elaborate on the green 1f and -1f values as I seem to have read something about it? Seems like some scalar that is used to resize the geometry within the frustum?
I actually want to be able to programmaticaly position geometry against the viewport borders within 3d space at any resolution/window for any arbitrary Android device.
Note: I have a flash background which uses a stage (visual area) with a known width x height at any resolution which makes it easy to position/scale assets either using absolute measurements or percentual measurements. I guess I'm trying to figure out how this system applies to OpenGL's perspective view.
I guess this post using gluUnproject answers the question.