Develop Reference

How Can I check current frame have 3d Model in Google Ar core

Hello All I am working on a task where I want to show the TextView if camera frame does not have any 3D model in google ar core. I am able to place the 3D model into the back camera ar scene view I just want when I move my camera to check if the current frame has a 3D model or not.
here is my code for placing the 3d Model
private fun onTapPlane(hitResult: HitResult, plane: Plane, motionEvent: MotionEvent) {
if (model == null) {
Toast.makeText(context, "Loading...", Toast.LENGTH_SHORT).show()
return
}
clearAllModel()
anchorNode = AnchorNode(hitResult.createAnchor()).apply {
// Create the transformable model and add it to the anchor.
addChild(TransformableNode(arFragment.transformationSystem).apply {
model?.isShadowCaster = false
model?.isShadowReceiver = false
translationController.isEnabled = true
scaleController.isEnabled = true
rotationController.isEnabled = true
renderable = model
// Add the View
})
}
// add the Anchor node
scene.addChild(anchorNode)
}
Here I am check when its detect the plane surface
private fun onUpdateListener(frameTime: FrameTime?) {
arSceneView.arFrame?.let { frame ->
val trackable = frame.getUpdatedTrackables(Plane::class.java).iterator()
if (trackable.hasNext()) {
val plane = trackable.next() as Plane
if (plane.trackingState == TrackingState.TRACKING) {
binding.tvPlane.text = ""
binding.ivImage.setImageResource(R.drawable.ic_back_camera_tutorial_hint)
} else {
Glide.with(requireContext()).load(R.raw.loop_new).into(binding.ivImage)
binding.tvPlane.text = ""
}
}
}
}
Hello All I am working on a task where I want to show the TextView if the camera frame does not have any 3D model in google ar core. I am able to place the 3D model into the back camera ar scene view I just want when I move my camera to check if the current frame has a 3D model or not.

localRotation is not rotating around the cube center

Sceneform 1.15 on Android
I want to make that the cube renderable is rotating around the own centar.
val anchorNode = AnchorNode().apply {
setParent(scene)
worldPosition = Vector3(2f, 3f, 0f)
}
scene.addChild(anchorNode)
dieNode = Node().apply {
setParent(anchorNode)
localRotation = Quaternion.eulerAngles(Vector3(10f,20f,60f))
name = "die"
renderable = it
}
I made an animator that is supposed to only rotate the cube around its own center.
private fun roll() {
val anim = createAnimator()
val node = scene.findByName("die")!!
anim.target = node
anim.setDuration(9000)
anim.start()
}
private fun createAnimator() : ObjectAnimator {
val o1 = Quaternion.eulerAngles(Vector3(-90f,180f,90f))
val animator = ObjectAnimator()
animator.setObjectValues(o1)
animator.setPropertyName("localRotation")
animator.setEvaluator(QuaternionEvaluator())
animator.setInterpolator(LinearInterpolator())
animator.setAutoCancel(true)
return animator
}
But, it happens that while cube is rotating, it is also moved which is not desired behavior.
Screenshots where one of the symetrically placed die in world should be only rotated:

The position is not centred for the Cube object. If it is off the centre with respect to the parent object, it will rotate around (0,0,0), if it lies off this axis - the object will move in a circle, not rotate.

ARCore Sceneform ViewRenderable rotates the image by random angle

I am placing a 2D image on vertical wall like a painting hanging on wall. But the rendered image is rotating by random angle. Its not properly aligned. I have integrated following code:
#RequiresApi(api = Build.VERSION_CODES.N)
private void createViewRenderable(WeakReference<ARRoomViewActivity> weakActivity) {
imageViewRenderable = new ImageView(this);
Picasso.get().load(paintingImageUrl)
.networkPolicy(NetworkPolicy.OFFLINE)
.into(imageViewRenderable);
ViewRenderable.builder()
.setView(this, imageViewRenderable)
.build()
.thenAccept(renderable -> {
ARRoomViewActivity activity = weakActivity.get();
if (activity != null) {
activity.renderable = renderable;
}
})
.exceptionally(
throwable -> {
return null;
});
}
private void addToScene(HitResult hitResult) {
if (renderable == null) {
return;
}
Anchor anchor = hitResult.createAnchor();
anchorNode = new AnchorNode(anchor);
anchorNode.setParent(arFragment.getArSceneView().getScene());
node = new Node();
node.setRenderable(renderable);
transformableNode = new TransformableNode(arFragment.getTransformationSystem());
node.setLocalRotation(Quaternion.axisAngle(new Vector3(-1f, 0, 0), -90f));
node.setLookDirection(new Vector3(0, 10f, 0));
transformableNode.setParent(anchorNode);
node.setParent(transformableNode);
transformableNode.select();
}
I am using following Sceneform version and also tried latest 1.17.0 but could not succeed
implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.7.0'

The orientation of anchors by default is such that one of the axes points somewhat towards the camera. This results in models placed on the floor to be "looking at" the user. However, models on the wall end up rotating arbitrarily. This can be fixed by setting the look direction of the node.
In your addToScene method, you can introduce an intermediate node that fixes the orientation relative to the anchor node. Children of the intermediate node will then be oriented correctly.
// Create an anchor node that will stay attached to the wall
Anchor anchor = hitResult.createAnchor();
anchorNode = new AnchorNode(anchor);
anchorNode.setParent(arFragment.getArSceneView().getScene());
// Create an intermediate node and orient it to be level by fixing look direction.
// This is needed specifically for nodes on walls.
Node intermediateNode = new Node();
intermediateNode.setParent(anchorNode);
Vector3 anchorUp = anchorNode.getUp();
intermediateNode.setLookDirection(Vector3.up(), anchorUp);
Note: the last two lines in the code sample above are the key to getting the orientation correct.

Added object on vertical plane always rotated in ARCore

I am adding an image on vertical plane in Sceneform ARFragment. But it always get rotated. The code is working fine on horizontal plane. My code for placing images on vertical Plane is as follow:
arFragment.setOnTapArPlaneListener { hitResult: HitResult,
plane: Plane,
motionEvent: MotionEvent ->
if(!isOnceTapedOnSurface) {
val anchor = hitResult.createAnchor()
val anchorNode = AnchorNode(anchor)
anchorNode.setParent(arFragment.arSceneView.scene)
andy = TransformableNode(arFragment.transformationSystem)
if(plane.type == Plane.Type.VERTICAL) {
val anchorUp = anchorNode.up
andy.setLookDirection(Vector3.up(), anchorUp)
}
andy.setParent(anchorNode)
andy.renderable = andyRenderable
andy.select()
// arFragment.arSceneView.planeRenderer.isVisible = false
isOnceTapedOnSurface = true
}
}

To fix this issue you can use the above solution. But you should rotate an object using world rotation. Don't use local rotation. We need to zero the rotation value. If you are using local rotation, the object will behave anchor(parent) rotation. So by using world rotation we can control the object.
String planeType = "";
//When tapping on the surface you can get the anchor orientation
if (plane.getType() == Plane.Type.VERTICAL){
planeType = "Vertical";
}else if (plane.getType() == Plane.Type.HORIZONTAL_UPWARD_FACING){
planeType = "Horizontal_Upward";
}else if (plane.getType() == Plane.Type.HORIZONTAL_DOWNWARD_FACING){
planeType = "Horizontal_Downward";
}else {
planeType = "Horizontal";
}```
// First set object world rotation zero
transformableNode.setWorldRotation(Quaternion.axisAngle(new Vector3(0, 0f, 0), 0));
// check plane type is vertical or horizontal if it is vertical below logic will work.
if (planeType.equals("Vertical")) {
Vector3 anchorUp = anchorNode.getUp();
transformableNode.setLookDirection(Vector3.up(), anchorUp);
}

To fix this issue you need to set public Pose getCenterPose(). It returns the pose of the center of the detected plane, defined to have the origin. The pose's transformed +Y axis will be point normal out of the plane, with the +X and +Z axes orienting the extents of the bounding rectangle.
anchor = mySession.createAnchor(plane.getCenterPose())
When the its trackable state is TRACKING, this pose is synced with the latest frame. When its trackable state is PAUSED, an identity pose will be returned.
Your code could be the following:
Anchor newAnchor;
for (Plane plane : mSession.getAllTrackables(Plane.class)) {
if(plane.getType() == Plane.Type.VERTICAL &&
plane.getTrackingState() == TrackingState.TRACKING) {
newAnchor = plane.createAnchor(plane.getCenterPose());
break;
}
}
One more thing from Google ARCore software engineers:
Keep objects close to anchors.
When anchoring objects, make sure that they are close to the anchor you are using. Avoid placing objects farther than a few meters from the anchor to prevent unexpected rotational movement due to ARCore's updates to world space coordinates.
If you need to place an object more than a few meters away from an existing anchor, create a new anchor closer to this position and attach the object to the new anchor.

Retrieve bounds of an augmented object

I'm attempting to anchor a ViewRenderable to a ModelRenderable so that it appears on the top right corner of the Renderable & I use the following snippet to place it.
//Sets the button north east of the node that it is attached to
viewRenderable.localPosition = Vector3(
parent.right.x,
parent.up.y,
parent.right.z
)
However, the viewRenderable doesn't really show up as intended. It rather shows up a few floats away from the top right corner of the ModelRenderable. Is there a way I can get the actual bounds of the ModelRenderable so that I can place it better? I want the end result something similar to the below picture.
What I have right now is this image:
My code:
//Creation of ViewRenderable
val shareButton = ViewRenderable.builder()
.setVerticalAlignment(ViewRenderable.VerticalAlignment.TOP)
.setHorizontalAlignment(ViewRenderable.HorizontalAlignment.RIGHT)
.setView(view.getARFragment().context, R.layout.share)
.build()
//Creation of ModelRenderable
val video = ModelRenderable.builder()
.setSource(view.getARFragment().context,
Uri.parse("chroma_key_video.sfb")).build()
//Adding Model to the scene | Extension method called on ParentNode
private fun Node.attachVideoNode(videoRenderable: ModelRenderable?) {
//Show another anchor object
val videoNode = Node()
//Attach the new node with the node that this method was called on
videoNode.setParent(this#attachVideoNode)
//Set local position
videoNode.localPosition = this.right //This works fine. The video is rendered next to another ModelRenderable.
// Set the scale of the node so that the aspect ratio of the video is correct.
val videoWidth = mediaPlayer?.videoWidth?.toFloat()
val videoHeight = mediaPlayer?.videoHeight?.toFloat()
videoHeight?.let {
videoWidth?.apply {
videoNode.localScale = Vector3(
(this#apply / it), 1.0f, 1.0f)
}
}
// Start playing the video when the first node is placed.
mediaPlayer?.apply {
if (!isPlaying) {
start()
// Wait to set the renderable until the first frame of the video becomes available.
// This prevents the renderable from briefly appearing as a black quad before the video
// plays.
texture?.surfaceTexture
?.setOnFrameAvailableListener { surfaceTexture ->
videoNode.renderable = videoRenderable
surfaceTexture.setOnFrameAvailableListener(null)
//Attach a share button
videoNode.attachShareButton()
ARApplication.log("The local rotation of this videoRenderable is ${videoNode.localRotation}")
}
} else
videoNode.renderable = videoRenderable
}
}
//Attaches a share button to any node that it is called on. | Extension method called on VideoNode
private fun Node.attachShareButton() {
//Create a new node to display the close button
var closeButton = Node()
closeButton.setParent(this)
ARApplication.log("The close button has been added to the scene at world coordinates: ${closeButton.worldPosition}")
//Set the close button as a renderable
closeButton.renderable = view.getViewRenderable()
}

I don't know exactly what you already got, but this is a solution for put a text on the ViewRenderable on the top and on the right
x x V
x O x
x x x
V - ViewRenderable
O - ModelRenderable
ViewRenderable.builder()
.setView(context, R.layout.VIEW_LAYOUT_XXXXX)
.setHorizontalAlignment(ViewRenderable.HorizontalAlignment.RIGHT) //Here is where the magic happens
.build()
.thenAccept(
(renderable) -> {
YOUR_NODE_XXXXX.setRenderable(renderable);
TextView textView = (TextView) renderable.getView();
textView.setText(YOUR_TEXT_XXXXX);
})
.exceptionally(
(throwable) -> {
throw new AssertionError(R.string.ERROR_MSG_XXXXX, throwable);
});
I let a _XXXXXin everything that you should change with your code.
Result should be like this
EDIT. Ok looking your code now this is what you should do:
Firs you need to inicialize your view (I think your "close button" should be your share button)
private fun Node.attachShareButton() {
if(shareButton == null){ //Create a new node to display the share button
var shareButton = Node()
shareButton.setParent(this)
shareButton.renderable = view.getViewRenderable()
shareButton.setLocalPosition(new Vector3(SIZE_OF_BUTTON, SIZE_OF_BUTTON, SIZE_OF_BUTTON));
}
if (infoCard == null) {
infoCard = new Node();
infoCard.setParent(this);
infoCard.setEnabled(false);
infoCard.setLocalPosition(new Vector3(0.0f, SIZE_OF_BUTTON*0.55f, 0.0f));
ViewRenderable.builder()
.setView(context, R.layout.share)
.setHorizontalAlignment(ViewRenderable.HorizontalAlignment.RIGHT)
.build()
.thenAccept(
(renderable) -> {
infoCard.setRenderable(renderable);
})
.exceptionally(
(throwable) -> {
throw new AssertionError(R.string.ERROR_MSG_XXXXX, throwable);
});
}
shareButton.onTap(HitTestResult hitTestResult, MotionEvent motionEvent){
if (infoCard == null) {
return;
}
infoCard.setEnabled(!infoCard.isEnabled()); //This show your card when the button is click/tap
}
}