Develop Reference

Off-center camera projection for 3d illusion using LibGDX

I'm trying to achieve this 3d pop out of the screen kind of effect using LibGDX on Android (the camera process described at the given link):
https://www.anxious-bored.com/blog/
but the result I get is stretched objects when the eye moves around.
I simulate the eye position using a gyroscope to get device rotation and assume a fixed distance from the device screen. I've also incorporated ARCore for eye tracking, but that also yields the same result.
Here is a GIF of what I'm seeing:
https://i.stack.imgur.com/YOf6n.gif
Anyone have any idea on what I'm doing wrong?
Here is the relevant code I'm using in Kotlin.
private fun updateCamera() {
val camera = // The Perspective camera instance
// Move the camera to the eye position, but keep the same camera direction
camera.view.setToLookAt(camera.direction, camera.up).mul(GDXHelperInstances.matrix4_1.setToTranslation(GDXHelperInstances.vector3_1.set(camera.position).scl(-1f)))
// Get the size of the screen in meters
val deviceHalfHeight = camera.viewportHeight / Gdx.graphics.ppcY * 0.01f
val deviceHalfWidth = camera.viewportWidth / Gdx.graphics.ppcX * 0.01f
// Get the device position and plane relative to the camera
val pos = Vector3(Vector3.Zero).mul(camera.view)
val plane = Plane(GDXHelperInstances.vector3_1.set(camera.direction).scl(-1f), Vector3.Zero)
// Calculate the bounds of the viewport in virtual space (the device screen dimensions in meters with center at Vector3.Zero) relative to the camera (view space)
val left = pos.x - deviceHalfWidth
val right = pos.x + deviceHalfWidth
val bottom = pos.y - deviceHalfHeight
val top = pos.y + deviceHalfHeight
val nearScale = camera.near / plane.distance(camera.position).absoluteValue
// Off-axis projection
camera.projection.setToProjection(left*nearScale, right*nearScale, bottom*nearScale, top*nearScale, camera.near, camera.far)
// Calculate new asymmetrical frustum
camera.combined.set(camera.projection)
Matrix4.mul(camera.combined.`val`, camera.view.`val`)
camera.invProjectionView.set(camera.combined)
Matrix4.inv(camera.invProjectionView.`val`)
camera.frustum.update(camera.invProjectionView)
}

Reduce tracking window using google mlkit vision samples

I would like to reduce the reduce bar code tracking window when using the google vision api. There are some answers here but they feel a bit outdated.
I'm using google's sample: https://github.com/googlesamples/mlkit/tree/master/android/vision-quickstart
Currently, I try to figure out if a barcode is inside my overlay box inside BarcodeScannerProcessor onSuccess callback:
override fun onSuccess(barcodes: List<Barcode>, graphicOverlay: GraphicOverlay) {
if(barcodes.isEmpty())
return;
for(barcode in barcodes) {
val center = Point(graphicOverlay.imageWidth / 2, graphicOverlay.imageHeight / 2)
val rectWidth = graphicOverlay.imageWidth * Settings.OverlayWidthFactor
val rectHeight = graphicOverlay.imageHeight * Settings.OverlayHeightFactor
val left = center.x - rectWidth / 2
val top = center.y - rectHeight / 2
val right = center.x + rectWidth / 2
val bottom = center.y + rectHeight / 2
val rect = Rect(left.toInt(), top.toInt(), right.toInt(), bottom.toInt())
val contains = rect.contains(barcode.boundingBox!!)
val color = if(contains) Color.GREEN else Color.RED
graphicOverlay.add(BarcodeGraphic(graphicOverlay, barcode, "left: ${barcode.boundingBox!!.left}", color))
}
}
Y-wise it works perfectly, but the X values from barcode.boundingBox e.g. barcode.boundingBox.left seems to have an offset. Is it based on what's being calculated in GraphicOverlay?
I'm expecting the value below to be close to 0, but the offset is about 90 here:
Or perhaps it's more efficient to crop the image according to the box?

Actually the bounding box is correct. The trick is that the image aspect ratio doesn't match the viewport aspect ratio so the image is cropped horizontally. Try to open settings (a gear in the top right corner) and choose an appropriate resolution.
For example take a look at these two screenshots. On the first one the selected resolution (1080x1920) matches my phone resolution so the padding looks good (17px). On the second screenshot the aspect ratio is different (1.0 for 720x720 resolution) therefore the image is cropped and the padding looks incorrect.
So the offset should be transformed from image coordinates to the screen coordinates. Under the hood GraphicOverlay uses a matrix for this transformation. You can use the same matrix:
for(barcode in barcodes) {
barcode.boundingBox?.let { bbox ->
val offset = floatArrayOf(bbox.left.toFloat(), bbox.top.toFloat())
graphicOverlay.transformationMatrix.mapPoints(offset)
val leftOffset = offset[0]
val topOffset = offset[1]
...
}
}
The only thing is that the transformationMatrix is private, so you should add a getter to access it.

As you know, the preview size of the camera is configurable at the settings menu. This configurable size specifies the graphicOverlay dimensions.
On the other hand, the aspect ratio of the CameraSourcePreview (i.e. preview_view in activity_vision_live_preview.xml) which is shown on the screen, does not necessarily equal to the ratio of the graphicOverlay. Because depends on the size of the phone's screen and the height that the parent ConstraintLayout allows occupying.
So, in the preview, based on the difference between the aspect ratio of graphicOverlay and preview_view, some part of the graphicOverlay might not be shown horizontally or vertically.
There are some parameters inside GraphicOverlay that can help us to adjust the left and top of the barcode's boundingBox in such a way that they start from 0 in the visible area.
First of all, they should be accessible out of the GraphicOverlay class. So, it's just enough to write a getter method for them:
GraphicOverlay.java
public class GraphicOverlay extends View {
...
/**
* The factor of overlay View size to image size. Anything in the image coordinates need to be
* scaled by this amount to fit with the area of overlay View.
*/
public float getScaleFactor() {
return scaleFactor;
}
/**
* The number of vertical pixels needed to be cropped on each side to fit the image with the
* area of overlay View after scaling.
*/
public float getPostScaleHeightOffset() {
return postScaleHeightOffset;
}
/**
* The number of horizontal pixels needed to be cropped on each side to fit the image with the
* area of overlay View after scaling.
*/
public float getPostScaleWidthOffset() {
return postScaleWidthOffset;
}
}
Now, it is possible to calculate the left and top difference gap using these parameters like the following:
BarcodeScannerProcessor.kt
class BarcodeScannerProcessor(
context: Context
) : VisionProcessorBase<List<Barcode>>(context) {
...
override fun onSuccess(barcodes: List<Barcode>, graphicOverlay: GraphicOverlay) {
if (barcodes.isEmpty()) {
Log.v(MANUAL_TESTING_LOG, "No barcode has been detected")
}
val leftDiff = graphicOverlay.run { postScaleWidthOffset / scaleFactor }.toInt()
val topDiff = graphicOverlay.run { postScaleHeightOffset / scaleFactor }.toInt()
for (i in barcodes.indices) {
val barcode = barcodes[i]
val color = Color.RED
val text = "left: ${barcode.boundingBox!!.left - leftDiff} top: ${barcode.boundingBox!!.top - topDiff}"
graphicOverlay.add(MyBarcodeGraphic(graphicOverlay, barcode, text, color))
logExtrasForTesting(barcode)
}
}
...
}
Visual Result:
Here is the visual result of the output. As it's obvious in the pictures, the gap between both left & top of the barcode and the left and top of the visible area is started from 0. In the case of the left picture, the graphicOverlay is set to the size of 480x640 (aspect ratio ≈ 1.3334) and for the right one 360x640 (aspect ratio ≈ 1.7778). In both cases, on my phone, the CameraSourcePreview has a steady size of 1440x2056 pixels (aspect ratio ≈ 1.4278), so it means that the calculation truly reflected the position of the barcode in the visible area.
(note that the aspect ratio of the visible area in one experiment is lower than that of graphicOverlay, and in another experiment, greater: 1.3334 < 1.4278 < 1.7778. So, the left values and top values are adjusted respectively.)

Use x/y coordinates to set bounds of a moving particle

I have two bitmaps that I draw onto the center of a canvas:
One is only a background, it's a spirit level in top view which doesnt move. The second one is a bitmap looking like a "air bubble". When the user tilts the phone, the sensors read the tilt and the air bubble moves according to the sensor values along the x-axis. However, I need to make sure that the air bubble doesnt move too far, e.g out of the background-bitmap.
So I tried to which x coordinate the bubble can travel to,
before I have to set xPos = xPos -1 using trial and error
This works fine on my device.
To clarify: On my phone, the air bubble could move to the coordinate x = 50 from the middle of the screen. This would be the point, where the bitmap is at the very left of the background spirit level.
On a larger phone, the position x = 50 is too far left, and therefore looking like the air bubble travelled out of the water level.
Now I've tried following:
I calculated the area in % in which the air bubble can move. Let's say that
is 70% of the entire width of the bitmap. So I tried to calculate the two x boundary values:
leftBoundary = XmiddlePoint - (backgroundBitmap.getWidth() * 0.35);
rightBoundary = XmiddlePoint + (backgroundBitmap.getWidth() * 0.35);
...which doesnt work when testing with different screen sizes :(
Is it possible to compensate for different screen sizes and densities using absolute coordinates or do I have to rethink my idea?
If you need any information that I forgot about, please let me know. If this question has already been answered, I would appreciate a link :) Thanks in advance!
Edit:
I load my bitmaps like this:
private Bitmap backgroundBitmap;
private static final int BITMAP_WIDTH = 1898;
private static final int BITMAP_HEIGHT = 438;
public class SimulationView extends View implements SensorEventListener{
public SimulationView(Context context){
Bitmap map = BitmapFactory.decodeResource(getResources, R.mipmap.backgroundImage);
backgroundBitmap = Bitmap.createScaledBitmap(map, BITMAP_WIDTH, BITMAP_HEIGHT, true;
}
and draw it like this:
protected void onDraw(Canvas canvas){
canvas.drawBitmap(backgroundBitmap, XmiddlePoint - BITMAP_WIDTH / 2, YmiddlePont - BITMAP_HEIGHT / 2, null);
}
backgroundBitmap.getWidth() and getHeight() prints out the correct sizes.
Calculating like mentioned above would return following boundaries:
DisplayMetrics displayMetrics = new DisplayMetrics();
((Activity) getContext()).getWindowManager().getDefaultDisplay().getMetrics(displayMetrics);
int width = displayMetrics.widthPixels;
//which prints out width = 2392
xMiddlePoint = width / 2;
// = 1196
leftBoundary = xMiddlePoint - (BITMAP.getWidth()* 0.35);
// = 531,7
However, when I use trial and error, the right x coordinate seems to be at around 700.

I've come across a great explanation on how to fix my issue here.
As user AgentKnopf explained, you have to scale coordinates or bitmaps like this:
X = (targetScreenWidth / defaultScreenWidth) * defaultXCoordinate
Y = (targetScreenHeight / defaultScreenHeight) * defaultYCoordinate
which, in my case, translates to:
int defaultScreenWidth = 1920;
int defaultXCoordinate = 333;
DisplayMetrics displayMetrics = new DisplayMetrics();
((Activity) getContext()).getWindowManager().getDefaultDisplay().getMetrics(displayMetrics);
displayWidth = displayMetrics.widthPixels;
leftBoundary = (displayWidth / defaultScreenWidth) * defaultXCoordinates

Libgdx - box2d polygonshape doesn't fit its sprite

I have a body with a polygonshape created using .setasbox but when I run my game the box is a bit bigger than my sprite.
I know setasbox uses half height and half width, I used my scaling constant to convert meters to pixels and I know the sprite has the origin of the axis on the bottom left as well. Despite of that I still have a box with a width a bit larger than the sprite and this gap is the same however I change the size of the box...
This is the code I use to create my box (160 is the constant to scale meters to pixels):
public Block(World w, float halfWidth, float halfHeight, Vector2 position, Texture tex){
world = w;
bodyd = new BodyDef();
bodyd.type = BodyDef.BodyType.KinematicBody;
bodyd.gravityScale = 0;
shape = new PolygonShape();
shape.setAsBox(halfWidth, halfHeight);
fixtured = new FixtureDef();
fixtured.shape = shape;
fixtured.density = DENS;
fixtured.friction = FRIC;
fixtured.restitution = REST;
bodyd.position.set(new Vector2(position.x, position.y));
body = world.createBody(bodyd);
fixture = body.createFixture(fixtured);
body.setUserData(this);
texture = tex;
sprite = new Sprite(texture);
sprite.setSize(halfWidth * 2 * 160, halfHeight*2*160);
sprite.setPosition((body.getPosition().x - halfWidth) * 160, (body.getPosition().y - halfHeight) * 160);
}

can you try using Box2DSprite? its very easy..
https://bitbucket.org/dermetfan/libgdx-utils/wiki/net.dermetfan.gdx.graphics.g2d.Box2DSprite
http://www.java-gaming.org/index.php?topic=29843.0

I don't see anything wrong with your code
did you consider that the size that you put for your sprite is the size of the full sprite not the size of the block inside your sprite
I think this is why your brick sprite is smaller than your brick physic :
unless the your brick has the full size of the sprite then may be the problem is related to something else
hope that was helpful !

Phaser: Make background screen-width instead of content-width

I am making a game using the Phaser Framework for Android. I need to make the canvas fill the entire screen width and then flood it with background color, then put content in the center of screen. Currently, even though I already set the width of the canvas to window width, Phaser still reset it to content width, which is smaller, then filling only the content width with background.
How do I stop Phaser from using content width?
var gameObj = new Phaser.Game($(".parent").width(), 700, Phaser.CANVAS, 'parent');
gameObj.stage.backgroundColor = '#A6E5F5';
gameObj.load.image('prld', 'prl.png');
gameObj.scale.scaleMode = Phaser.ScaleManager.SHOW_ALL;
gameObj.scale.pageAlignHorizontally = true;
gameObj.scale.pageAlignVertically = true;
gameObj.scale.setScreenSize(true);
gameObj.scale.refresh();
gameObj.stage.backgroundColor = '#A6E5F5';
// Load Content Here
gameObj.load.image('h1', 'res1/h1.png');
gameObj.load.image('h2', 'res1/h2.png');
....
g = gameObj.add.group();
g.create(20, 20, 'h1');
g.create(60, 20, 'h2');
....

You need Phaser 2.2+ for this, but if you want the canvas to fill the entire screen space available you would do:
var game = new Phaser.Game("100%", "100%", Phaser.CANVAS, 'parent');
...
game.scale.scaleMode = Phaser.ScaleManager.RESIZE;
Also you should not use pageAlign if you're using 100% dimensions. You also don't need to call refresh or setScreenSize.
When working in the RESIZE scale mode you can add a function to your States called 'resize'. It will be called whenever the screen dimensions change (orientation event perhaps) and be passed 2 parameters: width and height. Use those to adjust your game content layout.
You may want to get the Phaser Scale Manager book, it covers all this in lots more detail (caveat: I wrote it, but it has a "pay what you want" price, including completely free)

i use this code in my game and it work very nice without changing in position of any elements
var targetWidth = 720; // the width of the game we want
var targetHeight = 480; // the height of the game we want
// additional ratios
//Small – 360x240
//Normal – 480x320
//Large – 720x480
//XLarge – 960x640
//XXLarge – 1440x960
var deviceRatio = (window.innerWidth/window.innerHeight); //device aspect ratio
var newRatio = (targetHeight/targetWidth)*deviceRatio; //new ratio to fit the screen
var newWidth = targetWidth*newRatio;
var newHeight = targetHeight;
var gameWidth = newWidth;
var gameHeight = newHeight;
var gameRendrer = Phaser.AUTO;
and this code for Boot state
game.scale.scaleMode = Phaser.ScaleManager.SHOW_ALL;
game.scale.pageAlignHorizontally = true;
game.scale.pageAlignVertically = true;
game.scale.forceLandscape = true;
game.scale.setScreenSize(true);
this for landscape mode if you want to make it for portrait change width and height like
var targetWidth = 480; // the width of the game we want
var targetHeight = 720; // the height of the game we want
also force it to portrait
this code work well in my games