Develop Reference

2D Position of marker

Hi there I'm using ARToolKit v6 to render marker based on nft jpg image it works very well but..
I need to get position of marker in screen(view) and then add a custom textView there is that possible ? how to get 2d position of marker based on projectionMatrix and modelViewMatrix ?
Or can I draw a text or image instead of Cube ?
code
#Override
public void draw() {
super.draw();
GLES20.glEnable(GLES20.GL_CULL_FACE);
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
GLES20.glFrontFace(GLES20.GL_CCW);
boolean show = false;
for (int trackableUID : trackableUIDs) {
if (ARToolKit.getInstance().queryMarkerVisible(trackableUIDs.get(trackableUID))) {
float[] projectionMatrix = ARToolKit.getInstance().getProjectionMatrix();
float[] modelViewMatrix = ARToolKit.getInstance().queryMarkerTransformation(trackableUIDs.get(trackableUID));
Log.i("INFOOOOO", projectionMatrix.toString());
Log.i("INFOOOOO", modelViewMatrix.toString());
cube.draw(projectionMatrix, modelViewMatrix);
show = true;
}
}
}

#sturner thanks so much , I found a solution below is the code if someone is looking for solution
final float[] projectionMatrix = ARToolKit.getInstance().getProjectionMatrix();
final float[] modelViewMatrix = ARToolKit.getInstance().queryMarkerTransformation(trackableUIDs.get(trackableUID));
if (view == null) {
view = new int[4];
view[0] = 0;
view[1] = 0;
view[2] = ARTrackingActivity.width;
view[3] = ARTrackingActivity.height;
}
int i = GLU.gluProject(0, 0, 0, modelViewMatrix, 0, projectionMatrix, 0, view, 0, floats, 0);
if (i == GLES20.GL_TRUE) {
// draw the object in screen
}

Using addMarker with ARToolKit

I am currently working on an android app, and I want to include ARToolKit to scan pictures and show 3d models.
So I added the ARToolKit 6 to my Android Studio and I created the program itself fast. Next I wanted to add markers with addMarker, so basically my code was this.
class ARTrackingRenderer extends ARRenderer {
private static final class Trackable{
String name;
float height;
Trackable(String name, float height)
{
this.name = name;
this.height = height;
}
}
private static final Trackable trackables[] = new Trackable[]{
new Trackable("Alterra_Ticket_1.jpg", 95.3f),
new Trackable("Alterra_Postcard_2.jpg", 95.3f),
new Trackable("Alterra_Postcard_3.jpg", 127.0f),
new Trackable("Alterra_Postcard_4.jpg", 95.3f)
};
private int trackableUIDs[] = new int[trackables.length];
private Cube cube;
private Cube cube2;
#Override
public boolean configureARScene() {
int i = 0;
for (Trackable trackable : trackables){
trackableUIDs[i] = ARToolKit.getInstance().addMarker("2d;Data/2d/" + trackable.name + ";" + trackable.height);
if (trackableUIDs[1] < 0) return false;
}
NativeInterface.arwSetTrackerOptionInt(NativeInterface.ARW_TRACKER_OPTION_2D_MAX_IMAGES, trackables.length);
return true;
}
//Shader calls should be within a GL thread. GL threads are onSurfaceChanged(), onSurfaceCreated() or onDravFrame()
//As the cube instantiates the shader during setShaderProgram call we need to create the cube here.
#Override
public void onSurfaceCreated(GL10 unused, javax.microedition.khronos.egl.EGLConfig config){
this.shaderProgram = new SimpleShaderProgram(new SimpleVertexShader(), new SimpleFragmentShader());
cube = new Cube(30.0f, 0.0f, 0.0f, 0.0f); //Das erste ist die Größe, das zweite die horizontale Ausrichtung, das dritte die vertikale und das vierte der Abstand vom boden aus
cube2 = new Cube(15.0f, 50.0f, 0.0f, 20.0f);
cube.setShaderProgram(shaderProgram);
cube2.setShaderProgram(shaderProgram);
super.onSurfaceCreated(unused, config);
}
#Override
public void draw(){
super.draw();
GLES20.glEnable(GLES20.GL_CULL_FACE);
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
GLES20.glFrontFace(GLES20.GL_CCW);
for (int trackableUID:trackableUIDs){
if (ARToolKit.getInstance().queryMarkerVisible(trackableUID)){
float[] projectionMatrix = ARToolKit.getInstance().getProjectionMatrix();
float[] modelVievMatrix = ARToolKit.getInstance().queryMarkerTransformation(trackableUID);
cube.draw(projectionMatrix, modelVievMatrix);
cube2.draw(projectionMatrix, modelVievMatrix);
}
}
}
}
I got the code from a Tutorial on YouTube. I want to add new markers. How can I do this? Or basically just how can I add new pictures, so it wont scan everytime the alterra pictures?
Thanks for all help.

This code:
private static final Trackable trackables[] = new Trackable[]{
new Trackable("Alterra_Ticket_1.jpg", 95.3f),
new Trackable("Alterra_Postcard_2.jpg", 95.3f),
new Trackable("Alterra_Postcard_3.jpg", 127.0f),
new Trackable("Alterra_Postcard_4.jpg", 95.3f)
};
Is responsible for adding the 'markers' called Trackable to ARToolKit. You can add your markers to the trackables array.
The .jpg files need to be available in the assets directory parallel to the Alterra*.jpg files.
Only .jpg files are possible.

How to render 3D object within a Libgdx Actor?

Most of the Libgdx tutorials I found show how to add 2D elements in a 3D world, but I would like to know how to the the opposite, adding 3D elements in a 2D Stage.
I tried adding a background image to the Stage, then adding to the Stage an Actor that renders the model batch and the 3D instances in its draw() method.
But instead, the image isn't drawn and part of the 3D object is hidden.
SimpleGame class
public class SimpleGame extends ApplicationAdapter {
Stage stage;
#Override
public void create () {
stage = new Stage();
InputMultiplexer im = new InputMultiplexer(stage);
Gdx.input.setInputProcessor( im );
Image background = new Image(new Texture("badlogic.jpg"));
background.setSize(stage.getWidth(), stage.getHeight());
stage.addActor(background);
setup();
}
private void setup() {
SimpleActor3D group = new SimpleActor3D();
group.setSize(stage.getWidth(), stage.getHeight());
group.setPosition(0, 0);
stage.addActor(group);
}
#Override
public void render () {
stage.act();
Gdx.gl.glClearColor(1, 1, 1, 1);
Gdx.gl.glViewport(0, 0, Gdx.graphics.getWidth(), Gdx.graphics.getHeight());
Gdx.gl.glClear( GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT );
stage.draw();
}
}
SimpleActor3D class
public class SimpleActor3D extends Actor {
public Environment environment;
public PerspectiveCamera camera;
public ModelBatch modelBatch;
public ModelInstance boxInstance;
public SimpleActor3D() {
environment = SimpleUtils.createEnvironment();
camera = SimpleUtils.createCamera();
boxInstance = SimpleUtils.createModelInstance(Color.GREEN);
modelBatch = new ModelBatch();
}
#Override
public void draw(Batch batch, float parentAlpha) {
Gdx.gl.glViewport((int)getX(), (int)getY(), (int)getWidth(), (int)getHeight());
modelBatch.begin(camera);
modelBatch.render( boxInstance, environment );
modelBatch.end();
super.draw(batch, parentAlpha);
}
}
SimpleUtils class
public class SimpleUtils {
public static Environment createEnvironment() {
Environment environment = new Environment();
environment.set( new ColorAttribute(ColorAttribute.AmbientLight, 0.4f, 0.4f, 0.4f, 1f) );
DirectionalLight dLight = new DirectionalLight();
Color lightColor = new Color(0.75f, 0.75f, 0.75f, 1);
Vector3 lightVector = new Vector3(-1.0f, -0.75f, -0.25f);
dLight.set( lightColor, lightVector );
environment.add( dLight ) ;
return environment;
}
public static PerspectiveCamera createCamera() {
PerspectiveCamera camera = new PerspectiveCamera(67, Gdx.graphics.getWidth(), Gdx.graphics.getHeight());
camera.position.set(10f, 10f, 10f);
camera.lookAt(0,0,0);
camera.near = 1f;
camera.far = 300f;
camera.update();
return camera;
}
public static ModelInstance createModelInstance(Color color) {
ModelBuilder modelBuilder = new ModelBuilder();
Material boxMaterial = new Material();
boxMaterial.set( ColorAttribute.createDiffuse(color) );
int usageCode = VertexAttributes.Usage.Position + VertexAttributes.Usage.ColorPacked + VertexAttributes.Usage.Normal;
Model boxModel = modelBuilder.createBox( 5f, 5f, 5f, boxMaterial, usageCode );
return new ModelInstance(boxModel);
}
}
What I would like :
What I have instead :
I have tried rendering the model batch directly in the ApplicationAdapter render() method and it works perfectly, so the problems must lie somewhere with the Stage but I can't find how.

I had the same problem but I needed to render 3d object only once so I came with an idea to render 3d model as a Sprite. In order to do that I rendered my model via modelBatch to frame buffer object instead of default screen buffer and then created a sprite from FBO color buffer.
Sample code below:
FrameBuffer frameBuffer = new FrameBuffer(Pixmap.Format.RGBA8888, Gdx.graphics.getBackBufferWidth(), Gdx.graphics.getBackBufferHeight(), true);
Sprite renderModel(ModelInstance modelInstance) {
frameBuffer.begin(); //Capture rendering to frame buffer.
Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT | (Gdx.graphics.getBufferFormat().coverageSampling ? GL20.GL_COVERAGE_BUFFER_BIT_NV : 0))
modelBatch.begin(camera);
modelBatch.render(modelInstance);
modelBatch.end();
frameBuffer.end();
return new Sprite(frameBuffer.getColorBufferTexture());
}
You can always update your sprite texture in a render loop with use of sprite.setTexture(); method. You can also create an Image from a texture -> new Image(frameBuffer.getColorBufferTexture()); and use it in Scene2d.

Integrating jpct-ae to Qualcomm's Vuforia engine in android

In Android, I am trying to integrate JPCT to Vuforia by using this tutorial:
http://www.jpct.net/wiki/index.php/Integrating_JPCT-AE_with_Vuforia
The first time the application is launched, it works, but when I go back and I touch "play" again, it crashes.
These are the errors in my LogCat when the application crashes:
FATAL EXCEPTION: main
java.lang.RuntimeException: [ 1362671862690 ] - ERROR: A texture with the name 'texture' has been declared twice!
at com.threed.jpct.Logger.log(Logger.java:189)
at com.threed.jpct.TextureManager.addTexture(TextureManager.java:138)
at com.qualcomm.QCARSamples.ImageTargets.ImageTargetsRenderer.<init> (ImageTargetsRenderer.java:78)
at com.qualcomm.QCARSamples.ImageTargets.ImageTargets.initApplicationAR(ImageTargets.java:807)
at com.qualcomm.QCARSamples.ImageTargets.ImageTargets.updateApplicationStatus(ImageTargets.java:649)
at com.qualcomm.QCARSamples.ImageTargets.ImageTargets.updateApplicationStatus(ImageTargets.java:641)
at com.qualcomm.QCARSamples.ImageTargets.ImageTargets.access$3(ImageTargets.java:598)
at com.qualcomm.QCARSamples.ImageTargets.ImageTargets$InitQCARTask.onPostExecute(ImageTargets.java:226)
at com.qualcomm.QCARSamples.ImageTargets.ImageTargets$InitQCARTask.onPostExecute(ImageTargets.java:1)
at android.os.AsyncTask.finish(AsyncTask.java:417)
at android.os.AsyncTask.access$300(AsyncTask.java:127)
at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:429)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:123)
at android.app.ActivityThread.main(ActivityThread.java:3691)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:507)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:847)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:605)
at dalvik.system.NativeStart.main(Native Method)
Here is Imagetargetsrenderer.java code
public class ImageTargetsRenderer implements GLSurfaceView.Renderer
{
public boolean mIsActive = false;
/** Reference to main activity **/
public ImageTargets mActivity;
/** Native function for initializing the renderer. */
public native void initRendering();
/** Native function to update the renderer. */
public native void updateRendering(int width, int height);
private World world=null;
private Light sun = null;
private Object3D cube = null;
private FrameBuffer fb = null;
private float[] modelViewMat=null;
private Camera cam=null;
private float fov=0;
private float fovy=0;
//private Camera cam=null;
private Object3D plane=null;
public ImageTargetsRenderer(ImageTargets activity){
this.mActivity = activity;
world = new World();
world.setAmbientLight(20, 20, 20);
sun = new Light(world);
sun.setIntensity(250, 250, 250);
// Create a texture out of the icon...:-)
Texture texture = new Texture(BitmapHelper.rescale(BitmapHelper.convert(mActivity.getResources().getDrawable(R.drawable.ic_launcher)), 64, 64));
TextureManager.getInstance().addTexture("texture", texture);
cube = Primitives.getCube(10);
cube.calcTextureWrapSpherical();
cube.setTexture("texture");
cube.strip();
cube.build();
world.addObject(cube);
cam = world.getCamera();
/*cam.moveCamera(Camera.CAMERA_MOVEOUT, 50);
cam.lookAt(cube.getTransformedCenter());*/
SimpleVector sv = new SimpleVector();
SimpleVector position=new SimpleVector();
position.x=0;
position.y=0;
position.z=-10;
cube.setOrigin(position);
sv.set(cube.getTransformedCenter());
sv.y -= 100;
sv.z -= 100;
sun.setPosition(sv);
MemoryHelper.compact();
}
/** Called when the surface is created or recreated. */
public void onSurfaceCreated(GL10 gl, EGLConfig config)
{
DebugLog.LOGD("GLRenderer::onSurfaceCreated");
// Call native function to initialize rendering:
initRendering();
// Call QCAR function to (re)initialize rendering after first use
// or after OpenGL ES context was lost (e.g. after onPause/onResume):
QCAR.onSurfaceCreated();
}
/** Called when the surface changed size. */
public void onSurfaceChanged(GL10 gl, int width, int height)
{
DebugLog.LOGD("GLRenderer::onSurfaceChanged");
// Call native function to update rendering when render surface
// parameters have changed:
updateRendering(width, height);
// Call QCAR function to handle render surface size changes:
QCAR.onSurfaceChanged(width, height);
if (fb != null) {
fb.dispose();
}
fb = new FrameBuffer(width, height);
}
/** The native render function. */
public native void renderFrame();
/** Called to draw the current frame. */
public void onDrawFrame(GL10 gl)
{
if (!mIsActive)
return;
// Update render view (projection matrix and viewport) if needed:
mActivity.updateRenderView();
//updateCamera();
// Call our native function to render content
renderFrame();
world.renderScene(fb);
world.draw(fb);
fb.display();
}
public void updateModelviewMatrix(float mat[]) {
modelViewMat = mat;
}
public void setFov(float fov_) {
fov = fov_;
}
public void setFovy(float fovy_) {
fovy = fovy_;
}
public void updateCamera() {
Matrix m = new Matrix();
m.setDump(modelViewMat);
cam.setBack(m);
cam.setFOV(fov);
cam.setYFOV(fovy);
}
}
Code for imagetargets.cpp
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *env, jobject obj)
{
const QCAR::CameraCalibration& cameraCalibration = QCAR::CameraDevice::getInstance().getCameraCalibration();
QCAR::Vec2F size = cameraCalibration.getSize();
QCAR::Vec2F focalLength = cameraCalibration.getFocalLength();
float fovyRadians = 2 * atan(0.5f * size.data[1] / focalLength.data[1]);
float fovRadians = 2 * atan(0.5f * size.data[0] / focalLength.data[0]);
jclass activityClass = env->GetObjectClass(obj);
jfloatArray modelviewArray = env->NewFloatArray(16);
jmethodID updateMatrixMethod = env->GetMethodID(activityClass, "updateModelviewMatrix", "([F)V");
jmethodID fovMethod = env->GetMethodID(activityClass, "setFov", "(F)V");
jmethodID fovyMethod = env->GetMethodID(activityClass, "setFovy", "(F)V");
// test
jclass newClass = env->GetObjectClass(obj);
jmethodID updateCameraMethod = env->GetMethodID(newClass, "updateCamera", "()V");
// Clear color and depth buffer
//glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// Get the state from QCAR and mark the beginning of a rendering section
QCAR::State state = QCAR::Renderer::getInstance().begin();
// Explicitly render the Video Background
QCAR::Renderer::getInstance().drawVideoBackground();
// Did we find any trackables this frame?
for(int tIdx = 0; tIdx < state.getNumTrackableResults(); tIdx++)
{
// Get the trackable:
const QCAR::TrackableResult* result = state.getTrackableResult(tIdx);
const QCAR::Trackable& trackable = result->getTrackable();
QCAR::Matrix44F modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(result- >getPose());
}
QCAR::Renderer::getInstance().end();
for(int tIdx = 0; tIdx < state.getNumTrackableResults(); tIdx++)
{
// Get the trackable:
const QCAR::TrackableResult* result = state.getTrackableResult(tIdx);
const QCAR::Trackable& trackable = result->getTrackable();
QCAR::Matrix44F modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(result- >getPose());
SampleUtils::rotatePoseMatrix(180.0f, 1.0f, 0, 0, &modelViewMatrix.data[0]);
// Passes the model view matrix to java
env->SetFloatArrayRegion(modelviewArray, 0, 16, modelViewMatrix.data);
env->CallVoidMethod(obj, updateMatrixMethod , modelviewArray);
env->CallVoidMethod(obj, updateCameraMethod);
env->CallVoidMethod(obj, fovMethod, fovRadians);
env->CallVoidMethod(obj, fovyMethod, fovyRadians);
}
env->DeleteLocalRef(modelviewArray);
}
What does that exception mean?

The beginning of your renderFrame method in ImageTarget.cpp should be this way:
jclass activityClass = env->GetObjectClass(obj);
jfloatArray modelviewArray = env->NewFloatArray(16);
jmethodID method = env->GetMethodID(activityClass, "updateModelviewMatrix", "([F)V");
This, I think, is gonna solve your error for activityClass was not declared in this scope
Comment out this line and test again. You don't need it anymore.
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
I would also recommend to comment out the renderFrame() method in onDrawFrame() to see if jPCT can render the cube if QCAR hasn't started the rendering earlier. (just for testing purpose)
Not to mention, QCAR initially changes the OpenGL states by default. Therefor, you have to enable some of them in order to render with jPCT. Check OpenGL State Changes in Video Background Renderer for more info.
I'm using this for OpenGL ES 1.x after I call renderFrame in onDrawFrame:
GL11 gl11 = (GL11) gl;
gl11.glEnable(GL11.GL_DEPTH_TEST);
gl11.glEnable(GL11.GL_CULL_FACE);
gl11.glTexEnvi(GL11.GL_TEXTURE_ENV, GL11.GL_TEXTURE_ENV_MODE, GL11.GL_MODULATE);
gl11.glEnable(GL11.GL_LIGHTING);
gl11.glEnable(GL11.GL_BLEND);

In order to see something before applying the matrix, you must first tell the camera to lookAt the object.
Camera cam = world.getCamera();
cam.moveCamera(Camera.CAMERA_MOVEOUT, 50);
cam.lookAt(cube.getTransformedCenter());
Be aware that you should remove these lines when you update the camera with the modelview matrix from the marker.
If you follow my tutorial you actually don't have to activate any OpenGL states to see something over the marker (although you might be interested in activating them as Sam Rad suggested, for other reasons).

Add this to Renderer for removing loaded Textures
public void cleanup()
{
TextureManager.getInstance().removeTexture("texture");
}
Call this from Activity while closing/pausing
protected void onPause()
{
mRenderer.cleanup();
}

ClassCaseException on GL11ExtensionPack. Rendering to texture on OpenGL android

I'm trying to render to a texture (really thought it would be easier than this!)
I found this resource: which seems to be exactly what I want
I'm getting a ClassCastException however, on GL11ExtensionPack gl11ep = (GL11ExtensionPack) gl;
Can anyone tell me why?
public void renderToTexture(GLRenderer glRenderer, GL10 gl)
{
boolean checkIfContextSupportsExtension = checkIfContextSupportsExtension(gl, "GL_OES_framebuffer_object");
if(checkIfContextSupportsExtension)
{
GL11ExtensionPack gl11ep = (GL11ExtensionPack) gl;
int mFrameBuffer = createFrameBuffer(gl,texture.getWidth(), texture.getHeight(), texture.getGLID());
gl11ep.glBindFramebufferOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES, mFrameBuffer);
gl.glClearColor(0f, 1f, 0f, 1f);
gl11ep.glBindFramebufferOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES, 0);
}
}
Issue fixed by removing setDebugFlags(GLSurfaceView.DEBUG_LOG_GL_CALLS);
However, its still not modifying the texture. I modified the code so all it does is clear the colour green, so the texture should become all green. Here is the createFrameBuffer method. I don't really understand everything thats going on in this :/
private int createFrameBuffer(GL10 gl, int width, int height, int targetTextureId) {
GL11ExtensionPack gl11ep = (GL11ExtensionPack) gl;
int framebuffer;
int[] framebuffers = new int[1];
gl11ep.glGenFramebuffersOES(1, framebuffers, 0);
framebuffer = framebuffers[0];
gl11ep.glBindFramebufferOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES, framebuffer);
int depthbuffer;
int[] renderbuffers = new int[1];
gl11ep.glGenRenderbuffersOES(1, renderbuffers, 0);
depthbuffer = renderbuffers[0];
gl11ep.glBindRenderbufferOES(GL11ExtensionPack.GL_RENDERBUFFER_OES, depthbuffer);
gl11ep.glRenderbufferStorageOES(GL11ExtensionPack.GL_RENDERBUFFER_OES,
GL11ExtensionPack.GL_DEPTH_COMPONENT16, width, height);
gl11ep.glFramebufferRenderbufferOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES,
GL11ExtensionPack.GL_DEPTH_ATTACHMENT_OES,
GL11ExtensionPack.GL_RENDERBUFFER_OES, depthbuffer);
gl11ep.glFramebufferTexture2DOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES,
GL11ExtensionPack.GL_COLOR_ATTACHMENT0_OES, GL10.GL_TEXTURE_2D,
targetTextureId, 0);
int status = gl11ep.glCheckFramebufferStatusOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES);
if (status != GL11ExtensionPack.GL_FRAMEBUFFER_COMPLETE_OES) {
//throw new RuntimeException("Framebuffer is not complete: " +
// Integer.toHexString(status));
Log.v("error","Frame buffer not complete");
} else {
gl11ep.glBindFramebufferOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES, 0);
return -1;
}
return framebuffer;
}

Calling setDebugFlags on a GLSurfaceView to wrap the GL instance has/had the unfortunate sideeffect to not implementing all GL* interfaces. Unfortunately the issue does not state in which version this is fixed.