I'm using gstreamer to retrieve a video feed (sent from a RaspPi), and I need to display it into Google Cardboard.
I based my work on the tutorial-3 example of gstreamer. I managed to display my video into a SurfaceView, by giving my Surface (retrieved from SurfaceView.SurfaceHolder.getSurface(), but I now need to connect it with the Google Cardboard.
If I do not mistake, Google Cardboard relies on some SurfaceTexture. So I thought it would be easy to simply get the Surface from the SurfaceTexture using the Surface(SurfaceTexture) constructor.
The problem is that it simply doesn't work. My Google cardboard application Cardboard Pasthrough example, and I haven't touched the OpenGL code, since I don't know anything about it.
While debugging, I found out that there is (at least) one problem with the code I'm using. It seems that the line
GLES20.glActiveTexture(GL_TEXTURE_EXTERNAL_OES);
GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture);
are causing me some trouble since GL_TEXTURE_EXTERNAL_OES isn't in the required range for the glActiveTexture methods (which requires a GL from 0 to GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS).
Here are my logs :
GLConsumer W [unnamed-12520-0] bindTextureImage: clearing GL error: 0x500
Adreno-ES20 W <core_glActiveTexture:348>: GL_INVALID_ENUM
So what's working right now ?
It seems that the video feed is received by gstreamer, that tries to update the Surface (I get some notifications about the onFrameAvailable of the SurfaceTexture that are being updated, and the error logs are only sent since there). However, the screens stays black, as if nothing is updated.
Here are the most interesting parts of my code :
#Override
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
CardboardView cardboardView = (CardboardView) findViewById(R.id.cardboard_view);
cardboardView.setRenderer(this);
setCardboardView(cardboardView);
// Initialize GStreamer and warn if it fails
try {
GStreamer.init(this);
} catch (Exception e) {
//Catch e...
}
mCamera = new float[16];
mView = new float[16];
mHeadView = new float[16];
//gstreamer stuff
nativeInit();
}
#Override
public void onSurfaceCreated(EGLConfig eglConfig) {
Log.d(TAG, "onSurfaceCreated start");
GLES20.glClearColor(0.5f, 0.1f, 0.1f, 0.5f);
ByteBuffer bb = ByteBuffer.allocateDirect(squareVertices.length * 4);
bb.order(ByteOrder.nativeOrder());
vertexBuffer = bb.asFloatBuffer();
vertexBuffer.put(squareVertices);
vertexBuffer.position(0);
ByteBuffer dlb = ByteBuffer.allocateDirect(drawOrder.length * 2);
dlb.order(ByteOrder.nativeOrder());
drawListBuffer = dlb.asShortBuffer();
drawListBuffer.put(drawOrder);
drawListBuffer.position(0);
ByteBuffer bb2 = ByteBuffer.allocateDirect(textureVertices.length * 4);
bb2.order(ByteOrder.nativeOrder());
textureVerticesBuffer = bb2.asFloatBuffer();
textureVerticesBuffer.put(textureVertices);
textureVerticesBuffer.position(0);
int vertexShader = loadGLShader(GLES20.GL_VERTEX_SHADER, vertexShaderCode);
int fragmentShader = loadGLShader(GLES20.GL_FRAGMENT_SHADER, fragmentShaderCode);
mProgram = GLES20.glCreateProgram(); // create empty OpenGL ES Program
GLES20.glAttachShader(mProgram, vertexShader); // add the vertex shader to program
GLES20.glAttachShader(mProgram, fragmentShader); // add the fragment shader to program
GLES20.glLinkProgram(mProgram);
checkGLError("Problem on line "+new Throwable().getStackTrace()[0].getLineNumber());
Log.d(TAG, "Surface created");
texture = createTexture();
initSurface(texture);
}
static private int createTexture()
{
Log.d(TAG + "_cardboard", "createTexture");
int[] texture = new int[1];
GLES20.glGenTextures(1,texture, 0);
checkGLError("GenTextures Problem on line "+new Throwable().getStackTrace()[0].getLineNumber());
GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture[0]);
checkGLError("BindTextures Problem on line "+new Throwable().getStackTrace()[0].getLineNumber());
GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES,
GL10.GL_TEXTURE_MIN_FILTER,GL10.GL_LINEAR);
GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES,
GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES,
GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES,
GL10.GL_TEXTURE_WRAP_T, GL10.GL_CLAMP_TO_EDGE);
checkGLError("Problem on line "+new Throwable().getStackTrace()[0].getLineNumber());
return texture[0];
}
//Give the surface to gstreamer.
private void initSurface(int texture) {
mSurface = new SurfaceTexture(texture);
mSurface.setOnFrameAvailableListener(this);
Log.d(TAG, "OnFrameAvailableListener set");
Surface toto = new Surface(mSurface);
nativeSurfaceInit(toto);
toto.release();
}
//When we need to render
#Override
public void onFrameAvailable(SurfaceTexture surfaceTexture) {
Log.d(TAG, "onFrameAvailable");
this.getCardboardView().requestRender();
}
//Display to cardboard
#Override
public void onNewFrame(HeadTransform headTransform) {
headTransform.getHeadView(mHeadView, 0);
// Build the camera matrix and apply it to the ModelView.
Matrix.setLookAtM(mCamera, 0, 0.0f, 0.0f, 0.01f, 0.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f);
float[] mtx = new float[16];
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
mSurface.updateTexImage();
mSurface.getTransformMatrix(mtx);
float[] test = new float[3];
headTransform.getEulerAngles(test, 0);
//if(networkThread != null){
// networkThread.setRegValue(test);
//}
}
#Override
public void onDrawEye(Eye eye) {
// Log.d(TAG, "onDrawEye");
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
GLES20.glUseProgram(mProgram);
Log.d(TAG, "trying to access " + GL_TEXTURE_EXTERNAL_OES +" out of " + GLES20.GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS);
GLES20.glActiveTexture(GL_TEXTURE_EXTERNAL_OES);
// checkGLError("Problem on line "+new Throwable().getStackTrace()[0].getLineNumber());
GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture);
// checkGLError("Problem on line "+new Throwable().getStackTrace()[0].getLineNumber());
mPositionHandle = GLES20.glGetAttribLocation(mProgram, "position");
GLES20.glEnableVertexAttribArray(mPositionHandle);
// checkGLError("Problem on line "+new Throwable().getStackTrace()[0].getLineNumber());
GLES20.glVertexAttribPointer(mPositionHandle, COORDS_PER_VERTEX, GLES20.GL_FLOAT,
false, vertexStride, vertexBuffer);
// checkGLError("Problem on line "+new Throwable().getStackTrace()[0].getLineNumber());
mTextureCoordHandle = GLES20.glGetAttribLocation(mProgram, "inputTextureCoordinate");
GLES20.glEnableVertexAttribArray(mTextureCoordHandle);
GLES20.glVertexAttribPointer(mTextureCoordHandle, COORDS_PER_VERTEX, GLES20.GL_FLOAT,
false, vertexStride, textureVerticesBuffer);
mColorHandle = GLES20.glGetAttribLocation(mProgram, "s_texture");
GLES20.glDrawElements(GLES20.GL_TRIANGLES, drawOrder.length,
GLES20.GL_UNSIGNED_SHORT, drawListBuffer);
// Disable vertex array
GLES20.glDisableVertexAttribArray(mPositionHandle);
GLES20.glDisableVertexAttribArray(mTextureCoordHandle);
Matrix.multiplyMM(mView, 0, eye.getEyeView(), 0, mCamera, 0);
}
For more code, here is a gist with the main two files : https://gist.github.com/MagicMicky/4caa3ac669215652e40f
edit: When trying to work with the camera app on gstreamer, the same errors are shown in the logcat as the one I described earlier. So this might be nothing of importance...
Related
I just started programming opengl-es 2.0 and I'm currently struggling to find an issue concerned with setting the color of a wavefront object im drawing (https://pastebin.com/cEvpj8rt). The drawing is working just fine until I start to manipulate the color at which point im being confronted with opengl error 1281 and I'm unable to pinpoint the cause in my code. I've broken down the shader code to what I believe is the bare minimum required for the fragment shader to work:
void main() {
gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0);
}
To eliminate any additional source of error I am setting the color with a constant value as can be seen above. I doubt the error lies with the simple code above but is concerned with the code in my adapted renderer implementation. (it is based on the renderer that came with a sample from the ar-core github repo. The full code of the initial renderer can be found here: https://github.com/google-ar/arcore-android-sdk/blob/master/samples/java_arcore_hello_ar/app/src/main/java/com/google/ar/core/examples/java/helloar/rendering/ObjectRenderer.java while the adapted version can be seen here: https://pastebin.com/9cmKVnLV) Below you can find an excerpt of the code responsible for setting up and drawing the object. I reckoned the issue to be connected to the texturing which is why I removed the code.
I know its a bit much to ask for help given my lack of understanding on the matter at hand but I'd be glad for any hint/advice at this point. The error occurs after the first draw in the following method:
public void draw(float[] cameraView, float[] cameraPerspective) {
multiplyMM(mModelViewMatrix, 0, cameraView, 0, mModelMatrix, 0);
multiplyMM(mModelViewProjectionMatrix, 0, cameraPerspective, 0, mModelViewMatrix, 0);
glUseProgram(mProgram);
glBindBuffer(GL_ARRAY_BUFFER, mVertexBufferId);
glVertexAttribPointer(mPositionAttribute, COORDS_PER_VERTEX,
GL_FLOAT, false, 0, mVerticesBaseAddress);
glVertexAttribPointer(mNormalAttribute, 3,
GL_FLOAT, false, 0, mNormalsBaseAddress);
glBindBuffer(GL_ARRAY_BUFFER, 0);
// Set the ModelViewProjection matrix in the shader.
glUniformMatrix4fv(mModelViewUniform, 1,
false, mModelViewMatrix, 0);
glUniformMatrix4fv(mModelViewProjectionUniform, 1,
false, mModelViewProjectionMatrix, 0);
glEnableVertexAttribArray(mPositionAttribute);
glEnableVertexAttribArray(mNormalAttribute);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, mIndexBufferId);
glDrawElements(GL_TRIANGLES, mIndexCount, GL_UNSIGNED_SHORT, 0);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0);
glDisableVertexAttribArray(mPositionAttribute);
glDisableVertexAttribArray(mNormalAttribute);
// this is where the error is detected
OpenGlHelper.checkGLError(TAG, "After draw");
}
Here the method which is used for initialization:
public void createOnGlThread(Context context) throws IOException {
InputStream objInputStream = context.getAssets()
.open(OBJ_ASSET_NAME);
Obj obj = ObjReader.read(objInputStream);
obj = ObjUtils.convertToRenderable(obj);
IntBuffer wideIndices = ObjData.getFaceVertexIndices(obj, 3);
FloatBuffer vertices = ObjData.getVertices(obj);
FloatBuffer texCoords = ObjData.getTexCoords(obj, 2);
FloatBuffer normals = ObjData.getNormals(obj);
ShortBuffer indices = ByteBuffer.allocateDirect(2 * wideIndices.limit())
.order(ByteOrder.nativeOrder()).asShortBuffer();
while (wideIndices.hasRemaining()) {
indices.put((short) wideIndices.get());
}
indices.rewind();
int[] buffers = new int[2];
glGenBuffers(2, buffers, 0);
mVertexBufferId = buffers[0];
mIndexBufferId = buffers[1];
// Load vertex buffer
mVerticesBaseAddress = 0;
mTexCoordsBaseAddress = mVerticesBaseAddress + 4 * vertices.limit();
mNormalsBaseAddress = mTexCoordsBaseAddress + 4 * texCoords.limit();
final int totalBytes = mNormalsBaseAddress + 4 * normals.limit();
glBindBuffer(GL_ARRAY_BUFFER, mVertexBufferId);
glBufferData(GL_ARRAY_BUFFER, totalBytes, null, GL_STATIC_DRAW);
glBufferSubData(GL_ARRAY_BUFFER, mVerticesBaseAddress,
4 * vertices.limit(), vertices);
glBufferSubData(GL_ARRAY_BUFFER, mTexCoordsBaseAddress,
4 * texCoords.limit(), texCoords);
glBufferSubData(GL_ARRAY_BUFFER, mNormalsBaseAddress,
4 * normals.limit(), normals);
glBindBuffer(GL_ARRAY_BUFFER, 0);
// Load index buffer
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, mIndexBufferId);
mIndexCount = indices.limit();
glBufferData(GL_ELEMENT_ARRAY_BUFFER, 2 * mIndexCount,
indices, GL_STATIC_DRAW);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0);
OpenGlHelper.checkGLError(TAG, "OBJ buffer load");
mProgram = glCreateProgram();
glAttachShader(mProgram, OpenGlHelper.loadGLShader(TAG, context,
GL_VERTEX_SHADER, R.raw.sphere_vertex));
glAttachShader(mProgram, OpenGlHelper.loadGLShader(TAG, context,
GL_FRAGMENT_SHADER, R.raw.sphere_fragment));
glLinkProgram(mProgram);
glUseProgram(mProgram);
OpenGlHelper.checkGLError(TAG, "Program creation");
mModelViewUniform = glGetUniformLocation(mProgram, "u_ModelView");
mModelViewProjectionUniform =
glGetUniformLocation(mProgram, "u_ModelViewProjection");
mPositionAttribute = glGetAttribLocation(mProgram, "a_Position");
mNormalAttribute = glGetAttribLocation(mProgram, "a_Normal");
OpenGlHelper.checkGLError(TAG, "Program parameters");
setIdentityM(mModelMatrix, 0);
}
I am drawing a simple 3D shape. The shape is read from the .obj file and the whole shape has one color.
I want to create a possibility to change the color. For example if I tap the button I want to make the structure red.
I've read a lot of tutorials but I can't figure out how to change the color (redraw element?) during the runtime. I've tried something like that in the onDrawFrame
gl.glEnableClientState(GL10.GL_COLOR_ARRAY);
gl.glMaterialfv(GL10.GL_FRONT_AND_BACK, GL10.GL_AMBIENT, colorBuffer);
gl.glDisableClientState(GL10.GL_COLOR_ARRAY);
But nothing is changing.
Where and how should I need to implement such action ?
//update
draw method
if(!initialized) {
init(gl);
initialized = true;
}
if(glCameraMatrixBuffer != null) {
glMatrixBuffer.put(glMatrix);
glMatrixBuffer.position(0);
//argDrawMode3D
gl.glMatrixMode(GL10.GL_MODELVIEW);
gl.glLoadIdentity();
//argDraw3dCamera
gl.glMatrixMode(GL10.GL_PROJECTION);
gl.glLoadMatrixf( glCameraMatrixBuffer );
gl.glMatrixMode(GL10.GL_MODELVIEW);
gl.glLoadMatrixf(glMatrixBuffer);
}
gl.glEnableClientState(GL10.GL_NORMAL_ARRAY);
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_COLOR_ARRAY);
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, vertexBuffer);
gl.glMaterialfv(GL10.GL_FRONT_AND_BACK, GL10.GL_AMBIENT, colorBuffer);
for(int i=0; i<parts.size(); i++){
TDModelPart t=parts.get(i);
Material m=t.getMaterial();
if(m!=null){
FloatBuffer a=m.getAmbientColorBuffer();
FloatBuffer d=m.getDiffuseColorBuffer();
FloatBuffer s=m.getSpecularColorBuffer();
gl.glMaterialfv(GL10.GL_FRONT_AND_BACK,GL10.GL_AMBIENT,a);
gl.glMaterialfv(GL10.GL_FRONT_AND_BACK,GL10.GL_SPECULAR,s);
gl.glMaterialfv(GL10.GL_FRONT_AND_BACK,GL10.GL_DIFFUSE,d);
}
gl.glNormalPointer(GL10.GL_FLOAT, 0, t.getNormalBuffer());
gl.glDrawElements(GL10.GL_TRIANGLES,t.getFacesCount(),GL10.GL_UNSIGNED_SHORT,t.getFaceBuffer());
}
gl.glDisableClientState(GL10.GL_COLOR_ARRAY);
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
gl.glDisableClientState(GL10.GL_NORMAL_ARRAY);
public final void onDrawFrame(GL10 gl) {
gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
if (DEBUG) {
gl = (GL10) GLDebugHelper.wrap(gl, GLDebugHelper.CONFIG_CHECK_GL_ERROR, log);
}
setupDraw2D(gl);
gl.glDisable(GL10.GL_DEPTH_TEST);
gl.glEnable(GL10.GL_TEXTURE_2D);
gl.glDisable(GL10.GL_LIGHTING);
gl.glBindTexture(GL10.GL_TEXTURE_2D, textureName);
//load new preview frame as a texture, if needed
if (frameEnqueued) {
frameLock.lock();
if (!isTextureInitialized) {
initializeTexture(gl);
} else {
//just update the image
//can we just update a portion(non power of two)?...seems to work
gl.glTexSubImage2D(GL10.GL_TEXTURE_2D, 0, 0, 0, previewFrameWidth, previewFrameHeight,
mode, GL10.GL_UNSIGNED_BYTE, frameData);
}
frameLock.unlock();
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_LINEAR);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
frameEnqueued = false;
}
gl.glColor4f(1, 1, 1, 1f);
//draw camera preview frame:
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, textureBuffer);
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, squareBuffer);
//draw camera square
gl.glDrawArrays(GL10.GL_TRIANGLE_STRIP, 0, 4);
gl.glDisableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
if (customRenderer != null) {
customRenderer.setupEnv(gl);
} else {
// gl.glEnable(GL10.GL_LIGHTING);
// gl.glLightfv(GL10.GL_LIGHT0, GL10.GL_AMBIENT, ambientLightBuffer);
// gl.glLightfv(GL10.GL_LIGHT0, GL10.GL_DIFFUSE, diffuseLightBuffer);
// gl.glLightfv(GL10.GL_LIGHT0, GL10.GL_SPECULAR, specularLightBuffer);
// gl.glLightfv(GL10.GL_LIGHT0, GL10.GL_POSITION, lightPositionBuffer);
// gl.glEnable(GL10.GL_LIGHT0);
}
matrixGrabber.getCurrentState(gl);
if (performAction) {
Log.d("ZMIANA_KOLORY", "akcja się wola");
int[] c = new int[]{255, 0, 0};
Vector<Float> v = markerInfo.getObjects().get(0).vectors;
ByteBuffer vBuf = ByteBuffer.allocateDirect(v.size() * 4);
vBuf.order(ByteOrder.nativeOrder());
float[] newColor = new float[c.length + 1];
for (int i = 0; i < c.length; i++) {
newColor[i] = (1.0f / 255) * c[i];
newColor[3] = 1.0f;
}
Log.d("ZMIANA_KOLORY", newColor[0] + " " + newColor[1] + " " + newColor[2] + " " + newColor[3] + " ");
FloatBuffer colorBuffer = vBuf.asFloatBuffer();
colorBuffer.put(newColor);
colorBuffer.position(0);
gl.glEnableClientState(GL10.GL_COLOR_ARRAY);
gl.glMaterialfv(GL10.GL_FRONT_AND_BACK, GL10.GL_AMBIENT, colorBuffer);
gl.glDisableClientState(GL10.GL_COLOR_ARRAY);
performAction = false;
// Log.d("PERFORM_ACTION", "wykonuję akcję: " + actionName);
// switch (actionName) {
// case "Highlight":
//
// for(ARObject obj : markerInfo.getObjects()) {
//// if(tmp.get)
//
// }
//
// actionName = null;
// performAction = false;
// break;
// }
}
markerInfo.draw(gl);
if (customRenderer != null) {
customRenderer.draw(gl);
}
if (takeScreenshot) {
Log.d("SCREENSHOT", "Sprawdzam touched");
Ray r = new Ray(gl, screenWidth, screenHeight, x, y);
Log.d("SCREENSHOT", "Near Coord =" + Arrays.toString(r.P0));
Log.d("SCREENSHOT", "Far Coord =" + Arrays.toString(r.P1));
takeScreenshot = false;
}
}
OpenGL is a state based drawing system. State that's not active/enabled at the very moment you make a drawing call has no effect. You're enabling the color array, set the pointer and then immediately after disable it again. For the color vertex array to be taken into account, it must be enabled.
However you also seem to use a texture there. And on top of that you're using the fixed function pipeline (FFP). There are a number of interactions between textures, colors and illumination which can all lead to the color not showing up or being not turning out as you'd expect.
Do yourself a favour and start using shaders. Understanding the way the FFP works is not hard; however it grew so many state switches and data paths, that it's become very cumbersome to work with. You can either write 40+ lines of code to set up the FFP as you desire, or you can write a single line of shader code to the same effect.
This is the code I'm using to render a Bitmap in Wikitude SDK class ArchitectView
// clear Screen and Depth Buffer, we have set the clear color as black.
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
// get handle to vertex shader's vPosition member
int mPositionHandle = GLES20.glGetAttribLocation(riGraphicTools.sp_Image, "vPosition");
// Enable generic vertex attribute array
GLES20.glEnableVertexAttribArray(mPositionHandle);
// Prepare the triangle coordinate data
GLES20.glVertexAttribPointer(mPositionHandle, 3,
GLES20.GL_FLOAT, false,
0, vertexBuffer);
// Get handle to texture coordinates location
int mTexCoordLoc = GLES20.glGetAttribLocation(riGraphicTools.sp_Image, "a_texCoord" );
// Enable generic vertex attribute array
GLES20.glEnableVertexAttribArray ( mTexCoordLoc );
// Prepare the texturecoordinates
GLES20.glVertexAttribPointer ( mTexCoordLoc, 2, GLES20.GL_FLOAT,
false,
0, uvBuffer);
// Get handle to shape's transformation matrix
int mtrxhandle = GLES20.glGetUniformLocation(riGraphicTools.sp_Image, "uMVPMatrix");
// Apply the projection and view transformation
GLES20.glUniformMatrix4fv(mtrxhandle, 1, false, m, 0);
// Get handle to textures locations
int mSamplerLoc = GLES20.glGetUniformLocation (riGraphicTools.sp_Image, "s_texture" );
// Set the sampler texture unit to 0, where we have saved the texture.
GLES20.glUniform1i ( mSamplerLoc, 0);
// Draw the triangle
GLES20.glDrawElements(GLES20.GL_TRIANGLES, indices.length,
GLES20.GL_UNSIGNED_SHORT, drawListBuffer);
// Disable vertex array
GLES20.glDisableVertexAttribArray(mPositionHandle);
GLES20.glDisableVertexAttribArray(mTexCoordLoc);
And this is how image being setup:
public void SetupImage(Bitmap bmp)
{
// Create our UV coordinates.
uvs = new float[] {
0.0f, 0.0f,
0.0f, 1.0f,
1.0f, 1.0f,
1.0f, 0.0f
};
// The texture buffer
ByteBuffer bb = ByteBuffer.allocateDirect(uvs.length * 4);
bb.order(ByteOrder.nativeOrder());
uvBuffer = bb.asFloatBuffer();
uvBuffer.put(uvs);
uvBuffer.position(0);
// Generate Textures, if more needed, alter these numbers.
int[] texturenames = new int[1];
GLES20.glGenTextures(1, texturenames, 0);
// Bind texture to texturename
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, texturenames[0]);
// Set filtering
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
// Load the bitmap into the bound texture.
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bmp, 0);
// We are done using the bitmap so we should recycle it.
bmp.recycle();
}
The problem is I can't render texture inside ArchitectView, but rendering it without any issues on default android's GLSurfaceView.
You're not supposed to write any OpenGL code inside the ArchitectView. If you need some custom drawing, please write a plugin for the Wikitude SDK and use the startRender/endRender method to draw your OpenGL content (Only these two methods guarantee a valid OpenGL context bound).
I am new to opengl es on android and learning opengl by doing some examples. I am using two program to draw 3 objects. The following code loads the texture and draws a square. But it is appearing as a black square instead of applying the texture to the body.
my fragment-shader code
precision mediump float;
uniform sampler2D u_Texture;
varying vec2 v_TexCoordinate;
void main() {
gl_FragColor = texture2D(u_Texture, v_TexCoordinate);
}
my vertex-shader code
attribute vec2 a_TexCoordinate;
varying vec2 v_TexCoordinate;
attribute vec4 a_Position;
uniform mat4 u_Matrix;
void main() {
gl_Position = u_Matrix * a_Position;
v_TexCoordinate = a_TexCoordinate;
}
my object vertex buffer
float [] vBufferFloat = new float[] {
-0.2f, -0.2f, 1f,
0.2f, -0.2f, 1f,
0.2f, 0.2f, 1f,
-0.2f, 0.2f, 1f,
-0.2f, -0.2f, 1f,
};
my texture buffer
float [] texCoordinate = new float[] {
-0.2f, -0.2f,
0.2f, -0.2f,
0.2f, 0.2f,
-0.2f, 0.2f,
-0.2f, -0.2f,
};
my onSurfaceCreated && onDrawFrame code
public void onSurfaceCreated() {
cloudRendereProgram = ShaderHelper.createProgram(mContext, R.raw.sky_texture_vertex_shader, R.raw.sky_texture_fragment_shader);
cloudTextureId = Utils.loadTexture(mContext, com.elpis.gamecontroller.R.drawable.cloud);
aTextureLocation = GLES20.glGetAttribLocation(cloudRendereProgram, "a_TexCoordinate");
uMatrixLocation = GLES20.glGetUniformLocation(cloudRendereProgram, "u_Matrix");
aPositionLocation = GLES20.glGetAttribLocation(cloudRendereProgram, "a_Position");
uTextureLocation = GLES20.glGetUniformLocation(cloudRendereProgram, "u_Texture");
}
public void onDrawFrame() {
float [] mVMatrix = new float[16];
GLES20.glUseProgram(cloudRendereProgram);
GLES20.glVertexAttribPointer(aPositionLocation, 3, GLES20.GL_FLOAT, false, 0, vBuff.buffer);
GLES20.glEnableVertexAttribArray(aPositionLocation);
Matrix.multiplyMM(mVMatrix, 0, modelMatrix, 0, projectionMatrix, 0);
GLES20.glUniformMatrix4fv(uMatrixLocation, 1, false, mVMatrix, 0);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, this.cloudTextureId);
GLES20.glUniform1i(uTextureLocation, 0);
GLES20.glVertexAttribPointer(aTextureLocation, 2, GLES20.GL_FLOAT, false, 0, texBuff.buffer);
GLES20.glEnableVertexAttribArray(aTextureLocation);
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_FAN, 0, 5);
}
and my texture loader helper code
public static int loadTexture(Context ctx, int resId) {
final int [] textureHandle = new int[1];
GLES20.glGenTextures(1, textureHandle, 0);
if (textureHandle[0] == 0)
return 0;
final BitmapFactory.Options options = new Options();
options.inScaled = false;
final Bitmap imgTexture = BitmapFactory.decodeResource(ctx.getResources(), resId);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureHandle[0]);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST);
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, imgTexture, 0);
imgTexture.recycle();
return textureHandle[0];
}
When I run the android application all I see is a black square instead of seeing a texture of cloud. So, I would appreciate if anyone point me in the right direction.
Two quick questions; Is it valid to create multiple opengl program objects with different shaders and run them concurrently?
[UPDATE]
The problem was on the onDrawFrame(). I had to use vBuff.buffer.position(0) and texBuff.buffer.position(0) to be able to draw the texture correctly.
public void onDrawFrame() {
float [] mVMatrix = new float[16];
GLES20.glUseProgram(cloudRendereProgram);
// FIX
vBuff.buffer.position(0);
texBuff.buffer.position(0);
// END FIX
GLES20.glVertexAttribPointer(aPositionLocation, 3, GLES20.GL_FLOAT, false, 0, vBuff.buffer);
GLES20.glEnableVertexAttribArray(aPositionLocation);
Matrix.multiplyMM(mVMatrix, 0, modelMatrix, 0, projectionMatrix, 0);
GLES20.glUniformMatrix4fv(uMatrixLocation, 1, false, mVMatrix, 0);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, this.cloudTextureId);
GLES20.glUniform1i(uTextureLocation, 0);
GLES20.glVertexAttribPointer(aTextureLocation, 2, GLES20.GL_FLOAT, false, 0, texBuff.buffer);
GLES20.glEnableVertexAttribArray(aTextureLocation);
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_FAN, 0, 5);
}
You're not calling glEnableVertexAttribArray for your texture coords. Add this line to your onDrawFrame():
GLES20.glEnableVertexAttribArray(aTextureLocation);
Call glGenerateMipmap after you upload the texture texImage2D() in your loadTexture() function, to make sure all mip-map levels are valid:
glGenerateMipmap(GLES20.GL_TEXTURE_2D);
Also, move these calls from your surfaceCreated() function to the start of drawFrame():
aTextureLocation = GLES20.glGetAttribLocation(cloudRendereProgram, "a_TexCoordinate");
uMatrixLocation = GLES20.glGetUniformLocation(cloudRendereProgram, "u_Matrix");
aPositionLocation = GLES20.glGetAttribLocation(cloudRendereProgram, "a_Position");
uTextureLocation = GLES20.glGetUniformLocation(cloudRendereProgram, "u_Texture");
(it could be that these variables are not bound properly or the GL context is not yet properly set up in surfaceCreated())
A debugging tip for OpenGLES. Add this function to your code (it's from the Android OpenGLES samples):
public static void checkGlError(String glOperation) {
int error;
while ((error = GLES20.glGetError()) != GLES20.GL_NO_ERROR) {
Log.e(TAG, glOperation + ": glError " + error);
throw new RuntimeException(glOperation + ": glError " + error);
}
}
you can call it after each OpenGLES call, and pass in a String that can be whatever debugging message you want. If anything goes wrong then you'll get an exception instead of just silent failure which will leave you scratching your head trying to figure out what went wrong. Make sure to remove it from your final build to avoid force closes.
I'm new to OpenGL and I'm developing an Augmented-reality application for Android.
Until now, I was drawing white squares, perpendicular to the camera, pointing the user to the direction where a "Point of interest" would be.
Now, I'm trying to show some text into the squares.
I've read a lot and it seems that creating a texture with the text is the most direct and easiest approach, so I'm creating the textures as soon as I get data of the Points of interest and stick them to their squares. For that, I use bitmaps.
Let's see some code. Within my onDrawFrame method, I do something like this:
public void onDrawFrame(GL10 gl) {
// Get sensors matrix
...
//Clear Screen And Depth Buffer
gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
// Load remapped matrix
gl.glMatrixMode(GL10.GL_MODELVIEW);
// List of Points of interest (modified when some data is downloaded)
synchronized (poiList) {
if(mNewData){ // True if new data has been dowloaded)
if(textures != null) // Delete old textures
gl.glDeleteTextures(textures.length, textures, 0);
textures = loadGLTexture(gl, soapPoiList.getPoiList());
mNewData = false;
}
int i = 0;
// Iterate the list
for (PointOfInterest c : poiList) {
gl.glLoadIdentity();
gl.glLoadMatrixf(remappedRotationMatrix, 0);
// Get bearing
...
// Place polygon in the right place
gl.glRotatef(-bearing, 0, 0, 1);
gl.glTranslatef(0, ICONS_SIZE_RATIO, 0);
// Actually draws the polygon with text
c.draw(gl, textures[i]);
i++;
}
}
}
Where loadGLTextures is:
protected int[] loadGLTexture(GL10 gl, List<PointOfInterest> l){
int res[] = new int[l.size()];
gl.glGenTextures(res.length, res, 0);
int i = 0;
for(PointOfInterest p : l) {
Bitmap bitmap = Bitmap.createBitmap(256, 256, Bitmap.Config.RGB_565);
bitmap.eraseColor(Color.BLACK);
Canvas canvas = new Canvas(bitmap);
Paint textPaint = new Paint();
textPaint.setTextSize(35);
textPaint.setFakeBoldText(true);
textPaint.setAntiAlias(true);
textPaint.setARGB(255, 255, 255, 255);
// Draw the text centered
canvas.drawText(Float.toString(p.getDinstanceToOrigin()) + " m.", 10,35, textPaint);
// Bind the texture to our array
gl.glBindTexture(GL10.GL_TEXTURE_2D, res[i]);
// Create Nearest Filtered Texture
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_LINEAR);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_T, GL10.GL_CLAMP_TO_EDGE);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap, 0);
bitmap.recycle();
i++;
}
return res;
}
It basically creates a bitmap for each Point of Interest, and generate a texture with it. The texture will be applied over a white square later, as it is shown in this class:
public class PointOfInterest {
// MEMBERS ----------------------------------------------------------------
....
....
// OpenGL necessary variables
/** The buffer holding the vertices */
private FloatBuffer vertexBuffer;
/** The initial vertex definition */
private float vertices[] = {
-1.0f, 0.0f, -1.0f, //Bottom Left V1
-1.0f, 0.0f, 1.0f, //Top Left V2
1.0f, 0.0f, -1.0f, //Bottom Right V3
1.0f, 0.0f, 1.0f, //Top Right V4
};
private FloatBuffer textureBuffer;
private float texture[] = {
0.0f, 0.0f, // V1
1.0f, 0.0f, // V3
0.0f, 1.0f, // V2
1.0f, 1.0f // V4
};
// CONSTRUCTORS -----------------------------------------------------------
public PointOfInterest(Location originLocation){
currentLocation = originLocation;
mPoiLocation = new Location(LocationManager.GPS_PROVIDER);
ByteBuffer byteBuf = ByteBuffer.allocateDirect(vertices.length * 4);
byteBuf.order(ByteOrder.nativeOrder());
vertexBuffer = byteBuf.asFloatBuffer();
vertexBuffer.put(vertices);
vertexBuffer.position(0);
byteBuf = ByteBuffer.allocateDirect(texture.length * 4);
byteBuf.order(ByteOrder.nativeOrder());
textureBuffer = byteBuf.asFloatBuffer();
textureBuffer.put(texture);
textureBuffer.position(0);
}
// PUBLIC METHODS ---------------------------------------------------------
public void draw(GL10 gl, int textureId){
// Bind the previously generated texture
gl.glBindTexture(GL10.GL_TEXTURE_2D, textureId);
// Point to our buffers
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
// set the colour for the square
//gl.glColor4f(0.0f, 0.1f, 0.0f, 0.5f);
//Set the face rotation
gl.glFrontFace(GL10.GL_CW);
//Point to our vertex buffer
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, vertexBuffer);
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, textureBuffer);
//Draw the vertices as triangle strip
gl.glDrawArrays(GL10.GL_TRIANGLE_STRIP, 0, vertices.length / 3);
//Disable the client state before leaving
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
gl.glDisableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
}
....
....
}
I have tried to map the texture as it is taught here and here with no success. I really don't know what to do to get some letters drawn on the squares and am really lost here... Maybe the text is being drawn on the other face of the square, maybe the textures are not being generated... I don't know.
Any kind of help would be very appreciated.
Okay, I had forgotten to enable texture mapping. You can do that within any method that uses the GL10 object. I prefer to do it within the draw method of my objects, so any other object is not affected by the texture. It's as simple as this (just changed 2 lines, the ones that say NEW!!):
public void draw(GL10 gl, int textureId){
gl.glEnable(GL10.GL_TEXTURE_2D); //NEW !!! Enable Texture Mapping
// Bind the previously generated texture
gl.glBindTexture(GL10.GL_TEXTURE_2D, textureId);
// Point to our buffers
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
// set the colour for the square
//gl.glColor4f(0.0f, 0.1f, 0.0f, 0.5f);
//Set the face rotation
gl.glFrontFace(GL10.GL_CW);
//Point to our vertex buffer
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, vertexBuffer);
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, textureBuffer);
//Draw the vertices as triangle strip
gl.glDrawArrays(GL10.GL_TRIANGLE_STRIP, 0, vertices.length / 3);
//Disable the client state before leaving
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
gl.glDisableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
gl.glDisable(GL10.GL_TEXTURE_2D); ////NEW !!! Disable texture mapping
}