I am rendering camera data on my GLSurfaceView. Everything work fine . But the problem is the image stretch a bit in landscape mood. I have read this and tried that solution http://www.learnopengles.com/understanding-opengls-matrices . Please the check the "Adjusting to the screen’s aspect ratio" part of that link. I tried that solution . my code for that solution are attached below. These code are from part of my Renderer class which implements GLSurfaceView.Renderer interface :
private float[] vertices;
private float[] verticesFrontCameraPortrait = new float[]{1.f, -1.f, -1.f, -1.f, -1.f, 1.f, 1.f, 1.f};
private float[] verticesFrontCamera_90_Degree_Right = new float[]{-1.f, -1.f, -1.f, 1.f, 1.f, 1.f, 1.f, -1.f,};
private float[] verticesFrontCamera_270_Degree_right = new float[]{1.f, 1.f, 1.f, -1.f, -1.f, -1.f, -1.f, 1.f,};
#Override
public void onSurfaceChanged(GL10 gl10, int width, int height) {
GLES20.glViewport(0, 0, width, height);
float aspectRatio = (float) width / (float) height;
Matrix.orthoM(vertices, 0, -aspectRatio, aspectRatio, -1, 1, -1, 1);
}
My vertices variable hold the reference of any of the three vertice matrices according to device configuration. That works fine totally. But the problem occurs if try the work on onSurfaceChanged(GL10 gl10, int width, int height)
method. It crush my program saying ArrayIndexOutOfBoundsExceptio . But if i don't use these two lines :
float aspectRatio = (float) width / (float) height;
Matrix.orthoM(vertices, 0, -aspectRatio, aspectRatio, -1, 1, -1, 1);
everything work fine but the video/camera image data's are stretched on landscape mood. I don't want stretched image that are rendered on my GLSurfaceView.
Here is my full Renderer class .
/**
* IMPORTANT - Please read before changing
*
* This class render a NV21 (YV12) image byte array in GlSurfaceView using OpenGl.
* NV21 image format has 12 bits per pixel out of which 8 bits are for luminance (Y) and 4 bits
* are for chrominance (UV). So yBuffer size is same and uvBuffer size is of half
* of number of pixels. yTexture height and widht are also same as image height and width.
* First 2/3 of the input image array are Y values and last 1/3 are uv in the
* order v0u0v1u1... (altering v and u values) so on. So GL_LUMINANCE and GL_LUMINANCE_ALPHA format
* are used to pass yBuffer and uvBuffer respectively and fragment_shader takes U value from alpha channel
* and V value from red channel (could be green or blue channel with same result). uvTexture height
* and width are also 1/4 of the original image height and width
*
* GL_TEXTURE0 + 1 (GL_TEXTURE1) and GL_TEXTURE0 + 2 (GL_TEXTURE2) must be used for yTexture and uvTexture.
* If GL_TEXTURE0 is used for yTexture, it doesn't work in some devices.
*/
public class VideoCallImageRenderer implements GLSurfaceView.Renderer {
private final String TAG = "VideoCallImageRenderer";
private int cameraType; // 0 for backcamera, 1 for front camera, 2 for opponent camera
private Context context;
private int shaderProgram;
short[] indices = new short[]{0, 1, 2, 0, 2, 3};
// y texture handle
private int[] yTexture = new int[1];
// uv texture handle
private int[] uvTexture = new int[1];
// texture coordinate and vertices buffers
private FloatBuffer texCoordBuffer, vertexBuffer;
// indices buffer
private ShortBuffer indexBuffer;
// y and uv texture buffers
private ByteBuffer yBuffer, uvBuffer;
// image height and width
private int width = 0, height = 0;
// true when a valid image data is set. default value false.
private boolean render = false;
// position attribute location handle in vertex shader
private int positionLocation;
// texture coordinate attribute location handle in vertex shader
private int textureCoordinateLocation;
// y_texture sampler2D location handle in fragment shader
private int yTextureLocation;
// uv_texture sampler2D location handle in fragment shader
private int uvTextureLocation;
final private float bytePerPixel = 1.5f;
/*This value of vertices are changed to roatate the image: explanation
let 1.f, 1.f = A
-1.f, 1.f = B
-1.f, -1.f= C
1.f, -1.f = D
SO ABCD = Back camera in normal stage
if we want to rotate 90 degre right then value will be DABC
if we want to rotate 90 degre left then value will be BCDA
private float[] vertices = new float[]{
1.f, -1.f,
-1.f, -1.f,
-1.f, 1.f,
1.f, 1.f
};
*/
private float[] vertices;
private float[] verticesPortrait;
private float[] vertices_90_Degree_Right;
private float[] vertices_270_Degree_Right;
private int currentDeviceOrientation = -1;
private float[] verticesFrontCameraPortrait = new float[]{1.f, -1.f, -1.f, -1.f, -1.f, 1.f, 1.f, 1.f};
private float[] verticesFrontCamera_90_Degree_Right = new float[]{-1.f, -1.f, -1.f, 1.f, 1.f, 1.f, 1.f, -1.f,};
private float[] verticesFrontCamera_270_Degree_right = new float[]{1.f, 1.f, 1.f, -1.f, -1.f, -1.f, -1.f, 1.f,};
private float[] verticesBackCameraPortrait = new float[]{1.f, 1.f, -1.f, 1.f, -1.f, -1.f, 1.f, -1.f};
private float[] verticesBackCamera_90_Degree_Right = new float[]{1.f, -1.f, 1.f, 1.f, -1.f, 1.f, -1.f, -1.f,};
private float[] verticesBackCamera_270_Degree_Right = new float[]{-1.f, 1.f, -1.f, -1.f, 1.f, -1.f, 1.f, 1.f,};
private float[] verticesOpponentCamera = new float[]{1.f, 1.f, 1.f, -1.f, -1.f, -1.f, -1.f, 1.f};
private float[] verticeOpponentCamera_90_Degree_Right = new float[]{1.f, -1.f, -1.f, -1.f, -1.f, 1.f, 1.f, 1.f,};
private float[] verticeOpponentCamera_270_Degree_Right = new float[]{-1.f, 1.f, 1.f, 1.f, 1.f, -1.f, -1.f, -1.f,};
private float[] texCoords = new float[]{
0.f, 0.f,
0.f, 1.f,
1.f, 1.f,
1.f, 0.f
};
public VideoCallImageRenderer(Context context, int cameraType) {
this.context = context;
// initialize texture coordinate buffer
this.cameraType = cameraType;
if (cameraType == 0) {
verticesPortrait = verticesBackCameraPortrait;
vertices_90_Degree_Right = verticesBackCamera_90_Degree_Right;
vertices_270_Degree_Right = verticesBackCamera_270_Degree_Right;
} else if (cameraType == 1) {
verticesPortrait = verticesFrontCameraPortrait;
vertices_90_Degree_Right = verticesFrontCamera_90_Degree_Right;
vertices_270_Degree_Right = verticesFrontCamera_270_Degree_right;
} else {
verticesPortrait = verticesOpponentCamera;
vertices_90_Degree_Right = verticeOpponentCamera_90_Degree_Right;
vertices_270_Degree_Right = verticeOpponentCamera_270_Degree_Right;
}
ByteBuffer tcbb = ByteBuffer.allocateDirect(texCoords.length * 4);
tcbb.order(ByteOrder.nativeOrder());
texCoordBuffer = tcbb.asFloatBuffer();
texCoordBuffer.put(texCoords);
texCoordBuffer.position(0);
// initialize vertices buffer
vertices = verticesPortrait;
ByteBuffer vbb = ByteBuffer.allocateDirect(vertices.length * 4);
vbb.order(ByteOrder.nativeOrder());
vertexBuffer = vbb.asFloatBuffer();
vertexBuffer.put(vertices);
vertexBuffer.position(0);
// initialize indices buffer
ByteBuffer ibb = ByteBuffer.allocateDirect(indices.length * 2);
ibb.order(ByteOrder.nativeOrder());
indexBuffer = ibb.asShortBuffer();
indexBuffer.put(indices);
indexBuffer.position(0);
}
public void resetVertex(float[] new_vertices) {
Constants.debugLog(TAG, "resetVertex");
ByteBuffer vbb = ByteBuffer.allocateDirect(new_vertices.length * 4);
vbb.order(ByteOrder.nativeOrder());
vertexBuffer = vbb.asFloatBuffer();
vertexBuffer.put(new_vertices);
vertexBuffer.position(0);
}
public void setImageBuffer(final byte[] imageBytes, int height, int width, int iDeviceOrientation) {
// reinitialize texture buffers if width or height changes
Constants.debugLog(TAG, "setImageBuffer device Orientation == " + iDeviceOrientation);
if (currentDeviceOrientation == -1) {
currentDeviceOrientation = iDeviceOrientation;
}
if (iDeviceOrientation != currentDeviceOrientation) {
currentDeviceOrientation = iDeviceOrientation;
switch (currentDeviceOrientation) {
case 0:
vertices = verticesPortrait;
resetVertex(vertices);
break;
case 1:
vertices = vertices_90_Degree_Right;
resetVertex(vertices);
break;
case 3:
vertices = vertices_270_Degree_Right;
resetVertex(vertices);
break;
}
}
final boolean resolutionChanged = this.width != width || this.height != height;
if (resolutionChanged) {
this.width = width;
this.height = height;
final int numberOfPixels = this.height * this.width;
this.yBuffer = ByteBuffer.allocateDirect(numberOfPixels);
this.yBuffer.order(ByteOrder.nativeOrder());
this.uvBuffer = ByteBuffer.allocateDirect(numberOfPixels / 2);
this.uvBuffer.order(ByteOrder.nativeOrder());
}
this.render = updateYUVBuffers(imageBytes);
}
private boolean updateYUVBuffers(final byte[] imageBytes) {
final int numberOfPixels = this.height * this.width;
final int numberOfExpectedBytes = (int) (numberOfPixels * this.bytePerPixel);
if (imageBytes != null && imageBytes.length != (int) (numberOfPixels * this.bytePerPixel)) {
return false;
}
// put image bytes into texture buffers
yBuffer.put(imageBytes, 0, numberOfPixels);
yBuffer.position(0);
uvBuffer.put(imageBytes, numberOfPixels, numberOfPixels / 2);
uvBuffer.position(0);
return true;
}
#Override
public void onSurfaceCreated(GL10 gl10, EGLConfig eglConfig) {
createShader();
positionLocation = GLES20.glGetAttribLocation(shaderProgram, "a_position");
textureCoordinateLocation = GLES20.glGetAttribLocation(shaderProgram, "a_texCoord");
// generate y texture
GLES20.glEnable(GLES20.GL_TEXTURE_2D);
yTextureLocation = GLES20.glGetUniformLocation(shaderProgram, "y_texture");
GLES20.glGenTextures(1, yTexture, 0);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0 + 1);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, yTexture[0]);
// generate uv texture
GLES20.glEnable(GLES20.GL_TEXTURE_2D);
uvTextureLocation = GLES20.glGetUniformLocation(shaderProgram, "uv_texture");
GLES20.glGenTextures(1, uvTexture, 0);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0 + 2);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, uvTexture[0]);
// clear display color
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
}
#Override
public void onSurfaceChanged(GL10 gl10, int width, int height) {
GLES20.glViewport(0, 0, width, height);
float aspectRatio = (float) width / (float) height;
Matrix.orthoM(vertices, 0, -aspectRatio, aspectRatio, -1, 1, -1, 1);
}
#Override
public void onDrawFrame(GL10 gl10) {
// clear display
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
if (render) {
GLES20.glUseProgram(shaderProgram);
GLES20.glVertexAttribPointer(positionLocation, 2,
GLES20.GL_FLOAT, false,
0, vertexBuffer);
GLES20.glVertexAttribPointer(textureCoordinateLocation, 2, GLES20.GL_FLOAT,
false,
0, texCoordBuffer);
GLES20.glEnableVertexAttribArray(positionLocation);
GLES20.glEnableVertexAttribArray(textureCoordinateLocation);
// create and update y texture
GLES20.glActiveTexture(GLES20.GL_TEXTURE0 + 1);
GLES20.glUniform1i(yTextureLocation, 1);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, this.width,
this.height, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, this.yBuffer);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
// create and update uv texture
GLES20.glActiveTexture(GLES20.GL_TEXTURE0 + 2);
GLES20.glUniform1i(uvTextureLocation, 2);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE_ALPHA, this.width / 2,
this.height / 2, 0, GLES20.GL_LUMINANCE_ALPHA, GLES20.GL_UNSIGNED_BYTE, this.uvBuffer);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
// render image
GLES20.glDrawElements(GLES20.GL_TRIANGLES, indices.length,
GLES20.GL_UNSIGNED_SHORT, indexBuffer);
GLES20.glDisableVertexAttribArray(positionLocation);
GLES20.glDisableVertexAttribArray(textureCoordinateLocation);
}
}
void createShader() {
int vertexShader = loadShader(GLES20.GL_VERTEX_SHADER,
CallConstants.readRawTextFile(context, R.raw.vertex_shader));
int fragmentShader = loadShader(GLES20.GL_FRAGMENT_SHADER,
CallConstants.readRawTextFile(context, R.raw.fragment_shader));
shaderProgram = GLES20.glCreateProgram();
GLES20.glAttachShader(shaderProgram, vertexShader);
GLES20.glAttachShader(shaderProgram, fragmentShader);
GLES20.glLinkProgram(shaderProgram);
GLES20.glUseProgram(shaderProgram);
int[] linkStatus = new int[1];
GLES20.glGetProgramiv(shaderProgram, GLES20.GL_LINK_STATUS, linkStatus, 0);
if (linkStatus[0] != GLES20.GL_TRUE) {
Log.e("Render", "Could not link program: ");
Log.e("Render", GLES20.glGetProgramInfoLog(shaderProgram));
GLES20.glDeleteProgram(shaderProgram);
shaderProgram = 0;
}
// free up no longer needed shader resources
GLES20.glDeleteShader(vertexShader);
GLES20.glDeleteShader(fragmentShader);
}
public int loadShader(int type, String shaderCode) {
int shader = GLES20.glCreateShader(type);
GLES20.glShaderSource(shader, shaderCode);
GLES20.glCompileShader(shader);
return shader;
}
}
Related
If i change the background of GLES20.glClearColor(1.0f, 1.0f, 0.0f, 1.0f) and then try to draw a texture, colors of this last changes unexpectedly. This is the png file:
The result of the application when i try simply to display it is this one:
Im using this code:
public class GLRenderer implements GLSurfaceView.Renderer {
private static final String TAG = "MyGLRenderer";
private float[] vertices = {
-1f, -1f,
1f, -1f,
-1f, 1f,
1f, 1f
};
private float[] textureVertices = {
0f, 1f,
1f, 1f,
0f, 0f,
1f, 0f
};
private final String vertexShaderCode =
"attribute vec4 aPosition;" +
"attribute vec2 aTexPosition;" +
"varying vec2 vTexPosition;" +
"void main() {" +
" gl_Position = aPosition;" +
" vTexPosition = aTexPosition;" +
"}";
private final String fragmentShaderCode =
"precision mediump float;" +
"uniform sampler2D uTexture;" +
"varying vec2 vTexPosition;" +
"void main() {\n" +
"vec4 color = texture2D(uTexture, vTexPosition);\n"+
//"if(color.r == 0.0 && color.g == 0.0 && color.b == 0.0)\n"+
// "color = vec4(1.0,0.5,0.5,1.0);"+
// "discard;"+
" gl_FragColor = color;" +
"}";
private FloatBuffer verticesBuffer;
private FloatBuffer textureBuffer;
private int vertexShader;
private int fragmentShader;
private int program;
private Bitmap bmp;
private int textures[] = new int[2];
// mMVPMatrix is an abbreviation for "Model View Projection Matrix"
private final float[] mMVPMatrix = new float[16];
private final float[] mProjectionMatrix = new float[16];
private final float[] mViewMatrix = new float[16];
private final float[] mRotationMatrix = new float[16];
public GLRenderer() {
bmp=Bitmap.createBitmap(513,912, Bitmap.Config.ARGB_8888);
}
#Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
GLES20.glClearColor(1.0f, 1.0f, 0.0f, 1.0f);
checkGlError("glClearColor");
setup();
}
#Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
GLES20.glViewport(0, 0, width, height);
float ratio = (float) width / height;
Matrix.frustumM(mProjectionMatrix, 0, -ratio, ratio, -1, 1, 3, 7);
}
#Override
public void onDrawFrame(GL10 gl) {
Log.d("Drawing_Frame","Working");
float[] scratch = new float[16];
// Draw background color
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
// Set the camera position (View matrix)
Matrix.setLookAtM(mViewMatrix, 0, 0, 0, -3, 0f, 0f, 0f, 0f, 1.0f, 0.0f);
// Calculate the projection and view transformation
Matrix.multiplyMM(mMVPMatrix, 0, mProjectionMatrix, 0, mViewMatrix, 0);
// Draw Bitmap
drawBinaryImage(bmp,textures[0]);
Matrix.setRotateM(mRotationMatrix, 0, 0, 0, 0, 1.0f);
Matrix.multiplyMM(scratch, 0, mMVPMatrix, 0, mRotationMatrix, 0);
}
private void setup(){
GLES20.glGenTextures(2, textures, 0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textures[0]);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bmp, 0);
//GLES20.glBindTexture(GL);
initializeBuffers();
initializeProgram();
}
private void initializeBuffers() {
ByteBuffer buff = ByteBuffer.allocateDirect(vertices.length * 4);
buff.order(ByteOrder.nativeOrder());
verticesBuffer = buff.asFloatBuffer();
verticesBuffer.put(vertices);
verticesBuffer.position(0);
buff = ByteBuffer.allocateDirect(textureVertices.length * 4);
buff.order(ByteOrder.nativeOrder());
textureBuffer = buff.asFloatBuffer();
textureBuffer.put(textureVertices);
textureBuffer.position(0);
}
private void initializeProgram() {
vertexShader = GLES20.glCreateShader(GLES20.GL_VERTEX_SHADER);
GLES20.glGetShaderInfoLog(vertexShader);
checkGlError("glCreateShader");
GLES20.glShaderSource(vertexShader, vertexShaderCode);
GLES20.glCompileShader(vertexShader);
fragmentShader = GLES20.glCreateShader(GLES20.GL_FRAGMENT_SHADER);
GLES20.glShaderSource(fragmentShader, fragmentShaderCode);
GLES20.glCompileShader(fragmentShader);
program = GLES20.glCreateProgram();
GLES20.glAttachShader(program, vertexShader);
GLES20.glAttachShader(program, fragmentShader);
GLES20.glLinkProgram(program);
checkGlError("glLinkProgram");
}
public void updateTexture(Bitmap bmp){
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bmp, 0);
}
private void drawBinaryImage(Bitmap bmp,int texture){
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
GLES20.glUseProgram(program);
//Changes Here original Line GLES20.glDisable(GLES20.GL_BLEND);
GLES20.glDisable(GLES20.GL_CULL_FACE);
GLES20.glDisable(GLES20.GL_DEPTH_TEST);
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE);
int positionHandle = GLES20.glGetAttribLocation(program, "aPosition");
int textureHandle = GLES20.glGetUniformLocation(program, "uTexture");
int texturePositionHandle = GLES20.glGetAttribLocation(program, "aTexPosition");
//Log.d("GL_SETUP",positionHandle+" , "+textureHandle);
GLES20.glVertexAttribPointer(texturePositionHandle, 2, GLES20.GL_FLOAT, false, 0, textureBuffer);
GLES20.glEnableVertexAttribArray(texturePositionHandle);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, texture);
Log.d("FILTER_APPLY","Applying");
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER,GLES20.GL_LINEAR);
GLES20.glUniform1i(textureHandle, 0);
GLES20.glVertexAttribPointer(positionHandle, 2, GLES20.GL_FLOAT, false, 0, verticesBuffer);
GLES20.glEnableVertexAttribArray(positionHandle);
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
checkGlError("glDrawArrays");
}
public void setBitmap(Bitmap bitmap){
updateTexture(bitmap);
this.bmp = bitmap;
}
public static void checkGlError(String glOperation) {
int error;
while ((error = GLES20.glGetError()) != GLES20.GL_NO_ERROR) {
Log.e(TAG, glOperation + ": glError " + error);
throw new RuntimeException(glOperation + ": glError " + error);
}
}
}
The framebuffer is cleared by GLES20.glClearColor(1.0f, 1.0f, 0.0f, 1.0f);.
RGBA(1, 1, 0, 1) is yellow. This causes that before rendering the texture, the entire framebuffer is filled in yellow.
The texture contains a blue color RGBA(0, 0, 1, 1) and a black color RGBA(0, 0, 0, 1).
When the quad with the texture is drawn, then blending is enabled with the following function:
(see Blending and glBlendFunc)
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE);
At blending the current color of the framebuffer is mixed by the actually drawn color. With the above setup this done by the following function:
destinationColor = sourceColor * 1 + destinationColor * 1
In the regions where the texture is blue, the final color becomes white:
(1, 1, 0) * 1 + (0, 0, 1) * 1 = (1, 1, 1)
In the regions where the texture is black, the color in the framebuffer stays yellow:
(1, 1, 0) * 1 + (0, 0, 0) * 1 = (1, 1, 0)
I have a NV21 (YUV420) camera video which I'm applying on it a fragment shader in order to get some filter effects and YUV to RGB convertion as well.
Everything is working except the bad performance.
My fragment shader is a bit heavy because it has many textur2D() calls.
The original frame resolution is 480x640 pixels, and I noticed that if I'm setting the viewport to this original size (instead of fullscreen size) it is working good and fluently.
So basically I need first to render the frame and processing it in a frameBuffer (FBO) with that original size and then (after the shader's work done) scale it to the fullscreen size using viewport (1080x1920 mostly), And it means that the "heavy" processing work would be applied on much less fragments.
I've found out some tutorials and similar questions here how to achieve that, but unfortunately I've got no luck with that. (Got some black screens or GL_INVALID_OPERATION and etc')...
Any help would be much appreciated.
Also, another (optional) performance tweak that I don't know how to deal with (if it's possible) is to combined somehow these 3 textures (Y_tex, U_tex and V_tex) to a single texture which be uniformed to the shader as a single sampler and then I can make just one texture2D() call in the shader in order to get the current YUV values and convert them to RGB values.
This is my renderer code:
static class MyRenderer implements GLSurfaceView.Renderer
{
int mTextureIds[] = new int[3];
float[] mScaleMatrix = new float[16];
private FloatBuffer mVertexBuffer;
private FloatBuffer mTextureBuffer;
private ShortBuffer mDrawListBuffer;
boolean mVideoFitEnabled = true;
boolean mVideoDisabled = false;
// number of coordinates per vertex in this array
static final int COORDS_PER_VERTEX = 3;
static final int TEXTURECOORDS_PER_VERTEX = 2;
static float mXYZCoords[] = {
-1.0f, 1.0f, 0.0f, // top left
-1.0f, -1.0f, 0.0f, // bottom left
1.0f, -1.0f, 0.0f, // bottom right
1.0f, 1.0f, 0.0f // top right
};
static float mUVCoords[] = {
0, 0, // top left
0, 1, // bottom left
1, 1, // bottom right
1, 0 // top right
};
private short mVertexIndex[] = {0, 1, 2, 0, 2, 3}; // order to draw vertices
private final String vertexShaderCode =
"uniform mat4 uMVPMatrix;"
+ "attribute vec4 aPosition;\n"
+ "attribute vec2 aTextureCoord;\n"
+ "varying vec2 vTextureCoord;\n"
+ "void main() {\n"
+ " gl_Position = uMVPMatrix * aPosition;\n"
+ " vTextureCoord = aTextureCoord;\n"
+ "}\n";
private final String fragmentShaderCode =
"precision mediump float;\n"
+ "uniform sampler2D Ytex;\n"
+ "uniform sampler2D Utex,Vtex;\n"
+ "varying vec2 vTextureCoord;\n"
+ "void main(void) {\n"
+ " float nx,ny,r,g,b,y,u,v;\n"
+ " mediump vec4 txl,ux,vx;"
+ " nx=vTextureCoord[0];\n"
+ " ny=vTextureCoord[1];\n"
+ " y=texture2D(Ytex,vec2(nx,ny)).r;\n"
+ " u=texture2D(Utex,vec2(nx,ny)).r;\n"
+ " v=texture2D(Vtex,vec2(nx,ny)).r;\n"
+ " y=1.1643*(y-0.0625);\n"
+ " u=u-0.5;\n"
+ " v=v-0.5;\n"
+ " r=y+1.5958*v;\n"
+ " g=y-0.39173*u-0.81290*v;\n"
+ " b=y+2.017*u;\n"
// --> Bilateral blur filter code HERE <--
+ " gl_FragColor=vec4(r,g,b,1.0);\n"
+ "}\n";
ReentrantLock mFrameLock = new ReentrantLock();
Frame mCurrentFrame;
private int mProgram;
private int mTextureWidth;
private int mTextureHeight;
private int mViewportWidth;
private int mViewportHeight;
public MyRenderer()
{
ByteBuffer bb = ByteBuffer.allocateDirect(mXYZCoords.length * 4);
bb.order(ByteOrder.nativeOrder());
mVertexBuffer = bb.asFloatBuffer();
mVertexBuffer.put(mXYZCoords);
mVertexBuffer.position(0);
ByteBuffer tb = ByteBuffer.allocateDirect(mUVCoords.length * 4);
tb.order(ByteOrder.nativeOrder());
mTextureBuffer = tb.asFloatBuffer();
mTextureBuffer.put(mUVCoords);
mTextureBuffer.position(0);
ByteBuffer dlb = ByteBuffer.allocateDirect(mVertexIndex.length * 2);
dlb.order(ByteOrder.nativeOrder());
mDrawListBuffer = dlb.asShortBuffer();
mDrawListBuffer.put(mVertexIndex);
mDrawListBuffer.position(0);
}
#Override public void onSurfaceCreated(GL10 gl, EGLConfig config)
{
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
int vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexShaderCode);
int fragmentShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentShaderCode);
mProgram = GLES20.glCreateProgram(); // create empty OpenGL ES Program
GLES20.glAttachShader(mProgram, vertexShader); // add the vertex shader to program
GLES20.glAttachShader(mProgram, fragmentShader); // add the fragment shader to program
GLES20.glLinkProgram(mProgram);
int positionHandle = GLES20.glGetAttribLocation(mProgram, "aPosition");
int textureHandle = GLES20.glGetAttribLocation(mProgram, "aTextureCoord");
GLES20.glVertexAttribPointer(positionHandle, COORDS_PER_VERTEX, GLES20.GL_FLOAT, false, COORDS_PER_VERTEX * 4, mVertexBuffer);
GLES20.glEnableVertexAttribArray(positionHandle);
GLES20.glVertexAttribPointer(textureHandle, TEXTURECOORDS_PER_VERTEX, GLES20.GL_FLOAT, false, TEXTURECOORDS_PER_VERTEX * 4, mTextureBuffer);
GLES20.glEnableVertexAttribArray(textureHandle);
GLES20.glUseProgram(mProgram);
int i = GLES20.glGetUniformLocation(mProgram, "Ytex"); // GLES20.glUniform3i(i, 0, 1, 2);
GLES20.glUniform1i(i, 0); /* Bind Ytex to texture unit 0 */
i = GLES20.glGetUniformLocation(mProgram, "Utex");
GLES20.glUniform1i(i, 1); /* Bind Utex to texture unit 1 */
i = GLES20.glGetUniformLocation(mProgram, "Vtex");
GLES20.glUniform1i(i, 2); /* Bind Vtex to texture unit 2 */
mTextureWidth = 0;
mTextureHeight = 0;
}
static void initializeTexture(int name, int id, int width, int height)
{
GLES20.glActiveTexture(name);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, id);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, width, height, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, null);
}
void setupTextures(Frame frame)
{
if (mTextureIds[0] != 0)
{
GLES20.glDeleteTextures(3, mTextureIds, 0);
}
GLES20.glGenTextures(3, mTextureIds, 0);
int w = frame.getWidth();
int h = frame.getHeight();
int hw = (w + 1) >> 1;
int hh = (h + 1) >> 1;
initializeTexture(GLES20.GL_TEXTURE0, mTextureIds[0], w, h);
initializeTexture(GLES20.GL_TEXTURE1, mTextureIds[1], hw, hh);
initializeTexture(GLES20.GL_TEXTURE2, mTextureIds[2], hw, hh);
mTextureWidth = frame.getWidth();
mTextureHeight = frame.getHeight();
}
void updateTextures(Frame frame)
{
int width = frame.getWidth();
int height = frame.getHeight();
int half_width = (width + 1) >> 1;
int half_height = (height + 1) >> 1;
int y_size = width * height;
int uv_size = half_width * half_height;
ByteBuffer bb = frame.getBuffer();
bb.clear(); // If we are reusing this frame, make sure we reset position and limit
if (bb.remaining() == y_size + uv_size * 2)
{
bb.position(0);
GLES20.glPixelStorei(GLES20.GL_UNPACK_ALIGNMENT, 1);
GLES20.glPixelStorei(GLES20.GL_PACK_ALIGNMENT, 1);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureIds[0]);
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, width, height, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, bb);
bb.position(y_size);
GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureIds[1]);
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, half_width, half_height, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, bb);
bb.position(y_size + uv_size);
GLES20.glActiveTexture(GLES20.GL_TEXTURE2);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureIds[2]);
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, half_width, half_height, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, bb);
int i = GLES20.glGetUniformLocation(mProgram, "width");
GLES20.glUniform1f(i, (float) mTextureWidth);
i = GLES20.glGetUniformLocation(mProgram, "height");
GLES20.glUniform1f(i, (float) mTextureHeight);
}
else
{
mTextureWidth = 0;
mTextureHeight = 0;
}
}
#Override public void onSurfaceChanged(GL10 gl, int width, int height)
{
GLES20.glViewport(0, 0, width, height);
mViewportWidth = width;
mViewportHeight = height;
}
#Override public void onDrawFrame(GL10 gl)
{
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
mFrameLock.lock();
if (mCurrentFrame != null && !mVideoDisabled)
{
GLES20.glUseProgram(mProgram);
if (mTextureWidth != mCurrentFrame.getWidth() || mTextureHeight != mCurrentFrame.getHeight())
{
setupTextures(mCurrentFrame);
}
updateTextures(mCurrentFrame);
Matrix.setIdentityM(mScaleMatrix, 0);
float scaleX = 1.0f, scaleY = 1.0f;
float ratio = (float) mCurrentFrame.getWidth() / mCurrentFrame.getHeight();
float vratio = (float) mViewportWidth / mViewportHeight;
if (mVideoFitEnabled)
{
if (ratio > vratio)
{
scaleY = vratio / ratio;
}
else
{
scaleX = ratio / vratio;
}
}
else
{
if (ratio < vratio)
{
scaleY = vratio / ratio;
}
else
{
scaleX = ratio / vratio;
}
}
Matrix.scaleM(mScaleMatrix, 0, scaleX * (mCurrentFrame.isMirroredX() ? -1.0f : 1.0f), scaleY, 1);
int mMVPMatrixHandle = GLES20.glGetUniformLocation(mProgram, "uMVPMatrix");
GLES20.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, mScaleMatrix, 0);
GLES20.glDrawElements(GLES20.GL_TRIANGLES, mVertexIndex.length, GLES20.GL_UNSIGNED_SHORT, mDrawListBuffer);
}
mFrameLock.unlock();
}
public void displayFrame(Frame frame)
{
mFrameLock.lock();
if (this.mCurrentFrame != null)
{
this.mCurrentFrame.recycle();
}
this.mCurrentFrame = frame;
mFrameLock.unlock();
}
public static int loadShader(int type, String shaderCode)
{
int shader = GLES20.glCreateShader(type);
GLES20.glShaderSource(shader, shaderCode);
GLES20.glCompileShader(shader);
return shader;
}
public void disableVideo(boolean b)
{
mFrameLock.lock();
mVideoDisabled = b;
if (mVideoDisabled)
{
if (this.mCurrentFrame != null)
{
this.mCurrentFrame.recycle();
}
this.mCurrentFrame = null;
}
mFrameLock.unlock();
}
public void enableVideoFit(boolean enableVideoFit)
{
mVideoFitEnabled = enableVideoFit;
}
}
Eventually I've figured it out thanks to a very talent guy who helped me with it.
Here is my renderer class which has now a frameBuffer with 2 rendering passes:
static class MyRenderer implements GLSurfaceView.Renderer
{
int mTextureIds[] = new int[4];
float[] mScaleMatrix = new float[16];
float[] mFilterScaleMatrix = new float[16];
private FloatBuffer mVertexBuffer;
private FloatBuffer mTextureBuffer;
private ShortBuffer mDrawListBuffer;
private IntBuffer frameBuffer;
boolean mVideoFitEnabled = true;
boolean mVideoDisabled = false;
// number of coordinates per vertex in this array
static final int COORDS_PER_VERTEX = 3;
static final int TEXTURECOORDS_PER_VERTEX = 2;
static float mXYZCoords[] = {
-1.0f, 1.0f, 0.0f, // top left
-1.0f, -1.0f, 0.0f, // bottom left
1.0f, -1.0f, 0.0f, // bottom right
1.0f, 1.0f, 0.0f // top right
};
static float mUVCoords[] = {
0, 0, // top left
0, 1, // bottom left
1, 1, // bottom right
1, 0 // top right
};
private short mVertexIndex[] = {0, 1, 2, 0, 2, 3}; // order to draw vertices
private final String vertexShaderCode =
"uniform mat4 uMVPMatrix;"
+ "attribute vec4 aPosition;\n"
+ "attribute vec2 aTextureCoord;\n"
+ "varying vec2 vTextureCoord;\n"
+ "void main() {\n"
+ " gl_Position = uMVPMatrix * aPosition;\n"
+ " vTextureCoord = aTextureCoord;\n"
+ "}\n";
private final String fragmentShaderCode =
"YUV to RGB Conversion shader HERE";
private final String frameBufferShader =
"MY filter effect shader HERE";
ReentrantLock mFrameLock = new ReentrantLock();
Frame mCurrentFrame;
private int mProgram;
private int mProgramFilter;
private int mTextureWidth;
private int mTextureHeight;
private int mViewportWidth;
private int mViewportHeight;
public MyRenderer()
{
ByteBuffer bb = ByteBuffer.allocateDirect(mXYZCoords.length * 4);
bb.order(ByteOrder.nativeOrder());
mVertexBuffer = bb.asFloatBuffer();
mVertexBuffer.put(mXYZCoords);
mVertexBuffer.position(0);
ByteBuffer tb = ByteBuffer.allocateDirect(mUVCoords.length * 4);
tb.order(ByteOrder.nativeOrder());
mTextureBuffer = tb.asFloatBuffer();
mTextureBuffer.put(mUVCoords);
mTextureBuffer.position(0);
ByteBuffer dlb = ByteBuffer.allocateDirect(mVertexIndex.length * 2);
dlb.order(ByteOrder.nativeOrder());
mDrawListBuffer = dlb.asShortBuffer();
mDrawListBuffer.put(mVertexIndex);
mDrawListBuffer.position(0);
frameBuffer = IntBuffer.allocate(1);
}
#Override public void onSurfaceCreated(GL10 gl, EGLConfig config)
{
GLES20.glDisable(GLES20.GL_DEPTH_TEST);
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
int vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexShaderCode);
int fragmentShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentShaderCode);
int filterVertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexShaderCode);
int filterFragmentShader = loadShader(GLES20.GL_FRAGMENT_SHADER, frameBufferShader);
mProgram = GLES20.glCreateProgram(); // create empty OpenGL ES Program
GLES20.glAttachShader(mProgram, vertexShader); // add the vertex shader to program
GLES20.glAttachShader(mProgram, fragmentShader); // add the fragment shader to program
GLES20.glLinkProgram(mProgram);
mProgramFilter = GLES20.glCreateProgram(); // create empty OpenGL ES Program
GLES20.glAttachShader(mProgramFilter, filterVertexShader); // add the vertex shader to program
GLES20.glAttachShader(mProgramFilter, filterFragmentShader); // add the fragment shader to program
GLES20.glLinkProgram(mProgramFilter);
int positionHandle = GLES20.glGetAttribLocation(mProgram, "aPosition");
int textureHandle = GLES20.glGetAttribLocation(mProgram, "aTextureCoord");
GLES20.glVertexAttribPointer(positionHandle, COORDS_PER_VERTEX, GLES20.GL_FLOAT, false, COORDS_PER_VERTEX * 4, mVertexBuffer);
GLES20.glEnableVertexAttribArray(positionHandle);
GLES20.glVertexAttribPointer(textureHandle, TEXTURECOORDS_PER_VERTEX, GLES20.GL_FLOAT, false, TEXTURECOORDS_PER_VERTEX * 4, mTextureBuffer);
GLES20.glEnableVertexAttribArray(textureHandle);
GLES20.glUseProgram(mProgram);
int i = GLES20.glGetUniformLocation(mProgram, "Ytex");
GLES20.glUniform1i(i, 3); /* Bind Ytex to texture unit 0 */
i = GLES20.glGetUniformLocation(mProgram, "Utex");
GLES20.glUniform1i(i, 1); /* Bind Utex to texture unit 1 */
i = GLES20.glGetUniformLocation(mProgram, "Vtex");
GLES20.glUniform1i(i, 2); /* Bind Vtex to texture unit 2 */
GLES20.glUseProgram(mProgramFilter);
i = GLES20.glGetUniformLocation(mProgramFilter, "Ytex");
GLES20.glUniform1i(i, 0);
mTextureWidth = 0;
mTextureHeight = 0;
}
static void initializeTexture(int name, int id, int width, int height)
{
GLES20.glActiveTexture(name);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, id);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, width, height, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, null);
}
void setupTextures(Frame frame)
{
if (mTextureIds[0] != 0)
{
GLES20.glDeleteTextures(4, mTextureIds, 0);
}
GLES20.glGenTextures(4, mTextureIds, 0);
int w = frame.getWidth();
int h = frame.getHeight();
int hw = (w + 1) >> 1;
int hh = (h + 1) >> 1;
initializeTexture(GLES20.GL_TEXTURE0, mTextureIds[0], w, h);
initializeTexture(GLES20.GL_TEXTURE1, mTextureIds[1], hw, hh);
initializeTexture(GLES20.GL_TEXTURE2, mTextureIds[2], hw, hh);
GLES20.glGenFramebuffers(1, frameBuffer);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, frameBuffer.get(0));
GLES20.glActiveTexture(GLES20.GL_TEXTURE3);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureIds[3]);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, w, h, 0, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, null);
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, mTextureIds[3], 0);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
mTextureWidth = frame.getWidth();
mTextureHeight = frame.getHeight();
GLES20.glUseProgram(mProgramFilter);
int i = GLES20.glGetUniformLocation(mProgramFilter, "width");
GLES20.glUniform1f(i, (float) mTextureWidth);
i = GLES20.glGetUniformLocation(mProgramFilter, "height");
GLES20.glUniform1f(i, (float) mTextureHeight);
}
void updateTextures(Frame frame)
{
int width = frame.getWidth();
int height = frame.getHeight();
int half_width = (width + 1) >> 1;
int half_height = (height + 1) >> 1;
int y_size = width * height;
int uv_size = half_width * half_height;
ByteBuffer bb = frame.getBuffer();
bb.clear(); // If we are reusing this frame, make sure we reset position and limit
if (bb.remaining() == y_size + uv_size * 2)
{
bb.position(0);
GLES20.glPixelStorei(GLES20.GL_UNPACK_ALIGNMENT, 1);
GLES20.glPixelStorei(GLES20.GL_PACK_ALIGNMENT, 1);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureIds[0]);
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, width, height, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, bb);
bb.position(y_size);
GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureIds[1]);
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, half_width, half_height, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, bb);
bb.position(y_size + uv_size);
GLES20.glActiveTexture(GLES20.GL_TEXTURE2);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureIds[2]);
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, half_width, half_height, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, bb);
}
else
{
mTextureWidth = 0;
mTextureHeight = 0;
}
}
#Override public void onSurfaceChanged(GL10 gl, int width, int height)
{
/// GLES20.glViewport(0, 0, width, height);
mViewportWidth = width;
mViewportHeight = height;
}
#Override public void onDrawFrame(GL10 gl)
{
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
mFrameLock.lock();
if (mCurrentFrame != null && !mVideoDisabled)
{
if (mTextureWidth != mCurrentFrame.getWidth() || mTextureHeight != mCurrentFrame.getHeight())
{
setupTextures(mCurrentFrame);
}
updateTextures(mCurrentFrame);
/// Step 1: Smoothing Filter - Render to FrameBuffer [pass 1]
Matrix.setIdentityM(mFilterScaleMatrix, 0);
GLES20.glViewport(0, 0, mTextureWidth, mTextureHeight);
GLES20.glUseProgram(mProgramFilter);
int mMVPFilterMatrixHandle = GLES20.glGetUniformLocation(mProgramFilter, "uMVPMatrix");
GLES20.glUniformMatrix4fv(mMVPFilterMatrixHandle, 1, false, mFilterScaleMatrix, 0);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, frameBuffer.get(0));
GLES20.glDrawElements(GLES20.GL_TRIANGLES, mVertexIndex.length, GLES20.GL_UNSIGNED_SHORT, mDrawListBuffer);
/// Step 2: Draw + RGB Conversion - Render to screen [pass 2]
Matrix.setIdentityM(mScaleMatrix, 0);
float scaleX = 1.0f, scaleY = 1.0f;
float ratio = (float) mCurrentFrame.getWidth() / mCurrentFrame.getHeight();
float vratio = (float) mViewportWidth / mViewportHeight;
if (mVideoFitEnabled)
{
if (ratio > vratio)
{
scaleY = vratio / ratio;
}
else
{
scaleX = ratio / vratio;
}
}
else
{
if (ratio < vratio)
{
scaleY = vratio / ratio;
}
else
{
scaleX = ratio / vratio;
}
}
Matrix.scaleM(mScaleMatrix, 0, scaleX * (mCurrentFrame.isMirroredX() ? -1.0f : 1.0f), scaleY, 1);
GLES20.glUseProgram(mProgram);
GLES20.glViewport(0, 0, mViewportWidth, mViewportHeight);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
int mMVPMatrixHandle = GLES20.glGetUniformLocation(mProgram, "uMVPMatrix");
GLES20.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, mScaleMatrix, 0);
GLES20.glDrawElements(GLES20.GL_TRIANGLES, mVertexIndex.length, GLES20.GL_UNSIGNED_SHORT, mDrawListBuffer);
}
mFrameLock.unlock();
}
public void displayFrame(Frame frame)
{
mFrameLock.lock();
if (this.mCurrentFrame != null)
{
this.mCurrentFrame.recycle();
}
this.mCurrentFrame = frame;
mFrameLock.unlock();
}
public static int loadShader(int type, String shaderCode)
{
int shader = GLES20.glCreateShader(type);
GLES20.glShaderSource(shader, shaderCode);
GLES20.glCompileShader(shader);
return shader;
}
public void disableVideo(boolean b)
{
mFrameLock.lock();
mVideoDisabled = b;
if (mVideoDisabled)
{
if (this.mCurrentFrame != null)
{
this.mCurrentFrame.recycle();
}
this.mCurrentFrame = null;
}
mFrameLock.unlock();
}
public void enableVideoFit(boolean enableVideoFit)
{
mVideoFitEnabled = enableVideoFit;
}
}
What you are trying to do is called downsampling.What you need to do is to render first into a small FBO, where both the viewport and the FBO attachment are of the same size. In that pass you can apply your blur effect. Then you can render or blit it to another FBO of the original size to get the texture scaled back.It's important to note that depending on your blue technique, the upscaled result may have noticeable degraded quality.
As some OpenGL API calls may have different namings in Android SDK,here is the general pseudo-code for what you need to do:
Let's w = original width and h = original height.
1.Create a custom FBO with texture attachment of the size w/2 and h/2 (if you plan to downsample at half the original resolution.
2.Attach the texture to the FBO.Bind the FBO for write.
3.set glViewport() to the same size as the FBO texture attachment.
4.Render a full screen quad.Apply your effect during this pass.
5.Second pass:Bind back the default framebuffer(alternatively - another custom FBO if you have other rendering stage later).If you just want to blit,then make sure you bind the fist FBO for read and not for write.
6.Bind the texture attached to the FBO from the previous pass to the sampler unit.
7.Set the viewport to w and h size.Draw the full screen quad.Do in the fragment shader whatever you need.(Gamma corrections,blending etc..)
That's it.What is nice in this technique is that OpenGL does the upscale/downscale for you automatically with the filtering type you select for the texture being involved in the process.
And on the side note:If you have performance issues this is not a good idea to write a renderer in Java.Though it is possible your issues are due to wrong API usage or inefficient algorithm application for the blur effect.
I'm having an issue rendering textures in openGL ES 2 on Android. The image is being drawn but the texture ism't wrapping correctly by the look of it.
I have tried all the usual things to fix the issue but nothing has worked.
Here's how one of the images should look:
But here's how they look on the screen:
Ignore the black border that's part of the texture.
Here is my Texture class:
public class HFTexture {
private int width;
private int height;
private int textureId;
private HFGame game;
private String textureFile;
public HFTexture(HFGame game, String textureFile) {
this.game = game;
this.textureFile = textureFile;
//load();
}
public void load() {
int[] texIds = new int[1];
GLES20.glGenTextures(1, texIds, 0);
textureId = texIds[0];
InputStream in;
try {
in = game.getFileManager().getAsset(textureFile);
Bitmap bitmap = BitmapFactory.decodeStream(in);
width = bitmap.getWidth();
height = bitmap.getHeight();
bind();
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bitmap, 0);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, 0);
bitmap.recycle();
} catch(IOException ex) {
}
}
public void bind() {
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureId);
}
public void activate(HFShader shader, int texture) {
GLES20.glActiveTexture(texture);
bind();
GLES20.glUniform1i(shader.getHandle("sampler0"), 0);
}
public void delete() {
bind();
int[] textureIds = {textureId};
GLES20.glDeleteTextures(1, textureIds, 0);
}
}
Here is my Vertices class:
public class Vertices {
private FloatBuffer vertexBuffer;
private FloatBuffer normalBuffer;
private FloatBuffer texCoordBuffer;
private ShortBuffer indexBuffer;
private final int VERTEX_COUNT;
private final int VERTEX_STRIDE;
private final int VERTEX_SIZE = 3;
private final int NORMAL_STRIDE;
private final int NORMAL_SIZE = 3;
private final int TEXTURE_COORD_STRIDE;
private final int TEXTURE_COORD_SIZE = 2;
private final int INDEX_COUNT;
public Vertices(float[] vertices, float[] normals, float[] texCoords, short[] indices) {
VERTEX_STRIDE = VERTEX_SIZE * 4;
NORMAL_STRIDE = NORMAL_SIZE * 4;
TEXTURE_COORD_STRIDE = TEXTURE_COORD_SIZE * 4;
VERTEX_COUNT = vertices.length;
INDEX_COUNT = indices.length;
ByteBuffer bb = ByteBuffer.allocateDirect(VERTEX_COUNT * VERTEX_STRIDE);
bb.order(ByteOrder.nativeOrder());
vertexBuffer = bb.asFloatBuffer();
vertexBuffer.put(vertices);
vertexBuffer.position(0);
bb = ByteBuffer.allocateDirect(normals.length * NORMAL_STRIDE);
bb.order(ByteOrder.nativeOrder());
normalBuffer = bb.asFloatBuffer();
normalBuffer.put(normals);
normalBuffer.position(0);
bb = ByteBuffer.allocateDirect(texCoords.length * TEXTURE_COORD_STRIDE);
bb.order(ByteOrder.nativeOrder());
texCoordBuffer = bb.asFloatBuffer();
texCoordBuffer.put(texCoords);
texCoordBuffer.position(0);
bb = ByteBuffer.allocateDirect(indices.length * 2);
bb.order(ByteOrder.nativeOrder());
indexBuffer = bb.asShortBuffer();
indexBuffer.put(indices);
indexBuffer.position(0);
}
public void bind(HFShader shader) {
int positionHandle = shader.getHandle("position");
int normalHandle = shader.getHandle("normal");
int texCoordHandle = shader.getHandle("texCoord");
GLES20.glEnableVertexAttribArray(positionHandle);
GLES20.glVertexAttribPointer(
positionHandle, VERTEX_SIZE,
GLES20.GL_FLOAT, false,
VERTEX_STRIDE, vertexBuffer);
GLES20.glEnableVertexAttribArray(normalHandle);
GLES20.glVertexAttribPointer(
normalHandle, NORMAL_SIZE,
GLES20.GL_FLOAT, false,
NORMAL_STRIDE, normalBuffer);
GLES20.glEnableVertexAttribArray(texCoordHandle);
GLES20.glVertexAttribPointer(
texCoordHandle, TEXTURE_COORD_SIZE,
GLES20.GL_FLOAT, false,
TEXTURE_COORD_STRIDE, vertexBuffer);
}
public void unbind(HFShader shader) {
int positionHandle = shader.getHandle("position");
int normalHandle = shader.getHandle("normal");
int texCoordHandle = shader.getHandle("texCoord");
GLES20.glDisableVertexAttribArray(positionHandle);
GLES20.glDisableVertexAttribArray(normalHandle);
GLES20.glDisableVertexAttribArray(texCoordHandle);
}
public void draw() {
if(indexBuffer != null) {
GLES20.glDrawElements(GLES20.GL_TRIANGLES, INDEX_COUNT, GLES20.GL_UNSIGNED_SHORT, indexBuffer);
} else {
GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, VERTEX_COUNT);
}
}
}
And here is my Vertex data:
float[] verts = {
-(width / 2f), (height / 2f), 0f, // index 0
-(width / 2f), -(height / 2f), 0f, // index 1
(width / 2f), -(height / 2f), 0f, // index 2
(width / 2f), (height / 2f), 0f // index 3
};
float[] norms = {
0.0f, 0.0f, -1.0f,
0.0f, 0.0f, -1.0f,
0.0f, 0.0f, -1.0f,
0.0f, 0.0f, -1.0f
};
float[] texCoords = {
0f, 1f,
0f, 0f,
1f, 0f,
1f, 1f
};
short[] indices = {
0,1,2,2,3,0
};
I've tried adding the clamp to edge texture parameters as well but that didn't seem to help. Have I just put the vertex and texture coords in the wrong order or is there something I'm missing altogether?
You are setting your vertex buffer instead of your texture coord buffer for the texture co-ordinates:
GLES20.glEnableVertexAttribArray(texCoordHandle);
GLES20.glVertexAttribPointer(
texCoordHandle, TEXTURE_COORD_SIZE,
GLES20.GL_FLOAT, false,
TEXTURE_COORD_STRIDE, vertexBuffer); // <-- here
should be:
GLES20.glEnableVertexAttribArray(texCoordHandle);
GLES20.glVertexAttribPointer(
texCoordHandle, TEXTURE_COORD_SIZE,
GLES20.GL_FLOAT, false,
TEXTURE_COORD_STRIDE, texCoordBuffer);
i create object 3d in android using tutorial from learnopengles, and i create cube from the lesson six of that tutorial (texture filtering), after that i want replace the cube with my object (i create the strawberry object). i want my object can display in the view, so i parsing the my object (my object use extension file .obj) to my renderer class, but the object in view is displaying random triangle object.
this is my parsing code :
public ObjLoader(Context mActivityContext) {
FileReader fr;
String str;
ArrayList<Float> tempModelVertices = new ArrayList<Float>();
ArrayList<Float> tempTextureVertices = new ArrayList<Float>();
ArrayList<Float> tempNormalVertices = new ArrayList<Float>();
ArrayList<Integer> facesM = new ArrayList<Integer>();
ArrayList<Integer> facesT = new ArrayList<Integer>();
ArrayList<Integer> facesN = new ArrayList<Integer>();
try {
fr = new FileReader(new File("model/straw_obj"));
BufferedReader br = new BufferedReader(fr);
while((str = br.readLine())!=null){
if(str.startsWith("f")){
String[] strAr = str.replaceAll("f", "").trim().split(" ");
for(String s : strAr){
String[] cornerAr = s.split("/");
facesM.add(Integer.parseInt(cornerAr[0].trim())-1);
facesT.add(Integer.parseInt(cornerAr[1].trim())-1);
facesN.add(Integer.parseInt(cornerAr[2].trim())-1);
}
}
else if(str.startsWith("vt")){
String[] strAr = str.replaceAll("vt", "").trim().split(" ");
tempTextureVertices.add(Float.valueOf(strAr[0].trim()));
tempTextureVertices.add(-1*Float.valueOf(strAr[1].trim()));
}
else if(str.startsWith("vn")){
String[] strAr = str.replaceAll("vn", "").trim().split(" ");
tempNormalVertices.add(Float.valueOf(strAr[0].trim()));
tempNormalVertices.add(Float.valueOf(strAr[1].trim()));
tempNormalVertices.add(Float.valueOf(strAr[2].trim()));
}
else if(str.startsWith("v")){
String[] strAr = str.replaceAll("v", "").trim().split(" ");
tempModelVertices.add(Float.valueOf(strAr[0].trim()));
tempModelVertices.add(Float.valueOf(strAr[1].trim()));
tempModelVertices.add(Float.valueOf(strAr[2].trim()));
}
}
//Log.v(LOG_TAG, "v :"+ String.valueOf(v) + "vt :"+ String.valueOf(vt) + "vn :"+ String.valueOf(vn) + "f :"+ String.valueOf(f));
} catch (IOException e) {
// TODO Auto-generated catch block
Log.v(TAG, "error");
}
Log.v(TAG, "vt " + String.valueOf(tempTextureVertices.size()) + " vn " + String.valueOf(tempNormalVertices.size()) + " v " + String.valueOf(tempModelVertices.size()));
ModelPositionData = new float[facesM.size()];
ModelTextureCoordinateData = new float[facesT.size()];
ModelNormalData = new float[facesN.size()];
for(int i=0; i<facesM.size(); i++){
ModelPositionData[i] = tempModelVertices.get(facesM.get(i));
}
for(int i=0; i<facesT.size(); i++){
ModelTextureCoordinateData[i] = tempTextureVertices.get(facesT.get(i));
}
for(int i=0; i<facesN.size(); i++){
ModelNormalData[i] = tempNormalVertices.get(facesN.get(i));
}
}
and this is how i create the glsurface renderer
public class TesterRenderer implements GLSurfaceView.Renderer{
private static final String TAG = "TesterRenderer";
private final Context mActivityContext;
/**
* Store the model matrix. This matrix is used to move models from object space (where each model can be thought
* of being located at the center of the universe) to world space.
*/
private float[] mModelMatrix = new float[16];
/**
* Store the view matrix. This can be thought of as our camera. This matrix transforms world space to eye space;
* it positions things relative to our eye.
*/
private float[] mViewMatrix = new float[16];
/** Store the projection matrix. This is used to project the scene onto a 2D viewport. */
private float[] mProjectionMatrix = new float[16];
/** Allocate storage for the final combined matrix. This will be passed into the shader program. */
private float[] mMVPMatrix = new float[16];
/** Store the accumulated rotation. */
private final float[] mAccumulatedRotation = new float[16];
/** Store the current rotation. */
private final float[] mCurrentRotation = new float[16];
/** A temporary matrix. */
private float[] mTemporaryMatrix = new float[16];
/**
* Stores a copy of the model matrix specifically for the light position.
*/
private float[] mLightModelMatrix = new float[16];
/** Store our model data in a float buffer. */
private final FloatBuffer mModelPositions;
private final FloatBuffer mModelNormals;
private final FloatBuffer mModelTextureCoordinates;
// private final FloatBuffer mModelTextureCoordinatesForPlane;
/** This will be used to pass in the transformation matrix. */
private int mMVPMatrixHandle;
/** This will be used to pass in the modelview matrix. */
private int mMVMatrixHandle;
/** This will be used to pass in the light position. */
private int mLightPosHandle;
/** This will be used to pass in the texture. */
private int mTextureUniformHandle;
/** This will be used to pass in model position information. */
private int mPositionHandle;
/** This will be used to pass in model normal information. */
private int mNormalHandle;
/** This will be used to pass in model texture coordinate information. */
private int mTextureCoordinateHandle;
/** How many bytes per float. */
private final int mBytesPerFloat = 4;
/** Size of the position data in elements. */
private final int mPositionDataSize = 3;
/** Size of the normal data in elements. */
private final int mNormalDataSize = 3;
/** Size of the texture coordinate data in elements. */
private final int mTextureCoordinateDataSize = 2;
/** Used to hold a light centered on the origin in model space. We need a 4th coordinate so we can get translations to work when
* we multiply this by our transformation matrices. */
private final float[] mLightPosInModelSpace = new float[] {0.0f, 0.0f, 0.0f, 1.0f};
/** Used to hold the current position of the light in world space (after transformation via model matrix). */
private final float[] mLightPosInWorldSpace = new float[4];
/** Used to hold the transformed position of the light in eye space (after transformation via modelview matrix) */
private final float[] mLightPosInEyeSpace = new float[4];
/** This is a handle to our cube shading program. */
private int mProgramHandle;
/** This is a handle to our light point program. */
private int mPointProgramHandle;
/** These are handles to our texture data. */
private int mTextureDataHandle;
// private int mGrassDataHandle;
/** Temporary place to save the min and mag filter, in case the activity was restarted. */
private int mQueuedMinFilter;
private int mQueuedMagFilter;
// These still work without volatile, but refreshes are not guaranteed to happen.
public volatile float mDeltaX;
public volatile float mDeltaY;
public TesterRenderer(final Context activityContext)
{
mActivityContext = activityContext;
ObjLoader obj = new ObjLoader(mActivityContext);
mModelPositions = ByteBuffer.allocateDirect(obj.ModelPositionData.length * mBytesPerFloat)
.order(ByteOrder.nativeOrder()).asFloatBuffer();
mModelPositions.put(obj.ModelPositionData).position(0);
mModelNormals = ByteBuffer.allocateDirect(obj.ModelNormalData.length * mBytesPerFloat)
.order(ByteOrder.nativeOrder()).asFloatBuffer();
mModelNormals.put(obj.ModelNormalData).position(0);
mModelTextureCoordinates = ByteBuffer.allocateDirect(obj.ModelTextureCoordinateData.length * mBytesPerFloat)
.order(ByteOrder.nativeOrder()).asFloatBuffer();
mModelTextureCoordinates.put(obj.ModelTextureCoordinateData).position(0);
}
#Override
public void onSurfaceCreated(GL10 glUnused, EGLConfig config)
{
// Set the background clear color to black.
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
// Use culling to remove back faces.
GLES20.glEnable(GLES20.GL_CULL_FACE);
// Enable depth testing
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
// The below glEnable() call is a holdover from OpenGL ES 1, and is not needed in OpenGL ES 2.
// Enable texture mapping
// GLES20.glEnable(GLES20.GL_TEXTURE_2D);
// Position the eye in front of the origin.
final float eyeX = 0.0f;
final float eyeY = 0.0f;
final float eyeZ = -0.5f;
// We are looking toward the distance
final float lookX = 0.0f;
final float lookY = 0.0f;
final float lookZ = -5.0f;
// Set our up vector. This is where our head would be pointing were we holding the camera.
final float upX = 0.0f;
final float upY = 1.0f;
final float upZ = 0.0f;
// Set the view matrix. This matrix can be said to represent the camera position.
// NOTE: In OpenGL 1, a ModelView matrix is used, which is a combination of a model and
// view matrix. In OpenGL 2, we can keep track of these matrices separately if we choose.
Matrix.setLookAtM(mViewMatrix, 0, eyeX, eyeY, eyeZ, lookX, lookY, lookZ, upX, upY, upZ);
final String vertexShader = RawResourceReader.readTextFileFromRawResource(mActivityContext, R.raw.per_pixel_vertex_shader_tex_and_light);
final String fragmentShader = RawResourceReader.readTextFileFromRawResource(mActivityContext, R.raw.per_pixel_fragment_shader_tex_and_light);
final int vertexShaderHandle = ShaderHelper.compileShader(GLES20.GL_VERTEX_SHADER, vertexShader);
final int fragmentShaderHandle = ShaderHelper.compileShader(GLES20.GL_FRAGMENT_SHADER, fragmentShader);
mProgramHandle = ShaderHelper.createAndLinkProgram(vertexShaderHandle, fragmentShaderHandle,
new String[] {"a_Position", "a_Normal", "a_TexCoordinate"});
// Define a simple shader program for our point.
final String pointVertexShader = RawResourceReader.readTextFileFromRawResource(mActivityContext, R.raw.point_vertex_shader);
final String pointFragmentShader = RawResourceReader.readTextFileFromRawResource(mActivityContext, R.raw.point_fragment_shader);
final int pointVertexShaderHandle = ShaderHelper.compileShader(GLES20.GL_VERTEX_SHADER, pointVertexShader);
final int pointFragmentShaderHandle = ShaderHelper.compileShader(GLES20.GL_FRAGMENT_SHADER, pointFragmentShader);
mPointProgramHandle = ShaderHelper.createAndLinkProgram(pointVertexShaderHandle, pointFragmentShaderHandle,
new String[] {"a_Position"});
// Load the texture
mTextureDataHandle = TextureHelper.loadTexture(mActivityContext, R.drawable.strawberry_texture);
GLES20.glGenerateMipmap(GLES20.GL_TEXTURE_2D);
// mGrassDataHandle = TextureHelper.loadTexture(mActivityContext, R.drawable.noisy_grass_public_domain);
// GLES20.glGenerateMipmap(GLES20.GL_TEXTURE_2D);
if (mQueuedMinFilter != 0)
{
setMinFilter(mQueuedMinFilter);
}
if (mQueuedMagFilter != 0)
{
setMagFilter(mQueuedMagFilter);
}
// Initialize the accumulated rotation matrix
Matrix.setIdentityM(mAccumulatedRotation, 0);
}
#Override
public void onSurfaceChanged(GL10 glUnused, int width, int height)
{
// Set the OpenGL viewport to the same size as the surface.
GLES20.glViewport(0, 0, width, height);
// Create a new perspective projection matrix. The height will stay the same
// while the width will vary as per aspect ratio.
final float ratio = (float) width / height;
final float left = -ratio;
final float right = ratio;
final float bottom = -1.0f;
final float top = 1.0f;
final float near = 1.0f;
final float far = 1000.0f;
Matrix.frustumM(mProjectionMatrix, 0, left, right, bottom, top, near, far);
}
#Override
public void onDrawFrame(GL10 glUnused)
{
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
// Do a complete rotation every 10 seconds.
long time = SystemClock.uptimeMillis() % 10000L;
long slowTime = SystemClock.uptimeMillis() % 100000L;
float angleInDegrees = (360.0f / 10000.0f) * ((int) time);
float slowAngleInDegrees = (360.0f / 100000.0f) * ((int) slowTime);
// Set our per-vertex lighting program.
GLES20.glUseProgram(mProgramHandle);
// Set program handles for cube drawing.
mMVPMatrixHandle = GLES20.glGetUniformLocation(mProgramHandle, "u_MVPMatrix");
mMVMatrixHandle = GLES20.glGetUniformLocation(mProgramHandle, "u_MVMatrix");
mLightPosHandle = GLES20.glGetUniformLocation(mProgramHandle, "u_LightPos");
mTextureUniformHandle = GLES20.glGetUniformLocation(mProgramHandle, "u_Texture");
mPositionHandle = GLES20.glGetAttribLocation(mProgramHandle, "a_Position");
mNormalHandle = GLES20.glGetAttribLocation(mProgramHandle, "a_Normal");
mTextureCoordinateHandle = GLES20.glGetAttribLocation(mProgramHandle, "a_TexCoordinate");
// Calculate position of the light. Rotate and then push into the distance.
Matrix.setIdentityM(mLightModelMatrix, 0);
Matrix.translateM(mLightModelMatrix, 0, 0.0f, 0.0f, -2.0f);
Matrix.rotateM(mLightModelMatrix, 0, angleInDegrees, 0.0f, 1.0f, 0.0f);
Matrix.translateM(mLightModelMatrix, 0, 0.0f, 0.0f, 3.5f);
Matrix.multiplyMV(mLightPosInWorldSpace, 0, mLightModelMatrix, 0, mLightPosInModelSpace, 0);
Matrix.multiplyMV(mLightPosInEyeSpace, 0, mViewMatrix, 0, mLightPosInWorldSpace, 0);
// Draw a cube.
// Translate the cube into the screen.
Matrix.setIdentityM(mModelMatrix, 0);
Matrix.translateM(mModelMatrix, 0, 0.0f, 0.0f, -7.0f);
// Set a matrix that contains the current rotation.
Matrix.setIdentityM(mCurrentRotation, 0);
Matrix.rotateM(mCurrentRotation, 0, mDeltaX, 0.0f, 1.0f, 0.0f);
Matrix.rotateM(mCurrentRotation, 0, mDeltaY, 1.0f, 0.0f, 0.0f);
mDeltaX = 0.0f;
mDeltaY = 0.0f;
// Multiply the current rotation by the accumulated rotation, and then set the accumulated rotation to the result.
Matrix.multiplyMM(mTemporaryMatrix, 0, mCurrentRotation, 0, mAccumulatedRotation, 0);
System.arraycopy(mTemporaryMatrix, 0, mAccumulatedRotation, 0, 16);
// Rotate the cube taking the overall rotation into account.
Matrix.multiplyMM(mTemporaryMatrix, 0, mModelMatrix, 0, mAccumulatedRotation, 0);
System.arraycopy(mTemporaryMatrix, 0, mModelMatrix, 0, 16);
// Set the active texture unit to texture unit 0.
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
// Bind the texture to this unit.
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureDataHandle);
// Tell the texture uniform sampler to use this texture in the shader by binding to texture unit 0.
GLES20.glUniform1i(mTextureUniformHandle, 0);
// Pass in the texture coordinate information
GLES20.glEnableVertexAttribArray(mTextureCoordinateHandle);
mModelTextureCoordinates.position(0);
GLES20.glVertexAttribPointer(mTextureCoordinateHandle, mTextureCoordinateDataSize, GLES20.GL_FLOAT, false,
0, mModelTextureCoordinates);
drawModel();
// Draw a plane
Matrix.setIdentityM(mModelMatrix, 0);
Matrix.translateM(mModelMatrix, 0, 0.0f, -2.0f, -5.0f);
Matrix.scaleM(mModelMatrix, 0, 25.0f, 1.0f, 25.0f);
Matrix.rotateM(mModelMatrix, 0, slowAngleInDegrees, 0.0f, 1.0f, 0.0f);
// Set the active texture unit to texture unit 0.
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
// Bind the texture to this unit.
//GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mGrassDataHandle);
// Tell the texture uniform sampler to use this texture in the shader by binding to texture unit 0.
GLES20.glUniform1i(mTextureUniformHandle, 0);
// Pass in the texture coordinate information
GLES20.glEnableVertexAttribArray(mTextureCoordinateHandle);
drawModel();
GLES20.glUseProgram(mPointProgramHandle);
drawLight();
}
public void setMinFilter(final int filter)
{
if (mTextureDataHandle != 0)
{
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureDataHandle);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, filter);
// GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mGrassDataHandle);
// GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, filter);
}
else
{
mQueuedMinFilter = filter;
}
}
public void setMagFilter(final int filter)
{
if (mTextureDataHandle != 0)
{
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureDataHandle);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, filter);
// GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mGrassDataHandle);
// GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, filter);
}
else
{
mQueuedMagFilter = filter;
}
}
private void drawModel()
{
// Pass in the position information
GLES20.glEnableVertexAttribArray(mPositionHandle);
mModelPositions.position(0);
GLES20.glVertexAttribPointer(mPositionHandle, mPositionDataSize, GLES20.GL_FLOAT, false,
0, mModelPositions);
// Pass in the normal information
GLES20.glEnableVertexAttribArray(mNormalHandle);
mModelNormals.position(0);
GLES20.glVertexAttribPointer(mNormalHandle, mNormalDataSize, GLES20.GL_FLOAT, false,
0, mModelNormals);
// This multiplies the view matrix by the model matrix, and stores the result in the MVP matrix
// (which currently contains model * view).
Matrix.multiplyMM(mMVPMatrix, 0, mViewMatrix, 0, mModelMatrix, 0);
// Pass in the modelview matrix.
GLES20.glUniformMatrix4fv(mMVMatrixHandle, 1, false, mMVPMatrix, 0);
// This multiplies the modelview matrix by the projection matrix, and stores the result in the MVP matrix
// (which now contains model * view * projection).
Matrix.multiplyMM(mTemporaryMatrix, 0, mProjectionMatrix, 0, mMVPMatrix, 0);
System.arraycopy(mTemporaryMatrix, 0, mMVPMatrix, 0, 16);
// Pass in the combined matrix.
GLES20.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, mMVPMatrix, 0);
// Pass in the light position in eye space.
GLES20.glUniform3f(mLightPosHandle, mLightPosInEyeSpace[0], mLightPosInEyeSpace[1], mLightPosInEyeSpace[2]);
// Draw the cube.
GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, 36);
}
/**
* Draws a point representing the position of the light.
*/
private void drawLight()
{
final int pointMVPMatrixHandle = GLES20.glGetUniformLocation(mPointProgramHandle, "u_MVPMatrix");
final int pointPositionHandle = GLES20.glGetAttribLocation(mPointProgramHandle, "a_Position");
// Pass in the position.
GLES20.glVertexAttrib3f(pointPositionHandle, mLightPosInModelSpace[0], mLightPosInModelSpace[1], mLightPosInModelSpace[2]);
// Since we are not using a buffer object, disable vertex arrays for this attribute.
GLES20.glDisableVertexAttribArray(pointPositionHandle);
// Pass in the transformation matrix.
Matrix.multiplyMM(mMVPMatrix, 0, mViewMatrix, 0, mLightModelMatrix, 0);
Matrix.multiplyMM(mTemporaryMatrix, 0, mProjectionMatrix, 0, mMVPMatrix, 0);
System.arraycopy(mTemporaryMatrix, 0, mMVPMatrix, 0, 16);
GLES20.glUniformMatrix4fv(pointMVPMatrixHandle, 1, false, mMVPMatrix, 0);
// Draw the point.
GLES20.glDrawArrays(GLES20.GL_POINTS, 0, 1);
}
}
can someone help me fix this ?
It looks like there is a problem with the way you reorder the coordinates based on the indices in the faces:
for(int i=0; i<facesM.size(); i++){
ModelPositionData[i] = tempModelVertices.get(facesM.get(i));
}
Each position consists of 3 coordinates. This loop copies only one value per position, though. It should look something like this:
for(int i=0; i<facesM.size(); i++){
ModelPositionData[3 * i ] = tempModelVertices.get(3 * facesM.get(i) );
ModelPositionData[3 * i + 1] = tempModelVertices.get(3 * facesM.get(i) + 1);
ModelPositionData[3 * i + 2] = tempModelVertices.get(3 * facesM.get(i) + 2);
}
You will also need to adjust the allocation accordingly:
ModelPositionData = new float[3 * facesM.size()];
and make the equivalent changes for the normals and texture coordinates.
Guys i need your help again :)
MainRenderer.java:
package com.example.galaga2d;
import java.nio.ByteBuffer;
import java.nio.FloatBuffer;
import java.nio.ByteOrder;
import java.util.Random;
import java.util.Vector;
import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.opengles.GL10;
import android.content.Context;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.opengl.GLES20;
import android.opengl.GLSurfaceView.Renderer;
import android.opengl.GLUtils;
public class MainRenderer implements Renderer {
Random rand = new Random();
float chance = 0.0f;
private Context context;
public Ship playerShip = new Ship();
Vector<Asteroid> asteroidVector = new Vector<Asteroid>();
public MainRenderer(Context context) {
this.context = context;
}
#Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
//! TEXTURES
playerShip.loadGLTexture(gl, this.context);
gl.glEnable(GL10.GL_TEXTURE_2D);
gl.glEnable(GL10.GL_BLEND);
gl.glBlendFunc(GL10.GL_ONE, GL10.GL_SRC_COLOR);
//gl.glShadeModel(GL10.GL_SMOOTH);
//! TEXTURES
gl.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
}
#Override
public void onDrawFrame(GL10 gl) {
gl.glClear(GL10.GL_COLOR_BUFFER_BIT);
gl.glLoadIdentity();
chance = rand.nextFloat() * (100.0f - 1.0f) + 1.0f;
if (chance <= 4.0f) {
asteroidVector.addElement(new Asteroid());
}
if (playerShip.life != 0) {
playerShip.draw(gl);
gl.glLoadIdentity();
for (int i = 0; i < asteroidVector.size(); i++) {
if(asteroidVector.elementAt(i).textured == 0) {
asteroidVector.elementAt(i).loadGLTexture(gl, this.context);
asteroidVector.elementAt(i).textured |= 1;
//gl.glLoadIdentity();
} else {
asteroidVector.elementAt(i).textured &= ~1;
}
}
for (int i = 0; i < asteroidVector.size(); i++) {
asteroidVector.elementAt(i).collisionCheck();
asteroidVector.elementAt(i).draw(gl);
if (asteroidVector.elementAt(i).Y > 480.0f) {
asteroidVector.remove(i);
}
gl.glLoadIdentity();
}
} else {
gl.glClearColor(0.0f, 0.0f, 1.0f, 1.0f);
}
}
#Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
gl.glViewport(0, 0, width, height);
gl.glMatrixMode(GL10.GL_PROJECTION);
gl.glLoadIdentity();
gl.glOrthof(0, width, height, 0, 1, -1);
gl.glViewport(0, 0, width, height);
gl.glMatrixMode(GL10.GL_MODELVIEW);
gl.glLoadIdentity();
}
// --------------------------------------------------------------------------------
class Ship {
public int life = 3; // Количество жизней игрока
public FloatBuffer ShipVertexBuffer; // Vertex буффер
public FloatBuffer ShipTextureBuffer; // Texture буффер
public float X = 100.0f; // Начальные координаты игрока по X
public float Y = 300.0f; // Начальные координаты игрока по Y
//! TEXTURES
private int[] textures = new int[1];
//! TEXTURES
public float ShipVerticles[] = { // Вертикли прямоугольника - корабль
0, 0,
0, 30,
30, 0,
30, 30
};
//! TEXTURES
public float ShipTextures[] = { // Разметка наложения текстуры, соответствует
0.0f, 0.0f, // разметке вертиклей
0.0f, 1.0f,
1.0f, 0.0f,
1.0f, 1.0f
};
//! TEXTURES
public Ship() {
//! Буффер вертексов
ByteBuffer bb = ByteBuffer.allocateDirect(36);
bb.order(ByteOrder.nativeOrder());
ShipVertexBuffer = bb.asFloatBuffer();
ShipVertexBuffer.put(ShipVerticles);
ShipVertexBuffer.position(0);
//! TEXTURES
bb = ByteBuffer.allocateDirect(ShipTextures.length * 4);
bb.order(ByteOrder.nativeOrder());
ShipTextureBuffer = bb.asFloatBuffer();
ShipTextureBuffer.put(ShipTextures);
ShipTextureBuffer.position(0);
//! TEXTURES
}
public void loadGLTexture(GL10 gl, Context context) {
// loading texture
Bitmap bitmap = BitmapFactory.decodeResource(context.getResources(),
R.drawable.ship);
// generate one texture pointer
gl.glGenTextures(1, textures, 0);
// ...and bind it to our array
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
// create nearest filtered texture
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_NEAREST);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
// Use Android GLUtils to specify a two-dimensional texture image from our bitmap
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap, 0);
// Clean up
bitmap.recycle();
}
public void draw(GL10 gl) {
//! TEXTURE
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
//! TEXTURE
gl.glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
gl.glTranslatef(playerShip.X, playerShip.Y, 0.0f);
gl.glVertexPointer(2, GL10.GL_FLOAT, 0, ShipVertexBuffer);
//! TEXTURE
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, ShipTextureBuffer);
//! TEXTURE
gl.glDrawArrays(GL10.GL_TRIANGLE_STRIP, 0, 4);
//! TEXTURE
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
gl.glDisableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
//! TEXTURE
}
}
class Asteroid {
private float colorR = rand.nextFloat()* (1.0f - 0.3f) + 0.3f;
private float colorG = rand.nextFloat()* (1.0f - 0.3f) + 0.3f;
private float colorB = rand.nextFloat()* (1.0f - 0.3f) + 0.3f;
private float X = rand.nextFloat() * (300.0f - 1.0f) + 1.0f;
private float Y = -30.0f;
private float size = rand.nextFloat() * (30.0f - 20.0f) + 20.0f;
private float speed = rand.nextFloat() * (10.0f - 1.0f) + 1.0f;
private int collision = 0;
private int textured = 0;
private FloatBuffer AsteroidVertexBuffer;
private FloatBuffer AsteroidTextureBuffer;
//! TEXTURES
private int[] textures = new int[1];
//! TEXTURES
public float AsteroidVerticles[] = {
0, 0, // лево низ
0, size, // лево вверх
size, 0, // право низ
size, size // право вверх
};
//! TEXTURES
public float AsteroidTextures[] = {
0.0f, 0.0f,
0.0f, 1.0f,
1.0f, 0.0f,
1.0f, 1.0f
};
//! TEXTURES
public Asteroid() {
ByteBuffer bb = ByteBuffer.allocateDirect(36);
bb.order(ByteOrder.nativeOrder());
AsteroidVertexBuffer = bb.asFloatBuffer();
AsteroidVertexBuffer.put(AsteroidVerticles);
AsteroidVertexBuffer.position(0);
//! TEXTURES
bb = ByteBuffer.allocateDirect(AsteroidTextures.length * 4);
bb.order(ByteOrder.nativeOrder());
AsteroidTextureBuffer = bb.asFloatBuffer();
AsteroidTextureBuffer.put(AsteroidTextures);
AsteroidTextureBuffer.position(0);
//! TEXTURES
}
public void collisionCheck() {
float result = (float)Math.sqrt(Math.pow((playerShip.X-X), 2)+Math.pow((playerShip.Y-Y), 2));
if (result < size)
{
if(collision == 0)
{
playerShip.life = playerShip.life - 1;
collision |= 1;
}
} else {
collision &= ~1;
}
}
public void loadGLTexture(GL10 gl, Context context) {
// loading texture
Bitmap bitmap = BitmapFactory.decodeResource(context.getResources(),
R.drawable.asteroid);
// generate one texture pointer
gl.glGenTextures(1, textures, 0);
// ...and bind it to our array
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
// create nearest filtered texture
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_NEAREST);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
// Use Android GLUtils to specify a two-dimensional texture image from our bitmap
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap, 0);
// Clean up
bitmap.recycle();
}
public void draw(GL10 gl) {
Y += speed;
//! TEXTURE
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
//! TEXTURE
gl.glColor4f(colorR, colorG, colorB, 1.0f);
gl.glTranslatef(X, Y, 0.0f);
gl.glVertexPointer(2, GL10.GL_FLOAT, 0, AsteroidVertexBuffer);
//! TEXTURE
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, AsteroidTextureBuffer);
//! TEXTURE
gl.glDrawArrays(GL10.GL_TRIANGLE_STRIP, 0, 4);
//! TEXTURE
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
gl.glDisableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
//! TEXTURE
}
}
// --------------------------------------------------------------------------------
}
Every frame drawing object asteroid by chance:
chance = rand.nextFloat() * (100.0f - 1.0f) + 1.0f;
if (chance <= 4.0f) {
asteroidVector.addElement(new Asteroid());
}
Thats mean we need to load texture for all new objects we draw every second, but we dont need to load texture for one object many times, and i add flag to check is object textured or not:
for (int i = 0; i < asteroidVector.size(); i++) {
if(asteroidVector.elementAt(i).textured == 0) {
asteroidVector.elementAt(i).loadGLTexture(gl, this.context);
asteroidVector.elementAt(i).textured |= 1;
//gl.glLoadIdentity();
} else {
asteroidVector.elementAt(i).textured &= ~1;
}
}
After object created and textured, we need to delete it if he go over screen border, so i do this:
for (int i = 0; i < asteroidVector.size(); i++) {
asteroidVector.elementAt(i).collisionCheck();
asteroidVector.elementAt(i).draw(gl);
//! THIS
if (asteroidVector.elementAt(i).Y > 480.0f) {
asteroidVector.remove(i);
}
//! THIS
gl.glLoadIdentity();
}
But that not enough, because tuxture buffer dont clear, and after 10-20 seconds on application running i have see some lagging and low fps.
The question is - How i can clear texture buffer or memory? To fix lagging and low fps?
I suppose you could do that, but then again you shouldn't be getting yourself into a situation where you actually need to in the first place.
As it now stands, you're creating a new texture object (and never freeing it!) for every Asteroid that's ever constructed, which is absolutely not necessary - a great way to fix that is to use a shared texture object for all of your Asteroid instances.
In a static function (called once), simply decode the image resource to raw pixel data -> glGenTextures -> glBindTexture -> glTexParam... calls ->GLUtils.TexImage2D -> free the (client-side) decoded raw pixel data and store the texture id (given by glGenTextures) into a static variable within the Asteroid class. Having done this, you can just bind to this shared texture object at the rendering stage (ie. draw), and proceed as you normally would.
As said before, never calling glDeleteTextures is the second part of your problem, although now that you're (hopefully) using a shared texture object for your Asteroid instances, it becomes much less important.
(Furthermore, even the vertex/texcoord buffer can be shared between instances, provided they are identical :P)