I can't get the texture tied to a SurfaceTexture to display in Unity.
Update 4: Based on the pipeline in update 1 (surface->external texture via surface texture -> fbo -> texture 2d) I know the SurfaceTexture isn't properly converting its surface to a texture. I can get correctly drawn pictures from its surface via pixelcopy and I can confirm my FBO drawing to texture2d pipeline works with some test colors. So the question is, why can't the SurfaceTexture convert its surface to a texture?
I generate a Texture in Java and pass its pointer back to Unity:
public void initGLTexture()
{
Log.d("Unity", "initGLTexture");
int textures[] = new int[1];
GLES20.glGenTextures(1, textures, 0);
mTextureId = textures[0];
GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mTextureId);
GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
}
I create a SurfaceTexture from the id (in Java):
mSurfaceTexture = new SurfaceTexture(mTextureId);
mSurfaceTexture.setDefaultBufferSize(512, 512);
I use a third-party library, GeckoView, to render onto the Surface of the SurfaceTexture. I call the following method from Unity's OnRenderObject() to keep all GL rendering on the same thread:
mSurfaceTexture.updateTexImage();
I know the above code allows proper drawing onto the surface.
I call the following in Unity to load the texture:
_imageTexture2D = Texture2D.CreateExternalTexture(
512,512,TextureFormat.RGBA32,false,true,(IntPtr) mTextureId);
_rawImage.texture = _imageTexture2D;
Why does the RawImage with the texture applied show only this sprite-looking thing, which should be a webpage?
Update 1: So I've been working on the hypothesis of: use Gecko to draw to the Surface, and use a SurfaceTexture to render this surface to a GL_TEXTURE_EXTERNAL_OES. Since I can't display this on Unity (not sure why) I am drawing this texture to a frame buffer and copying the pixels in the framebuffer to a GL_TEXTURE_2D. I am getting a web page in the texture_2d (in the emulator with an imageview and glReadPixels). However, when I import the work into Unity to test if the pipeline is okay thus far I just get a black screen. I CAN get images of the surface via the PixelCopy api.
Here is my FBO overview code - my rendering code comes from grafika's texture2D program:
// bind display buffer
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, mFrameBufferId);
GlUtil.checkGlError("glbindframebuffer");
// unbind external texture to make sure it's fresh
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0);
GlUtil.checkGlError("glunbindexternaltex");
// bind source texture (done in drawFrame as well )
GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mOffscreenTextureId);
GlUtil.checkGlError("glBindFramebuffer");
// draw to frame buffer
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f); // again, only really need to
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT); // clear pixels outside rect
mFullScreen.drawFrame(mOffscreenTextureId, mIdentityMatrix);
// unbind source texture
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D,0);
GlUtil.checkGlError("glBindTexture2d");
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0);
GlUtil.checkGlError("glunbindexternaltex");
// make sure we're still bound to fbo
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, mFrameBufferId);
GlUtil.checkGlError("glBindTexture2d");
// copy pixels from frame buffer to display texture
GLES20.glCopyTexImage2D(GLES20.GL_TEXTURE_2D,0,GLES20.GL_RGBA,0,0,512,512,0);
// read pixels from the display buffer to imageview for debugging
BitmapDisplay.mBitmap = SavePixels(0,0,512,512);
// unbind texture
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D,0);
Here's my player settings > other:
Update 2: Possible pipeline to try: call the draw function of the external texture to FBO (attached to Unity's texture_2d) in C++ via this interface.
Update 3: Calling the Java functions from native code that are responsible for drawing the texture from the SurfaceTexture to the FBO to Unity's texture via the GL.IssuePluginEvent produce a black texture as in the first update. It will show images in the emulator but not in Unity.
I had to do a similar task a couple of months ago and found out that the correct pipeline is creating a texture in Unity, obtaining a native pointer in C and finally updating it in the Java layer.
Please take a look at this sample project, it should give you a different perspective.
https://github.com/robsondepaula/unity-android-native-camera
Regards
Related
Alright I made an application that uses a FBO to render to a texture and later uses this texture to display it on a quad.
That's the code I am using to initialize the FBO and the texture:
IntBuffer intBuffer = IntBuffer.allocate(1);
GLES20.glGenFramebuffers(1, intBuffer);
fbo = intBuffer.get(0);
GLES20.glGenTextures(1, intBuffer);
tex = intBuffer.get(0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, tex);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, width, height, 0, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, null);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fbo);
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, tex, 0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, 0);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
(1. code in onDrawFrame())
The problem is that I don't know where to put the code, so at first I put the code in the onDrawFrame() method. When I do that the texturing works fine and I get the result that I want. But since I am creating a new FBO and texture every frame, after some time The application starts to lag and at the end the emulator crashes.
(2. code in onSurfaceChanged())
After noticing that behaviour I decided to put the code in the onSurfaceChanged() method. This time the application didn't crash but the entire texture is black. I tried to clear the texture or draw a white quad over it but nothing worked. And when I try to get an error from GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER) or GLES20.glGetError() it says that everything is fine.
I also tried to put the code in the onDrawFrame() method surrounded with a if-clause so that it's only executed once. The result is also the black texture.
(3. code in onDrawFrame() and onSurfaceChanged())
The last thing I tried was to put the code in both methods. I wanted to reduce the code in the onDrawFrame() method until I find the codeline that makes the texture work fine. But when I started the application the quad flickered, changing from the black texture to the right texture.
There is no problem with the texture displaying since it works with the one method. I also tried to add a depth-buffer to the code but the result is the same. So since I don't need it in my application I commented it out.
Is there anyone who can say what's going on? If someone has a application that works but doesn't know what's the problem with mine it would be nice if that person could upload the project because I didn't find a tutorial that has a android project up for download.
Alright thanks to Rabbid76 for giving me a checklist to find my problem. I tried to use the fbo as a texture. This is the right binding: GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, tex). In the tutorial I have read the FBO was also bound. I still don't know why it works when I initialize the FBO in the onDrawFrame() method and use it then as texture.
I just heard from a user who says that my (Android OpenGL ES 2.0) app (a game) won't run on his HTC 1X+ handset. All he gets is the music and the banner ad at the top of the screen and nothing else.
Unfortunately I don't have an HTC 1X+ to test this on.
Some notes:
Although my textures are not power of 2, I'm only using GLES20.GL_CLAMP_TO_EDGE
From what I've read, the HTC 1X+ has a max Texture Size of 2048 x 2048 and it gets it's resources from the XHDPI folder (annoyingly), even so, I have only 1 texure that exceeds that size, all other objects displayed on my app's opening page use textures much smaller than this max amount, so something should be displayed.
I'm not using texture compression of any kind
My app runs quite happily on the 15 (aprox) other devices I, and others have tested it on - just the 1x (so far) is giving problems.
Can anyone point out some common issues with OpenGL ES 2.0 that could be causing these textures not to be rendered? Are there any quirks with certain Android versions or devices?
I haven't yet posted any code simply because the app works on most devices, and I'm not sure which parts of the code would be helpful, but if any code is required, please just ask.
Edit - including texture loading code
public static int LoadTexture(GLSurfaceView view, Bitmap imgTex){
//Array for texture
int textures[] = new int[1];
try {
//texture name
GLES20.glGenTextures(1, textures, 0);
//Bind textures
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textures[0]);
//Set parameters for texture
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
//Apply texture
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, imgTex, 0);
//clamp texture
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T,GLES20.GL_CLAMP_TO_EDGE);
} catch (Exception e){
}
//Increase texture count
textureCount++;
//Return texture
return textures[0];
}
Have you checked the sampler properties?
The settings look correct, but why not also specify the GL_TEXTURE_WRAP_S setting? Use GLES20.glTexParameteri for integer values.
What type of bitmap are you using?
Try to force the internal format to GL_RGBA, GL_BGR, GL_RGB
Do you properly unbind or bind to a different texture?
Other texture settings may be causing havoc in other parts of the code..
Do you specify the correct texture unit in your shader?
Print out the shader's sampler attribute position so you know this is correct, and make sure to bind the texture to it explicitly during rendering.
Background:
Android native camera app uses a OpenGL_1.0 context to display camera preview and gallery pictures. Now I want to add a live-filter on the native camera preview.
To add a live-filter on my own camera app preview is simple --- just use the OpenGL_2.0 to do the image-processing and display. Since OpenGL_1.0 doses't support image-process and somehow it is used for display in the Android native camera app. *I now want to create a new GL context based on OpenGL_2.0 for image-processing and pass the processed image to the other GL context based on OpenGL_1.0 for display.*
Problem:
The problem is how to transfer the processed image from the GL-context-process (based on OpenGL_2.0) to the GL-context-display (based on OpenGL_1.0). I have tried to use FBO: first copy the image pixels from texture in GL-context-process and then set them back to another texture in GL-context-display. But copy pixels from texture is quite slow, typically takes hundreds of milliseconds. That is too slow for camera preview.
*Is there a better way to transfer textures from one GL context to another? Especially, when one GL context is based on OpenGL_2.0 while the other is based on OpenGL_1.0.*
I have found a solution using EGLImage. Just in case someone finds it useful:
Thread #1 that loads a texture:
EGLContext eglContext1 = eglCreateContext(eglDisplay, eglConfig, EGL_NO_CONTEXT, contextAttributes);
EGLSurface eglSurface1 = eglCreatePbufferSurface(eglDisplay, eglConfig, NULL); // pbuffer surface is enough, we're not going to use it anyway
eglMakeCurrent(eglDisplay, eglSurface1, eglSurface1, eglContext1);
int textureId; // texture to be used on thread #2
// ... OpenGL calls skipped: create and specify texture
//(glGenTextures, glBindTexture, glTexImage2D, etc.)
glBindTexture(GL_TEXTURE_2D, 0);
EGLint imageAttributes[] = {
EGL_GL_TEXTURE_LEVEL_KHR, 0, // mip map level to reference
EGL_IMAGE_PRESERVED_KHR, EGL_FALSE,
EGL_NONE
};
EGLImageKHR eglImage = eglCreateImageKHR(eglDisplay, eglContext1, EGL_GL_TEXTURE_2D_KHR, reinterpret_cast<EGLClientBuffer>(textureId), imageAttributes);
Thread #2 that displays 3D scene:
// it will use eglImage created on thread #1 so make sure it has access to it + proper synchronization etc.
GLuint texture;
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
// texture parameters are not stored in EGLImage so don't forget to specify them (especially when no additional mip map levels will be used)
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, eglImage);
// texture state is now like if you called glTexImage2D on it
Reference:
http://software.intel.com/en-us/articles/using-opengl-es-to-accelerate-apps-with-legacy-2d-guis
https://groups.google.com/forum/#!topic/android-platform/qZMe9hpWSMU
I hava a class in an Android App that holds a byte array as data source for a texture. I use a Framebuffer to render some stuff onto that texture and then render the texture on the screen. This works perfectly.
However, I can do this with 151 textures only. Instance #152 generates this error:
:0: PVRSRVAllocDeviceMem: Error 1 returned
:0: ComputeFrameBufferCompleteness: Can't create render surface.
Here is the code snippet (Constructor):
// Texture image bytes
imgBuf=ByteBuffer.allocateDirect(TEXEL_X*TEXEL_Y*3);
imgBuf.position(0);
// Fill the texture with an arbitrary color, so we see something
byte col=(byte)(System.nanoTime()%255);
for (int ii=0; ii<imgBuf.capacity(); ii+=3)
{ imgBuf.put(col);
imgBuf.put((byte)(col*3%255));
imgBuf.put((byte)(col*7%255));
}
imgBuf.rewind();
// Generate the texture
GLES20.glGenTextures(1,textureID,0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D,textureID[0]);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D,GLES20.GL_TEXTURE_WRAP_S,
GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T,
GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D,
GLES20.GL_TEXTURE_MIN_FILTER,GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D,GLES20.GL_TEXTURE_MAG_FILTER,
GLES20.GL_LINEAR);
// Associate a two-dimensional texture image with the byte buffer
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D,0,GLES20.GL_RGB,TEXEL_X,
TEXEL_Y,0,GLES20.GL_RGB,GLES20.GL_UNSIGNED_BYTE,imgBuf);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D,0);
// Get framebuffer for later rendering to this texture
GLES20.glGenFramebuffers(1,frameBufID,0);
And here is the problem (Render to texture)
If I leave out this part, displaying hundreds of such textures works well, but then I cannot render anyting onto the texture :( If I keep it, it works fine with 151 textures.
// Bind frame buffer and specify texture as color attachment
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER,frameBufID[0]);
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER,GLES20.GL_COLOR_ATTACHMENT0,GLES20.GL_TEXTURE_2D,textureID[0],0);
// Check status
int status=GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER);
Log.i(TAG,texNum+":"+status);
// Render some stuff on the texture
// ......
// (It does not matter. The status check fails even without rendering anything here)
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER,GLES20.GL_COLOR_ATTACHMENT0,GLES20.GL_TEXTURE_2D,0,0);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER,0);
I hope somebody can shed light upon this.
Thanks,
Ru
You are most probably using a device with PowerVR SGX540 GPU, they have this problem on android. Not on iOS, so this seems like a driver issue despite what their support says on their forum:
http://forum.imgtec.com/discussion/3026/glcheckframebufferstatus-returns-gl-framebuffer-unsupported-when-creating-too-many-fbo
If you really need to have more than 152 textures rendered to then you have to read out the pixels with glReadPixels(), delete the texture and fbo, and create a new texture by providing the data to glTexImage2D
I am programming an Android 2d game using opengl es 2.0. After I draw my sprites to the backbuffer I draw lights to a FBO and try to blend it to the back buffer again.
When I draw the FBO to the framebuffer, even trasparent without any color, the framerates drops from 60 to 30 on a Samsung Galaxy w (it has an adreno 205 as gpu). I searched everywhere and tried everything, even if I draw a single sprite on the scene and blend a trasparent FBO texture to the screen the framerate drops. I tried other games with lighting effects on that phone and they run fine, almost every game is fine on that phone, I believe they use the framebuffer as well.
On the Galaxy SII (mali 400 gpu) runs fine, I am quite new to opengl so I believe I am making a mistake somewhere, I share my code.
// Create a framebuffer and renderbuffer
GLES20.glGenFramebuffers(1, fb, offset);
GLES20.glGenRenderbuffers(1, depthRb, offset);
// Create a texture to hold the frame buffer
GLES20.glGenTextures(1, renderTex, offset);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, renderTex[offset]);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA,
screenWidth, screenHeight, 0,
GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE,
null);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S,
GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T,
GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER,
GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER,
GLES20.GL_LINEAR);
//bind renderbuffer
GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, depthRb[offset]);
GLES20.glRenderbufferStorage(GLES20.GL_RENDERBUFFER, GLES20.GL_DEPTH_COMPONENT16,
screenWidth, screenHeight);
// bind the framebuffer
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fb[offset]);
// specify texture as color attachment
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0,
GLES20.GL_TEXTURE_2D, renderTex[offset], 0);
// specify depth_renderbufer as depth attachment
GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_DEPTH_ATTACHMENT,
GLES20.GL_RENDERBUFFER, depthRb[0]);
// Check FBO status.
int status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER);
if ( status == GLES20.GL_FRAMEBUFFER_COMPLETE )
{
Log.d("GLGame framebuffer creation", "Framebuffer complete");
}
// set default framebuffer
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
I do this once on surface creation. Not sure if is correct. I keep the texture and framebuffer ids to switch to them when I need.
My drawing code:
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureId);
ShaderProgram program = glgame.getProgram();
//put vertices in the floatbuffer
mTriangleVertices.put(vertices, 0, len);
mTriangleVertices.flip();
GLES20.glVertexAttribPointer(program.POSITION_LOCATION, 2, GLES20.GL_FLOAT, false,
TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices);
//preparing parameter for texture position
mTriangleVertices.position(TRIANGLE_VERTICES_DATA_UV_OFFSET);
GLES20.glEnableVertexAttribArray(program.POSITION_LOCATION);
//preparing parameter for texture coords
GLES20.glVertexAttribPointer(program.TEXTURECOORD_LOCATION, 2, GLES20.GL_FLOAT,
false, TRIANGLE_VERTICES_DATA_STRIDE_BYTES,
mTriangleVertices);
//set projection matrix
GLES20.glEnableVertexAttribArray(program.TEXTURECOORD_LOCATION);
GLES20.glUniformMatrix4fv(program.MATRIX_LOCATION, 1, false, matrix, 0);
//draw triangle with indices to form a rectangle
GLES20.glDrawElements(GLES20.GL_TRIANGLES, numSprites * 6, GLES20.GL_UNSIGNED_SHORT,
indicesbuf);
//clear buffers
mTriangleVertices.clear();
mVertexColors.clear();
Everything is rendered on screen correctly, but the performance are ruined just when I draw the FBO texture.
Thank you very much for your help. I worked very hard on this and didn't find a solution.
According to qualcomm docs, you need to glclear after every glbindframebuffer, this is a problem related to tiled architecture, if you are switching framebuffers, data need to get copied from fastmem to normal memory to save current framebuffer and from slowmem to fast mem to get contents of newly binded frame, in case you are clearing just after glbind no data is copied from slowmem to fastmem and you are saving time, but you need to redesign your render pipeline often, so it will avoid reading data back and forth between slow and fast memory, so try to do glclear after each bind and it should help, you can also use adreno profiler to get additional information about problematic calls, but i doubt it will help with adreno200 i am trying to get two buffers for blur and i am ending with 10fps, bindframebuffer call can take up to 20msec if its not cleared, if it is it should end at 2ms.