I have a ByteArray and want to create a bitmap out of it using OpenGL. In Android, there is decodeByteArray() method which returns a Bitmap object that can be drawn on ImageView.
What is the equivalent method available in OpenGL?
There is no equivalent function in OpenGL (ES) since it is a pure API and not made for decoding byte arrays.
However, if you want to apply some texture onto your model you might use an approach like this:
gl.glGenTextures(1, textures, 0);
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER,GL10.GL_NEAREST);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER,GL10.GL_LINEAR);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_S,GL10.GL_REPEAT);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_T,GL10.GL_REPEAT);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, texture, 0);
texture.recycle();
Related
My goal is to add some text info in the video output file obtained after recording a video with the Camera2 API (e. g. date/time, user id etc.). I have checked some references about how to do this using camera API but I didn't find any information about how to do achieve this with Camera2 API. Can anyone help me?
This is what I found for camera API
The link that you provided about how to achieve your solution using Camera API will work also for Camera2 API. You should generate a GLSurfaceView with the information that you want to achieve together with a GLSurfaceView.Renderer to process each frame of your camera with OpenGL.
After configure your surface you should generate a new Surface from your SurfaceTexture:
Surface videoSurface = new Surface(surfaceGLTexture);
After that you can use createCaptureSession together with your Surface and a CameraCaptureSession.StateCallback()to generate a video preview using CameraDevice.TEMPLATE_RECORD in your CaptureRequest.Builder.
Take a look at grafica. Take a look specifically at TextureMovieEncoder.java. Replace the code in private void drawBox(int posn) with the code found in the best answer to this question:
Draw text in OpenGL ES
copied here for reference:
// Create an empty, mutable bitmap
Bitmap bitmap = Bitmap.createBitmap(256, 256, Bitmap.Config.ARGB_4444);
// get a canvas to paint over the bitmap
Canvas canvas = new Canvas(bitmap);
bitmap.eraseColor(0);
// get a background image from resources
// note the image format must match the bitmap format
Drawable background = context.getResources().getDrawable(R.drawable.background);
background.setBounds(0, 0, 256, 256);
background.draw(canvas); // draw the background to our bitmap
// Draw the text
Paint textPaint = new Paint();
textPaint.setTextSize(32);
textPaint.setAntiAlias(true);
textPaint.setARGB(0xff, 0x00, 0x00, 0x00);
// draw the text centered
canvas.drawText("Hello World", 16,112, textPaint);
//Generate one texture pointer...
gl.glGenTextures(1, textures, 0);
//...and bind it to our array
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
//Create Nearest Filtered Texture
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_NEAREST);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
//Different possible texture parameters, e.g. GL10.GL_CLAMP_TO_EDGE
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_S, GL10.GL_REPEAT);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_T, GL10.GL_REPEAT);
//Use the Android GLUtils to specify a two-dimensional texture image from our bitmap
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap, 0);
//Clean up
bitmap.recycle();
The same approach works, pretty much - the camera2 API wants a Surface to draw to, but you can create one from a SurfaceTexture:
Surface s = new Surface(mSurfaceTexture);
Then you can pass this Surface to the camera2 CameraDevice.createCaptureSession() call; see for example Camera2Video for a basic recording app to modify.
From your GL rendering, you have to then send the data to a video encoder.
FFMPEG provides too many thing that you can do what you want. Please try this tool. May be it will help you. I suggest you. Please check this link:
https://stackoverflow.com/a/38299320/3992798
onPreviewFrame only gets called when the preview frames from the camera get displayed. I'm processing the image as an open gl texture using the same technique here:
http://nhenze.net/?p=172&cpage=1#comment-8424
But it seems like a waste to render the preview to the screen just so I can draw over it with my textured image. Is there a better way to get the pixels from the camera than during the call to onPreviewFrame?
yes, you can use a SurfaceTexture. There's some mild trickiness here as it needs to be an external texture not the normal 2D texture.
This means that if you render the texture with ES2 you need some like
#extension GL_OES_EGL_image_external : require
uniform samplerExternalOES s_texture;
in the fragment shader.
Example code:
int[] textures = new int[1];
// generate one texture pointer and bind it as an external texture.
GLES20.glGenTextures(1, textures, 0);
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textures[0]);
// No mip-mapping with camera source.
GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
GL10.GL_TEXTURE_MIN_FILTER,
GL10.GL_LINEAR);
GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
// Clamp to edge is only option.
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
GL10.GL_TEXTURE_WRAP_T, GL10.GL_CLAMP_TO_EDGE);
int texture_id = textures[0];
SurfaceTexture mTexture = new SurfaceTexture(texture_id);
mTexture.setOnFrameAvailableListener(this);
Camera cam = Camera.open();
cam.setPreviewTexture(mTexture);
Why do my textures seemingly take up so much space?
My app, which uses opengl heavily, produces the following heap stats* during operation:
Used heap dump 1.8 MB
Number of objects 49,447
Number of classes 2,257
Number of class loaders 4
Number of GC roots 8,551
Format hprof
JVM version
Time 1:08:15 AM GMT+02:00
Date Oct 2, 2011
Identifier size 32-bit
But, when I use the task manager on my phone to look at the ram use of my application it says my app uses 44.42MB. Is there any relationship between heap size use and ram use? I think much of that 42MB must be my open GL textures, but I can't figure out why they take up so much space, because on disk all the files together take only take up 24MB (and they are not all loaded at the same time). And I'm even making many of them smaller by resizing the bitmap prior to texture loading. I also dynamically create some textures, but also destroy those textures after use.
I am using OpenGL 1.0 and Android 2.2, typical code that I use to load a texture looks like this:
static int set_gl_texture(Bitmap bitmap){
bitmap = Bitmap.createScaledBitmap(bitmap, 256, 256, true);
// generate one texture pointer
mGL.glGenTextures(1, mTextures, 0);
mGL.glBindTexture(GL10.GL_TEXTURE_2D, mTextures[0]); // A bound texture is
// an active texture
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0,GL10.GL_RGBA, bitmap, 0);
// create nearest filtered texture
mGL.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER,
GL10.GL_LINEAR); // This is where the scaling algorithms are
mGL.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER,
GL10.GL_LINEAR); // This is where the scaling algorithms are
mGL.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_S,
GL10.GL_CLAMP_TO_EDGE);
mGL.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_T,
GL10.GL_CLAMP_TO_EDGE);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap, 0);
bitmap.recycle();
Log.v("GLSurfaceView", "Loading Texture Finished, Error Codes:"+mGL.glGetError());
return mTextures[0];
}
Code which I use to load bitmaps looks like the following:
public int load_texture(int res_id){
if(mBitmapOpts==null){
mBitmapOpts = new BitmapFactory.Options();
mBitmapOpts.inScaled = false;
}
mBtoLoad = BitmapFactory.decodeResource(MyApplicationObject.getContext().getResources(),res_id, mBitmapOpts);
assert mBtoLoad != null;
return GraphicsOperations.set_gl_texture(mBtoLoad);
}
*hprof file analyzed using mat, same data is generated by eclipse ddms
PNGs are compressed images. In order for OpenGL to use them, the pngs must be decompressed. This decompression will increase the memory size.
You may want to decrease the size of some of the textures somehow. Maybe instead of using 512x512 images, use 256x256 or 128x128. Some textures that you use may not need to be so large since they are going onto a mobile device with a limited screen size.
I am now able to load a texture to the project I have made but it gave me results that I was not expecting. Instead of loading my 91 tiles and having them have crate textures, and having a picture of a puppy be at 10x and 10 y, everything is the puppy.
This is my draw code
#Override
public void onDrawFrame(GL10 gl) {
// TODO Auto-generated method stub
gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
gl.glLoadIdentity();
for(int i =0;i<91;i++)
{
myfloortiles[i].draw(gl);
}
gl.glLoadIdentity();
gl.glOrthof(0, width, height, 0, 0, 1);
gl.glDisable(GL10.GL_DEPTH_TEST);
// gl.glClear(GL10.GL_COLOR_BUFFER_BIT);
gl.glTranslatef(10, 10, 1);
bitmap=BitmapFactory.decodeResource(context.getResources(), R.drawable.puppy);
int textureID;
int[] temp = new int[1];
gl.glGenTextures(1, temp, 0);
textureID = temp[0];
gl.glBindTexture(GL10.GL_TEXTURE_2D, textureID);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER,
GL10.GL_NEAREST);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER,
GL10.GL_NEAREST); // GL_LINEAR for quality.
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_S,
GL10.GL_CLAMP_TO_EDGE);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_T,
GL10.GL_CLAMP_TO_EDGE);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap, 0);
bitmap.recycle();
}
Of course I have to be doing something wrong. Is it that I have to make a 3d object and then render the texture to it? Or is there another way? Also how would I have an animation if i chose the 3d object with texture path? Would I swap in new images when I want it to draw a different image for the interface? Is my place the image in a specific location logic correct? Or is that also terribly incorrect also?
Technically not a mistake, but a problem is generating texture names and creating a texture object in the drawing function. Texture creating is a one-time thing:
You generate a texture name, which is a handle by which you refer to the texture later (glGenTextures)
You bind that name (glBindTexture)
You upload the image (glTexImage and/or glTexSubImage)
For drawing the actual textured meshes you
enable texturing (*glEnable(GL_TEXTURE_…)*)
bind the texture(s) (glBindTexture)
draw the mesh(es)
In your code you're going through the whole texture creation phase for every redraw. The last created texture is also the bound texture and thus will be subsequently applied to the following drawing calls. Which are about to happen in the next call of onDrawFrame
On a side note: You clearly have not understood how the matrix stack works. You mixed calls of glOrtho, glLoadIdentity, glTranslate and so on without any sensible structure. You need to get the understanding of those straigt, too. I'd say even before you tackle texturing.
I've created an opengl surface and everything works fine, however when I try to draw text onto it using the following method:
public void loadFPSTexture(GL10 gl){
Bitmap bitmap = Bitmap.createBitmap(256, 256, Bitmap.Config.RGB_565);
bitmap.eraseColor(Color.BLACK);
Canvas canvas = new Canvas(bitmap);
Paint textPaint = new Paint();
textPaint.setTextSize(35);
textPaint.setFakeBoldText(true);
textPaint.setAntiAlias(true);
textPaint.setARGB(255, 255, 255, 255);
canvas.drawText("FPS "+reportedFramerate, 10,35, textPaint);
gl.glGenTextures(1, texturesFPS, 0);
gl.glBindTexture(GL10.GL_TEXTURE_2D, texturesFPS[0]);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_LINEAR);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_T, GL10.GL_REPEAT);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap, 0);
bitmap.recycle();
}
Then used in my onDraw function with:
gl.glPushMatrix();
gl.glEnable(GL10.GL_TEXTURE_2D);
gl.glBindTexture(GL10.GL_TEXTURE_2D, texturesFPS[0]);
gl.glTranslatef(-surfaceSize.x/1.5f, surfaceSize.y/1.5f, 0.0f);
gl.glScalef(10, 10, 1.0f);
gl.glColor4f(1.0f, 1.0f, 1.0f, saturation_head);
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
gl.glVertexPointer(3, GL10.GL_FLOAT, 0,vertexBuffer);
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, textureFPSBuffer);
gl.glDrawElements(GL10.GL_TRIANGLES, indices.length,GL10.GL_UNSIGNED_SHORT, indexBuffer);
gl.glDisableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
gl.glDisable(GL10.GL_TEXTURE_2D);
gl.glPopMatrix();
I get a weird crash and the surface is unable to be drawn after this point. Logcat shows a constant stream of the following:
E/Adreno200-EGL( 2578): eglLockWindowSurface: failed to map the memory for fd=32 offs=1536000
E/SurfaceFlinger( 2578): GL error 0x0505
E/Adreno200-EGL( 2578): eglLockWindowSurface: failed to map the memory for fd=32 offs=1536000
E/Adreno200-EGL( 2578): egliSwapWindowSurface: oglSwapBuffer failed
E/SurfaceFlinger( 2578): eglSwapBuffers: EGL error 0x3003 (EGL_BAD_ALLOC)
I'm not sure why this is happening? any help would be much appreciated!
The trouble is that I was generating textures repeatedly without ever deleting them. Simply adding one line before generation is enough to prevent a memory leak (it seems there is no need to check that a texture has already been generated):
gl.glDeleteTextures(1, texturesFPS, 1);
gl.glGenTextures(1, texturesFPS, 1);
Simple as that :)
Looks like you're creating a new texture each time you call loadFPSTexture(), and never release it. After some time this will cause you to run out of memory, which could explain the EGL_BAD_ALLOC in your logs.
It would be better to create your bitmap, canvas and texturesFPS variables just once, and reuse them in the loadFPSTexture() function. In that case, you should probably use GLUtils.texSubImage2D() instead of GLUtils.texImage2D(), to upload new bitmap data to the existing texture.
I don't have comment access yet (frequent other 'help' sites, just getting up on SO), but wanted to echo (and upvoted) svdree's note, and add more details -- because other people starting with GLES will certainly hit similar issues.
Your bitmap, canvas, and texture (and paint!) should be created once, wherever you set up your gles resources to begin with. They should be deleted when you clean up resources for the app. Unless you are resizing the bitmap/texture, recreating is thrashing memory (cpu and gpu) unnecessarily.
The initial creation of the texture, you'd use the GLUtils.texImage2D function to prepare the texture (can just upload the bitmap as-is, don't care about the data). That ensures the texture is allocated by the driver, proper width/height buffer ready for later updates.
Rendering the fps might then look more like:
s_bitmap.eraseColor(Color.BLACK);
s_canvas.drawText("FPS "+reportedFramerate, 10, 35, s_textPaint);
gl.glBindTexture(GL10.GL_TEXTURE_2D, s_texturesFPS[0]);
GLUtils.texSubImage2D(GL10.GL_TEXTURE_2D, 0, 0, 0, s_bitmap);
... and that's it. An order of magnitude faster, and obviously much cleaner. :-)
You can make it even faster beyond that, for example only erase/fill the bitmap rectangle where the fps is being drawn, and then use gl.glTexSubImage2D directly to upload only those rows the text is rendered to (saving you clearing, and uploading, say 220 extra rows of data that isn't changing...).
Hope that helps!