I'm working on this photo app that uses OpenGL 2.0 with a Renderer, an off-screen GLSurfaceView and some shader scripts (*.fsh and *.vsh).
after loading the shader scripts from Assets folder, preparing the GL surface and context, etc, etc we finally call GLES20.glDrawArrays(GLES20.GL_TRIANGLE_FAN, 0, 4); and it works quite nicely and it generates the bitmaps with the effects.
The problem, OF COURSE, is the memory limitations and any large enough bitmap (regardless of device, not so big for old Gingerbread and very large images for the Nexus 10) and it will produce and OutOfMemoryException.
I'm not so knowledgeable in OpenGL and the way I know to deal with very large amounts of data is to use a stream so it's not necessary to hold it all in memory.
So the question is, is there a way to do apply an openGl shader/renderer through a Stream instead of a in-memory Bitmap ? If yes, any pointer to a link or base procedure?
Not exactly sure what you mean by Stream but here's another solution. Split rendering up into multiple passes. Fore instance, if you have a 512x512 texture and a corresponding quad to texture but can only afford to upload a 256x256 due to memory restrictions do the following:
split up the texture into 4 chunks
create a single, fitting texture object
for each chunk
upload the current chunk into the tex objects data store
draw 1/4 of the quad, e.g. top-left and texture accordingly
Note that the above example assume a 512x512 texture and screen-size. In any case, I think you get the idea.
Obviously, this is the usual memory/performance trade-off.You circumvent memory restrictions by using more bandwidth for transfers and do more rendering.
Note: I'm a desktop GL guy and I'm not quite sure how memory is split up betweem the GPU and the rest, or if there even is some dedicated VRAM. I assume you've got a limited amount available for GL resources which is even smaller than the overall system memory.
Related
I would like to know if there is any kind of limitation on the texture size that can be used in any Android Opengl Es 2.0 projects. I understand that having a huge texture of size 4096x4096 is a bit meaning less as it is rendered on a small screen. But What if the requirement is to switch between many textures at run time? And If I want to have a texture atlas to do a quick single upload instead of multiple smaller texture upload. Please let me know your ideas in this regards.
Also I am sure there has to be a limitation on the size of image that can be processed by a device, as the memory on the device is limited. But I would like to know if it is resolution based or is it size based. I mean if a device has a limitation of 1024x1024 image size can it handle a compressed texture of size 2048x2048 that would be of same size approx as uncompressed 1024x1024.
Also please let me know on an general basis usually how much the limitation on texture size or resolution normal devices running android 2.2 and above would be.
Also please let me know if there are any best practices when handling high resolution images in opengles 2.0 to get best performance in both load time and also run time.
There is a hardware limitation on the texture sizes. To manually look them up, you can go to a site such as glbenchmark.com (Here displaying details about google galaxy nexus).
To automatically find the maximum size from your code, you can use something like:
int[] max = new int[1];
gl.glGetIntegerv(GL10.GL_MAX_TEXTURE_SIZE, max, 0); //put the maximum texture size in the array.
(For GL10, but the same method exists for GLES20)
When it comes to the processing or editing of an image you usually use an instance of Bitmap when working in android. This holds the uncompressed values of your image and is thus resolution dependant. However, it is recommended that you use compressed textures for your openGL applications as this improves the memory-use efficiency (note that you cannot modify these compressed textures).
From the previous link:
Texture compression can significantly increase the performance of your
OpenGL application by reducing memory requirements and making more
efficient use of memory bandwidth. The Android framework provides
support for the ETC1 compression format as a standard feature [...]
You should take a look at this document which contains many good practices and hints about texture loading and usage. The author explicitly writes:
Best practice: Use ETC for texture compression.
Best practice: Make sure your geometry and texture resolutions are
appropriate for the size they're displayed at. Don't use a 1k x 1k
texture for something that's at most 500 pixels wide on screen. The
same for geometry.
I created a movie player based on FFmpeg. It works fine. The decoding is quite fast, on LG P970 (Cortex A8 with Neon) I have an average 70 fps with 640 x 424 resolution video stream including YUV2RGB conversion. However, there is one bottleneck. It is drawing on Canvas.
I use jnigraphics native library to fill picture data into the bitmap in the native side and then I draw this bitmap on Canvas in SurfaceView. It is quite simple and common approach, but the drawing takes 44 ms for bitmap with 640 x 424 resolution which reduces fps to 23 and makes this technique unusable... It takes a lot more then the whole A/V frame decoding!
Is there any method how to draw bitmaps significantly faster? I would prefer to render completely in the native code using OpenGLES 2, but I have read it also could be slow. So what now?...
How can I render bitmaps as fast as possible?
Draw them in GLES1.x. You do not need to use GLES2 as you will have no use, or at least not in the context of your question, for shaders which would be the general primary reason of using GLES2.x. So for simplicity sake, GLES1.x would be ideal. All you need to do is draw the bytebuffer to the screen. On my Galaxy S (Vibrant) this takes about 3ms. The size of the byte[] in my wallpaper is 800x480x3 or 1152000 which is significantly larger than what you are working with.
I believe this guide should point you in the correct direction.
http://qdevarena.blogspot.com/2009/02/how-to-load-texture-in-android-opengl.html
As for the notion of accessing canvas from native code, I would just avoid that altogether and follow an OpenGL implementation by offloading everything to GPU as much as possible.
I recall during the Replica Island presentation during GoogleIO the designer said that using the OpenGL 'draw_texture' extension glDrawTexfOES was the fastest way to blit to the screen, and significantly faster than drawing just regular quads with textures attached (I'm assuming you're using OpenGL).
You can't rotate the texture, but it doesn't sound like you need that.
In OpenGL ES 1.1, I would like to take multiple texture Ids and combine them into a single textureId. Then I would be able to use this resulting texture multiple times in the future. My texture sources could be transparent PNGs that I want to stack together. This would be a huge optimization since I wouldn't have to render multiple textures every frame.
I have seen examples like the wiki Texture_Combiners, but it doesn't seem like the results are reusable.
Also, if there is a way to mask an image with another into a reusable texture, that would be extremely helpful too.
What you want to do is render to texture. If you're writing for iOS you're guaranteed that the OES framebuffer extension will be available, so you can use that. If you're writing for Android or another platform then the extension may be available but isn't guaranteed. If it isn't available you can fall back on glCopyTexImage2D.
So in the first case you'd create a frame buffer which has a texture as its colour buffer. Render to that then switch to another frame buffer and you can henceforth draw from the texture.
In the second you'd draw into whatever frame buffer you have, then use glCopyTexImage2D to copy from the current colour buffer into a texture. This will be a little slower because it's a copy, but it'll still probably be a lot faster than reading back the rendered content and then uploading it yourself.
ES 2.0 makes the functions contained in the framebuffer extension mandatory, so ES 2.0 capable GPUs are very likely to support the extension.
I would like to know if there is any kind of limitation on the texture size that can be used in any Android Opengl Es 2.0 projects. I understand that having a huge texture of size 4096x4096 is a bit meaning less as it is rendered on a small screen. But What if the requirement is to switch between many textures at run time? And If I want to have a texture atlas to do a quick single upload instead of multiple smaller texture upload. Please let me know your ideas in this regards.
Also I am sure there has to be a limitation on the size of image that can be processed by a device, as the memory on the device is limited. But I would like to know if it is resolution based or is it size based. I mean if a device has a limitation of 1024x1024 image size can it handle a compressed texture of size 2048x2048 that would be of same size approx as uncompressed 1024x1024.
Also please let me know on an general basis usually how much the limitation on texture size or resolution normal devices running android 2.2 and above would be.
Also please let me know if there are any best practices when handling high resolution images in opengles 2.0 to get best performance in both load time and also run time.
There is a hardware limitation on the texture sizes. To manually look them up, you can go to a site such as glbenchmark.com (Here displaying details about google galaxy nexus).
To automatically find the maximum size from your code, you can use something like:
int[] max = new int[1];
gl.glGetIntegerv(GL10.GL_MAX_TEXTURE_SIZE, max, 0); //put the maximum texture size in the array.
(For GL10, but the same method exists for GLES20)
When it comes to the processing or editing of an image you usually use an instance of Bitmap when working in android. This holds the uncompressed values of your image and is thus resolution dependant. However, it is recommended that you use compressed textures for your openGL applications as this improves the memory-use efficiency (note that you cannot modify these compressed textures).
From the previous link:
Texture compression can significantly increase the performance of your
OpenGL application by reducing memory requirements and making more
efficient use of memory bandwidth. The Android framework provides
support for the ETC1 compression format as a standard feature [...]
You should take a look at this document which contains many good practices and hints about texture loading and usage. The author explicitly writes:
Best practice: Use ETC for texture compression.
Best practice: Make sure your geometry and texture resolutions are
appropriate for the size they're displayed at. Don't use a 1k x 1k
texture for something that's at most 500 pixels wide on screen. The
same for geometry.
Is it possible to create a MIP map in open gl es 1.x that only loads the texture resolution it is currently rendering?
So instead of loading all the textures resolutions from the largest to the smallest at once, have the mipmap only store the one it is currently rendering. Then have the gl load the new resolution textures as I zoom in and out. This way I could load many large textures onto a surface and zoom out to view them all at once without having any VM budget issues.
If gl doesn't have a way to do this, is it possible to override the onDraw function to determine what level of the mipmap is being requested to be rendered so I can manually load a new texture?
As far as I am aware, there is no functionality for that. When you define that a texture object has MipMaps then you have to fill all of them. If you dont then that counts as an error and undefined behaviour as a result - usually you get a black rendering when using that texture.
Even if you could do it, you wouldn't really want to; uploading data for texturing can be a slow process especially on mobile platform, so performance would suffer a lot. Finally MipMaps usually work with linear interpolation, blending between the different available resolutions of the texture data uploaded - at least two sizes would be required.