I'm following this tutorial
https://github.com/mattdesl/lwjgl-basics/wiki/ShaderLesson4
to create mask technique in libgdx. Everything works fine as the example code.
But the example code only use
a set of (background, mask, foreground) texture. When applying in my case with different set of textures, I cannot get it works unless
using batch.flush() after batch.draw(tex0,...).
The cycle is:
batch.begin()
tex1 = foreground1;
tex2 = mask1;
tex1.bind(1);
tex2.bind(2);
Gdx.gl.glActiveTexture(GL20.GL_TEXTURE0);
batch.draw(tex0, ...)
batch.flush();
tex1 = foreground2;
tex2 = mask2;
tex1.bind(1);
tex2.bind(2);
Gdx.gl.glActiveTexture(GL20.GL_TEXTURE0);
batch.draw(tex0, ...)
batch.flush();
batch.end();
If I don't use batch.flush(), all masks and foregrounds will be identical. But using batch.flush() severely reducing performance when I test on my Android phone.
Thanks in advance, any help is highly appreciated
Related
it's the first time for me that I ask help here. I will try to be as precise as possible in my question.
I am trying to develop a shape detection app for Android.
I first identified the algorithm which works for my case playing with Python. Basically for each frame I do this:
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, lower_color, upper_color)
contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
#here I filter my results
by this algorithm I am able to run the analysis realtime on videos having a frame rate of 120fps.
So I tryied to implement the same algorithm on Android Studio, doing the following for each Frame:
Imgproc.cvtColor(frameInput, tempFrame, Imgproc.COLOR_BGR2HSV);
Core.inRange(tempFrame,lowColorRoi,highColorRoi,tempFrame);
List<MatOfPoint> contours1 = new ArrayList<MatOfPoint>();
Imgproc.findContours(tempFrame /*.clone()*/, contours1, new Mat(), Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
for(MatOfPoint c : contours1){
//here I filter my results
}
and I see that only the findContour function takes 5-600ms to be performed at each iteration (I noticed that it takes also more using tempFrame.clone()), allowing more or less to run the analysis with only 2fps.
This speed is not acceptable at all of course. Do you have any suggestion about how to improve this speed? 30-40fps would be already a good target for me.
I will really appreciate any help from you all. Many thanks in advance.
I would suggest trying to do your shape analysis on a lower resolution version of the image, if that is acceptable. I often see directly proportional timing with number of pixels of the image and the number of channels of the image - so if you can halve the width and height it could be a 4 times performance improvement. If that works, likely the first thing to do is a resize, then all subsequent calls have a smaller burden.
Next, be careful using OpenCV in Java/Kotlin because there is a definite cost to marshalling over the JNI interface. You could write the majority of your code in native C++, and then make just a single call across JNI to a C++ function that handles all of the shape analysis at once.
Im currently trying to display a video frame using opengl.
So far it works but I have some color problem.
Im using this as my
Reference for my logic
I have this code
//YUV420SP data
uint8_t *decodedBuff = AMediaCodec_getOutputBuffer(d->codec, status, &bufSize);
buildTexture(decodedBuff, decodedBuff+w*h, decodedBuff+w*h, w, h);
renderFrame();
but it displays with wrong color.
decodedBuff = Y
decodedBuff+w*h = U
decodedBuff+w*h*5 = V
but this separation formula is for YUV420P.
Do you guys happen to know whats for YUV420SP?
Your help is very much appreciated
If you are doing it this way you are doing it wrong. You should never manually read raw data from video surfaces in fragment shaders.
Generate a SurfaceTexture, bind it to an OpenGL ES texture, and use EGL_image_external to access the texture via an external image sampler.
This will give you direct access to the video data in your shader, including automatic handling of the memory format and color conversion, in many cases for "free" because it's backed by GPU hardware acceleration.
I'm happily using the SpriteBatch class of the LibGDX Framework.
My aim is to modify the representation of the sprite through a shader.
batch = new SpriteBatch(2, shaderProgram);
I copied the default shader from the SpriteBatch Class and added another
uniform Sampler 2d
+ "uniform sampler2D u_Texture2;\n"//
Is there a working way to give the texture to the shader.
Doing it like this, allways ends up in a ClearColor Screen.
batch.begin();
texture2.bind(1);
shaderProgram.setUniformi("u_Texture2", 1);
batch.draw(spriteTexture,positions[0].x,positions[0].y);
batch.draw(spriteTexture,positions[1].x,positions[1].y);
batch.end();
Each texture alone is working. Drawing manually with the help of the Mesh Class works as expected. So what can i do to use the convenience of SpriteBatch?
THX for Help
I guess the problem there is related to texture bindings. SpriteBatch asumes that the active texture unit will be 0, so it makes a call to
lastTexture.bind(); instead of lastTexture.bind(0);
The problem is that the active unit you are giving it is 1 (as you call texture2.bind(1); in your code). so, texture unit 0 is never bound, and may be thats causing the blank screen.
For instance, i would add a Gdx.GL20.glActiveTexture(0); before the draw calls. I'm not quite sure it will solve the problem, but it's a start!
EDIT:
I try my suggested solution and it works! :D. To be clearer, you should be doing this:
batch.begin();
texture2.bind(1);
shaderProgram.setUniformi("u_Texture2", 1);
Gdx.gl.glActiveTexture(GL10.GL_TEXTURE0);//This is required by the SpriteBatch!!
//have the unit0 activated, so when it calls bind(), it has the desired effect.
batch.draw(spriteTexture,positions[0].x,positions[0].y);
batch.draw(spriteTexture,positions[1].x,positions[1].y);
batch.end();
I'm running into trouble with LibGdx developing an 3D Android app. I metioned it briefly in my answer to my own question, but I have been unable to figure out why I cannot get transparency from .png images.
The textures are transparent, I've checked in my OpenGL3+ framework on windows and they show transparent in the correct parts.
My (simple placeholder) fragment shader code ends with:
vec4 finalColor = ambient + diffuse;
finalColor.a = colorMap.a;
gl_FragColor = finalColor;
I've tried finalColor.a = 0.1; which doesn't work.
I've added this to my code but to no avail:
Gdx.gl20.glEnable(GL20.GL_DEPTH_TEST);
Gdx.gl20.glBlendFunc(GL20.GL_ALPHA, GL20.GL_ONE_MINUS_SRC_ALPHA);
Gdx.gl20.glEnable(GL20.GL_BLEND);
I've even checked and changed:
AndroidApplicationConfiguration cfg = new AndroidApplicationConfiguration();
cfg.r = cfg.g = cfg.b = cfg.a = 8;
Which I though was the number of bits per channel.
Any help at all on this would be appreciated, I just hope I've not overlooked anything.
It's not a gigantic issue, and I could more or less complete my project without transparency, but would really like to know why I cant get it to work.
GL_ALPHA is not a legal option for glBlendFunc. You either want GL_SRC_ALPHA (most likely), or GL_DST_ALPHA.
glGetError would have been helpful to you here, you should start using it!
http://www.khronos.org/opengles/documentation/opengles1_0/html/glBlendFunc.html
I am trying to load textures as follows:
private Texture mTexture;
...
public Textures(final BaseGameActivity activity, final Engine engine) {
this.mTexture = new Texture(2048, 1024,
TextureOptions.BILINEAR_PREMULTIPLYALPHA);
this.mBackgroundTextureRegion = TextureRegionFactory.createFromAsset(
this.mTexture, activity, "img/back.png", 0, 0);
this.mSwingBackTextureRegion = TextureRegionFactory.createFromAsset(
this.mTexture, activity, "img/player.png", 836, 0);
...
I want to load more than 200 textures. However, the current method that I am using is too long.
Are there faster methods to complete it?
I am working in GLES1.
The easiest way to do it is with Texture Packer, found here
This allows you to add multiple image files in to one easy to load spritesheet. The engine loads this spritesheet in to a texture and creates a class that lets you easily reference each image from that spreadsheet. Turn 200 TextureRegions in to 1 TexturePack.
I'm using GLES2 and I'm not sure where the source files are for GLES1. Poke around the forums and you should be able to find out how to use them. There has been plenty of talk about it.
There is a texture packer built in AndEngine which does this automagically. Try searching the AndEngine forum.
http://www.andengine.org/forums/