Issue partially resolved, leaving previous post & code here for reference, new issue (stated in title) after the strong text at the bottom
I am trying to use colour picking to identify objects in OpenGL on Android. My code works fine for the most part, all objects are assigned unique colours in a 0.00f, 0.00f, 0.00f format (Alpha is always 0), and when clicking on the objects they are identified (Most of the time) using glreadpixels and converting/comparing the values.
The problem only occurs when using certain colour values. For instance, if an object is given the colour 0.0f, 0.77f, 1.0f (RGB), it will not colour solidly. It will colour parts of the object 0.0,0.76,1.0 or 0.0,0.78.1.0. I thought it might be colour blending so I coloured every object in the scene this colour but the same thing happened, this also eliminates any lighting issues which I thought might be another cause (despite not implementing light explicitly to my knowledge). This issue occurs on a few colours, not just the one stated.
How can I tell the object or the renderer to colour these objects solidly exactly as stated, instead of a blend of the colours either side of it?
The colours are not coming through as stated, if a color of R:0.0f G:0.77f B1.0f is passed to glUniform4v & glDrawArrays, it is drawn (and read by glreadpixels) as R:0.0f G:0.78f B1.0f. This is not the only value with which this happens, this is just an example.
Any suggestions for a fix are appreciated
There are at least two aspects that can come in your way of getting exactly the expected color:
Dithering
Dithering for color output is enabled by default. Based on what I've seen, it doesn't typically seem to come into play (or at least not in a measurable way) if you're rendering to targets with 8 bits per component. But it's definitely very noticeable when you're rendering to a lower color depth, like RGB565.
The details of what exactly happens as the result of dithering appears to be very vendor dependent.
Dithering is enabled by default. For typical use, that's probably a good thing, because you only care about the visual appearance. And the whole idea of dithering is obviously to enhance the visual quality. But if you rely on getting controllable and predictable values for your output colors, like it's necessary in your picking case, you should always disable dithering:
glDisable(GL_DITHER);
Precision
As you're already aware based on your code, precision is a big concern here. You obviously can't expect to get exactly the same floating point value back as the one you originally specified for the color.
The primary loss of precision comes from the conversion of the color value to a normalized value in the color buffer. With 8 bits/component color depth, the precision of that value is 1.0/255.0. Which means that you should be fine with generating and comparing values with a precision of 0.01.
Another source of precision loss is the shader processing. Since you specify mediump for the precision in the shader code, which gives you at least about 10 bits of precision, that also looks like it should not be harmful.
One possibility is that you didn't actually get a configuration with 8-bit color components. This would also be consistent with the visual dithering effect. Say if you got a RGB565 surface, your observed precision starts to make sense.
For example, with RGB565, if you pass in 0.77 for the green component, the value is multiplied with 63 (2^6 - 1) during fixed point conversion, which gives 48.51. Now, the spec says:
Values are converted (by rounding to nearest) to a fixed-point value with m bits, where m is the number of bits allocated to the corresponding R, G, B, A, or depth buffer component.
The nearest value for 48.51 is 49. But if you lose any kind of precision somewhere on the way, it could very easily become 48.
Now, when these values are converted back to float while you read them back, they are divided by 63.0. If the value in the framebuffer was 48, the result is 0.762, which you would round to in your code 0.76. If it was 49, the result is 0.777, which rounds to 0.78.
So in short:
Be very careful about what kind of precision you can expect.
I think you might have an RGB565 framebuffer.
Also, using multiples of 0.01 for the values does not look like an ideal strategy because it does not line up with the representation in the framebuffer. I would use multiples of 2^b - 1, where b is the number of bits in the color components. Use those values when specifying colors, and apply the matching quantization when you compare the values you read back with the expected value.
Related
I have started reading about RenderScript in android. I am following this particular blog.
While reading I come across the part called Setting floating point precision
It might seem as a noob question, but why do we need to change floating point precision? What benefit we get? Anything related to RenderScript in particular?
These precisions are for the compute part of renderscript. Generally they will not affect rendering in which you will get GL precision which is much lower than IEEE 754 in general, but you shouldn't use that since the graphics part of renderscript is deprecated.
Essentially, you should use rs_fp_relaxed since that will get you onto the highest range of mobile GPU and SIMD-supporting CPU devices.
rs_fp_relaxed enables flush to zero for denorms and round to zero operation. This affects the answer when you do math on half, float and double types. Although you should also avoid double if you want to be accelerated by mobile gpus and also not take a speed hit even on devices which natively support doubles.
I recommend checking out the wiki pages on floats: https://en.wikipedia.org/wiki/Single-precision_floating-point_format
The gist is floats are stored in two parts the exponent and the significand similar to scientific notation of 1.23 * 10^13. When the exponent is all 0s, then your number is denormal. So if your calculation results in a value where the exponent is 0, then the significand will also end up being zero instead of the actual value. For float32 the specific values are 1.1754942E-38 (0x7ffff) to 1.4E-45 (0x1) and the corresponding negative values.
Round to zero comes in when you do math with two floating point numbers an implementation will not calculate the extra digit of precision to know which way to round the last bit so you can be off by 1 ulp from a round-to-even implementation. Generally 1 ulp is quite small but the absolute difference depends on where your value lies in the real number space. For example 1.0 is encoded as 0x3f800000. A 1 ulp error could give you 0x3f800001 which is converted to 1.0000001.
Precision is basically what it is. It tells how precise things on screen will be drawn. In some cases floating point precision might be insignificant on its own or in comparison to other thing like memory or performance. If you have a device with small screen and low memory you don't need double precision to draw a model.
I have a decalbatch and some decal in it. Lets say 50. I have a groupstrategy and a cuszom shader.
My probleme is when I do something in that shader than my all particles responding to that. So when I change the alpha on the shader than it changes on all particle decal.
How can I change one by one on that shader?
Thanks
Shader uniforms and constants affect everything in the batch.
If you want to continue doing this with shader uniforms, you could flush the batch and then submit more decals each time you change the value of the parameter, but you would need to keep them sorted for transparent decals to look correct. You could do this by creating a GroupStrategy that sorts all the decals and then assigns them groups in ascending order from far to near, creating a new group each time the affected parameter is different.
The above has the potential to cause a lot of batch flushing which could cause a performance hit. An alternative is to use the existing vertex attributes to encode your data per decal. However, the only one that's really available is vertex color, since you need the texture coordinates and position attributes as is. So you can only put data into the color of the decal, if you aren't using color for tinting purposes.
A third possibility is to use a library that allows more customization than DecalBatch, such as FlexBatch. FlexBatch can be used sort of like a DecalBatch, but you can define whatever vertex attributes you need.
I am working on chroma key (green screen) filter for android using opengl; the only difference is I am trying to replace not only green background but any color passed by user. I have been able to replace the color but the problem is that it also replaces color from object where light intensity is very high.
Can anyone help me to reduce the light glare from texture so that my filter can work as expected?
Or any reference greenscreen filter which works perfectly.
Anything will be welcomed.
EDIT : I have added screenshot to explain the situation, Here I tried to replace red background with these clouds, it worked for all area excluding the one having glare of light in it. I can overcome this by increasing the tolerance value but then that will make it to replace some yellow pixels from the object.
Algorithm wise just matching RGB colors is going to be difficult.
The first thing to note is that in your case you are really just looking at some form of luminance-sensitive value, not a pure chroma value. Bright pixels will always have a strong response in R, G, and B channels, so simple thresholding isn't going to give a reliable workarond here.
If you extracted a luminance-independent chroma value (like a YUV encoding would do) then you could isolate "redness", "greenness", and "blueness" independent of the brightness of the color.
However, this is still going to have edge cases. For example, what happens if your bottle is red, or has a red label? To solve this you need some form of geometry feature extraction (e.g. sorbel edge detect over the image), and then color replace only within a specific identified region.
None of this is trivial - accurate object detection and recognition is very hard and still very much under active research (e.g. think Tesla cruise control trying to spot and avoid objects while driving).
I'm trying to figure a way to recreate or at least have a similar result to the max clamp blend equation in OpenGl es 2.0 on Android devices.
Unfortunately, glBlendEquation(GL_MAX_EXT) is not supported on Android. GL_MAX enum is defined in the gl header in Android but when executing, the result is a GL_INVALID_ENUM, 0x0500 error.
I have a solution using shaders and off screen textures where each render ping-pongs back and forth between textures using the shader to calculate the max pixel value.
However, this solution isn't fast enough for any real time execution on most Android devices.
So given this limitation, is there any way to recreate a similar result using just different blend equations and blend factors?
I have tried many blend function combinations, the closest have been:
glBlendFunction(GL_SRC_ALPHA, GL_ONE_MINUS_ALPHA) : This comes close but textures become too transparent. Textures with low alpha values are difficult to see.
glBlendFunction(GL_ONE_MINUS_DST_ALPHA, GL_SRC_ALPHA) : This also comes some what close but the alpha accumulates too much and the colors become darker than intended.
If you could do GL_MAX blending without needing a special blend function... OpenGL would never have added it in the first place. So your options are to do without or to use your shader method.
I'm looking at SurfaceFlinger, i.e. the the code that is doing the composition in Android, and I have trouble understanding some of the OpenGL ES 1.0 calls as I've only programmed ES 2.0 code.
Here goes the piece of code that interests me:
glTexEnvx(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);
glColor4x(0x10000, 0x10000, 0x10000, 0x10000);
glEnable(GL_BLEND);
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
What it is supposed to be doing is blending a texture into the buffer.
Why does one need the glTextEnvx and glColor4x calls? Isn't glBlendFunc enough for achieving blending?
I know my question is naive, but I still don't understand what is glTexEnvx for even after reading the docs.
glTexEnv() sets the texture environment mode. GL_REPLACE tells the renderer to skip the current color (for example, from glColor4()) and instead tells the renderer to use your texture's colors for every corresponding pixel. If you, instead of GL_REPLACE, use GL_MODULATE, then your glColor4() call will be included together with the texture's colors when the renderer sets a pixel's color.
Your glColor4() call shouldn't be doing anything (when using GL_REPLACE) that can be seen on your object.
About your glBlendFunc() arguments:
GL_ONE is using the current color that comes from your incoming primitive, which we call source. GL_ONE_MINUS_SRC_ALPHA is multiplying the destination (which is the currently stored pixel in the frame buffer) by (1 - source alpha value).
Normally, when you're not using textures, you achieve a transparent effect from color primitives when the glColor4() contains an alpha value where 1 equals fully opaque and 0 fully transparent.