I want to render a very large hollow diamond. In the fragment shader, if a point is inside the diamond, it is not rendered, otherwise it is rendered. The program runs very well on PC, but it does not work on Android phones. I guess it should be caused by accuracy problems. The size of the rhombus is changed. Sometimes it is too large, exceeding the mediump value in GLSL. Is there any other way to simply render a hollow diamond on the phone screen? The main code is as below:
uniform mediump vec4 user_DiamondCenter;
uniform mediump vec4 user_DiamondSize;
varying mediump vec4 v_worldPos;
lowp vec4 fragmentColor = v_fragmentColor;
lowp vec4 srcColor = texture2D(CC_Texture0, v_texCoord);
mediump vec4 testPoint = (v_worldPos - user_DiamondCenter);
lowp float fIn = step(user_DiamondSize.y * user_DiamondSize.x * 0.5f, abs(testPoint.x * user_DiamondSize.y) + abs(testPoint.y * user_DiamondSize.x));
fragmentColor.a = fragmentColor.a * fIn;
gl_FragColor = fragmentColor * srcColor;
The user_DiamondSize may be very large, so it cause the problem. Is there any solution, or any other simple way to render this large hollow diamond? The camera is moving, and sometimes it is necessary to render the border of this diamond.
If the camera is within the object, and the object is not rendered only on phone, then the obvious issue is in Face culling.
glDisable(GL_CULL_FACE)
This will disable culling so you can check if that is the issue
Related
I'm pretty new to opengl-es and currently have a problem rendering my video output on a screen in Unity.
I was developing a video player project with Unity. I bought the EasyMovieTexture plugin and replaced the video player module with another open-source video player (Ijkplayer) years ago which worked fine all the time.
Now I want to replace it with newer VLC using libvlcjni. I compiled and just replaced the old Ijkplayer but it didn't work as I expected. The screen just started flashing one color from every video frame but the video was going and the audio track was normally playing too.
Screenshot - A test scene with a screen, sorry there's mistake with texcoord but only color flashing
I'd like to provide some further information hope that I can find some help (Sorry if I have some mistakes or misunderstandings):
As far as I know, these video player modules need a Surface (Or SurfaceTexture) in the Android layer and the video decoder will work as a data producer. The SurfaceTexture consumes data from the producer and converts it to the texture with the type of GL_TEXTURE_EXTERNAL_OES which can be directly consumed and displayed with components like TextureView. But this texture data cannot be consumed in the Unity layer unless I use a GLSL shader directly sampling my OES texture data, like this:
// Unity shader with GLSL
GLSLPROGRAM
#pragma only_renderers gles3
#include "UnityCG.glslinc"
// ...
// Ignoring vertex shader
// ...
in vec2 textureCoord;
layout(binding = 0) uniform samplerExternalOES _MainTex;
out vec4 fragColor;
void main()
{
fragColor = texture(_MainTex, textureCoord);
}
ENDGLSL
My approach was to convert the texture to GL_TEXTURE_2D with the native library which came along with the EasyMovieTexture plugin. Here I cannot provide the source code of this .so library but I've decompiled it in IDAPro and I know it can work along with GLES and render the external texture data to another 2d texture using FrameBuffer Object and external shader program.
Here is a random example to explain the procedure, it is NOT the accurate code from the binary: FilterFBOTexture.java
Though I cannot edit the .so file, luckily it was reading two external files as the shader program:
// vertex shader
attribute highp vec3 inVertex;
attribute mediump vec3 inNormal;
attribute mediump vec2 inTexCoord;
uniform highp mat4 MVPMatrix;
uniform mediump vec2 TexCoordMove;
varying mediump vec2 TexCoord;
void main()
{
highp vec4 vPos = vec4(0,0,0,1);
vPos.x = ( inTexCoord.x * 2.0 - 1.0 );
vPos.y = ( inTexCoord.y * 2.0 - 1.0 );
gl_Position = vPos;
mediump vec4 vec4Temp = vec4(inTexCoord.x - TexCoordMove.x,inTexCoord.y - TexCoordMove.y,0,1.0);
vec4Temp = MVPMatrix * vec4Temp;
vec4Temp.xyz = vec4Temp.xyz / vec4Temp.w;
TexCoord = vec4Temp.xy;
}
// fragment shader
#extension GL_OES_EGL_image_external : require
uniform samplerExternalOES sTexture;
uniform lowp float AlphaValue;
varying mediump vec2 TexCoord;
void main()
{
lowp vec4 color = texture2D(sTexture, TexCoord) ;
color = vec4(color.rgb, color.a * AlphaValue);
gl_FragColor = color;
}
I don't know whether I have to check the vertex shader or just dive into the source code of libvlcjni so that I can correctly render my video output. Any idea will be grateful, thanks.
Update 2022-11-4:
I turned around and began to use VLC for Android.
Big appreciation to #mfkl, I created an issue days ago on the VLC repo.
https://code.videolan.org/videolan/vlc-unity/-/issues/164
The problem still remains but at least I can work for something now.
On Android devices with Mali-400 GPU (Samsung Galaxy S II, Samsung Galaxy S3 Mini, Samsung Galaxy Note II), at random times the screen will start showing repeated frames.
Example from 0:51 until 1:01 on the following video https://www.youtube.com/watch?v=5-p6Oy0BZmg
It seems as if new frames aren't being rendered, and what was in the old buffer is being shown again. The game continues to advance behind the repeated frames.
This doesn't happen on other GPUs.
I read about using glFlush or glFinish, but GLSurfaceView takes care of this when doing eglSwapBuffers after onDrawFrame.
I've read about quirks of Mali-400, like is better to use varying for texture coords, or to use lowp, but it doesn't help. Here are the shaders for reference:
Vertex shader:
// Vertex Shader
attribute vec4 position;
attribute vec4 colorModelIn;
attribute vec4 colorVertexIn;
varying lowp vec4 colorOut;
uniform mat4 modelViewProjectionMatrix;
uniform mat4 modelMatrix;
attribute vec2 TexCoordIn;
varying lowp vec2 TexCoordOut;
uniform bool bUseVertexColor;
void main()
{
if( bUseVertexColor ){
colorOut = colorVertexIn * colorModelIn;
} else {
colorOut = colorModelIn;
}
TexCoordOut = TexCoordIn;
gl_Position = modelViewProjectionMatrix * modelMatrix * position;
}
Fragment shader:
// Fragment shader
varying lowp vec4 colorOut;
varying lowp vec2 TexCoordOut;
uniform sampler2D Texture;
uniform bool bUseTexture;
void main()
{
if( bUseTexture ){
gl_FragColor = colorOut * texture2D(Texture, TexCoordOut);
} else {
gl_FragColor = colorOut;
}
}
I'm aware that these shaders aren't optimal, and that I'm going down the path of reproducing the fixed pipeline.
The rendering goes back to normal after some time, or after touching the screen. The only reason I can think for going back to normal when touching is that I use color-coding to detect the touched object. I render an image to the back buffer and glReadPixels from it. Then, overwrite the back buffer with the normal game image.
I'm out of ideas on how to attack this problem.
EDIT
After following Muzza's advice I started to log GL errors. glGetInteger and glBindBuffer report out-of-memory.
Above I said that the problem solves itself after a while. When that happens these appear in the logs:
01-23 21:57:52.956: D/WebView(9860): onSizeChanged - w:480 h:75
01-23 21:57:53.126: D/TilesManager(9860): new EGLContext from framework: 40e00bd0
01-23 21:57:53.126: D/GLWebViewState(9860): Reinit shader
01-23 21:57:53.171: D/GLWebViewState(9860): Reinit transferQueue
This can happen if the OpenGL state becomes invalid in some way. The graphics drivers can just skip frames entirely. Check Logcat to see if there is any output from the drivers, and add glGetError() calls throughout your code to see if any error comes up there.
I would like to be able to pass more per-vertex-data to my own custom shaders in kivy than the usual vertex coords + texture coords. Specifically, I would like to pass a value that says which animation frame should be used in selecting the texture coords.
I found an example (http://shadowmint.blogspot.com/2013/10/kivy-textured-quad-easy-right-no.html), and succeeded in changing the format of the vertices passed to a mesh using an argument to the constructor of the Mesh, like this:
Mesh(mode = 'triangles', fmt=[('v_pos', 2, 'float'),
('v_tex0', 2, 'float'),
('v_frame_i', 1, 'float')]
I can then set the vertices to be drawn to something like this:
vertices = [x-r,y-r, uvpos[0],uvpos[1],animationFrame,
x-r,y+r, uvpos[0],uvpos[1]+uvsize[1],animationFrame,
x+r,y-r, uvpos[0]+uvsize[0],uvpos[1],animationFrame,
x+r,y+r, uvpos[0]+uvsize[0],uvpos[1]+uvsize[1],animationFrame,
x+r,y-r, uvpos[0]+uvsize[0],uvpos[1],animationFrame,
x-r,y+r, uvpos[0],uvpos[1]+uvsize[1],animationFrame,
]
..this works well when I run in Ubuntu, but when I run on my android device the drawn texture either doesn't draw, or it looks like the vertex or texture coordinate data is corrupt / not aligned or something.
Here is my shader code in case that is relevant. Again, this all behaves as I want it to when I run in ubuntu, but not when I run on android device.
---VERTEX SHADER---
#ifdef GL_ES
precision highp float;
#endif
/* vertex attributes */
attribute vec2 v_pos;
attribute vec2 v_tex0;
attribute float v_frame_i; // for animation
/* uniform variables */
uniform mat4 modelview_mat;
uniform mat4 projection_mat;
uniform vec4 color;
uniform float opacity;
uniform float sqrtNumFrames; // the width/height of the sprite-sheet
uniform float frameWidth;
/* Outputs to the fragment shader */
varying vec4 frag_color;
varying vec2 tc;
void main() {
frag_color = color * vec4(1.0, 1.0, 1.0, opacity);
gl_Position = projection_mat * modelview_mat * vec4(v_pos.xy, 0.0, 1.0);
float f = round(v_frame_i);
tc = v_tex0;
float w = (1.0/sqrtNumFrames);
tc *= w;
tc.x += w*mod(f,sqrtNumFrames); //////////// I think that the problem might
tc.y += w*round(f / sqrtNumFrames); ///////////// be related to this code, here?
}
---FRAGMENT SHADER---
#ifdef GL_ES
precision highp float;
#endif
/* Outputs from the vertex shader */
varying vec4 frag_color;
varying vec2 tc;
/* uniform texture samplers */
uniform sampler2D texture0;
uniform vec2 player_pos;
uniform vec2 window_size; // in pixels
void main (void){
gl_FragColor = frag_color * texture2D(texture0, tc);
}
I wonder if it may have to do with a version of GLSL and int / float math (in particular in identifying which image from the sprite sheet to draw, see the comments in the glsl code. One version is running on my desktop and another on the device?
Any suggestions for things to experiment with would be much appreciated!
After looking at the log from the running version on the android device (a Moto X phone), I saw that the custom shader was not linking. This appeared to be due to the use of the function round(x), which I replaced with floor(x+0.5) in both cases, and the shader now works on the phone and my desktop properly.
I think the problem is that the version of GLSL supported on the phone and on my PC are different..but I am not 100% certain about this.
I'm using a fragment shader that uses dFdy dFdx functions to calculate the normal of the
face to view in a flat appearance. This shader has been running ok in gles 2.0 and 3.0. Inexplicably, shader don't work in Android 4.4 ( KitKat - gles3.0 ).
(Solved!!.. individual derivatives for each component, solve the problem).
In order check error, i prepared these shaders :
//Vertex Shader
#version 300 es
precision highp float;
precision highp int;
uniform mat4 PMatrix; //Projection Matrix (varies according to camera)
uniform mat4 MVMatrix; //Model View Matrix (no change)
in vec3 vPosition;
out vec3 vPos;
main()
{
gl_Position=PMatrix * MVMatrix * vec4(vPosition.xyz,1.0);
vPos = (MVMatrix * vec4(vPosition.xyz,1.0)).xyz;
}
// Fragment shader
#version 300 es
#extension GL_OES_standard_derivatives : enable
precision highp float;
precision highp int;
in vec3 vPos;
main()
{
// don't run correctly
// vec3 fdx = dFdx(vPos);
// vec3 fdy = dFdy(vPos);
// ***Solved!*** this run correctly in KitKat
vec3 fdx = vec3(dFdx(vPos.x),dFdx(vPos.y),dFdx(vPos.z));
vec3 fdy = vec3(dFdy(vPos.x),dFdy(vPos.y),dFdy(vPos.z));
vec3 N = normalize(cross(fdx,fdy));
fragColor = vec4(N,1.0);
}
Drawing a cube, in Android<4.4 colors remain fixed for each side, independently of camera position (correct, the color identifies each normal-face). In Android 4.4 , colors vary if you move the camera.
Analyzing each derived separately :
1.- fragColor=vec4(normalize( fdx ), 1.0); colors are constantly changing (wrong)
2.- fragColor=vec4(normalize( fdy ), 1.0); the colors remain quasi-stable (quasi-ok)
A bug in the implementation of these features in Android 4.4?
We are doing something wrong?
You said that this was originally a GLES2 shader, and you are using highp in a fragment shader unconditionally? That is a disaster waiting to happen because GLES2 implementations are not required to support highp in fragment shaders. Likewise, support for dFdx (...), dFdy (...) and fwidth (...) is optional.
You need to check for GL_OES_standard_derivatives and GL_FRAGMENT_PRECISION_HIGH in the GLES2 implementation of this fragment shader.
To that end, you might consider the accuracy hint for derivatives:
GL_FRAGMENT_SHADER_DERIVATIVE_HINT_OES (GLES2, if the extension is supported)
GL_FRAGMENT_SHADER_DERIVATIVE_HINT (GLES3)
Individual derivatives in each component solve the problem in KitKat:
// Replace vec3 fdx = dFdx(vPos) by:
vec3 fdx = vec3(dFdx(vPos.x),dFdx(vPos.y),dFdx(vPos.z));
// Replace vec3 fdy = dFdy(vPos) by:
vec3 fdy = vec3(dFdy(vPos.x),dFdy(vPos.y),dFdy(vPos.z));
I've seen similar things on some AMD desktop setups.
Where dFdy( .xyz ) worked fine on NVIDIA/intel I had to do the derivate per-component to get it correctly for some AMD cards.
I'm optimizing a game that works for both, iPhone and Android. I'm using 4 shader to draw the scene and I noticed that if I change one of them to another the fps goes from 32 to 42, even though there's only 1 sprite being drawn with that shader, and the only difference in this 2 shaders is just a product in the fragmente shader.
These are the shaders:
default-2d-tex.shader
#ifdef GL_ES
precision highp float;
precision lowp int;
#endif
#ifdef VERTEX
uniform mat4 umvp;
attribute vec4 avertex;
attribute vec2 auv;
varying vec2 vuv;
void main()
{
// Pass the texture coordinate attribute to a varying.
vuv = auv;
// Here we set the final position to this vertex.
gl_Position = umvp * avertex;
}
#endif
#ifdef FRAGMENT
uniform sampler2D map0;
uniform vec4 ucolor;
varying vec2 vuv;
void main()
{
gl_FragColor = texture2D(map0, vuv) * ucolor;
}
#endif
default-2d-tex-white.shader
#ifdef GL_ES
precision highp float;
precision lowp int;
#endif
#ifdef VERTEX
uniform mat4 umvp;
attribute vec4 avertex;
attribute vec2 auv;
varying vec2 vuv;
void main()
{
// Pass the texture coordinate attribute to a varying.
vuv = auv;
// Here we set the final position to this vertex.
gl_Position = umvp * avertex;
}
#endif
#ifdef FRAGMENT
uniform sampler2D map0;
varying vec2 vuv;
void main()
{
gl_FragColor = texture2D(map0, vuv);
}
#endif
Again,
If I modify default-2d-tex.shader and remove the product "* ucolor", the fps goes from 32 to 42, and I'm using it for just one sprite in the scene!
Is this normal? Why is this shader being so slow and how can I improve it?
EDIT:
I see this performance slowdown on both iPod and Android in an equal ratio. Both are PowerVr SGX GPUs (iPod 3rd gen and Samsung Galaxy SL -PowerVR SGX 530-). iOS version is 4.1 and Android is 2.3.3
The sprite I'm drawing is scaled to fill the screen (scaled to 4x) and I'm drawing it once per frame. It's taken from a texture map so the texture is actually larger (1024x1024) but the portion taken is 80x120. Alpha blending is enabled.
EDIT 2
I made a mistake. The sprite is scaled 11x: its 32x48.
If I don't draw that sprite at all, fps goes to 45. I'm drawing a lot of sprites in the scene, why is that one taking so much time? Could it be because it's scaled so much?
When you remove "* ucolor" from the abovementioned code many things happen:
uniform ucolor becomes unused
GLSL compiler removes this uniform from the active uniforms set
when you have you program linked the ID of the missing uniform
becomes -1
when you do glUniform4fv( -1, value ) for the removed uniform it
just exits instantly, without any update
This is pretty much less work compared to having "* ucolor" in place.
And additionally, this is one less operation per fragment in the fragment shader.
Your problem is in your fragment shader in default-2d-tex.shader. You have
uniform vec4 ucolor;
That means, that each of your color's components (RGBA) will be converted to 32-bits float value. And this drops performance heavily.
Should be:
uniform lowp vec4 ucolor;