I'm writing crossplatform application. It should run on Android devices.
I want to use dFdx/dFdy for antialiasing. But, unfortunately, glsl es 2.0 does not support derivatives.
Can I replace dFdx/dFdy with something? I.E. 1/sprite_width, 1/sprite_height in screen pixels.
As I said, I need this to work on android devices. And I saw that my device support GL_OES_standard_derivatives, which allow it to use this functions. Does all android opengl es 2.0 devices support it?
As I said many Opengl ES 2.0 devices support GL_OES_standard_derivatives extension.
But for thoose who don't, I made this workaround:
float myFunc(vec2 p){
return p.x*p.x - p.y; // that's our function. We want derivative from it.
}
// this calculated in vertex shader
// width/height = triangle/quad width/height in px;
vec2 pixel_step = vec2(1/width, 1/height);
float current = myFunc(texCoord);
float dfdx = myFunc(texCoord + pixel_step.x) - current;
float dfdy = myFunc(texCoord + pixel_step.y) - current;
float fwidth = abs(dx) + abs(dy); // from khronos doc #http://www.khronos.org/registry/gles/extensions/OES/OES_standard_derivatives.txt
P.S. I get very close results to the glsl built ins, A little bit more blurry (in my shader). To fix this I added multiply pixel_step on 1/1.75. If someone knows why, let me know.
Related
I'm an experienced native iOS developer making my first foray into Android through Unity. I'm trying to set up a custom shader, but I'm having some trouble with the Normal maps. I've got them working perfectly in the Unity simulator on my computer, but when I build to an actual device (Samsung Galaxy S8+), the Normal maps don't work at all.
I'm using Mars as my test case. Here's the model running in the simulator on my computer:
And here's a screenshot from my device, running exactly the same code.
I've done a LOT of research, and apparently using Normal maps on Android with Unity is not an easy thing. There are a lot of people asking about it, but almost every answer I've found has said the trick is to override the texture import settings, and force it to be "Truecolor" which seems to be "RGBA 32 Bit" according to Unity's documentation. This hasn't helped me, though.
Another thread suggested reducing the Asino Level to zero, and another suggested turning off Mip Maps. I don't know what either of those are, but neither helped.
Here's my shader code, simplified but containing all references to Normal mapping:
void surf (Input IN, inout SurfaceOutputStandard o) {
half4 d = tex2D (_MainTex , IN.uv_MainTex);
half4 n = tex2D (_BumpMap , IN.uv_BumpMap);
o.Albedo = d.rgb;
o.Normal = UnpackNormal(n);
o.Metallic = 0.0;
o.Smoothness = 0.0;
}
I've seen some threads suggesting replacements for the "UnpackNormal()" function in the shader code, indicating that it might not be the thing to do on Android or mobile in general, but none of the suggested replacements have changed anything for better or worse: the normal maps continue to work in the simulator, but not on the device.
I've even tried making my own normal maps programmatically from a grayscale heightmap, to try to circumvent any import settings I may have done wrong. Here's the code I used, and again it works in the simulator but not on the device.
public Texture2D NormalMap(Texture2D source, float strength = 10.0f) {
Texture2D normalTexture;
float xLeft;
float xRight;
float yUp;
float yDown;
float yDelta;
float xDelta;
normalTexture = new Texture2D (source.width, source.height, TextureFormat.RGBA32, false, true);
for (int y=0; y<source.height; y++) {
for (int x=0; x<source.width; x++) {
xLeft = source.GetPixel (x - 1, y).grayscale * strength;
xRight = source.GetPixel (x + 1, y).grayscale * strength;
yUp = source.GetPixel (x, y - 1).grayscale * strength;
yDown = source.GetPixel (x, y + 1).grayscale * strength;
xDelta = ((xLeft - xRight) + 1) * 0.5f;
yDelta = ((yUp - yDown) + 1) * 0.5f;
normalTexture.SetPixel(x,y,new Color(xDelta,yDelta,1.0f,yDelta));
}
}
normalTexture.Apply();
return normalTexture;
}
Lastly, in the Build Settings, I've got the Platform set to Android and I've tried it using Texture Compression set to both "Don't Override" and "ETC (default)". The former was the original setting and the latter seemed to be Unity's suggestion both by the name and in the documentation.
I'm sure there's just some flag I haven't checked or some switch I haven't flipped, but I can't for the life of me figure out what I'm doing wrong here, or why there would be such a stubborn difference between the simulator and the device.
Can anyone help a Unity newbie out, and show me how these damn Normal maps are supposed to work on Android?
Check under:
Edit -> Project Settings -> Quality
Android is usually set to Fastest.
I have an opengles 3.1 application that renders fine on the desktop but does not render on android.
The bit that goes wrong is when i have uniform buffer objects. In the vertex shader I have the below for example
layout (std140, binding = 0) uniform matrixUbo
{
mat4 projection;
mat4 view;
};
This works ok using deskop drivers but on android it fails. The version of opengles I am testing on is 3.2 compatible and the function calls are available in android.
I have tried both setting the bindings in the vertex shader and setting them using the glUniformBlockBinding method and both don't work on android (but both work on the desktop).
If I don't use those to matrix then the objects do render ok (I can see them ok on my android phone) but when I include those matrix nothing is drawn which tells me the matrix are full of zero's.
Is there anything special that needs to be done for UBO's to be supported on android?
I'm happy to provide more information as required.
To answer my own question, they are supported on android opengl es 3.1 but when you update the data you need to use a ByteBuffer not a FloatBuffer even though th function calls support it. Strange issue and a pain to debug!!
I am using OpenGL ES to run some shaders on Android.
On some older/cheap devices they do not support highp precision so the shader output is incorrect.
I need to know when the app starts if the device can support high precision. That way I can tell the user "forget it, your device does not support high precision floats" rather than have it output garbage for them.
I found this query code online, but it seems to only be for WebGL
var highp = gl.getShaderPrecisionFormat(gl.FRAGMENT_SHADER, gl.HIGH_FLOAT);
var highpSupported = highp.precision != 0;
Does anyone have a way I can query an android device (KitKat or higher) to see what precision the GLES shaders will support?
This is the final code I now use, but contents of range and precision are always -999 no matter where I run the code in my app. Before, during or after the GLSurfaceView has been created and GLES output has run.
IntBuffer range = IntBuffer.allocate(2);
IntBuffer precision = IntBuffer.allocate(1);
range.put(0,-999);
range.put(1,-999);
precision.put(0,-999);
android.opengl.GLES20.glGetShaderPrecisionFormat(android.opengl.GLES20.GL_FRAGMENT_SHADER, android.opengl.GLES20.GL_HIGH_FLOAT,range,precision);
String toastText="Range[0]="+String.valueOf(range.get(0))+" Range[1]="+String.valueOf(range.get(1))+" Precision[0]="+String.valueOf(precision.get(0));
Toast.makeText(getApplicationContext(),toastText, Toast.LENGTH_SHORT).show();
The above code always returns -999 for all 3 values, and the kronos doco states if an error occurs then the values will be unchanged. So it looks like there is an error or I am not calling it at the right time.
I have been trying to port this shader to Mobile Device. I am doing this on android device in OpenGL ES 2.0. here is fragment shader code from above site for reference:
void main(void)
{
// clamp pixel posiiton in [-1,1]
vec2 p = -1.0 + 2.0 * gl_FragCoord.xy / iResolution.xy;
vec2 uv;
// calculate angle of current pixel from origin
// atan return values are in [-pi, pi]
float a = atan(p.y,p.x);
// distance of point from origin
//float r = sqrt(dot(p,p));
float power = 7.0;
// http://en.wikipedia.org/wiki/Minkowski_distance
float r = pow( pow(p.x*p.x,power) + pow(p.y*p.y,power), 1.0/(2.0*power) );
// add global time for a moving tunnel
uv.x = .2/r + iGlobalTime/2.0;
uv.y = a/(3.1416);
// multiplication by r to give a darkened effect in center
vec3 col = texture2D(iChannel0, uv).xyz * (1.0-r);
//vec3 col = vec3(uv.y, 0.0,0.0);
gl_FragColor = vec4(col,1.0);
}
I am getting following result on my mobile phone(Moto G, Samsung Galaxy S Advance). Note how texture is flat and kind of clamped in middle
and following when same code is run on emulator(Nexus 5 API 21) (with emulated host gpu option)
Which is the expected output.
My texture wrapping mode is set to GL_REPEAT. What might be wrong ?
This is a precision issue. mediump, which is the highest precision that is guaranteed to be supported in ES 2.0 shaders, has a floating point magnitude range of [2^-14, 2^14], as listed in the table on page 33 of the spec ("The OpenGL ES Shading Language", version 1.00, which can be found at https://www.khronos.org/registry/gles/).
The following sequence of statements will quickly produce underflow:
float power = 7.0;
float r = pow( pow(p.x*p.x,power) + pow(p.y*p.y,power), 1.0/(2.0*power) );
Looking at this sub-expression:
pow(p.x*p.x,power)
Based on the calculation/comment at the start of your shader, the range of p.x is [-1, 1]. Using 0.1 as the value:
pow(0.1 * 0.1, 7.0) = pow(0.01, 7.0) = 10^-14
This is right around the limit for a medium value to underflow. So these sub-expressions will underflow to zero any time either p.x or p.y approaches the range [-0.1, 0.1].
It's not obvious what the best workaround is. Some ideas to try:
Use highp precision if your device supports it. highp support is optional in ES 2.0, and only available if GL_FRAGMENT_PRECISION_HIGH is defined.
Try a lower exponent than 7.0, and see if the visual result still meets your requirement.
Use some form of clamping for the pixels that would underflow. You could either discard them, or color them black. While it will not render he wall in the center, it would at least avoid the ugly artifacts.
BACKGROUND:
I want to add live filter based on the code of Android camera app. But the architecture of Android camera app is based on OpenGL ES 1.x. I need to use shader to custom our filter implementation. However, it is too difficult to update the camera app to OpenGL ES 2.0. Then I have to find some other methods to implement live filter instead of OpenGL. I decided to use render script after some research.
PROBLEM:
I have wrote a demo of a simple filter by render script. It shows that the fps is much lower than implementing it by OpenGL. About 5 fps vs 15 fps.
QUESTIONS:
The Android official offsite says: The RenderScript runtime will parallelize work across all processors available on a device, such as multi-core CPUs, GPUs, or DSPs, allowing you to focus on expressing algorithms rather than scheduling work or load balancing. Then why is render script implementation slower?
If render script cannot satisfy my requirement, is there a better way?
CODE DETAILS:
Hi I am in the same team with the questioner. We want to write a render-script based live-filter camera. In our test-demo-project, we use a simple filter: a YuvToRGB IntrinsicScript added with a overlay-filter ScriptC script.
In the OpenGL version, we set the camera data as textures and do the image-filter-procss with shader. Like this:
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureYHandle);
GLES20.glUniform1i(shader.uniforms.get("uTextureY"), 0);
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, mTextureWidth,
mTextureHeight, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE,
mPixelsYBuffer.position(0));
In the RenderScript version, we set the camera data as Allocation and do the image-filter-procss with script-kernals. Like this:
// The belowing code is from onPreviewFrame(byte[] data, Camera camera) which gives the camera frame data
byte[] imageData = datas[0];
long timeBegin = System.currentTimeMillis();
mYUVInAllocation.copyFrom(imageData);
mYuv.setInput(mYUVInAllocation);
mYuv.forEach(mRGBAAllocationA);
// To make sure the process of YUVtoRGBA has finished!
mRGBAAllocationA.copyTo(mOutBitmap);
Log.e(TAG, "RS time: YUV to RGBA : " + String.valueOf((System.currentTimeMillis() - timeBegin)));
mLayerScript.forEach_overlay(mRGBAAllocationA, mRGBAAllocationB);
mRGBAAllocationB.copyTo(mOutBitmap);
Log.e(TAG, "RS time: overlay : " + String.valueOf((System.currentTimeMillis() - timeBegin)));
mCameraSurPreview.refresh(mOutBitmap, mCameraDisplayOrientation, timeBegin);
The two problems are :
(1) RenderScript process seems slower than OpenGL process.
(2) According to our time-log, the process of YUV to RGBA which uses intrinsic script is very quick, takes about 6ms; but the process of overlay which uses scriptC is very slow, takes about 180ms. How does this happen?
Here is the rs-kernal code of the ScriptC we use(mLayerScript):
#pragma version(1)
#pragma rs java_package_name(**.renderscript)
#pragma stateFragment(parent)
#include "rs_graphics.rsh"
static rs_allocation layer;
static uint32_t dimX;
static uint32_t dimY;
void setLayer(rs_allocation layer1) {
layer = layer1;
}
void setBitmapDim(uint32_t dimX1, uint32_t dimY1) {
dimX = dimX1;
dimY = dimY1;
}
static float BlendOverlayf(float base, float blend) {
return (base < 0.5 ? (2.0 * base * blend) : (1.0 - 2.0 * (1.0 - base) * (1.0 - blend)));
}
static float3 BlendOverlay(float3 base, float3 blend) {
float3 blendOverLayPixel = {BlendOverlayf(base.r, blend.r), BlendOverlayf(base.g, blend.g), BlendOverlayf(base.b, blend.b)};
return blendOverLayPixel;
}
uchar4 __attribute__((kernel)) overlay(uchar4 in, uint32_t x, uint32_t y) {
float4 inPixel = rsUnpackColor8888(in);
uint32_t layerDimX = rsAllocationGetDimX(layer);
uint32_t layerDimY = rsAllocationGetDimY(layer);
uint32_t layerX = x * layerDimX / dimX;
uint32_t layerY = y * layerDimY / dimY;
uchar4* p = (uchar4*)rsGetElementAt(layer, layerX, layerY);
float4 layerPixel = rsUnpackColor8888(*p);
float3 color = BlendOverlay(inPixel.rgb, layerPixel.rgb);
float4 outf = {color.r, color.g, color.b, inPixel.a};
uchar4 outc = rsPackColorTo8888(outf.r, outf.g, outf.b, outf.a);
return outc;
}
Renderscript does not use any GPU or DSPs cores. That is a common misconception encouraged by Google's deliberately vague documentation. Renderscript used to have an interface to OpenGL ES, but that has been deprecated and has never been used for much beyond animated wallpapers. Renderscript will use multiple CPU cores, if available, but I suspect Renderscript will be replaced by OpenCL.
Take a look at the Effects class and the Effects demo in the Android SDK. It shows how to use OpenGL ES 2.0 shaders to apply effects to images without writing OpenGL ES code.
http://software.intel.com/en-us/articles/porting-opengl-games-to-android-on-intel-atom-processors-part-1
UPDATE:
It's wonderful when I learn more answering a question than asking one and that is the case here. You can see from the lack of answers that Renderscript is hardly used outside of Google because of its strange architecture that ignores industry standards like OpenCL and almost non-existent documentation on how it actually works.
Nonetheless, my answer did evoke a rare response from the Renderscrpt development team which includes only one link that actually contains any useful information about renderscript - this article by Alexandru Voica at IMG, the PowerVR GPU vendor:
http://withimagination.imgtec.com/index.php/powervr/running-renderscript-efficiently-with-powervr-gpus-on-android
That article has some good information which was new to me. There are comments posted there from more people who are having trouble getting Renderscript code to actually run on the GPU.
But, I was incorrect to assume that Renderscript is no longer being developed at Google. Although my statement that "Renderscript does not use any GPU or DSPs cores." was true until just fairly recently, I have learned that this has changed as of one of the Jelly Bean releases.
It would have been great if one of the Renderscript developers could have explained that. Or even if they had a public webpage that explains that or that lists
which GPUs are actually supported and how can you tell if your code actually gets run on a GPU.
My opinion is that Google will replace Renderscript with OpenCL eventually and I would not invest time developing with it.