I am developing a game for android using the android NDK Vulkan APIs. The code is, for the most part, in C++14. For most cases things work fine, however, on some devices, I have this problem where the x and y coordinates are switched. I draw on what I think of as the top of the screen, and it draws the objects on the side. Also, when I do anything with the view point (the view matrix), x and y are reversed. If I move the view point in the x direction, it actually moves in the y direction.
Also, the width and height reported by the swap chain are reversed. So that if I plug these values into the perspective matrix like so:
glm::perspective(glm::radians(60.0f), swapchainRetrievedWidth / (float) swapchainRetrievedHeight, 0.1f, 10.0f);
it will draw horribly skewed objects. But if I reverse the width and height, like so:
glm::perspective(glm::radians(60.0f), swapchainRetrievedHeight / (float) swapchainRetrievedWidth, 0.1f, 10.0f);
The objects look fine.
One device where this happens on is using an Adreno 530, API version 1.0.49, driver version: 35.143.1455, OS: android 8.0, phone vendor: HTC. For this device, these symptoms only occur if the device is using the split screen mode with the device held in landscape orientation (the app forces portrait mode). I've seen this happen on other devices too and in the full screen (not split screen) mode. So, I don't think it is the way I reinitialized the swap chain, pipeline, depth buffer, render pass and command buffers when the screen size changes. Since the screen would not change size for the devices where this problem occurs in full screen mode.
Am I doing something wrong? Is there a bug? I am willing to give more information on this problem, but do not know what is needed.
I tried the same thing in OpenGLES 2.0 on the same device in the same circumstances and these symptoms do not occur. Thanks for all your support and help.
(Answer based on discussion in comments)
This happens when you set VkSwapchainCreateInfo::preTransform to something other than VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR, but don't actually apply that transform during rendering. The safe thing to do is to always use VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR, but if your window is being rotated by the system compositor this is suboptimal from a performance/power point of view.
It's more efficient to look at what transform the system compositor is applying (VkSurfaceCapabilitiesKHR::currentTransform), apply that transform yourself during rendering, and let the compositor know you did so by setting VkSwapchainCreateInfo::preTransform.
Related
I want to horizontally invert frames coming to Surface object made from TextureView. What I can do is to set transformation matrix to this TextureView instance, whereas I postScale by -1 for x and 1 for y (leave unchanged), and than postTranslate dx on the full width of the view and leave dy unchanged (0F).
But the problem appears when I rotate my device by 90 degrees (horizontally) with screen rotation off: the image is rotated by 180 degrees, and it, of course, makes perfect sense, because x and y axises did not change.
How it can be solved? Is it possible to play with the translation matrix in such a way to resolve this problem? Or may be with OpenGL ES tools?
P.S.: strangely, but rotation is done two times two times more comparing to the device rotation itself, e.g. when I rotate the device by 90 degrees - preview is rotated by 180.
P.S.S.: I tried to invert the preview using Matrix's setPolyToPoly method... and have got exactly the same result.
P.S.S.S.: Also, played with open gl to achieve the goal using simple scale and rotation transformations for the model and projection matricies, and have got exact the same result!
Update:
These are screenshots that describe default behavior of the front camera - frame inversion is applied by HAL by default and I can't read the text; still, whether I rotate the device or not - "frame orientation" does not change:
And these are screenshots when I apply, e.g., Matrix.scaleM(modelMatrix, 0, -1F, 1F, 1F); and then apply this matrix to every coordinate that comes into vertex shader, so I can now read the text because I applied the inversion myself so HAL's inversion with my custom inversion will result into inversion absence, but when I rotate my device (with device orientation change on rotation disabled, of course, and that's the point) - I'll see myself flipped upside down, and that's, of course, make perfect sense, because device's coordinate system won't change. Still, I want to be able to avoid image rotation somehow on device rotation itself (like in the default mode - whether the device is rotated or not - the preview image just "does not care"), and to still be able to read the text (I mean, like in portrait mode).
Do you run your tests on front or back camera? Usually only the front camera frame needs to be flipped horizontally, not the back camera.
Anyway if you need different transformations based on device orientation, you need to detect orientation changes in your Activity. Camera doesn't do anything by itself when device is rotated, you will get the same results whatever transformation method you choose.
Update #2
I just realised that this problem can be solved if transformation is applied at the right moment in the pipeline. I will show a working example, it uses OpenGL extensively but I hope you can find something of use. See this code fragment, it uses a vertex shader to transform the coordinates. In drawFrame() there's a call to getTransformMatrix(). Instead of performing this call, get a matrix from this method I wrote when performing a similar task.
I have three different OBJs in my scene, a body, and a shirt and pant simulated over that body (rendered in that order).
Rendering was showing inner shirt outside the pants at some points in the form of some 'holes' on my test android devices while it works just fine on the desktops.
I guessed that some points are very close to each other and hence tried highp for precision and it started working fine on some of my devices (Surprisingly it doesn't work on an year old Nexus!)
Q. Have I identified the correct problem or it could be because of any other possible reason as well. Is there any way I can solve this issue on all devices ?
Q. Can I somehow at least get to know which GPUs will have this problem so that I can target my APK accordingly ?
Using :
Android 5.0
OpenGL ES 3.0
Edit:
Just in case its of any help, when rotating the scene, or zooming in-out, these holes show a 'twinkling behavior'.
highp support is not mandatory, and AFAICT it's also not an 'extension', so you can't query it and support for it is also not recorded in GLES capability databases like http://opengles.gpuinfo.org/
You can query the precision of the shader using glGetShaderPrecisionFormat ( http://docs.gl/es3/glGetShaderPrecisionFormat )
Of course it's up to your application to know what actual precision is actually needed. And this is at runtime, no way to know in advance.
Alright, so I seem to have solved this.
I quote from the opengl archives :
12.040 Depth buffering seems to work, but polygons seem to bleed through polygons that are in front of them. What's going on?
You may have configured your zNear and zFar clipping planes in a way
that severely limits your depth buffer precision. Generally, this is
caused by a zNear clipping plane value that's too close to 0.0. As the
zNear clipping plane is set increasingly closer to 0.0, the effective
precision of the depth buffer decreases dramatically. Moving the zFar
clipping plane further away from the eye always has a negative impact
on depth buffer precision, but it's not one as dramatic as moving the
zNear clipping plane.
Same page has lots of details about bounding boxes and related issues. Nice read.
I had a huge size BB. I reduced the size and moved camera nearer to the objects, the issue is resolved from all the devices.
While you are promoting my Android project, I discovered a strange.
I can display the map in the ocean Android OpenGL ES 2D graphics.
So, to be used only to determine the phase order of the object, the value is reduced to about 0.0001 Z-axis.
I tried over 1000 times the size of the object In the meantime.
Then, a phenomenon depending on the zoom in / zoom out, some objects flickering occurred.
Why such problems occur??
It is the problem of the target terminal-specific this can not be resolved if?
Or is it a problem of Android OpenGL ES itself?
***More....
The photo below is what you screen shot every time the screen of the actual device.
***I occurs when such a phenomenon to zoom in / zoom out each time.
I assume what you are experiencing is z-fighting: http://en.wikipedia.org/wiki/Z-fighting
This results due to the fact that your objects are too close together so that the z-buffer for certain pixels can't distinguish between which pixel is below or above the other.
You have three choices now:
1) Adjust your projection, specifically adjust znear and zfar values. Read more here: http://www.opengl.org/archives/resources/faq/technical/depthbuffer.htm
2) Increase the distance between both objects
3) Since you are drawing a 2D scene, you might use orthogonal projection. In that case it might be worth not to use depth buffering at all and just draw the objects from back to front (Painters Algorithm, http://en.wikipedia.org/wiki/Painters_algorithm).
I'm writing an Android and iOS engine in C++ and currently focusing on Android with the NDK.
I'd like to render to a viewport of a smaller size (say 600x360) and automatically upscale this to the native rez (say 800x480.) Currently the smaller viewport displays in a lower corner of my screen with black regions.
My problem is I don't know of a simple way to do this transparently using the NDK. There is a GLSurfaceview.setScaleX (and Y) function in API level 11, which would be perfect, but doesn't exist in API level 9, which I am targeting. Another bad solution is to render to a FBO and blit that to the screen as a final step.
I am considering simply story a scaling matrix and asking the user of the engine (for now just me) to always multiply vertices by this when drawing to the screen. This would be similar to using glPushMatrix.
I searched for a while and couldn't find a good solution. Does anyone know how to help?
What you can do is get the SurfaceHolder from GLSurfaceView, GLSurfaceView.getHolder() and then set the resolution you desire by calling SurfaceHolder.setFixedSize(width, height).
In my case the GLSurfaceView has a FrameLayout root which fills the screen, I am not sure if thats required - I have it because I add other elements on top - but if you set the size and it doesnt fill the screen then you know what's missing!
Using a FrameBuffer is also a valid way and you could draw some cool effects with it as well, the way above is just faster when the only thing you want to do is scale the rendering down (or possibly up? I haven't tried).
I've got an image that is 800 by 300, which I know is the width of my test platform's resolution (HTC Desire at 800x480). When I try to draw this image to the screen it scales oddly. It spills over the left hand side of the screen and fills almost all the vertical.
I'm using code like this:
canvas.drawBitmap( screen[1], new Rect(0,0,800,300), new Rect(0,0,800,300), null);
For some reason
width_x = canvas.getWidth();
width_y = canvas.getHeight();
reports my resolution as 533 by 320. Now I assume this is for the expletive-deleted fascinating scaling system Android uses so apps appear the same size on all phones but I want to ignore this. I'm writing a game, so want to handle scaling and positioning myself - for instance using more screen estate if it becomes available. Best Android practice may be suitable for an icon based application, but I would like to draw to absolute pixel positions, and get absolute resolution information for the screen.
Therefore my question is this - is this absolutely impossible? If it is completely contraindicated because Android has a simple and effective system in place to do this then I would be interested to know what it is. Dpi is not relevant to my game design (just like if this was a pc game, it would be irrelevant)
Perhaps my screen actually 533 by 320 unless I specify a resolution somehow? I tried using the scaling values from width and height and the image was the correct size on screen, roughly, but had jagged edges because some sort of scaling had occurred. I therefore did not have access to all the pixels my screen is capable of displaying.
To scale it I used something like
canvas.drawBitmap( screen[1], new Rect(0,0,800,300), new Rect((int)(0.0f),(int)(0.0f),(int)(533.0f),(int)(187.5f)), null); // numbers
Just whacked in for testing - ratio equivalent to reported screen resolution. Looks horrid.
Android is not doing anything to mess with your perceived resolution - you are working with 800x480 pixels on that Desire.
Are you working in a fullscreen, custom View and overriding onDraw? Are you working with a SurfaceView? We need to know these things before we can help you with your problem.
Assuming you are doing the above, you should be able to draw your bitmap to the screen without any scaling using Canvas.drawBitmap(Bitmap bitmap, float left, float top, Paint paint). In your case, that would look something like canvas.drawBitmap( screen[1], 0.0f, 0.0f, null); to put it in the upper left corner of your canvas.
In performance-sensitive apps (like games), you don't want to use the source/destination Rect version of drawBitmap() during your draw loop, since it will do the scaling during every iteration of the loop. Instead, draw a scaled/cropped version of your original bitmap to another member bitmap, and draw that one in the loop, using the x/y offset version of drawBitmap linked above.
If you want to know the amount of screen real estate you're working with (and you definitely should, if you're doing any custom drawing), you'll want to override either onSizeChanged() or surfaceChanged(), depending on implementation.
You should really check out the API demos, there are some great examples of how to do exactly what you're trying to do in there.
Good luck!
I fixed one of my problems - in the manifest file the OS I was targeting was set up incorrectly - switching it to 4 (i.e. 1.6) seemed to fix the values I was getting for height and width, at least for the HTC. Emulator is more problematic, but at least its a major step in the right direction. For your info, I'm working in full screen, landscape mode (fixed), with overridden functions for pretty much everything. (Including onDraw, surfaceChanged, and so forth)
If I can get the absolute width and height I can write my own code for loading the correct assets and using the correct scaling for screen positioning - DPI isn't an issue so hopefully that won't stray too far from suggested guidelines.