Writing directly to the framebuffer no longer works. Is there anyway to write to the display in the NDK? I might use ANativeWindow but that requires an existing surface. Is there a better way? Or is the only way to create a surface, natively, and then use ANativeWindow?
The display is owned by SurfaceFlinger and Hardware Composer, so unless you're planning to halt the Android framework you will need to work through them. (See the graphics architecture doc for more details.)
If you're developing a stand-alone command that is running as "shell" or "root", and you don't mind using non-public interfaces, you can just ask SurfaceFlinger for a window and draw on that. As of 5.0 "Lollipop" the old GLES tests were updated to work this way. See this answer for pointers; the San Angeles demo is illustrative.
If you're developing a regular app, you have to create a Surface and render to that through ANativeWindow. Regular apps aren't allowed exclusive access to the displays.
Related
I recently switched from using glBufferData to glMapBufferRange which gives me direct access to GPU memory rather than copying the data from CPU to GPU every frame.
This works just fine and in OpenGL ES 3.0 I do the following per frame:
Get a pointer to my GPU buffer memory via glMapBufferRange.
Directly update my buffer using this pointer.
Use glUnmapBuffer to unmap the buffer so that I can render.
But some Android devices may have at least OpenGL ES 3.1 and, as I understand it, may also have the EXT_buffer_storage extension (please correct me if that's the wrong extension ?). Using this extension it's possible to set up persistent buffer pointers which do not require mapping/unmapping every frame using the GL_MAP_PERSISTENT_BIT flag. But I can't figure out or find much online in the way of how to access these features.
How exactly do I invoke glMapBufferRange with GL_MAP_PERSISTENT_BIT set in OpenGL ES 3.1 on Android ?
Examining glGetString(GL_EXTENSIONS) does seem to show the extension is present on my device, but I can't seem to find GL_MAP_PERSISTENT_BIT anwhere, e.g. in GLES31 or GLES31Ext, and I'm just not sure how to proceed.
The standard Android Java bindings for OpenGL ES only expose extensions that are guaranteed to be supported by all implementations on Android. If you want to expose less universally available vendor extensions you'll need to roll your own JNI bindings, using eglGetProcAddress() from native code compiled with the NDK to fetch the entry points.
For this one you want the extension entry point glBufferStorageEXT().
On desktop OSes, OpenGL function loaders like GLEW, GLAD, etc. are used to load functions at runtime. But what about on Android? How are functions loaded? I've looked at a few apps and all of them seem to depend on EGL and GLES. But AFAIK EGL isn't a loading library, but an interface. Well, an interface to an interface as GLES is actually an interface.
This leads to another question: How come Android uses EGL when it is generally not used on desktops?
Back when I used android a bit, you could either link to the GLES 2.0 library, or you could link to the GLES 3.0 library, so kinda as if they provide the function pointers for you. ish. If you used GLES3.0, but the phone you ran it on only supported 2.0, your app would not load. To work around this, I always linked to GLES 2.0, and wrote my own function loader using eglGetProcAddress to extract the GLES3.0 API if available. This is pretty much how function loaders on windows/linux work (using wglGetProcAddress or glxGetProcAddress).
GLES has always been a stripped down version of the full blown desktop GL. It has always targeted a smaller subset of the full blown API (removing all of the legacy cruft), which in turn simplifies the OpenGL driver code somewhat, which in turn should reduce memory usage and save a little bit of battery life. Basically it's just more suited to use on a low power system where battery life is a concern.
How come Android uses EGL when it is generally not used on desktops?
It was widely used on embedded electronics prior to Android, so I suspect it was path of least resistance for a new smartphone OS as all GPU vendors had an EGL implementation already.
I have a setup with OpenGL ES 2.0 and EGL on Android 4.4.2 (API level 19).
My goal is to access the buffer of the window (the default framebuffer in OpenGL terms) directly from the CPU / user space.
I have tried using ANativeWindow_fromSurface to get ANativeWindow from the Surface of a GLSurfaceView. Then trying to get access to the buffer with ANativeWindow_lock fails with status -22. Logcat gives
03-25 10:50:25.363: E/BufferQueue(171): [SurfaceView](this:0xb8d5d978,id:32,api:1,p:6488,c:171) connect: already connected (cur=1, req=2)
From this discussion it seems you can't do that with GLSurfaceView, because EGL has already acquired the surface.
How could you get to the memory of the window? Can you somehow do it through an EGLSurface? I am willing to use android::GraphicBuffer, even tough it is not part of the NDK.
If this is not possible, can you use the other direction, by first creating an android::GraphicBuffer and then binding it to an EGLSurface and the displayed window?
Android devices may not have a framebuffer (i.e. /dev/graphics/fb). It's still widely used by the recovery UI, but it's being phased out.
If it does have a framebuffer, it will be opened and held by the Hardware Composer unless the app framework has been shut down. Since you're trying to use the NDK, I assume the framework is still running.
If your NDK code is running as root or system, you can request a top window from SurfaceFlinger. The San Angeles demo provides an example.
Additional information can be found here, here, and here. If you want to work with graphics at a low level, you should also read the graphics architecture doc.
This is not doable with just NDK API, you will need to pull-in some OS headers, that are not guaranteed to be stable.
You will need to subclass ANativeWindow, similarly to what is done in frameworks/native/include/ui/FramebufferNativeWindow.h.
However you may need to construct your own buffer queue using own-created android::GraphicBuffer objects, and properly respond to all dequeue() and enqueue() requests.
On enqueue() you will need to sync (GPU renders asynchronously) and than map enqueued buffer to CPU memory.
Note that this approach may be underperformant, due to explicit GPU<->CPU sync needed.
How to draw some stuff before surfacefligner start on android?
In this situation, it's some kind of traditional Linux with framebuffer device, so directly access framebuffer should be OK.
How about to use HWComposer directly and how about to use egl directly?
If SurfaceFlinger isn't running, you can just open the framebuffer device and write to it (assuming you're on a device that has a framebuffer device).
For an example of this, see the implementation of the "recovery" UI. The key file there is minui/graphics.c. The current implementation relies on libpixelflinger.
Using OpenGL ES / EGL will be a bit more tricky. Some of the early GLES tests, such as San Angeles, use the FramebufferWindow class, but that uses a fair bit of the framework. (FWIW, an upcoming release is expected to deprecate FramebufferWindow and switch the OpenGL tests that use them to a new library that talks to SurfaceFlinger.)
Update: the upcoming release happened, and you can see the replacement for FramebufferWindow ("WindowSurface") here.
If SurfaceFlinger isn't running you can talk to HardwareComposer directly, using the internal interface. There are some old tests that exercise it, but I don't know if they still work. The code in SurfaceFlinger is probably a better example at this point. Only one process can open HardwareComposer at a time, so SurfaceFlinger must not be running.
I have a cross platform mobile application that is:
90% written in cross platform C++
8% of it is platform dependent (the GUI code). For iOS, we use Objective C. For Android we use Java.
2% of it is embedded Webviews (WebKit + Webhooks).
Finding a proper test framework has been challenging.
We can test a lot with the C++ Google Test Framework. Nevertheless, we want to do end-to-end tests (like Selenium) that drags/double taps around the OpenGL view as well as the Webview.
I have looked at Appium and Selenium which can test everything except the OpenGL interface (which fires of the WebvViews) but I cannot find any examples for OpenGL tests.
Is there a good framework for testing OpenGL apps?
I have written unit-tests for C++ (Qt on Linux) & OpenGL before. I don't know any reason it shouldn't work for Java too.
The things which worked for me are:
Abstract your OpenGL context provider so the rest of your code is independent of it. In my case the main app used Qt's QGLWidget, but the unittests used a pbuffer-based one which I could create with no windowing infrastructure at all (other than a designated X11 DISPLAY). Later I added an "offscreen Mesa" (pure software OpenGL implementation) so they'd even work on a headless build machine with no GPU at all.
Keep your OpenGL code independent of your GUI code. In my case the OpenGL "rendering engine" didn't know anything about Qt classes (e.g mouse events). Define your own testable API which isn't tied to any specific GUI concepts, and write tests for it.
In the unittests, read the contents back from the framebuffer using glReadPixels and either hit them with some assertions about which pixels should be particular values, or go down the regression-testing route and compare the framebuffer capture with a stored image you know is good (either from manual verification, or because it's produced from some other reference model).
Allow a bit of fuzziness in any image regression testing; most OpenGL implementations produce slightly different output.
(I didn't do this, but...) Ideally you want to be able to test the GUI layer by asserting that it makes the expected sequence of calls to your rendering engine in response to GUI activity. If it does that, and you're confident of your render layer because of the above tests... well, no need to actually do the rendering. So create a suitable mock object of your rendering layer for use when GUI testing (I'd imagine tests like "mouse drag from here to there results in a call to the rendering layer to set a particular transform matrix"... stuff like that).
Hope it helps! I found it on..
ref.Automated testing for OpenGL application