I am getting confused with EGL.
My GLSurfaceView creates an EGLContext. Now I create a shared context. Now I need to use a EGLExtension.
The Method I have to use is called (>=API18):
EGLExt.eglPresentationTimeANDROID(android.opengl.EGLDisplay display, android.opengl.EGLSurface surface, long time);
The Problem is, that the GLSurfaceView does only creates javax.microedition.khronos.egl.EGLContext s.
Which tells me, NOT to use GLSurfaceView. So I tried TextureView, which is slightly similar, with the difference that you have to handle your own EGL stuff. Which is good for that purpose.
But:
The TextureView is slower, at least it looked like that, so I recorded some diagrams with the Method Profiler:
Here the TextureView with own EGL Handling:
The Thread on the top is a clock that wakes the Thread in the middle, which renders onto the TextureView. The main Thread will be called after that, for redrawing the TextureView.
... and here the GLSurfaceView with their own EGL Handling
The clock is in the middle this time, it calls the Thread on the top to render my image into a framebuffer, which I give directly into the SurfaceView (RENDERMODE_WHEN_DIRTY) and call requestRender to request the view to render.
As you can see with a short look already that with the GLSurfaceView it looks way cleaner that with the TextureView.
On both Examples I havn't had anything else on the screen and they rendered exactly the same Meshes with the same shader.
To my question:
Is there a way to use GLSurfaceView with EGL14 Contexts?
Did I do something wrong?
What you probably want to do is use a plain SurfaceView.
Here's the short version:
SurfaceView has two parts, the Surface and a bit of fake stuff in the View. The Surface gets passed directly to the surface compositor (SurfaceFlinger), so when you draw on it with OpenGL there's relatively little overhead. This makes it fast, but it also makes it not play quite right with the View hierarchy, because the Surface is on one layer and the View-based UI is on a different layer.
TextureView also has two parts, but the part you draw on lives behind the scenes (that's where the SurfaceTexture comes in). When the frame is complete, the stuff you drew is blitted onto the View layer. The GPU can do this quickly, but "some work" is always slower than "no work".
GLSurfaceView is a SurfaceView with a wrapper class that does all the EGL setup and inter-thread messaging for you.
Edit: the long version is available here.
If you can do the GL/EGL setup and thread management yourself -- which, if you're now running on a TextureView, you clearly can -- then you should probably use a plain SurfaceView.
Having said all that, it should be possible to make your original code work with GLSurfaceView. I expect you want to call eglPresentationTimeANDROID() on the EGL context that's shared with the GLSurfaceView, not from within GLSurfaceView itself, so it doesn't matter that GLSurfaceView is using EGL10 internally. What matters for sharing the context is the context client version (e.g. GLES2 vs. GLES3), not the EGL interface version used to configure the context.
You can see examples of all of this working in Grafika. In particular:
"Show + capture camera" uses a GLSurfaceView, the camera, and the video encoder. Note the EGL context is shared. The example is convoluted and somewhat painful, mostly because it's deliberately trying to use GLSurfaceView and a shared EGL context. (Update: note this issue about race conditions with shared contexts.)
"Play video (TextureView)" and "Basic GL in TextureView" show TextureView in action.
"Record GL app with FBO" uses a plain SurfaceView.
Thanks to fadden! It worked as expected.
To everyone who thinks about doing something similar:
It has advantages AND disadvantages using the (GL)SurfaceView to render images on it.
My testresults in the post above do not have anything else on the screen than the rendered image itself.
If you have other UI elements on the screen, especially if they get updated frequently, you should reconsider my choice of prefering the (GL)SurfaceView.
The SurfaceView creates a new window in the Android Windowsystem. Its advantage is, that if the SurfaceView gets refreshed, only this window will be refreshed. If you additionally update UI Elements (which are in another window of the windowsystem), then both refresh operations block themselfes (especially when ui drawing is hardwaresupported) because opengl cannot handle multi thread drawing properly.
For such a case it could be better using the TextureView, cause it's not another window of the Android Windowsystem. so if you refresh your View, all UI elements get refreshed as well. (Probably) everything in one Thread.
Hope I could help some of you!
Related
I'm working on a video app where user can watch a video, open it il fullscreen if needed and come back to default view and so on. I was using ExoPlayer and recently switch to default MediaPlayer due to the upcoming explanation.
I need to change "on the fly" the Surface of the player. I need to use the same player to display video among activities, with no delay to display the image. Using Exoplayer, the decoder wait for the next keyframe to draw pixels on the empty Surface.
So I need to use the same Surface so I don't need to push a new surface each time, just attachign the surface to a View parent. The Surface can stay the same but if I detach the SurfaceView to retrieve it from another activity and reattach it, the inner Surface is destroyed.
So is there a way to keep the same Surface across different activities ? With a Service ?
I know the question is a bit weird to understand, I will explain specified part is request in comment.
The Surface associated with a SurfaceView or TextureView will generally be destroyed when the Activity stops. It is possible to work around this behavior.
One approach is built into TextureView, and is described in the architecture doc, and demonstrated in the "double decode" activity in Grafika. The goal of the activity is to continue playing a pair of videos while the activity restarts due to screen rotation, not pausing at all. If you follow the code you can see how the return value from onSurfaceTextureDestroyed() is used to keep the SurfaceTexture alive, and how TextureView#setSurfaceTexture() attaches the SurfaceTexture to the new View. There's a bit of a trick to it -- the setSurfaceTexture() needs to happen in onCreate(), not onSurfaceTextureAvailable() -- but it's reasonably straightforward.
The example uses MediaCodec output for video playback, but it'll work equally well with anything that takes a Surface for output -- just create a Surface from the SurfaceTexture.
If you don't mind getting ankle-deep into OpenGL ES, you can just create your own SurfaceTexture, independent of Views and Activities, and render it yourself to the current SurfaceView. Grafika's "texture from camera" activity does this with live video from the camera (though it doesn't try to preserve it across Activity restarts).
I am creating a layout of type FrameLayout, in which I am adding two views. Two views are objects of GLSurfaceView and SurfaceView respectively. According to Android Developers Documentation regarding SurfaceView,
"The surface is Z ordered so that it is behind the window holding its SurfaceView; the SurfaceView punches a hole in its window to allow its surface to be displayed."
It works well for me and SurfaceView always stays behind my GLSurfaceView (used for opneGL drawings). But resuming after external event the behavior is odd for a following configuration,
Android Version: 4.3
Device Model Number : Nexus 7
Kernel Version 3.4.0.g1f57c39
Jun 13
Build Number: JWR66N
For this configuration, resuming after external event puts my GLSurfaceView behind SurfaceView. In other words, SurfaceView is placed at top in ZOrder and my OpenGL drawings are no more visible. On versions greater that Android 4.3, this behavior is not seen.
I can replicate this behavior on all versions by calling SurfaceView's following method with true as a parameter.
void setZOrderOnTop
Is this known issue. Anybody can help me on this?
Regards,
Sumedh
SurfaceViews have two parts, the Surface and the View. The Surface is a completely independent layer. The View is there so the UI layout code has something to work with. Generally the View is just transparent black, so you can see through to whatever is behind it.
GLSurfaceView is just SurfaceView with some code to manage EGL contexts and threading. Underneath it's just a SurfaceView. So if you have both a SurfaceView and a GLSurfaceView, and they have the same dimensions and Z-order, then one of them is going to "win" and the other is going to "lose" because they're trying to occupy the same space at the same time. There is no defined value for which one will "win", so inconsistent behavior is expected.
One way to avoid clashes is to leave one set to the default Z, and call setZOrderMediaOverlay() on the other. The "media overlay" is still behind the UI, but above the default Surface position. If you use setZOrderOnTop(), the Surface will be positioned above the UI as well.
The upper Surface will need to be rendered with transparent pixels if you want to see something behind it (the same way that the View needs to be transparent to see the Surface).
The most efficient way to avoid this issue is to not have this issue: use one SurfaceView for everything, rendering all of your non-UI-element content to it. This requires a bit more work (and probably a SurfaceTexture) if you're rendering video or showing a camera preview on one of the Surfaces.
You can find some examples in Grafika. The "multi-surface exerciser" demonstrates three overlapping SurfaceViews rendered in software, overlapping with UI elements. Other activities show ways to work with Surfaces, GLES, the camera, and video.
See also the Android System-Level Graphics Architecture doc, which explains all this in much greater detail.
Dont use "setZOrderOnTop" as true. That will get it over all the other layouts.
If you are using multiple surfaceviews. use this for each surfaceview
yourSurfaceView.setZOrderMediaOverlay(true);
then set this setZOrderOnTop as false for the surfaceview you initiated later and wanted it to get back to the other surfaceviews
secondSurfaceview.setZOrderOnTop(false);
I am trying to generate movie using MediaMuxer. The Grafika example is an excellent effort, but when i try to extend it, I have some problems.
I am trying to draw some basic shapes like square, triangle, lines into the Movie. My openGL code works well if I draw the shapes into the screen but I couldn't draw the same shapes into the video.
I also have questions about setting up openGL matrix, program, shader and viewport. Normally, there are methods like onSurfaceCreated and onSurfaceChanged so that I can setup these things. What is the best way to do it in GeneratedMovie?
Anybody has examples of writing into video with more complicated shapes would be welcome
The complexity of what you're drawing shouldn't matter. You draw whatever you're going to draw, then call eglSwapBuffers() to submit the buffer. Whether you draw one flat-shaded triangle or 100K super-duper-shaded triangles, you're still just submitting a buffer of data to the video encoder or the surface compositor.
There is no equivalent to SurfaceView's surfaceCreated() and surfaceChanged(), because the Surface is created by MediaCodec#createInputSurface() (so you know when it's created), and the Surface does not change.
The code that uses GeneratedMovie does some fairly trivial rendering (set scissor rect, call clear). The code in RecordFBOActivity is what you should probably be looking at -- it has a bouncing rect and a spinning triangle, and demonstrates three different ways to deal with the fact that you have to render twice.
(The code in HardwareScalerActivity uses the same GLES routines and demonstrates texturing, but it doesn't do recording.)
The key thing is to manage your EGLContext and EGLSurfaces carefully. The various bits of GLES state are held in the EGLContext, which can be current on only one thread at a time. It's easiest to use a single context and set up a separate EGLSurface for each Surface, but you can also create separate contexts (with or without sharing) and switch between them.
Some additional background material is available here.
I have 2 Activitiys which use OpenGL for drawing. At a transition from one activity to the next I get an unsightly empty screen filled with my OpenGL clear colour (so its not as bad as a black screen).
I wish to effectively transition seamlessly between Activitys, but there are several high load regions when a GLSurfaceView is created. The main issue is texture loading as this is slowest.
Is there anyway to double buffer between Activitys so that the last Activity view is frozen until I explicitly tell my next Activity to draw? I want transitions to be seamless?
Moving everything into one GLSurfaceView instance isn't really an option I want to consider.
You can use setRenderMode( RENDERMODE_WHEN_DIRTY) in your GLSurfaceView, so the surface only will be redraw when you call requestRender().
This way, anything that you draw before calling another surface view will only be cleared when you request a new draw.
You can back to the continuous drawing by setting render mode as RENDERMODE_CONTINUOUSLY.
It is hard to do it in Android 2.x because of its OpenGL ES. Also, it is not recommended that you use two OpenGL in one applications if you are in render continously. If so, to control them easily, you will need RENDERMODE_WHEN_DIRTY.
If you use it in Android 4.x, TextureView is an optional to do it.
TextureView is as same as GLSurfaceView but with View compatible, it means that you can use ViewAnimation for TextureView.
When is it necessary, or better to use a SurfaceView instead of a View?
Views are all drawn on the same GUI thread which is also used for all user interaction.
So if you need to update GUI rapidly or if the rendering takes too much time and affects user experience then use SurfaceView.
A few things I've noted:
SurfaceViews contain a nice rendering mechanism that allows threads to update the surface's content without using a handler (good for animation).
Surfaceviews cannot be transparent, they can only appear behind other elements in the view hierarchy.
I've found that they are much faster for animation than rendering onto a View.
For more information (and a great usage example) refer to the LunarLander project in the SDK
's examples section.
updated 05/09/2014
OK. We have official document now. It talked all I have mentioned, in a better way.
Read more detailed here.
Yes, the main difference is surfaceView can be updated on the background thread. However, there are more you might care.
surfaceView has dedicate surface buffer while all the view shares one surface buffer that is allocated by ViewRoot. In another word, surfaceView cost more resources.
surfaceView cannot be hardware accelerated (as of JB4.2) while 95% operations on normal View are HW accelerated using openGL ES.
More work should be done to create your customized surfaceView. You need to listener to the surfaceCreated/Destroy Event, create an render thread, more importantly, synchronized the render thread and main thread. However, to customize the View, all you need to do is override onDraw method.
The timing to update is different. Normal view update mechanism is constraint or controlled by the framework:You call view.invalidate in the UI thread or view.postInvalid in other thread to indicate to the framework that the view should be updated. However, the view won't be updated immediately but wait until next VSYNC event arrived. The easy approach to understand VSYNC is to consider it is as a timer that fire up every 16ms for a 60fps screen. In Android, all the normal view update (and display actually but I won't talk it today), is synchronized with VSYNC to achieve better smoothness. Now,back to the surfaceView, you can render it anytime as you wish. However, I can hardly tell if it is an advantage, since the display is also synchronized with VSYNC, as stated previously.
The main difference is that SurfaceView can be drawn on by background theads but Views can't.
SurfaceViews use more resources though so you don't want to use them unless you have to.
A SurfaceView is a custom view in Android that can be used to drawn inside it.
The main difference between a View and a SurfaceView is that a View is drawn in the
UI Thread, which is used for all the user interaction.
If you want to update the UI rapidly enough and render a good amount of information in
it, a SurfaceView is a better choice.
But there are a few technical insides to the SurfaceView:
1. They are not hardware accelerated.
2. Normal views are rendered when you call the methods invalidate or postInvalidate(), but this does not mean the view will be
immediately updated (A VSYNC will be sent, and the OS decides when
it gets updated. The SurfaceView can be immediately updated.
3. A SurfaceView has an allocated surface buffer, so it is more costly
One of the main differences between surfaceview and view is that to refresh the screen for a normal view we have to call invalidate method from the same thread where the view is defined. But even if we call invalidate, the refreshing does not happen immediately. It occurs only after the next arrival of the VSYNC signal. VSYNC signal is a kernel generated signal which happens every 16.6 ms or this is also known as 60 frame per second. So if we want more control over the refreshing of the screen (for example for very fast moving animation), we should not use normal view class.
On the other hand in case of surfaceview, we can refresh the screen as fast as we want and we can do it from a background thread. So refreshing of the surfaceview really does not depend upon VSYNC, and this is very useful if we want to do high speed animation. I have few training videos and example application which explain all these things nicely. Please have a look at the following training videos.
https://youtu.be/kRqsoApOr9U
https://youtu.be/Ji84HJ85FIQ
https://youtu.be/U8igPoyrUf8
Why use SurfaceView and not the classic View class...
One main reason is that SurfaceView can rapidly render the screen.
In simple words a SV is more capable of managing the timing and render animations.
To have a better understanding what is a SurfaceView we must compare it with the View class.
What is the difference... check this simple explanation in the video
https://m.youtube.com/watch?feature=youtu.be&v=eltlqsHSG30
Well with the View we have one major problem....the timing of rendering animations.
Normally the onDraw() is called from the Android run-time system.
So, when Android run-time system calls onDraw() then the application cant control
the timing of display, and this is important for animation. We have a gap of timing
between the application (our game) and the Android run-time system.
The SV it can call the onDraw() by a dedicated Thread.
Thus: the application controls the timing. So we can display the next bitmap image of the animation.