Synchronization of Android Views on top of GLSurfaceView - android

Is there a good way to draw standard Android View objects on top of a GLSurfaceView and synchronize the movement between the two layers?
My layout looks like
<FrameLayout>
<RelativeLayout>
<MyCustomGLSurfaceView>
</FrameLayout>
The MyCustomGLSurfaceView implements a camera and this camera is moved when the user touches the screen appropriately.
At the end of the onDrawFrame() callback in the renderer I invoke a callback to tell the RelativeLayout that the camera has moved and that it should update the location of any subviews as necessary.
To try and synchronize the two layers I am doing the following:
Set renderMode = RENDERMODE_WHEN_DIRTY within MyCustomGLSurfaceView
Having MyCustomGLSurfaceView implement Choreographer.FrameCallback and in the doFrame(frameTimeNanos: Long) method calling requestRender()
onDrawFrame invokes a callback to indicate the camera has moved and the RelativeLayout updates the position of it’s subviews as necessary.
Delaying the position of the camera 1 frame within the MyCustomGLSurfaceView since it appears that the Android View objects aren't updated on the screen until the following frame whereas the MyCustomGLSurfaceView was updated immediately.
The theory is that by listening for Choreographer callbacks, the rendering of the MyCustomGLSurfaceView is happening at the same time as when the application is re-drawing the standard UI elements.
This seems to work reasonably well but there are noticeable stutters within the Android View object layer. Also positioning the elements in the Android View layer has some odd behaviour when doing it via setting margins (views get destroyed when they're off screen and don't always reappear and sometimes stop being updated at all). I have tried positioning them using translationX / translationY but then the synchronization between the two layers isn't as good.
Is there a good approach for achieving this synchronization?
I have thought that it might be better to use a TextureView as opposed to a GLSurfaceView since the TextureView is composited into the view hierarchy rather than being a separate surface, so I thought maybe synchronization will fall out naturally from that.

Related

Best solution SurfaceView vs View drawing?

I would like to display the score when user click on a button (make +1 fly to top of the screen) and I wonder if it is better to use a surfaceview or view?
See here for more information on the differences between SurfaceView and View.
For your case, View would probably be simpler and appropriate.
If your application does not require a significant amount of processing or frame-rate speed (perhaps for a chess game, a snake game, or another slowly-animated application), then you should consider creating a custom View component and drawing with a Canvas in View.onDraw(). The most convenient aspect of doing so is that the Android framework will provide you with a pre-defined Canvas to which you will place your drawing calls.
A SurfaceView, on the other hand, is not drawn on the main thread and instead takes place on a secondary thread:
The SurfaceView is a special subclass of View that offers a dedicated drawing surface within the View hierarchy. The aim is to offer this drawing surface to an application's secondary thread, so that the application isn't required to wait until the system's View hierarchy is ready to draw. Instead, a secondary thread that has reference to a SurfaceView can draw to its own Canvas at its own pace.
I think view fits this scenario better. You could translate a view to the top of the screen.
So far, I've only used SurfaceView for rendering background graphics. Of course you could do it using SurfaceView, but that would bring more complexity in terms of coding. You need to draw this "+1" and calculate its position from frame to frame.
Using a View, you could just apply an animation, set the duration, and that should do the trick.

SurfaceView hides other components on screen

I am creating a layout of type FrameLayout, in which I am adding two views. Two views are objects of GLSurfaceView and SurfaceView respectively. According to Android Developers Documentation regarding SurfaceView,
"The surface is Z ordered so that it is behind the window holding its SurfaceView; the SurfaceView punches a hole in its window to allow its surface to be displayed."
It works well for me and SurfaceView always stays behind my GLSurfaceView (used for opneGL drawings). But resuming after external event the behavior is odd for a following configuration,
Android Version: 4.3
Device Model Number : Nexus 7
Kernel Version 3.4.0.g1f57c39
Jun 13
Build Number: JWR66N
For this configuration, resuming after external event puts my GLSurfaceView behind SurfaceView. In other words, SurfaceView is placed at top in ZOrder and my OpenGL drawings are no more visible. On versions greater that Android 4.3, this behavior is not seen.
I can replicate this behavior on all versions by calling SurfaceView's following method with true as a parameter.
void setZOrderOnTop
Is this known issue. Anybody can help me on this?
Regards,
Sumedh
SurfaceViews have two parts, the Surface and the View. The Surface is a completely independent layer. The View is there so the UI layout code has something to work with. Generally the View is just transparent black, so you can see through to whatever is behind it.
GLSurfaceView is just SurfaceView with some code to manage EGL contexts and threading. Underneath it's just a SurfaceView. So if you have both a SurfaceView and a GLSurfaceView, and they have the same dimensions and Z-order, then one of them is going to "win" and the other is going to "lose" because they're trying to occupy the same space at the same time. There is no defined value for which one will "win", so inconsistent behavior is expected.
One way to avoid clashes is to leave one set to the default Z, and call setZOrderMediaOverlay() on the other. The "media overlay" is still behind the UI, but above the default Surface position. If you use setZOrderOnTop(), the Surface will be positioned above the UI as well.
The upper Surface will need to be rendered with transparent pixels if you want to see something behind it (the same way that the View needs to be transparent to see the Surface).
The most efficient way to avoid this issue is to not have this issue: use one SurfaceView for everything, rendering all of your non-UI-element content to it. This requires a bit more work (and probably a SurfaceTexture) if you're rendering video or showing a camera preview on one of the Surfaces.
You can find some examples in Grafika. The "multi-surface exerciser" demonstrates three overlapping SurfaceViews rendered in software, overlapping with UI elements. Other activities show ways to work with Surfaces, GLES, the camera, and video.
See also the Android System-Level Graphics Architecture doc, which explains all this in much greater detail.
Dont use "setZOrderOnTop" as true. That will get it over all the other layouts.
If you are using multiple surfaceviews. use this for each surfaceview
yourSurfaceView.setZOrderMediaOverlay(true);
then set this setZOrderOnTop as false for the surfaceview you initiated later and wanted it to get back to the other surfaceviews
secondSurfaceview.setZOrderOnTop(false);

Performance: How to prevent requestLayout() from laying out entire hierarchy

I have a pretty complex android-application. Already flattened view-hierarchies as far as possible, but I still have lags in the application. For example there is a menu with entries that collapse/expand by having their height set by a ValueAnimator. Typically the animation runs with a bit of a lag the first time, and smooth after this first pass.
I noticed that when i call "requestLayout()" on the Menu-Item, Android seems to do a layout-pass and multiple measure-passes through the entire hierarchy.
Since i know that although the Menu-Item(View) changes height, the Menu(View) itself doesn't, is there some way to tell this to the application?
Can i somehow perform this first pass that seems to lag myself so that it occurs after application start-up and not at the first touch-input?
Here's a sketch of the animation I'm doing:
I am not sure why a layout is being triggered in your animation but I am going to answer your question abstractly.
If you are calling requestLayout (either directly or indirectly) in your animation you are doing it WRONG.
requestLayout, for correctness and safety, does a full view traversal on the view hierarchy b/c conceptually changing bounding box of a node in the view hierarchy can result in change in the bounds of any other node. Not always the case but in general it could, thats why requestLayout is a full traversal.
All of this is just another way of saying requestLayout will eat away time from your 16.6 ms frame time slot and make your animation choppy. This is especially bad for deep and complex hierarchies with many RelativeLayouts which internally does two passes per level (thus potentially causing exponential passes on a subtree)
Now, if you want to animate change in dimension use setScale in a hardware layer. And at the end of the animation merrily call requestlayout and also destroy the layer (to free up memory).
Because its a layer, repeatedly calling setScale in your animation results in change of the texture on the GPU and as a result totally bypass the traversal mechanism of the view hierarchy. This should make it buttery smooth.
Your question looks like mine: Only relayout children and not all the tree
First, you can try to avoid complex view hierarchy for your view. If possible, explode views on views that doesn't depend on another one.
When an animation is performed, avoid any layout request. Start your animation with a delay if a layout request is pending.
Use hardware layers for animation if possible (maybe Android use it by default with ValueAnimator)

Android view animation on top of GLSurfaceView is slow

We've noticed that when you put Android views with view animation (nothing complex, just AlphaAnimation and TranslateAnimation) on top of a GLSurfaceView, the animation runs slowly (i.e. you see a lot of stuttering.) I am calling pause() on the GLSurfaceView, and I believe I've confirmed (through setting breakpoints) that the GL draw calls are not getting hit while the animation is playing, so I'm not sure where the slowness is coming from.
Does anyone know of a way around this? I know that on iPhone this also used to be a problem, but there was some OS update they made to fix the issue. They are short view animations (e.g. You Win!) so it's not the worst thing in the world, but it would be nice if there was some workaround.
The reason we are not doing the animations in GL is that they have to be able to run from any Activity in our game, and not all of our Activities have GLSurfaceViews.
Finally, if it matters, we am using the modified GLSurfaceView source from Replica Island http://code.google.com/p/replicaisland/
Drawing on top of a GLSurfaceView is slow, therefore animating is as well. You are forcing the framework to do more work to determine what part of the surface view is visible.
You should really consider doing these animations inside the surface view when you are using a surface view.
An alternative is to put the animation in a small window above your activity.

Difference between SurfaceView and View?

When is it necessary, or better to use a SurfaceView instead of a View?
Views are all drawn on the same GUI thread which is also used for all user interaction.
So if you need to update GUI rapidly or if the rendering takes too much time and affects user experience then use SurfaceView.
A few things I've noted:
SurfaceViews contain a nice rendering mechanism that allows threads to update the surface's content without using a handler (good for animation).
Surfaceviews cannot be transparent, they can only appear behind other elements in the view hierarchy.
I've found that they are much faster for animation than rendering onto a View.
For more information (and a great usage example) refer to the LunarLander project in the SDK
's examples section.
updated 05/09/2014
OK. We have official document now. It talked all I have mentioned, in a better way.
Read more detailed here.
Yes, the main difference is surfaceView can be updated on the background thread. However, there are more you might care.
surfaceView has dedicate surface buffer while all the view shares one surface buffer that is allocated by ViewRoot. In another word, surfaceView cost more resources.
surfaceView cannot be hardware accelerated (as of JB4.2) while 95% operations on normal View are HW accelerated using openGL ES.
More work should be done to create your customized surfaceView. You need to listener to the surfaceCreated/Destroy Event, create an render thread, more importantly, synchronized the render thread and main thread. However, to customize the View, all you need to do is override onDraw method.
The timing to update is different. Normal view update mechanism is constraint or controlled by the framework:You call view.invalidate in the UI thread or view.postInvalid in other thread to indicate to the framework that the view should be updated. However, the view won't be updated immediately but wait until next VSYNC event arrived. The easy approach to understand VSYNC is to consider it is as a timer that fire up every 16ms for a 60fps screen. In Android, all the normal view update (and display actually but I won't talk it today), is synchronized with VSYNC to achieve better smoothness. Now,back to the surfaceView, you can render it anytime as you wish. However, I can hardly tell if it is an advantage, since the display is also synchronized with VSYNC, as stated previously.
The main difference is that SurfaceView can be drawn on by background theads but Views can't.
SurfaceViews use more resources though so you don't want to use them unless you have to.
A SurfaceView is a custom view in Android that can be used to drawn inside it.
The main difference between a View and a SurfaceView is that a View is drawn in the
UI Thread, which is used for all the user interaction.
If you want to update the UI rapidly enough and render a good amount of information in
it, a SurfaceView is a better choice.
But there are a few technical insides to the SurfaceView:
1. They are not hardware accelerated.
2. Normal views are rendered when you call the methods invalidate or postInvalidate(), but this does not mean the view will be
immediately updated (A VSYNC will be sent, and the OS decides when
it gets updated. The SurfaceView can be immediately updated.
3. A SurfaceView has an allocated surface buffer, so it is more costly
One of the main differences between surfaceview and view is that to refresh the screen for a normal view we have to call invalidate method from the same thread where the view is defined. But even if we call invalidate, the refreshing does not happen immediately. It occurs only after the next arrival of the VSYNC signal. VSYNC signal is a kernel generated signal which happens every 16.6 ms or this is also known as 60 frame per second. So if we want more control over the refreshing of the screen (for example for very fast moving animation), we should not use normal view class.
On the other hand in case of surfaceview, we can refresh the screen as fast as we want and we can do it from a background thread. So refreshing of the surfaceview really does not depend upon VSYNC, and this is very useful if we want to do high speed animation. I have few training videos and example application which explain all these things nicely. Please have a look at the following training videos.
https://youtu.be/kRqsoApOr9U
https://youtu.be/Ji84HJ85FIQ
https://youtu.be/U8igPoyrUf8
Why use SurfaceView and not the classic View class...
One main reason is that SurfaceView can rapidly render the screen.
In simple words a SV is more capable of managing the timing and render animations.
To have a better understanding what is a SurfaceView we must compare it with the View class.
What is the difference... check this simple explanation in the video
https://m.youtube.com/watch?feature=youtu.be&v=eltlqsHSG30
Well with the View we have one major problem....the timing of rendering animations.
Normally the onDraw() is called from the Android run-time system.
So, when Android run-time system calls onDraw() then the application cant control
the timing of display, and this is important for animation. We have a gap of timing
between the application (our game) and the Android run-time system.
The SV it can call the onDraw() by a dedicated Thread.
Thus: the application controls the timing. So we can display the next bitmap image of the animation.

Categories

Resources