Is there a good way to draw standard Android View objects on top of a GLSurfaceView and synchronize the movement between the two layers?
My layout looks like
<FrameLayout>
<RelativeLayout>
<MyCustomGLSurfaceView>
</FrameLayout>
The MyCustomGLSurfaceView implements a camera and this camera is moved when the user touches the screen appropriately.
At the end of the onDrawFrame() callback in the renderer I invoke a callback to tell the RelativeLayout that the camera has moved and that it should update the location of any subviews as necessary.
To try and synchronize the two layers I am doing the following:
Set renderMode = RENDERMODE_WHEN_DIRTY within MyCustomGLSurfaceView
Having MyCustomGLSurfaceView implement Choreographer.FrameCallback and in the doFrame(frameTimeNanos: Long) method calling requestRender()
onDrawFrame invokes a callback to indicate the camera has moved and the RelativeLayout updates the position of it’s subviews as necessary.
Delaying the position of the camera 1 frame within the MyCustomGLSurfaceView since it appears that the Android View objects aren't updated on the screen until the following frame whereas the MyCustomGLSurfaceView was updated immediately.
The theory is that by listening for Choreographer callbacks, the rendering of the MyCustomGLSurfaceView is happening at the same time as when the application is re-drawing the standard UI elements.
This seems to work reasonably well but there are noticeable stutters within the Android View object layer. Also positioning the elements in the Android View layer has some odd behaviour when doing it via setting margins (views get destroyed when they're off screen and don't always reappear and sometimes stop being updated at all). I have tried positioning them using translationX / translationY but then the synchronization between the two layers isn't as good.
Is there a good approach for achieving this synchronization?
I have thought that it might be better to use a TextureView as opposed to a GLSurfaceView since the TextureView is composited into the view hierarchy rather than being a separate surface, so I thought maybe synchronization will fall out naturally from that.
I'm struggling to understand the process of drawing to SurfaceView and therefore the whole Surface/Canvas/Bitmap system, which is used in Android.
I've read all articles and API documentation pages, which I was able to find on android-developers site, a few tutorials of android graphics, LunarLander source code and this question.
Please tell me, which of these statements are true, which are not, and why.
Canvas has its own Bitmap attached to it. Surface has its own Canvas attached to it.
All View's of window share the same Surface and thus share the same Canvas.
SurfaceView is subclass of View, which, unlike other View's subclasses and View itself, has its own Surface to draw in.
There is also one additional question:
Why is there a need for a Surface class, if there is already a Canvas for high-level operations with bitmap. Give an example of a situation where Canvas is non-suitable for doing work which Surface can do.
Here are some definitions:
A Surface is an object holding pixels that are being composited to the screen. Every window you see on the screen (a dialog, your full-screen activity, the status bar) has its own surface that it draws in to, and Surface Flinger renders these to the final display in their correct Z-order. A surface typically has more than one buffer (usually two) to do double-buffered rendering: the application can be drawing its next UI state while the surface flinger is compositing the screen using the last buffer, without needing to wait for the application to finish drawing.
A window is basically like you think of a window on the desktop. It has a single Surface in which the contents of the window is rendered. An application interacts with the Window Manager to create windows; the Window Manager creates a Surface for each window and gives it to the application for drawing. The application can draw whatever it wants in the Surface; to the Window Manager it is just an opaque rectangle.
A View is an interactive UI element inside of a window. A window has a single view hierarchy attached to it, which provides all of the behavior of the window. Whenever the window needs to be redrawn (such as because a view has invalidated itself), this is done into the window's Surface. The Surface is locked, which returns a Canvas that can be used to draw into it. A draw traversal is done down the hierarchy, handing the Canvas down for each view to draw its part of the UI. Once done, the Surface is unlocked and posted so that the just drawn buffer is swapped to the foreground to then be composited to the screen by Surface Flinger.
A SurfaceView is a special implementation of View that also creates its own dedicated Surface for the application to directly draw into (outside of the normal view hierarchy, which otherwise must share the single Surface for the window). The way this works is simpler than you may expect -- all SurfaceView does is ask the window manager to create a new window, telling it to Z-order that window either immediately behind or in front of the SurfaceView's window, and positioning it to match where the SurfaceView appears in the containing window. If the surface is being placed behind the main window (in Z order), SurfaceView also fills its part of the main window with transparency so that the surface can be seen.
A Bitmap is just an interface to some pixel data. The pixels may be allocated by Bitmap itself when you are directly creating one, or it may be pointing to pixels it doesn't own such as what internally happens to hook a Canvas up to a Surface for drawing. (A Bitmap is created and pointed to the current drawing buffer of the Surface.)
Also please keep in mind that, as this implies, a SurfaceView is a pretty heavy-weight object. If you have multiple SurfaceViews in a particular UI, stop and think about whether this is really needed. If you have more than two, you almost certainly have too many.
Here is a very basic and simple conceptual overview of how interaction happens among the Window, Surface, Canvas, and Bitmap.
Sometimes, a visual representation helps a lot in understanding twisted concepts.
I hope this graphic could help someone.
A Bitmap is simply a wrapper for a collection of pixels. Think of it as an array of pixels with some other convenient functions.
The Canvas is simply the class that contains all the drawing methods. It is similar to the Graphics class in AWT/Swing if you are familiar with that. All the logic on how to draw a circle, or a box, etc is contained inside Canvas. A canvas draws on a Bitmap or an open GL container but there is no reason why in the future it could be extended to draw onto other types of rasters.
SurfaceView is a View that contains a Surface. A surface is similar to a bitmap (it has a pixel store). I do not know how it is implemented but I'd imagine it is a some kind of Bitmap wrapper with extra methods for things that are directly related to screen displays (That is the reason for a surface, a Bitmap is too generic). You can get a Canvas from your Surface which is really getting the Canvas associated with the underlying Bitmap.
Your questions.
1.Canvas has its own Bitmap attached to it. Surface has its own Canvas attached to it.
Yes, a canvas operates on a Bitmap (or an open GL panel). Surface gives you a Canvas that is operating on whatever Surface is using for its Bitmap style pixel store.
2.All View's of window share the same Surface and thus share the same Canvas.
No. You could have as many surface views as you want.
3.SurfaceView is subclass of View, which, unlike other View's subclasses and View itself, has its own Surface to draw in.
Yes. Just like ListView is a subclass of View that has its own List data structure. Each subclass of View does something different.
I used surfaceview to create a graph. This graph changes continuously over time.
When I use postInvalidate method to update graph, surfaceview is redrawn but I don't want to redraw xy-axis. what should I do?
I believe what you are looking for is the lockCanvas(Rect dirty) method from the SurfaceHolder class.
http://developer.android.com/reference/android/view/SurfaceHolder.html
Official documentation:
"Just like lockCanvas() but allows specification of a dirty rectangle. Every pixel within that rectangle must be written; however pixels outside the dirty rectangle will be preserved by the next call to lockCanvas()."
Therefore you can just change the graphs contents, leaving any pre-drawn axis
I have 2 Activitiys which use OpenGL for drawing. At a transition from one activity to the next I get an unsightly empty screen filled with my OpenGL clear colour (so its not as bad as a black screen).
I wish to effectively transition seamlessly between Activitys, but there are several high load regions when a GLSurfaceView is created. The main issue is texture loading as this is slowest.
Is there anyway to double buffer between Activitys so that the last Activity view is frozen until I explicitly tell my next Activity to draw? I want transitions to be seamless?
Moving everything into one GLSurfaceView instance isn't really an option I want to consider.
You can use setRenderMode( RENDERMODE_WHEN_DIRTY) in your GLSurfaceView, so the surface only will be redraw when you call requestRender().
This way, anything that you draw before calling another surface view will only be cleared when you request a new draw.
You can back to the continuous drawing by setting render mode as RENDERMODE_CONTINUOUSLY.
It is hard to do it in Android 2.x because of its OpenGL ES. Also, it is not recommended that you use two OpenGL in one applications if you are in render continously. If so, to control them easily, you will need RENDERMODE_WHEN_DIRTY.
If you use it in Android 4.x, TextureView is an optional to do it.
TextureView is as same as GLSurfaceView but with View compatible, it means that you can use ViewAnimation for TextureView.
I'm developing chess game for Android (androidchess.appspot.com). If I want to add animations, should I use custom View extending Canvas (I do this now), or custom View extending SurfaceView?
I haven't tried using a View to extend Canvas, but for my game I'm using the same method as the LunarLander example game:
public class CustomView extends SurfaceView implements SurfaceHolder.Callback
The usefulness of this is that it gives you handles for SurfaceHolder (so you can call up the canvas which is drawn to the screen), and the callbacks for surfaceCreated, surfaceChanged and surfaceDestroyed. That lets you do things like drawing a custom animation as soon as the surface is available or make sure that you don't try to draw to the canvas after it has been deactivated. Looking through LunarLander should show you how to use these properly.
Edit: I remembered another reason why using a SurfaceHolder was useful. This is because, as I mentioned above, it lets you get direct access to the canvas which is drawn to the screen. With a SurfaceHolder this is done not by overriding onDraw but by using something like Canvas canvas = mSurfaceHolder.lockCanvas(). (See LunarLander for exact syntax). The reason this is important is because it lets you control exactly when the drawing happens. If you can only work by overriding onDraw(), then the drawing doesn't happen until your program reaches a 'waiting' phase. In other words, you can't use invalidate() and onDraw() in a loop because the drawing won't happen until the loop finishes. And since you're likely to use loops for things like drawing a piece moving across the screen, this becomes a problem.
Note: It may be possible to avoid this problem by using multiple threads. I simply haven't tried that since it isn't required for my game; the only animation is has is fixed-length animations in response to user input rather than something continuously moving in the background, so I haven't experimented with multiple threads yet.