As we know, we can choose TextureView, SurfaceView and GLSurfaceView for android camera preview.
Which one is best choice for camera preview ? I'm focused on the camera performance.
From a performance perspective, SurfaceView is the winner.
With SurfaceView, frames come from the camera and are forwarded to the system graphics compositor (SurfaceFlinger) with no copying. In most cases, any scaling will be done by the display processor rather than the GPU, which means that instead of scanning the pixels once for scaling and again for scan-out, they're only scanned once.
GLSurfaceView is a SurfaceView with some wrapper classes that handle EGL setup and thread management. You can't use OpenGL ES on a Surface that is receiving camera frames, so you're doing extra work with no benefit. (The overhead is minor one-time setup, not per-frame, so you likely won't be able to measure the difference.)
TextureView receives the frames in a SurfaceTexture as an "external" OpenGL ES texture, then uses GLES to render them onto the app's UI surface. The scaling and rendering are performed by the GPU, and the result is then forwarded to SurfaceFlinger. This is the slowest option, but also the most flexible of the Views.
If you'd like to learn more about how the system works, see the Android Graphics Architecture document.
" The SurfaceView creates a new window in the Android Windowsystem. Its advantage is, that if the SurfaceView gets refreshed, only this window will be refreshed. If you additionally update UI Elements (which are in another window of the windowsystem), then both refresh operations block themselfes (especially when ui drawing is hardwaresupported) because opengl cannot handle multi thread drawing properly.
For such a case it could be better using the TextureView, cause it's not another window of the Android Windowsystem. so if you refresh your View, all UI elements get refreshed as well. (Probably) everything in one Thread.
Hope I could help some of you! "
Source : stackoverflow.com
GLSurfaceView is a SurfaceView with a wrapper class that does all the EGL setup and inter-thread messaging for you.
Its completely upto you what you put to use.. They have their pros and cons over eachother :)
Related
I'm using a Texture widget, rendering its content from native code using OpenGL ES. In native code I call ANativeWindow_fromSurface and from that create an EGL surface. AIUI what happens is:
The ANativeWindow represents the producer side of a buffer queue.
Calling eglSwapBuffers causes a texture to be sent to this queue.
Flutter receives the texture and renders it using Skia when the TextureLayer is painted.
The texture is scaled to match the size of the TextureLayer (the scaling happens in AndroidExternalTextureGL::Paint()).
I'm trying to figure out how to synchronise the OpenGL rendering. I think I can use the choreographer to synchronise with the display vsync, but I'm unclear on how much latency this bufferqueue-then-render-with-skia mechanism introduces. I don't see any means to explicitly synchronise my native code's generation of textures with the TextureLayer's painting of them.
The scaling appears to be a particularly tricky aspect. I would like to avoid it entirely, by ensuring that the textures the native code generates are always of the right size. However there doesn't appear to be any direct link between the size of the TextureLayer and the size of the Surface/ANativeWindow. I could use a SizeChangedLayoutNotifier (or one of various alternative hacks) to detect changes in the size and communicate them to the native code, but I think this would lag by at least a frame so scaling would still take place when resizing.
I did find this issue, which talks about similar resizing challenges, but in the context of using an OEM web view. I don't understand Hixie's detailed proposal in that issue, but it appears to be specific to embedding of OEM views so I don't think it would help with my case.
Perhaps using a Texture widget here is the wrong approach. It seems to be designed mainly for displaying things like videos and camera previews. Is there another way to host natively rendered, interactive OpenGL graphics in Flutter?
i know that TextureView is show up after ICS.
but, SurfaceView is not deprecated at ICS.
SurfaceView has hole-punching structure, so it has many limit point.
can't stack two SurfaceView and can't translate and etc..
why SurfaceView is not deprecated despite of TextureView is show up?
SurfaceView is faster, and can handle DRM-protected video.
The hole-punching structure is necessary because SurfaceView's Surface is handled directly by the system compositor. For TextureView, you draw on a Surface, which is converted to a GL texture within the app, which is rendered a second time by the app onto the View layer. So there's an extra copy.
For DRM-protected video, no user or system code -- not even the Linux kernel -- is allowed to see unencrypted pixels. Only the video decoder and the display hardware. Because SurfaceView just forwards references through, and doesn't touch the actual data, this works.
For more details, see the graphics architecture doc.
I plan to write a 2D drawing app with a zoomable user interface. With the app the user should be able to transform (translate and scale) drawn paths (and of course the UI). I assume that there will be up to 500 paths at the same time.
Now, I am wondering which view to start off with (View, SurfaceView or GLSurfaceView ) in order to provide acceptable performance. I read a couple of posts [1-6] including the once on android developer guide and am still not 100% sure which view to use.
According to the presentation at Google I/O 2009 (http://youtu.be/U4Bk5rmIpic?t=45m10s) and own experiences canvas (View and SurfaceView) doesn't seem to be performing fast enough when handling more than 100 paths.
Does anyone see a possibility in implementing the app using canvas or is OpenGL the way to go?
[1] Android: Deciding between SurfaceView and OpenGL (GLSurfaceView)
[2] SurfaceView or GLSurfaceview?
[3] Difference between SurfaceView and View?
[4] http://pierrchen.blogspot.jp/2014/03/anroid-graphics-surfaceview-all-you.html
[5] Difference between SurfaceView and ImageView
[6] Difference between SurfaceView and GLSurfaceView in Android
Because of your performance concerns, you will need to use hardware rendering. Displays keep getting denser, so touching every pixel in software is getting slower.
If you want to draw with a Canvas, you must render to a custom View, with hardware acceleration enabled. Canvas rendering on a Surface is always done with software.
SurfaceView and GLSurfaceView are nearly the same thing; GLSurfaceView just has some helper functions that take care of some of the housekeeping for you. You can use GLES to render on SurfaceView and TextureView. There are multiple examples of this in Grafika.
If you want to know what's going on under the hood, see the System-Level Graphics Architecture doc.
You may want to consider the use of an open-source 2D game engine. That'll handle the EGL setup, should provide code for GLES font rendering, and so on.
Is there any tradeoff of add some OpenGL to a "serious" (not game) Android app?
The reason why I want to use OpenGL, is to add some 3d behaviour to a few views.
According to this http://developer.android.com/guide/topics/graphics/opengl.html OpenGL 1.0 is available in every Android device and doesn't require modification of manifest file. So there will never be compatibility issues.
The only 2 things I can think about is 1. mantainability by other developers which can't OpenGL. And possible 2. Integration problems with other components / not well reusable (although, not sure).
Is there also anything else, unexpected things, overhead of some sort, complications, etc.?
Asking because it seems not to be a very popular practice, people seem to prefer to "fake" the 3d with 2d or give it up. Don't know if it's only because they don't want to learn OpenGL.
I use OpenGL for some visualization in a released app, and I have an uncaught exception handler in place to catch any exception coming from the GLThread and disable OpenGL the next time the app is run, since I had some crash reports in the internals of GLSurfaceView.java coming in from buggier devices. If the 3D rendering is not crucial to your app, this is one approach you can take so that users with these devices can continue to use the app.
From Android 3.0+ you can also preserve the EGL context by calling GLSurfaceView. setPreserveEGLContextOnPause(true);. You'll only really need to do this if your renderer is very expensive to initialize, and it only works if you're not destroying the GLSurfaceView in between (i.e. the default behavior of an activity when rotating the device). If you're not loading that many resources then initializing OpenGL is usually fast enough.
From the SurfaceView docs (emphasis mine):
The surface is Z ordered so that it is behind the window holding its SurfaceView; the SurfaceView punches a hole in its window to allow its surface to be displayed. The view hierarchy will take care of correctly compositing with the Surface any siblings of the SurfaceView that would normally appear on top of it. This can be used to place overlays such as buttons on top of the Surface, though note however that it can have an impact on performance since a full alpha-blended composite will be performed each time the Surface changes.
The advantage is that your GL thread can update the screen independently of the UI thread (i.e. it doesn't need to render to a texture and render the texture to the screen); the disadvantage is that something needs to composite your view with the screen. If you're lucky, this can be done in the "hardware composer"; otherwise it is done on the GPU and may be a bit wasteful of GPU resources (see For Butter or Worse: Smoothing Out Performance in Android UIs at 27:32 and 40:23).
If your view is small, it may be better to use a TextureView. This will render to a texture and render the texture as part of the normal view hierarchy which might be better, but can increase latency. The downside is it's only available since API level 14.
Can anyone tell me what the basic difference is between SurfaceView and GLSurfaceView? When should I use SurfaceView, and when should I use GLSurfaceView?
I read some already answered questions on Stack Overflow, but they did not satisfy my queries.
Any help would be appreciated.
A GLSurfaceView is a SurfaceView that you can render into with OpenGL. Choosing between them is simple:
If you're familiar with OpenGL and need what it provides, use a GLSurfaceView.
Otherwise, use a SurfaceView.
OpenGL is low-level. If you're not already familiar with it, it's an undertaking to learn. If you only need 2D drawing, SurfaceView uses the high-level, reasonably high-performance Canvas. It's very easy to work with.
Unless you have a strong reason to use a GLSurfaceView, you should use a regular SurfaceView. I would suggest that if you don't already know that you need GL, then you probably don't.
GLSurfaceView is the primary building block for 3D applications as View is for 2D applications. It is widely used not only in 3D games but also multimedia applications such as camera to create special preview affect.
GLSurfaceView extends SurfaceView and additionally owns a render thread and a render object set by the client. The render thread keeps running , continuously or on-demand, and delegates to the render object to draw frame using OpenGL API. For both SurfaceView and GLSurfaceView, rendering is performing in a separate thread other than main thread. The difference is with SurfaceView the rendering thread is created by client while with GLSurfaceView it is created by the system. What's more, GLSurfaceView will internally handle the synchronization between main thread and rendering thread.
For more, check out this and this
SurfaceView
AFAIK Canvas is Simple to implement and effective in 2D drawing but 3D drawing are not supported on it
GLSurfaceView
If you want to design some 3D Game then you shold go with GLSurfaceView and OGLES
Whats my experience is if you just want to do 2D processing then select Canvas because its easier to implement and effective compare to GLSurfaceView.