I have a special design requiring for the app I'm developing right now.
Right now, I have a third-party private video library which plays a video stream. The design of this screen includes a translucent panel overlaid on top of the video, blurring the portion of the video that lies behind.
Normally in order to blur the background, you are supposed to take a screenshot of the view behind, blur it and use it as an image for the foreground view.
In this case, the video keeps on playing, so the blurred image changes every frame. How would you implement this then?
A possible solution would be to create a thread, taking screenshots, cropping them and put them as a background. Even better if that view is a SurfaceView, I guess. But I'm wondering what would be the best approach in this case. Would a thread that is continually taking screenshots create a huge performance impact? Is it possible to feed a surfaceView buffer with these images?
Thanks!
A SurfaceView surface is a consumer of graphics buffers. You can't have two producers for one consumer, which means you can't send the video to it and draw on it at the same time.
You can have multiple layers; the SurfaceView surface is on a separate layer behind the View UI layer. So you could play the video to the SurfaceView's surface, and draw your blur rectangle on the SurfaceView's view. (Normally the SurfaceView's view is completely transparent, and is just used as a place-holder for layout purposes.)
Another option would be to render the video frame to a SurfaceTexture. You would then render that texture to the SurfaceView surface with GLES, and render the blur rectangle on top. You can find an example of treating live camera input as a GLES texture in Grafika ("texture from camera" activity). This has the additional advantage that, since you're not interacting with the View system -- the SurfaceView surface is composited by the system, not the app -- you can do it all on an independent thread.
In any event, rendering, grabbing a screenshot, and re-rendering is going to be slower than the options described above.
For more details about why things work the way they do, see the Android System-Level Graphics architecture doc.
Related
I wrote a video play view, it is a SurfaceView with a MediaPlayer instance. I attached the mediaplayer to the surface holder when surface created then start the play.
This is easy and everyone knows the details. But I want to draw a bitmap which is the first frame of the video to the surfaceview.Canvas is not a choice to draw the bitmap, because it will disable the mediaplayer to connect.
Since api level 14, we can new a surface with surfacetexture. So we can use opengl es to draw video frame and bitmap. But I am concerned about the performance.This way of playing video is more complicated and will it cause more overhead? Who can give me some advices?
You have a few options:
Use a FrameLayout to put a custom View (or maybe just an ImageView) on top of the SurfaceView. Draw your content there. When video playback starts, hide the View.
Connect GLES, draw the first frame, disconnect GLES, connect the MediaPlayer, play the movie. This is essentially what Grafika's PlayMovieSurfaceActivity does to clear the screen to black (see clearSurface()) before playing a movie.
As noted in your question, you can send the video to a SurfaceTexture, and then choose to render your content or render the image from the texture onto the SurfaceView.
#1 is the easiest. #3 adds complexity and is more expensive.
Working with a SurfaceView to make a 2d game i do not know who to put a background image in it efficiently.
I want to avoid drawing it each frame because it is an static image, any help?
The SurfaceView's Surface is a single layer. You can specify a dirty rect to reduce your draw area when you lock the Canvas, but you have to draw everything in the rect every frame -- background and all.
You could use a pair of SurfaceViews, one at the default Z-ordering, one at "media" depth. Use SurfaceView.setZOrderMediaOverlay() to do that. The layers will be composited by the system before being rendered.
If you were using OpenGL ES rather than Canvas, it'd generally be more efficient to just redraw the background from a texture each time. (If HWC runs out of overlays, that's essentially what SurfaceFlinger is doing anyway.)
See the "multi-surface test" in Grafika for an example of using multiple overlapping surfaces.
I am creating a layout of type FrameLayout, in which I am adding two views. Two views are objects of GLSurfaceView and SurfaceView respectively. According to Android Developers Documentation regarding SurfaceView,
"The surface is Z ordered so that it is behind the window holding its SurfaceView; the SurfaceView punches a hole in its window to allow its surface to be displayed."
It works well for me and SurfaceView always stays behind my GLSurfaceView (used for opneGL drawings). But resuming after external event the behavior is odd for a following configuration,
Android Version: 4.3
Device Model Number : Nexus 7
Kernel Version 3.4.0.g1f57c39
Jun 13
Build Number: JWR66N
For this configuration, resuming after external event puts my GLSurfaceView behind SurfaceView. In other words, SurfaceView is placed at top in ZOrder and my OpenGL drawings are no more visible. On versions greater that Android 4.3, this behavior is not seen.
I can replicate this behavior on all versions by calling SurfaceView's following method with true as a parameter.
void setZOrderOnTop
Is this known issue. Anybody can help me on this?
Regards,
Sumedh
SurfaceViews have two parts, the Surface and the View. The Surface is a completely independent layer. The View is there so the UI layout code has something to work with. Generally the View is just transparent black, so you can see through to whatever is behind it.
GLSurfaceView is just SurfaceView with some code to manage EGL contexts and threading. Underneath it's just a SurfaceView. So if you have both a SurfaceView and a GLSurfaceView, and they have the same dimensions and Z-order, then one of them is going to "win" and the other is going to "lose" because they're trying to occupy the same space at the same time. There is no defined value for which one will "win", so inconsistent behavior is expected.
One way to avoid clashes is to leave one set to the default Z, and call setZOrderMediaOverlay() on the other. The "media overlay" is still behind the UI, but above the default Surface position. If you use setZOrderOnTop(), the Surface will be positioned above the UI as well.
The upper Surface will need to be rendered with transparent pixels if you want to see something behind it (the same way that the View needs to be transparent to see the Surface).
The most efficient way to avoid this issue is to not have this issue: use one SurfaceView for everything, rendering all of your non-UI-element content to it. This requires a bit more work (and probably a SurfaceTexture) if you're rendering video or showing a camera preview on one of the Surfaces.
You can find some examples in Grafika. The "multi-surface exerciser" demonstrates three overlapping SurfaceViews rendered in software, overlapping with UI elements. Other activities show ways to work with Surfaces, GLES, the camera, and video.
See also the Android System-Level Graphics Architecture doc, which explains all this in much greater detail.
Dont use "setZOrderOnTop" as true. That will get it over all the other layouts.
If you are using multiple surfaceviews. use this for each surfaceview
yourSurfaceView.setZOrderMediaOverlay(true);
then set this setZOrderOnTop as false for the surfaceview you initiated later and wanted it to get back to the other surfaceviews
secondSurfaceview.setZOrderOnTop(false);
Am working in H264 video rendering in Android application using SurfaceView. It has one feature to take snapshot while rendering the video on surface view. Whenever I take a snapshot, I get the Transparent/Black screen only. I use getDrawingCache() method to capture the screen that returns a null value only. I use the below code to capture the screen.
SurfaceView mSUrfaceView = new SurfaceView(this); //Member variable
if(mSUrfaceView!=null)
mSUrfaceView.setDrawingCacheEnabled(true); // After video render on surfaceview i enable the drawing cache
Bitmap bm = mSUrfaceView.getDrawingCache(); // return null
Unless you're rendering H.264 video frames in software with Canvas onto a View, the drawing-cache approach won't work (see e.g. this answer).
You cannot read pixels from the Surface part of the SurfaceView. The basic problem is that a Surface is a queue of buffers with a producer-consumer interface, and your app is on the producer side. The consumer, usually the system compositor (SurfaceFlinger), is able to capture a screen shot because it's on the other end of the pipe.
To grab snapshots while rendering video you can render video frames to a SurfaceTexture, which provides both producer and consumer within your app process. You can then render the texture for display with GLES, optionally grabbing pixels with glReadPixels() for the snapshot.
The Grafika app demonstrates various pieces, though none of the activities specifically solves your problem. For example, "continuous capture" directs the camera preview to a SurfaceTexture and then renders it twice (once for display, once for video encoding), which is similar to what you want to do. The GLES utility classes include a saveFrame() function that shows how to use glReadPixels() to create a bitmap.
See also the Android System-Level Graphics Architecture document.
I have a video view. This view is contained inside a custom FrameLayout called VideoStructure, where I can put also a channel logo or things alike.
Under normal conditions, the video is hardware accelerated, so the view is (i suppose) really a transparent "black hole", while the video is decoded & rendered by the relevant hardware.
My question is, if I override draw() in the Video View's container (the VideoStructure extends FrameLayout in the image) to draw some stuff (ie. the circle in the image) OVER the video -I'm overriding draw(), not onDraw()- will this break the hardware acceleration? Can I expect a big performance hit for doing this?
It should have no effect on performance.
SurfaceViews have two parts, the "view" part, and the "surface" part. The "view" part is a transparent hole that fits in with the other views, the "surface" part is a completely independent layer that is composited with the view layer by the system. The video is being sent to the "surface" part.
If you override SurfaceView's "view" renderer, you'll get a hardware-accelerated Canvas for a View that is normally completely transparent (so if you erase it, you better use an alpha of zero and the correct transfer mode).
If you attempt to render on the "surface" part, by getting a Canvas from lockCanvas(), you will either fail (because the video effectively has it locked), or succeed and prevent video from being written to it.
The system compositor is going to have to blend the "view" and "surface" layers no matter what appears in the "view" layer, so making a few more pixels opaque isn't going to have a measurable impact.
Update: see the graphics architecture doc for more details on Surfaces and composition.