i know that TextureView is show up after ICS.
but, SurfaceView is not deprecated at ICS.
SurfaceView has hole-punching structure, so it has many limit point.
can't stack two SurfaceView and can't translate and etc..
why SurfaceView is not deprecated despite of TextureView is show up?
SurfaceView is faster, and can handle DRM-protected video.
The hole-punching structure is necessary because SurfaceView's Surface is handled directly by the system compositor. For TextureView, you draw on a Surface, which is converted to a GL texture within the app, which is rendered a second time by the app onto the View layer. So there's an extra copy.
For DRM-protected video, no user or system code -- not even the Linux kernel -- is allowed to see unencrypted pixels. Only the video decoder and the display hardware. Because SurfaceView just forwards references through, and doesn't touch the actual data, this works.
For more details, see the graphics architecture doc.
Related
To play a media using Android MediaPlayer or MediaCodec, most of the time, you use SurfaceView or GLSurfaceView (There is another way to achieve this using TextureView, but let's not talk about it here, since it's a bit different type of view)
And as far as I know, capturing the video frame from SurfaceView is not possible - you don't have access to hw overlay.
How about GLSurfaceView? Since we have access to YUV pixels (we're, right?), is it possible?
Can anyone point me where i can find a sample code to do it?
I don't think below explanation can work, because it's assuming the color format is RGBA, and in above case, I think it's YUV.
When using GLES20.glReadPixels on android, the data returned by it is not exactly the same with the living preview
Thank you and have a great day.
You are correct in that you cannot read back from a Surface. It's the producer side of a producer-consumer pair. GLSurfaceView is just a bunch of code wrapped around a SurfaceView that (in theory) makes working with GLES easier.
So you have to send the preview somewhere else. One approach is to send it to a SurfaceTexture, which converts every frame sent to its Surface into a GLES texture. The texture can then be rendered twice, once for display and once to an offscreen pbuffer that can be saved as a bitmap (just like this question).
I'm not sure why you don't want to talk about TextureView. It's a View that uses SurfaceTexture under the hood, and it provides a getBitmap() call that does exactly what you want.
As we know, we can choose TextureView, SurfaceView and GLSurfaceView for android camera preview.
Which one is best choice for camera preview ? I'm focused on the camera performance.
From a performance perspective, SurfaceView is the winner.
With SurfaceView, frames come from the camera and are forwarded to the system graphics compositor (SurfaceFlinger) with no copying. In most cases, any scaling will be done by the display processor rather than the GPU, which means that instead of scanning the pixels once for scaling and again for scan-out, they're only scanned once.
GLSurfaceView is a SurfaceView with some wrapper classes that handle EGL setup and thread management. You can't use OpenGL ES on a Surface that is receiving camera frames, so you're doing extra work with no benefit. (The overhead is minor one-time setup, not per-frame, so you likely won't be able to measure the difference.)
TextureView receives the frames in a SurfaceTexture as an "external" OpenGL ES texture, then uses GLES to render them onto the app's UI surface. The scaling and rendering are performed by the GPU, and the result is then forwarded to SurfaceFlinger. This is the slowest option, but also the most flexible of the Views.
If you'd like to learn more about how the system works, see the Android Graphics Architecture document.
" The SurfaceView creates a new window in the Android Windowsystem. Its advantage is, that if the SurfaceView gets refreshed, only this window will be refreshed. If you additionally update UI Elements (which are in another window of the windowsystem), then both refresh operations block themselfes (especially when ui drawing is hardwaresupported) because opengl cannot handle multi thread drawing properly.
For such a case it could be better using the TextureView, cause it's not another window of the Android Windowsystem. so if you refresh your View, all UI elements get refreshed as well. (Probably) everything in one Thread.
Hope I could help some of you! "
Source : stackoverflow.com
GLSurfaceView is a SurfaceView with a wrapper class that does all the EGL setup and inter-thread messaging for you.
Its completely upto you what you put to use.. They have their pros and cons over eachother :)
I'm using libvlc on and android app to play a network stream; the native code draws the image on a surfaceview.
Let's assume the video stops and I have no access to the native code, how can I detect if the SurfaceView is still moving (the video is not frozen)?
I tried getViewTreeObserver().addOnDrawListener(); and getViewTreeObserver().addOnPreDrawListener(); but they do not have the effect I'm looking for.
Thanks
You can't get that information from the SurfaceView, because the SurfaceView itself does not know.
The SurfaceView's job is to set up the Surface, and create a hole in the View layout that you can see through. Once it has done these things, it is no longer actively involved in the process of displaying video. The content flows from the decoder to the Surface, which is managed by SurfaceFlinger (the system graphics compositor).
This is, for example, how DRM video works. The app "playing" the video has no access to DRM-protected video frames. (Neither does SurfaceFlinger, actually, but that's a longer story.)
The best way to know if content is still arriving is to ask the video source if it is still sending you content. Another approach would be to change your SurfaceView to a TextureView, and provide an onSurfaceTextureUpdated() callback method.
I am not sure what exactly what you are trying to achieve here but you can see if surface view is rendering or not through implementing an interface called SurfaceHolder.Callback which gives you access to the following methods,
On Surface Created - This is called immediately after the surface is first created.
On Surface Changed - This is called immediately after any structural changes (format or size) have been made to the surface.
On Surface Destroyed - This is called immediately before a surface is being destroyed.
Take a look at the documentation for surface view. For SurfaceHolder take a look at this link. Basically in order to know
I am trying to decode video samples using MediaCodec API. I am using surfaceView to show rendered samples. If i press home button, app going into pause state and surface destroyed. When i coming back to resume state, new surfaceView reference is creating, but decoder is unable to pump samples on surfaceView. so screen appearing as black.
video configure:
videoDecoder.configure(format, surface, null, 0);
So how can i reconfigure videoDecoder in above statement. It is similar to the following problem
How to keep decoding alive during screen orientation?
The MediaCodec API does not currently (API 19) provide a way to replace the output Surface.
As in the other question you refer to, I think the way to deal with this will be to decode to a Surface that isn't tied to the view hierarchy (and, hence, doesn't get torn down when the Activity is destroyed).
If you direct the output of the MediaCodec to a SurfaceTexture, you can then render that texture onto the SurfaceView. This will require a bit of GLES code. You can find the necessary pieces in the Grafika sources, but there isn't currently a full implementation of what you want (e.g. PlayMovieActivity decodes video to a SurfaceTexture, but that ST is part of a TextureView, which will get torn down).
The additional rendering step will increase the GPU load, and won't work for DRM-protected video. For most devices and apps this won't matter.
See also the bigflake examples.
Update: I've added this to Grafika, with a twist. See the "Double decode" example. The output goes to a SurfaceTexture associated with a TextureView. If the screen is rotated (or, currently, blanked by hitting the power button), decoding continues. If you leave the activity with the "back" or "home" button, decoding stops. It works by retaining the SurfaceTexture, attaching it to the new TextureView.
Ideally I'd like to accomplish two goals:
Pass the Camera preview data to a MediaCodec encoder via a Surface. I can create the Surface using MediaCodec.createInputSurface() but the Camera.setPreviewDisplay() takes a SurfaceHolder, not a Surface.
In addition to passing the Camera preview data to the encoder, I'd also like to display the preview on-screen (so the user can actually see what they are encoding). If the encoder wasn't involved then I'd use a SurfaceView, but that doesn't appear to work in this scenario since SurfaceView creates its own Surface and I think I need to use the one created by MediaCodec.
I've searched online quite a bit for a solution and haven't found one. Some examples on bigflake.com seem like a step in the right direction but they take an approach that adds a bunch of EGL/SurfaceTexture overhead that I'd like to avoid. I'm hoping there is a simpler example or solution where I can get the Camera and MediaCodec talking more directly without involving EGL or textures.
As of Android 4.3 (API 18), the bigflake CameraToMpegTest approach is the correct way.
The EGL/SurfaceTexture overhead is currently unavoidable, especially for what you want to do in goal #2. The idea is:
Configure the Camera to send the output to a SurfaceTexture. This makes the Camera output available to GLES as an "external texture".
Render the SurfaceTexture to the Surface returned by MediaCodec#createInputSurface(). That feeds the video encoder.
Render the SurfaceTexture a second time, to a GLSurfaceView. That puts it on the display for real-time preview.
The only data copying that happens is performed by the GLES driver, so you're doing hardware-accelerated blits, which will be fast.
The only tricky bit is you want the external texture to be available to two different EGL contexts (one for the MediaCodec, one for the GLSurfaceView). You can see an example of creating a shared context in the "Android Breakout game recorder patch" sample on bigflake -- it renders the game twice, once to the screen, once to a MediaCodec encoder.
Update: This is implemented in Grafika ("Show + capture camera").
Update: The multi-context approach in "show + capture camera" approach is somewhat flawed. The "continuous capture" Activity uses a plain SurfaceView, and is able to do both screen rendering and video recording with a single EGL context. This is recommended.