I tried to apply some 3D transformation (such as setRotationX) on a surfaceview which is used for camera previewing, but only the frame changes and the content don't.
A SurfaceView has two parts, the Surface and the View. The Surface is a separate layer that is rendered and composited independently. The View part is, by default, a transparent rectangle that creates a "hole" in the View layer, so that you can see through the Views to the Surface behind it.
The transformation you mention (setRotationX()) is a View method, but the camera preview is sent to the Surface. That's why the frame changed but the preview itself didn't.
You can send your preview to a TextureView, which can take an arbitrary transformation matrix (setTransform()), by using the Camera.setPreviewTexture() method. Or you can send it through a SurfaceTexture to an OpenGL ES texture, which can be rendered on the SurfaceView's Surface, using whatever GLES transformations you want. For an example of the latter, see Grafika's "texture from Camera" Activity.
Related
Most of the code for generating videos using MediaCodec I've seen so far either use pure OpenGL or locking the Canvas from the MediaCodec-generated Surface and editing it. Can I do it with a mix of both?
For example, if I generate my frames in the latter way, is it possible to apply a Fragment Shader on the MediaCodec-generated Surface before or after editing the Surface's Canvas?
A Surface is the producer end of a producer-consumer pair. Only one producer can be connected at a time, so you can't use GLES and Canvas on the same Surface without disconnecting one and attaching the other.
Last I checked (Lollipop) there was no way to disconnect a Canvas. So switching back and forth is not possible.
What you would need to do is:
Create a Canvas backed by a Bitmap.
Render onto that Canvas.
Upload the rendered Bitmap to GLES with glTexImage2D().
Blit the bitmap with GLES, using your desired fragment shader.
The overhead associated with the upload is unavoidable, but remember that you can draw the Bitmap at a smaller resolution and let GLES scale it up. Because you're drawing on a Bitmap rather than a Surface, it's not necessary to redraw the entire screen for every update, so there is some opportunity to reduce Canvas rendering overhead.
All of the above holds regardless of what the Surface is connected to -- could be MediaCodec, SurfaceView, SurfaceTexture, etc.
I have two surfaces views in a frame layout, which also contains a linear layout with some buttons. One of the buttons should be able to capture and save an image of the two surfaceviews. One surfaceview is a camera preview and the other is an opengl surface with a square in it. How would you go about taking the picture and saving it?
You can't read data back from a SurfaceView Surface. See e.g. this answer.
The way that you "capture" it is by rendering it to something you can read the pixels from. In your case, you'd grab a frame from the camera, render that to an offscreen pbuffer, then render the square with OpenGL ES onto the same pbuffer, and then grab that with glReadPixels(). Essentially you perform the Surface composition yourself.
I want to create a mask on camera surface view. see image below. mask is resizable. image will be clicked only by unblurred area. can anybody give idea how to create suh mask? thanks in advance.
You can't draw on the Surface of the SurfaceView that is receiving the camera preview.
You have two basic options: draw on the View part of the SurfaceView, treating it as a custom view, or create a second SurfaceView and layer it on top of the camera surface.
For the latter, you would use setZOrderMediaOverlay() to position the Surface above the camera Surface layer but below the View UI layer. You can use Canvas or GLES to draw on it. You can find an example of an activity with three SurfaceViews in Grafika's "multi-surface test".
You can draw on a surfaceview with camera preview. You need to set the
setWillNotDraw(false);
in surfaceCreated override.
See: Extended SurfaceView's onDraw() method never called
So from there on, do all your drawing in onDraw as normal.
I'm not certain how to implement the object made from OpenGl into the surfaceView of the Camera. Is it possible to set the OpenGl objectinto a custom android camera that uses surfaceView. ?
No.
The SurfaceView's Surface has a producer-consumer relationship. In this case, the producer is the Camera, and the consumer is the system compositor (surfaceflinger). To draw on the Surface with OpenGL ES, you would need to attach OpenGL as the producer, and a Surface cannot have two producers at the same time.
What you can do instead is create a second SurfaceView, and call setZOrderMediaOverlay() to put it above the Camera-driven Surface but below the UI, then render on that with GLES. See Grafika's "multi-surface test" for a demonstration of overlapping SurfaceViews.
I'm learning from the Documentation of Suface, Suface Flinger, SurfaceView, SurfaceHolder and Bitmap. I have found following definitions:
Surface
Surface is a collection of pixels. A Surface is an object holding pixels that are being composited to the screen. Every window you see on the screen (a dialog, your full-screen activity, the status bar) has its own surface that it draws in to, and Surface Flinger renders these to the final display in their correct Z-order. A surface typically has more than one buffer (usually two) to do double-buffered rendering: the application can be drawing its next UI state while the surface flinger is compositing the screen using the last buffer, without needing to wait for the application to finish drawing.
SurfaceView
This class is used to present a live camera preview to the user.
A SurfaceView is a special implementation of View that also creates its own dedicated Surface for the application to directly draw into (outside of the normal view hierarchy, which otherwise must share the single Surface for the window)
Bitmap
A Bitmap is a wrapper for collection of pixels,it is simply an array of pixels with some other convenient functions.
Canvas
Canvas is a class that contains all the drawing methods. It is similar to Graphics class in AWT/Swing in Java. It has all the logic on how to draw, a box etc. A Canvas operates on a Bitmap.
But I'm not clear about SurfaceHolder and SurfaceFlinger and according to above definition bitmap is also container of pixel and Surface too.
Can you help to give me clear understanding the definition of all these objects?
A Canvas is a basic context for doing drawing with the graphics API. You can create your own Canvas wrapping a Bitmap for doing off-screen drawing, and of course the UI framework will pass Canvases to widgets for them to draw themselves into. All these widgets are subclasses of View. Or, if they can contain other widgets, then they subclass from ViewGroup (which is a subclass of View).
If you want to do high-frame-rate animations, then you need to subclass from SurfaceView (or its subclass GLSurfaceView, if you want to do on-screen OpenGL-ES rendering). Each SurfaceView has a SurfaceHolder, which has lockCanvas methods that you can call at any time to get a Canvas into which to draw an updated display and post back for the user to see immediately via unlockCanvasAndPost.
For more info, see 2D graphics concepts and Custom widget tips