I'm trying to do camera recording and drawings on top on Google Glass, using a LiveCard.
in a regular activity, this would be achieved by using a FrameLayout, with a SurfaceView for the camera preview 'in the back', and another View in front of it used for drawing.
but using a LiveCard, if one needs subsecond updates, one has to use the LiveCard itself as a Surface. according to the documentation: https://developers.google.com/glass/develop/gdk/reference/com/google/android/glass/timeline/LiveCard
If your application requires more frequent updates (several times per
second) or rendering more elaborate graphics than the standard widgets
support, enable direct rendering and add a SurfaceHolder.Callback to
the card's surface.
LiveCard liveCard; // initialized elsewhere
liveCard.setDirectRenderingEnabled(true);
liveCard.getSurfaceHolder().addCallback(callback); You can then draw directly on the surface inside a background thread or in response
to external events (for example, sensor or location updates). Use the
surfaceCreated and surfaceDestroyed methods to start and stop your
rendering logic when the card is displayed or hidden.
now I can either draw my on stuff on this Surface, or I can give this to the MediaRecorder as the camera preview service, but can't do both, as it will fail with an error
I wonder if anyone has ideas on how to make this work still?
the way I'd draw into the LiveCard myself is to 'manually' lock the canvas, and call FrameLayout.draw(canvas); one option would be to have a layout that contains two SurfaceViews - one for the camera preview, and one for my own drawings, and use the same approach. but, even if I define such a layout in XML, I can't get the SurfaceViews created (e.g. the appropriate SurfaceView callbacks are never called, and any attempt of drawing on them results in failure)
Related
I'm working on a video app where user can watch a video, open it il fullscreen if needed and come back to default view and so on. I was using ExoPlayer and recently switch to default MediaPlayer due to the upcoming explanation.
I need to change "on the fly" the Surface of the player. I need to use the same player to display video among activities, with no delay to display the image. Using Exoplayer, the decoder wait for the next keyframe to draw pixels on the empty Surface.
So I need to use the same Surface so I don't need to push a new surface each time, just attachign the surface to a View parent. The Surface can stay the same but if I detach the SurfaceView to retrieve it from another activity and reattach it, the inner Surface is destroyed.
So is there a way to keep the same Surface across different activities ? With a Service ?
I know the question is a bit weird to understand, I will explain specified part is request in comment.
The Surface associated with a SurfaceView or TextureView will generally be destroyed when the Activity stops. It is possible to work around this behavior.
One approach is built into TextureView, and is described in the architecture doc, and demonstrated in the "double decode" activity in Grafika. The goal of the activity is to continue playing a pair of videos while the activity restarts due to screen rotation, not pausing at all. If you follow the code you can see how the return value from onSurfaceTextureDestroyed() is used to keep the SurfaceTexture alive, and how TextureView#setSurfaceTexture() attaches the SurfaceTexture to the new View. There's a bit of a trick to it -- the setSurfaceTexture() needs to happen in onCreate(), not onSurfaceTextureAvailable() -- but it's reasonably straightforward.
The example uses MediaCodec output for video playback, but it'll work equally well with anything that takes a Surface for output -- just create a Surface from the SurfaceTexture.
If you don't mind getting ankle-deep into OpenGL ES, you can just create your own SurfaceTexture, independent of Views and Activities, and render it yourself to the current SurfaceView. Grafika's "texture from camera" activity does this with live video from the camera (though it doesn't try to preserve it across Activity restarts).
I am creating a layout of type FrameLayout, in which I am adding two views. Two views are objects of GLSurfaceView and SurfaceView respectively. According to Android Developers Documentation regarding SurfaceView,
"The surface is Z ordered so that it is behind the window holding its SurfaceView; the SurfaceView punches a hole in its window to allow its surface to be displayed."
It works well for me and SurfaceView always stays behind my GLSurfaceView (used for opneGL drawings). But resuming after external event the behavior is odd for a following configuration,
Android Version: 4.3
Device Model Number : Nexus 7
Kernel Version 3.4.0.g1f57c39
Jun 13
Build Number: JWR66N
For this configuration, resuming after external event puts my GLSurfaceView behind SurfaceView. In other words, SurfaceView is placed at top in ZOrder and my OpenGL drawings are no more visible. On versions greater that Android 4.3, this behavior is not seen.
I can replicate this behavior on all versions by calling SurfaceView's following method with true as a parameter.
void setZOrderOnTop
Is this known issue. Anybody can help me on this?
Regards,
Sumedh
SurfaceViews have two parts, the Surface and the View. The Surface is a completely independent layer. The View is there so the UI layout code has something to work with. Generally the View is just transparent black, so you can see through to whatever is behind it.
GLSurfaceView is just SurfaceView with some code to manage EGL contexts and threading. Underneath it's just a SurfaceView. So if you have both a SurfaceView and a GLSurfaceView, and they have the same dimensions and Z-order, then one of them is going to "win" and the other is going to "lose" because they're trying to occupy the same space at the same time. There is no defined value for which one will "win", so inconsistent behavior is expected.
One way to avoid clashes is to leave one set to the default Z, and call setZOrderMediaOverlay() on the other. The "media overlay" is still behind the UI, but above the default Surface position. If you use setZOrderOnTop(), the Surface will be positioned above the UI as well.
The upper Surface will need to be rendered with transparent pixels if you want to see something behind it (the same way that the View needs to be transparent to see the Surface).
The most efficient way to avoid this issue is to not have this issue: use one SurfaceView for everything, rendering all of your non-UI-element content to it. This requires a bit more work (and probably a SurfaceTexture) if you're rendering video or showing a camera preview on one of the Surfaces.
You can find some examples in Grafika. The "multi-surface exerciser" demonstrates three overlapping SurfaceViews rendered in software, overlapping with UI elements. Other activities show ways to work with Surfaces, GLES, the camera, and video.
See also the Android System-Level Graphics Architecture doc, which explains all this in much greater detail.
Dont use "setZOrderOnTop" as true. That will get it over all the other layouts.
If you are using multiple surfaceviews. use this for each surfaceview
yourSurfaceView.setZOrderMediaOverlay(true);
then set this setZOrderOnTop as false for the surfaceview you initiated later and wanted it to get back to the other surfaceviews
secondSurfaceview.setZOrderOnTop(false);
I have a special design requiring for the app I'm developing right now.
Right now, I have a third-party private video library which plays a video stream. The design of this screen includes a translucent panel overlaid on top of the video, blurring the portion of the video that lies behind.
Normally in order to blur the background, you are supposed to take a screenshot of the view behind, blur it and use it as an image for the foreground view.
In this case, the video keeps on playing, so the blurred image changes every frame. How would you implement this then?
A possible solution would be to create a thread, taking screenshots, cropping them and put them as a background. Even better if that view is a SurfaceView, I guess. But I'm wondering what would be the best approach in this case. Would a thread that is continually taking screenshots create a huge performance impact? Is it possible to feed a surfaceView buffer with these images?
Thanks!
A SurfaceView surface is a consumer of graphics buffers. You can't have two producers for one consumer, which means you can't send the video to it and draw on it at the same time.
You can have multiple layers; the SurfaceView surface is on a separate layer behind the View UI layer. So you could play the video to the SurfaceView's surface, and draw your blur rectangle on the SurfaceView's view. (Normally the SurfaceView's view is completely transparent, and is just used as a place-holder for layout purposes.)
Another option would be to render the video frame to a SurfaceTexture. You would then render that texture to the SurfaceView surface with GLES, and render the blur rectangle on top. You can find an example of treating live camera input as a GLES texture in Grafika ("texture from camera" activity). This has the additional advantage that, since you're not interacting with the View system -- the SurfaceView surface is composited by the system, not the app -- you can do it all on an independent thread.
In any event, rendering, grabbing a screenshot, and re-rendering is going to be slower than the options described above.
For more details about why things work the way they do, see the Android System-Level Graphics architecture doc.
I have 2 Activitiys which use OpenGL for drawing. At a transition from one activity to the next I get an unsightly empty screen filled with my OpenGL clear colour (so its not as bad as a black screen).
I wish to effectively transition seamlessly between Activitys, but there are several high load regions when a GLSurfaceView is created. The main issue is texture loading as this is slowest.
Is there anyway to double buffer between Activitys so that the last Activity view is frozen until I explicitly tell my next Activity to draw? I want transitions to be seamless?
Moving everything into one GLSurfaceView instance isn't really an option I want to consider.
You can use setRenderMode( RENDERMODE_WHEN_DIRTY) in your GLSurfaceView, so the surface only will be redraw when you call requestRender().
This way, anything that you draw before calling another surface view will only be cleared when you request a new draw.
You can back to the continuous drawing by setting render mode as RENDERMODE_CONTINUOUSLY.
It is hard to do it in Android 2.x because of its OpenGL ES. Also, it is not recommended that you use two OpenGL in one applications if you are in render continously. If so, to control them easily, you will need RENDERMODE_WHEN_DIRTY.
If you use it in Android 4.x, TextureView is an optional to do it.
TextureView is as same as GLSurfaceView but with View compatible, it means that you can use ViewAnimation for TextureView.
On some devices, onPreviewFrame is not called if no SurfaceView was set to display the camera preview.
However, I handle the camera in a service, so I can't set a SurfaceView but I don't want to have visible preview anyway.
How can this be done? Can I programmatically create a SurfaceView and set it with Camera::setPreviewDisplay?
This must be possible or not?
It works on almost every phone without a SurfaceView but not on HTC One X and Google Nexus One...
According to this question, creating a SurfaceView in code works fine. Though I don't think you can create it through a service.
Another approach is to create a 1px-1px SurfaceView inside a RelativeLayout and hiding it with some other view on top of it. (visibility should still be VISIBLE). We use this trick for Path camera UI where we render preview buffers through OpenGL and it works fine.
According to documentation readily configured, visible and displayed surface view is necessary to activate camera preview. It may be overlayued though
From API 11 on, you can use a SurfaceTexture instead of a SurfaceView to get frames from the camera. Then, instead of using Camera.setPreviewDisplay, simply use Camera.setPreviewTexture.
This answer as well as this one discuss this point.