We've been fighting with some problems relating to SurfaceViews for more than a week, and find no proper solution to them. We read the other questions in the forum regarding to similar problems (and even Mixare source code) but couldn't find an answer, so we hope you could help us somehow.
Scenario:
We've got
a SurfaceView for the Camera
a SurfaceView for an OpenGL layer, which goes on top of the camera.
another View, which shows some information about what we can see on the screen. This one goes on top of both SurfaceViews.
Problem:
No matter how hard we try, both SurfaceViews apparently don't get on well with each other. If we try to:
setContentView(mCameraPreview);
addContentView(mGLSurfaceView, new LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT));
addContentView(mInfoView, new LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT));
(which seems logical), everything goes as expected until we lock/unlock the phone. After that, the GLSurfaceView just disappears (not the InfoView, that one is still shown).
If, instead, we try to:
setContentView(mGLSurfaceView);
addContentView(mCameraPreview, new LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT));
addContentView(mInfoView, new LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT));
Then the problem is that the GLSurfaceView only appears after lock/unlock, and before that the screen is showing the camera and the InfoView.
We discovered that if we sleep the main thread for 4.6 seconds (or more) after executing onStart() in the activity that shows the views, the behaviour is as expected (both camera, glsurface and info views are shown, even after lock/unlock).
The thing is, we're looking for a more... elegant solution.
It seems to us that the problem is the camera taking more time than expected in Camera.open(), and so the camera View is added, the GLSurfaceView is added, and when the camera actually opens, it opens on top of the GLSurfaceView. Regarding to this, we used bringToFront() on GLSurfaceView, and got it on top of the info view, but after lock/unlock the camera still opened on top of it, leaving us with a screen with the camera preview only.
Any ideas? How can we show both SurfaceViews and the info view on top of them?
try this:
mLayout.addView(mRenderView);
mLayout.addView(mCustomSurfaceView);
// without this line, the camera preview cannot be displayed
// when running activity at first time.
mCustomSurfaceView.setZOrderMediaOverlay(true);
this worked for me :)
I had the same problem. As you hint at yourself: multiple SurfaceViews don't get along with each other in that their Z order is undefined.
On my Samsung Galaxy S2 the ordering is the same as you describe (don't know how it is on other phones).
The way I solved this, is checking for first time creation of the Activity in onCreate():
#Override
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
//...
if ( savedInstanceState == null )
{
initCamView();
initOpenGL();
}
else
{
initOpenGL();
initCamView();
}
//...
}
with:
private void initOpenGL()
{
mGLSurfaceView = new GLSurfaceView(this);
mGLSurfaceView.setEGLConfigChooser(8, 8, 8, 8, 16, 0);
mGLSurfaceView.getHolder().setFormat(PixelFormat.TRANSLUCENT);
mOGLRenderer = new OGLRenderer(this);
mGLSurfaceView.setRenderer(mOGLRenderer);
mRL.addView(mGLSurfaceView); // mRL is the Relative Layout
}
and:
private void initCamView()
{
mCamView = new CustomCameraView( this.getApplicationContext(),
this.getWindowManager() );
mRL.addView(mCamView); // mRL is the Relative Layout
}
Not the most elegant solution, but it's better than letting the thread sleep for 4.6 seconds.
It might also be possible to just lock the screen in a single orientation, but then you need to do a lot of ugly hacking to get the overlays rotated the right way.
Or, if you're only targeting Android 3.0 or above (API level 8), you can just show the camera in an OpenGL SurfaceTexture. http://developer.android.com/reference/android/hardware/Camera.html#setPreviewTexture(android.graphics.SurfaceTexture)
Each SurfaceView's surface has a Z-depth. There are three possible depths: the default (bottom), "media overlay" (above it), and "top" (which is above everything, including the View-based UI). If you have two overlapping surfaces at the same depth, one will win, but you can't reliably define which one. You might get consistent behavior on one device only to find that it works the other way on a different device.
The composition is done with hardware overlays, when possible. Many currently-popular devices have 4 overlay planes. Once you exceed that, the composition is done with the GPU, which is going to be more expensive. If you have a status bar, navigation bar, and View UI, that's three, so you only get one SurfaceView for cheap.
For the example in the question, it would be best to combine as many things as possible onto the surface being rendered with GLES. This can include camera output on API 11+. See the "Texture from Camera" activity in Grafika for an example. (Grafika also has a demo app with three overlapping SurfaceViews; see the "multi-surface test" activity.)
If you'd like to know more about why SurfaceView behaves the way it does, see the Android System-Level Graphics doc.
I had the same problem: I wanted a SurfaceView under a GLSurfaceView, the first to play video and the second to run a game.
Whatever I did, the z order between these 2 surface views appears to be random.
But I found the solution, thanks to this post: https://stackoverflow.com/a/6028907/1557915
The trick is to remove the use of layout, but instead use setContentView/addContentView.
Related
I have encountered the nasty Android SurfaceView behavior:
When you leave an Activity with SurfaceView and quickly go back, the previous content of SurfaceView doesn't disappear and is displayed under the new one.
See the example project that reproduces the problem, please.
https://github.com/t-artikov/surface-view-bug
I have found that this is an Android bug, which was fixed on Android P
https://issuetracker.google.com/issues/72624889
But I need to get it to work on earlier OS versions.
Will be glad to any ideas how to solve the issue. Maybe someone has already encountered it and knows a workaround.
Note: This problem wouldn't occur on an API 22 device. I used the Pixel 2 API 26, as the OP did, in order to reproduce it.
It appears that, under this condition, the SurfaceView saves its current contents to some sort of hidden intermediate surface, at certain times (like when a transition begins). The condition can't be corrected by replacing the SurfaceView, or by adjusting the background drawables of any ancestor views.
Your onDrawFrame() is clearing the buffer with a transparent color, which is letting this "intermediate surface" show through. If you don't need the transparency, you could just clear it to solid white:
GLES20.glClearColor(1, 1, 1, 1);
EDIT
I found myself coming back to this question, as it is a very troublesome bug. Since you need to keep the transparency, it turns out you can add this to SecondActivity, to avoid the problem:
#Override
protected void onStart()
{
overridePendingTransition(0,0);
super.onStart();
}
I also tried disabling hardware acceleration, but no luck there.
The project is written in as3, packaged using AIR 28.0, and run on an Android 7 device (when run on a windows desktop environment it works fine).
Objects that are hidden from view by a scrollRect are still receiving mouse events on my Android 7 (S7), but not when I run the same exact application in a desktop environment.
Is this an Android or AIR bug? Or am I doing something wrong?
Example:
EDIT: (Original code example was incomplete see edit below)
var button:Sprite = new Sprite();
button.graphics.beginFill(0x00FF11, 1);
button.graphics.drawRect(0, 0, 50, 50);
button.graphics.endFill();
addChild(button);
button.addEventListener(MouseEvent.CLICK, buttonClick);
var outer:Sprite = new Sprite();
var square:Sprite = new Sprite();
outer.cacheAsBitmap = true;
outer.addChild(square);
var rect:Rectangle = new Rectangle(0, 0, 50, 50);
outer.scrollRect = rect;
addChild(outer);
outer.y = 50;
square.graphics.lineStyle();
square.graphics.beginFill(0x000000, 1);
square.graphics.drawRect(0, 0, 100, 100);
square.graphics.endFill();
square.addEventListener(MouseEvent.MOUSE_DOWN, squareDown);
square.y = -50;
In this example the rectangle drawn in the inner object is only visible within the outer.scrollRect (so 50x50 instead of 100x100) and the button object is visible above it. When a click event happens over the visible button however the innerClick event will fire, not the buttonClick event.
It seems that the invisible portion of the inner container, hidden by the applied scrollRect, is only hidden visually and still blocks input events.
The application is being compiled and installed through my IDE (Flash Builder). I have tried different versions of my IDE and AIR SDK, with the same results.
Any help would be greatly appreciated.
EDIT:
I found instances where scrollRect worked properly so did further testing to narrow down the bug.
Removing the line:
outer.cacheAsBitmap = true;
... fixed the issue. Applying cacheAsBitmap to the square object itself also still worked fine.
For some reason applying cacheAsBitmap to the object with the scrollRect causes this bug, but applying cacheAsBitmap to objects that are children further down the line works as it should.
I'm not experienced enough to know the performance implications or why it is suggested to add cacheAsBitmap to the object with the scrollRect. Maybe someone else can inform.
The scrollRect property is probably the most performance-wise way (with the regular Flash content) to clip some content with a rectangular mask.
The cacheAsBitmap property allows to relieve Flash Player of need to render the content of the object unless something changes inside the object.
To put it simply. Imagine a page of richly illustrated fairy-tale book: an overcomplicated vector picture with lots of details, some texts with fancy fonts, etc. You put it on stage, Flash Player needs to render all of it: vector fills and strokes, fancy fonts. Then, you don't change anything over the course of several frames, Flash Player does not need to render anything, it is good. Then you shift this page 1px to the right and - guess what - Flash Player needs to render the whole thing again.
Any change under that page, or over that page, or to that page - anything withing that page's bounding box - will make FP to render the page's whole content again - which would take a pretty heavy toll on performance.
Using cacheAsBitmap allows FP to render it only once, so only internal changes to that page will make FP to render it again, anything else will make FP just use the cached version which is faster than rendering it over and over again. Which is, I agree, might be a solution to mobile devices for they are slow and CPU-lacking.
Hence if your app has elements that:
contain complicated vector shapes and text
are static or change quite rarely
are not overly large
then you can cache them with cacheAsBitmap = true to gain some performance at the expense of RAM.
However.
If you are building a store-grade product, not just a school project or something for personal amusement, I advise you to look into Gamua Starling because you'll never get your app any close to running smoothly with regular Flash content. As soon as you need to redraw the whole screen the FPS will drop to the bottom of the sea, because Flash is CPU-hungry while mobile devices are CPU-lacking. Starling, on the other hand, utilizes Stage3D and device's GPU resources so you can have something overly rich and animated running at 60 FPS with no sweat.
I am creating a layout of type FrameLayout, in which I am adding two views. Two views are objects of GLSurfaceView and SurfaceView respectively. According to Android Developers Documentation regarding SurfaceView,
"The surface is Z ordered so that it is behind the window holding its SurfaceView; the SurfaceView punches a hole in its window to allow its surface to be displayed."
It works well for me and SurfaceView always stays behind my GLSurfaceView (used for opneGL drawings). But resuming after external event the behavior is odd for a following configuration,
Android Version: 4.3
Device Model Number : Nexus 7
Kernel Version 3.4.0.g1f57c39
Jun 13
Build Number: JWR66N
For this configuration, resuming after external event puts my GLSurfaceView behind SurfaceView. In other words, SurfaceView is placed at top in ZOrder and my OpenGL drawings are no more visible. On versions greater that Android 4.3, this behavior is not seen.
I can replicate this behavior on all versions by calling SurfaceView's following method with true as a parameter.
void setZOrderOnTop
Is this known issue. Anybody can help me on this?
Regards,
Sumedh
SurfaceViews have two parts, the Surface and the View. The Surface is a completely independent layer. The View is there so the UI layout code has something to work with. Generally the View is just transparent black, so you can see through to whatever is behind it.
GLSurfaceView is just SurfaceView with some code to manage EGL contexts and threading. Underneath it's just a SurfaceView. So if you have both a SurfaceView and a GLSurfaceView, and they have the same dimensions and Z-order, then one of them is going to "win" and the other is going to "lose" because they're trying to occupy the same space at the same time. There is no defined value for which one will "win", so inconsistent behavior is expected.
One way to avoid clashes is to leave one set to the default Z, and call setZOrderMediaOverlay() on the other. The "media overlay" is still behind the UI, but above the default Surface position. If you use setZOrderOnTop(), the Surface will be positioned above the UI as well.
The upper Surface will need to be rendered with transparent pixels if you want to see something behind it (the same way that the View needs to be transparent to see the Surface).
The most efficient way to avoid this issue is to not have this issue: use one SurfaceView for everything, rendering all of your non-UI-element content to it. This requires a bit more work (and probably a SurfaceTexture) if you're rendering video or showing a camera preview on one of the Surfaces.
You can find some examples in Grafika. The "multi-surface exerciser" demonstrates three overlapping SurfaceViews rendered in software, overlapping with UI elements. Other activities show ways to work with Surfaces, GLES, the camera, and video.
See also the Android System-Level Graphics Architecture doc, which explains all this in much greater detail.
Dont use "setZOrderOnTop" as true. That will get it over all the other layouts.
If you are using multiple surfaceviews. use this for each surfaceview
yourSurfaceView.setZOrderMediaOverlay(true);
then set this setZOrderOnTop as false for the surfaceview you initiated later and wanted it to get back to the other surfaceviews
secondSurfaceview.setZOrderOnTop(false);
I am trying to show a dynamic OpenGL over the Android camera preview. I have used the typical framework suggested in other guides/sites to create an object which implements the GLSurfaceView.Renderer interface and use setContentView() and addContentView() after. The problem is that the onDrawImage is occurring behind the surface holder which displays the camera preview.
This is part of a larger application and I have separately tested all components in a side application. The only difference between the two is that the larger application has several activities preceding this one, while the side application uses just this activity as the main one.
Update
The answer is to switch the order of the setContentView and addContentView. If it is the main application, use the order in this post: opengl overlay on camera view; however, if you are coming from another activity, run:
setContentView(theCameraCallbackView)
addContentView(theOpenGLView, new LayoutParams( LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT ))
If anyone can provide some reasoning behind this, it would be greatly appreciated! I spend about eight hours on this two-line swap so it would be great to know what the cause of this was (and hopefully prevent similar issues for others).
On some devices, onPreviewFrame is not called if no SurfaceView was set to display the camera preview.
However, I handle the camera in a service, so I can't set a SurfaceView but I don't want to have visible preview anyway.
How can this be done? Can I programmatically create a SurfaceView and set it with Camera::setPreviewDisplay?
This must be possible or not?
It works on almost every phone without a SurfaceView but not on HTC One X and Google Nexus One...
According to this question, creating a SurfaceView in code works fine. Though I don't think you can create it through a service.
Another approach is to create a 1px-1px SurfaceView inside a RelativeLayout and hiding it with some other view on top of it. (visibility should still be VISIBLE). We use this trick for Path camera UI where we render preview buffers through OpenGL and it works fine.
According to documentation readily configured, visible and displayed surface view is necessary to activate camera preview. It may be overlayued though
From API 11 on, you can use a SurfaceTexture instead of a SurfaceView to get frames from the camera. Then, instead of using Camera.setPreviewDisplay, simply use Camera.setPreviewTexture.
This answer as well as this one discuss this point.