I've written an OpenGL ES 2 app using NativeActivity that renders around 6 textured triangles that you can drag around using touch input. The app is rendering as fast as possible, calling glClear() before and eglSwapBuffers() after rendering. As expected the frame rate is capped to the display frequency of around 60 fps and the scene moves around smoothly. However, after some time (10 to 60 seconds) the movement starts to stutter even though the frame rate is still 60 fps. The device I'm using is OnePlus 3.
At first I suspected the culprit was battery saving mode, but investigation revealed no correlation between cpus' frequencies and the stutter.
I did notice something interesting though, the stutter starts when queueBuffer() execution time goes up.
This is the systrace before stutter:
This is the systrace after stutter:
This is the systrace during the transition from before to after:
How do I stop the app from stuttering?
I still don't know the exact cause of the stuttering, but I managed to find a work-around: Only render frames on Choreographer.postFrameCallback and set eglPresentationTimeANDROID to two vsyncs in the future. This avoids filling up the buffer queue, which seems to be one of the causes of the stuttering.
As a bonus it also improved the touch input latency a little.
More info here: Minimize Android GLSurfaceView lag
Related
I am using MediaCodec to play 1080p#60fps video. This is on freescale SabreSD platform with Android Lollipop 5.1.
Initially because of BufferQueue Synchronous Mode, the FPS was way below 60.I could now manage to play at 70FPS by changing the BufferQueue to Asynchronous as in JB.
Now the next challenge I am facing is the video lags and FPS drops drastically to 40 when I start interacting with the screen (pulling down notification bar , pressing volume button etc).
So I ran rafika MultiSurfaceActivity and Record GL, I can see all the test play smoothly when no screen is touched or disturbed, but as soon as I start scrolling the notification bar from top and continue that for long time, the fps gets reduced to 35-40FPS.
I have confirmed the same test on Kitkat 4.4.2 and JB 4.2.2 and they seems to work fine.
Same behaviour when playing MP4 from Gallery. The video gets stuck and lags a lot when we start playing with Notification bar
Can anyone explain what has change from Kitkat to Lollipop which can cause this issue (VSync, Triple Buffering ?).
Regurgitating a bit from the Grafika issue tracker:
The bouncing ball is software-rendered, so anything that soaks up CPU time is going to make it slow down. On devices with medium-grade CPUs and big displays (e.g. Nexus 10) it never gets close to 60fps. So a slowdown while you are playing with the nav bar doesn't surprise me, but if it continues to be slow even after you stop playing with the nav bar, then that's a little weird.
Video playback should be less affected, as that does less with the CPU.
Investigation into such problems usually begins by using systrace to capture traces in "good" and "bad" states, and comparing the two.
The key point of BufferQueue "async mode" is to allow frames to drop if the consumer can't keep up with the producer. It's primarily meant for SurfaceTexture, where producer and consumer are in the same app, potentially on the same thread, so having the producer stall waiting for the consumer could cause the program to hang. I'm not sure what you mean by needing it to exceed 60fps, but I would guess you're throwing frames at the display faster than it can render them... so you're not really increasing the frame rate, you're just using the BufferQueue to drop the frames instead of using Choreographer to decide when you need to drop them yourself.
In any event, I left Google back in June 2014, well before Lollipop was completed. If something works correctly on KitKat but weirdly on Lollipop, I'm afraid I can't provide much insight. If you can reproduce the behavior easily, it might be worth capturing a video that demonstrates the problem (point a second smart phone at the device exhibiting the problem, so they can see how you manipulate the device) and filing a bug on http://b.android.com/.
Some traces uploaded by the OP:
https://www.dropbox.com/s/luwovq7ohozccdy/Lollipop_bad.zip
https://www.dropbox.com/s/zkv0aqw0shecpw2/lollipop_good.zip
https://www.dropbox.com/s/g7qe01xvmfyvpak/kitkat.zip
Looking at the kitkat trace, something weird is going on in SurfaceFlinger. The main thread is sitting in postFrameBuffer for a very long time (23-32ms). It eventually wakes up, and the CPU row suggests it was waiting on activity from a "galcore daemon", which I'm not familiar with (seems particular to Vivante GPU).
The lollipop traces only show the CPU rows, as if the capture were done without the necessary tags. I don't believe the systrace capture command changed significantly between kitkat and lollipop, so I'm puzzled as to why the user-space-initiated logging would vanish but the kernel thread scheduling stuff would remain. Make sure you have sched gfx view specified.
The newer lollipop traces only have about a second of good data. When you see "Did Not Finish" it means a "start" record had no matching "end" record. You can increase the systrace logging buffer size with the -b flag. I think there's enough there though.
Looking at the /system/bin/surfaceflinger row you can see that, in the "good" trace, postFrameBuffer usually finishes in about 16ms, but it's still waiting on galcore. Zoom in on 388ms (use WASD keys). At 388.196ms, on the CPU 2 row, you can see galcore do something. Right after it completes, the thin line at the top of the surfaceflinger row changes from light grey (sleeping) to green (running). At 388.548ms, again on CPU 2, galcore runs again, and right after that on the surfaceflinger row you see queueBuffer start to execute.
The "bad" trace looks identical. For example, you can see two galcore executions at 101.146ms and 101.666ms, with what appear to be similar effects on the surfaceflinger row. The key difference is the time spent in postFrameBuffer, which is around 16ms for "good" and around 30ms for "bad".
So this doesn't appear to be a behavioral shift; rather, things are taking longer and deadlines are being missed.
As far as I can tell, SurfaceFlinger is being held up by galcore daemon. This is true in both "good" and "bad" cases. To see what the timing should look like you can run systrace on a Nexus device, or compare to traces from other devices (e.g. the one in this case study or this SO question). If you zoom in you can see doComposition executing in a few milliseconds, and postFrameBuffer finishing in a few tenths of a millisecond.
Summing up: you don't have good and bad, you have bad and worse. :-) I don't know what galcore is, but you'll likely need to have a conversation with the GPU OEM.
I've been developing a recording component of our app, which records GL renderings on a GLSurfaceView using MediaCodec and Muxer. I found a nice set of examples by fadden (like bigflake and Grafika... Thanks so much fadden), and tried those stuff out. I've built a recorder based on the game recording model of Android Breakout, because our app used GLSurfaceView to do the rendering.
It seemed working well on my Nexus 7 (2013 with 4.4.2 and 5.0.2). I was able to record rendered screens, and those encoded MP4s were played well on other devices. However, when a bit more complicated rendering got involved, it started dropping frames. Well, it's not that complicated, as it takes only around 4 ms to render the frame. So, I've been trying to identify the source of the problem.
My pseudo code are very similar to the Android Breakout Recorder. OnDrawFrame() looks like the following.
GLES20.glViewport(mViewportXoff, mViewportYoff, mViewportWidth, mViewportHeight);
drawFrame();
GameRecorder recorder = GameRecorder.getInstance();
if (recorder.isRecording() && recordThisFrame()) {
saveRenderState();
// switch to recorder state
recorder.makeCurrent();
recorder.getProjectionMatrix(mProjectionMatrix);
recorder.setViewport();
// render everything again
drawFrame();
recorder.swapBuffers();
restoreRenderState();
}
I've tried to measure performance of line-by-line calls and found that recorder.swapBuffers() takes around 10 ms on Nexus 7. Moving buffers could take that much time, which I think is reasonable. Another one takes longer than expected was recorder.makeCurrent(), which takes around another 10 ms.
I was also trying to measure the call intervals of onDrawFrame(). When recording was off (with recordThisFrame() setting to be false), I've been consistently getting 16.7 ~ 17 ms, which is expected. When recordThisFrame() is set to alternate between true and false to record every other frame, I've been getting around 22 ~ 25 ms (when recording) and 4~10 ms. Even after changing the drawFrame() call to simple glclear(), I got the same result. I tried this on Android Breakout Recorder (fadden's original code), and got the same result as well.
I was trying the "Record GL app with FBO" activity in Grafika, and I found it gets more consistent 16-17 ms intervals even with the recordings. I was tempted to replace GLSurfaceView with SurfaceView as in the Record GL app in Grafika, but that change does not seems to be a viable option for now.
Is the makeCurrent() call that much costly? Is there anyone experiencing similar interval issues? Any thoughts would be appreciated.
I'd like to calculate FPS to detect performance issue of an application based on existing Android profiling tool .
I noted that on Systrace, it can record the length of performTraversals. As far as I know, performTraversals performs measure, layout and draw, which include most of jobs when updating a frame. So can performTraversals be representative enough to measure whether a frame will take 60 ms to update?
I also noted that Systrace record the time spending on SurfaceFlinger. I know SurfaceFlinger served for rendering purpose, but I don't know the exact beginning point and ending point of a frame. Should I also considering the time spent on SurfaceFlinger to the frame rate? (Though I do observe that SurfaceFlinger perform more frequently than performTraversals, which means SurfaceFlinger may not necessarily follow performTraversals. It will also be triggered in other scenarios.)
P.S. I'm aware of the sysdump gfxinfo, but it can only record 128 frames(~2 seconds), while what I want may last much longer.
Systrace is not useful for measuring FPS overall, but you can do that trivially with a frame counter and System.nanoTime(). If you're not hitting your target framerate, though, it can help you figure out why not.
The official docs provide some useful pointers, but there's a lot of information and the interactions can be complex. The key things to know are:
The device display panel generates a vsync signal. You can see that on the VSYNC line. Every time it transitions between 1 and 0 is a refresh.
The vsync wakes surfaceflinger, which gathers up the incoming buffers for the various windows and composites them (either itself using OpenGL ES, or through the Hardware Composer).
If your app was running faster than the panel refresh rate (usually 60fps), it will have blocked waiting for surfaceflinger (in, say, eglSwapBuffers()). Once surfaceflinger acquires the buffer, the app is free to continue and generate another frame.
Unless you're rendering offscreen, you can't go faster than surfaceflinger.
As of Android 4.3 (API 18) you can add your own events to the systrace output using the android.os.Trace class. Wrapping your draw method with trace markers can be extremely informative. You have to enable their tag with systrace to see them.
If you want to be running at 60fps, your rendering must finish in well under 16.7ms. If you see a single invocation of performTraversals taking longer than that, you're not going to hit maximum speed.
I'm developing a GL live wallpaper that uses very little CPU and only modest GPU. On my older test phone, it can run at a full 58fps or so most of the time. But occasionally the effects ramp up, and then the render times jitter between 16ms and 50ms per frame. For example, it'll render several frames at 16ms, slide up to 50ms over a dozen frames or so, render several more frames at 50ms, then slide back down to 16ms and repeat. I discovered that if I set the CPU governor to "performance" (or "conservative", curiously enough) instead of the default "ondemand" it'll render with full effects at full speed. Alternatively, if I leave the governor alone and insert a busy loop in my code (increment a variable 100,000 times per frame) that bumps my CPU usage up enough to transition to a higher clock rate and render smoothly as well.
So it seems on this phone my app is bottlenecked by the GPU, but only when it throttles down. Now, I wouldn't mind if the GLSurfaceView rendered at a slower rate according to the GPU clock, but my problem here is that I'm getting the bursts of alternating high and low frame rates which makes my animation look fluid/frameskippy/fluid/frameskippy/etc. several times per second. It seems like the GPU clock is ramping up and down like crazy?
I got a visible improvement by using RENDERMODE_WHEN_DIRTY and calling requestRender() on a strictly timed thread, but the darn GPU keeps ramping up and down. Why won't it either render as fast as it can at the slower clock, or just jump to and STAY AT the higher clock?
The best solution I've come up with so far is using a sliding window to detect the average frame update time, then applying the difference from the target frame time until the two values converge. The time between render updates is slower but at least it's roughly constant. So that works in theory, but it takes several seconds to reach a steady state and it looks bad in the meantime.
I think a third option might be to cannibalize the GLSurfaceView source and make a custom version. From what I understand, the blocking GL calls are made in there, so it would be much easier for me to time render calls and react accordingly. I'm not very comfortable attempting that though because there's a lot of code in there that I'd have to spend a lot of time understanding before I could even begin to mess with it. Plus I'd then have to worry about how well version X of GLSurfaceView plays with any version Y of Android.
So, with all that said, do I have any other options here? Is there an easier fix to this?
try fixing the frame rate by pausing the thread (thread sleep) for the remaining time to reach a constant frame rate.
Our Android game has an issue which appears unique to the Galaxy S2.
Occasionally the render will stutter. By this I mean it basically seems to render the last two frames (as though its swapping the last two render buffers without updating either).
What's really odd about this is that the game continues to update, so say the stutter lasts for 2 seconds, the game will have progressed 2 seconds behind the scenes.
This is odd because our code is basically like this:
function Update()
DoGameLogic()
DoRender()
So this means that if our the game has updated, the game has also rendered. The maximum delta time is capped to 1 frame so there must have been more than one Update and thus multiple renders during the stutter.
My current theory is that on most devices the game lags during render, but on the S2 the render calls are executed but they "fall through" without updating the render buffer.
Has anyone run into this problem? I would really appreciate any suggestions about what this could be.
We found out what the problem is.
The Galaxy S 2 was for some reason running out of GL memory. This wasn't apparent on the devices we were testing with, but on other devices it would crash on some Open GL call - not the offending call mind you.
Eventually we tracked it down to using Point sprite VBO's. As the S 2 is a powerful device, we replaced Point sprites with Quad's mimicking point sprites as a workaround.
Incidentally, SoundPool would also run out of memory on this device, requiring another workaround.