Related
For a while now I've experienced an intermittent 'stuttering' of the sprites that are in motion within my Android Game. It's a fiarly simple 2D OpenGL ES 2.0 game. (This is an ongoing problem which I have re-visited many times).
In my game loop, I have 2 'timers' - one which will log the number of frames in the previous second, and another which counts the time (in Milliseconds) from the end of the current onDrawFrame iteration to the start of the next.
This is what I've found:
When not rendering anything, I get 60fps (for the most part), and every time onDrawFrame is called, it reports itself as taking longer that 16.667ms. Now, If I render something (doesn't matter if it's 1 quad or 100 quads, the result is the same), I get 60fps (for the most part) but now, only about 20% of onDrawFrame calls report themselves as taking longer than 16.667ms from the last call.
I don't really understand why this happens, firstly, why, when onDrawFrame isn't doing anything, is it called so 'slowly' - and more importantly, why does any GL call (one simple quad), still make the time between onDrawFrame calls longer than 16.667ms (albeit much less frequently).
I should say that when onDrawFrame reports taking longer than 16.667ms from the last iteration, it is almost always accompanied by a drop in FPS (to 58 or 59), but not all of the time, sometimes, the FPS stays constant. And conversely, sometimes when the FPS drops, onDrawFrame is called within 16.667ms of the last iteration completing.
So......
I'm trying to fix my game-loop and eradicate these 'stutters' - some other things to note:
When I do method profiling, it shows glSwapBuffers, sometimes taking a long time
When I do a GL Trace, most scenes its says renders in less than 1ms, but sometimes the odd frame takes 3.5-4ms - same scene. Nothing changes apart from the time it takes
Almost every time a frame is dropped, or onDrawFrame reports a long delay (or both), there is a visual glitch, but not every time. Big visual glitches seems to coincide with multiple 'delayed' onDrawFrame calls and /or dropped frames.
I don't think this is a scene complexity issue for 2 reasons: 1) even if I render my scene twice, it doesn't make the problem any worse, I still for the most part, get 60FPS with the occasional drop, just as before and 2), even if I strip the scene bare, I still get the problem.
I obviously am misunderstanding something, so a push in the right direction would be appreciated.
OnDrawFrame
#Override
public void onDrawFrame(GL10 gl) {
startTime = System.nanoTime();
fps++;
totalTime = System.nanoTime() - timeSinceLastCalled;
if (totalTime > 16667000) {
Log.v("Logging","Time between onDrawFrame calls: " + (totalTime /(double)1000000));
}
//Grab time
newTime = System.currentTimeMillis() * 0.001;
frameTime = newTime - currentTime; //Time the last frame took
if (frameTime > 0.25)
frameTime = 0.25;
currentTime = newTime;
accumulator += frameTime;
while (accumulator >= dt){
saveGameState();
updateLogic();
accumulator -= dt;
}
interpolation = (float) (accumulator / dt);
Matrix.multiplyMM(mMVPMatrix, 0, mProjMatrix, 0, mVMatrix, 0);
render(interpolation);
if (startTime > lastSecond + 1000000000) {
lastSecond = startTime;
Log.v("Logging","fps: "+fps);
fps=0;
}
endTime = startTime;
timeSinceLastCalled = System.nanoTime();
}
This game loop above is the one featured in this excellent article.
Some thoughts:
Don't use System.currentTimeMillis() for timing things. It's based on the wall clock, which can be updated by the network. Use System.nanoTime(), which is based off the monotonic clock.
See this appendix for some notes on game loops. Queue-stuffing is fine for many things, but understand that you're not exactly working off of VSYNC, so timings will tend to be inaccurate.
Some devices (notably those based on qcom SOCs) reduce CPU speed when they think they're idle. Always take timings while actively moving your finger around on the touch screen.
If you want to debug frame rate issues you need to be using systrace. The traceview profiling isn't that useful here.
See Grafika's "record GL app" Activity for an example of a simple GLES app that drops frames, but adjusts the animation such that it's rarely noticeable.
I have just started using the OpenGL ES 2 for android for my little game and have encountered a problem on redrawing the screen on each frame.
I have setup a loop on my Renderer's onDrawFrame, just a simple [ updateGameLogic() -> drawGame() ] or Thread.sleep() loop based on the time lapsed from last drawGame call.
Currently the updateGameLogic() method simply translate the camera to the +ve X direction (the game is 2d).
In the drawGame() call, I first clear my screen with GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT). Then I have 3 glBindTexture and glDrawElements calls for drawing 3 categories of objects with different texture atlas.
Here comes the problem, in between each frame drawn on screen, there is a blink of the previous frame appearing which is undesired and makes the game look dizzy. Precisely, say the game is just about to draw frame 3 from frame 2, right before frame 2 vanish and frame 3 appearing, there is a split moment where frame 1 is displayed.
I thought this may be due to the way the GLSurfaceView is buffered by the system so I experimented with calling multiple glClear before drawing but everything stays the same. Would be grateful if someone can provide some explanation / solution to the problem, and what I have done wrong, thanks. (basically paragraph 2 to 4 is all my code so I have not posted it, unless requested)
From the clarification in the comment, it sounds like you have something like this in your code:
public void onDrawFrame(GL10 gl) {
long currentTime = SystemClock.elapsedRealtime();
long deltaTime = currentTime - mLastFrameTime;
if (deltaTime < 33) {
Thread.sleep(33 - deltaTime);
return;
}
mLastFrameTime = currentTime;
updateGameLogic(deltaTime);
drawGame();
}
This will indeed cause problems. When onDrawFrame() is called, you have to render a frame. You can't just return without drawing anything. The caller will assume that you rendered a frame in any case, and it will end up being presented on the screen. If you decide not to render anything, whatever happened to be in the surface you were supposed to draw to will be presented. There's no telling what this will be, but it's quite likely that it's an old frame from 2-3 frames earlier.
If you want to artificially throttle the frame rate, e.g. to save power, unfortunately there's no very good way to do this in Android. Using sleeps in onDrawFrame() is kind of dirty (and inherently unreliable, IMHO), but it might be necessary in this case. The key is that either before or after you sleep, you still need to render a frame. As a first attempt, I would try tweaking the above to something like this:
public void onDrawFrame(GL10 gl) {
long currentTime = SystemClock.elapsedRealtime();
long deltaTime = currentTime - mLastFrameTime;
if (deltaTime < 33) {
Thread.sleep(33 - deltaTime);
currentTime = SystemClock.elapsedRealtime();
deltaTime = currentTime - mLastFrameTime;
}
mLastFrameTime = currentTime;
updateGameLogic(deltaTime);
drawGame();
}
Note that while there is still an artificial delay, there is no early return in the code anymore.
There are probably more robust variations of this idea for throttling the redraws to 30 fps. Some searching on SO or the rest of the internet should reveal previous discussions.
I'm measuring time interval for looped animations/particles/etc in my Android app. App is a live wallpaper so it doesn't prevent system from scaling clock down for power saving.
Because of this all methods to measure time intervals between last frames doesn't measure monotonic time - I experience animations inconsistently slowing down and speeding up.
I've used all possible methods to retrieve system time - all of them are not monotonic (even SystemClock.elapsedRealtime() and System.nanoTime() which are guaranteed to be monotonic but nope they are not).
If I hold a finger on screen to prevent power saving all animations are smooth.
This issue is very noticeable on Nexus10, slightly noticeable on Nexus7 1st gen and on Nexus4 it is not present at all.
Here is code excerpt to measure time:
public void onDrawFrame(GL10 glUnused) {
// calculate timers for animations based on difference between lastFrameTime and SystemClock.elapsedRealtime()
...
// save last timestamp for this frame
lastFrameTime = SystemClock.elapsedRealtime();
}
Expanding a little on what I think you're doing based on the outline of your code, your onDrawFrame() method in pseudo code looks like this:
deltaTime = SystemClock.elapsedRealtime() - lastFrameTrime
// point 1, see text below
update animation based on deltaTime
draw frame
// point 2, see text below
lastFrameTime = SystemClock.elapsedRealtime()
The problem with this sequence is that you lose the time between point 1 and point 2 from your total animation time. You need to make sure that the total sum of all your deltaTime values, which are the times applied to your animation, covers the entire wall clock time of the amimation. With the logic you use, this is not the case. You only add up the times between the end of one call to the start of the next call. You do not account for the time needed to execute the method, which can be significant.
This can be fixed with a slight change in the sequence:
currentTime = SystemClock.elapsedRealtime()
deltaTime = currentTime - lastFrameTime
lastFrameTime = currentTime
update animation based on deltaTime
draw frame
The key point is that you only call elapsedRealtime() once per frame.
Hey all I'm at a crossroads with my app that I've been working on.
It's a game and an 'arcade / action' one at that, but I've coded it using Surfaceview rather than Open GL (it just turned out that way as the game changed drastically from it's original design).
I find myself plagued with performance issues and not even in the game, but just in the first activity which is an animated menu (full screen background with about 8 sprites floating across the screen).
Even with this small amount of sprites, I can't get perfectly smooth movement. They move smoothly for a while and then it goes 'choppy' or 'jerky' for a split second.
I noticed that (from what I can tell) the background (a pre-scaled image) is taking about 7 to 8 ms to draw. Is this reasonable? I've experimented with different ways of drawing such as:
canvas.drawBitmap(scaledBackground, 0, 0, null);
the above code produces roughly the same results as:
canvas.drawBitmap(scaledBackground, null, screen, null);
However, if I change my holder to:
getHolder().setFormat(PixelFormat.RGBA_8888);
The the drawing of the bitmap shoots up to about 13 MS (I am assuming because it then has to convert to RGB_8888 format.
The strange thing is that the rendering and logic move at a very steady 30fps, it doesn't drop any frames and there is no Garbage Collection happening during run-time.
I've tried pretty much everything I can think of to get my sprites moving smoothly
I recently incorporated interpolation into my gameloop:
float interpolation = (float)(System.nanoTime() + skipTicks - nextGameTick)
/ (float)(skipTicks);
I then pass this into my draw() method:
onDraw(interpolate)
I have had some success with this and it has really helped smooth things out, but I'm still not happy with the results.
Can any one give me any final tips on maybe reducing the time taken to draw my bitmaps or any other tips on what may be causing this or do you think it's simply a case of Surfaceview not being up to the task and therefore, should I scrap the app as it were and start again with Open GL?
This is my main game loop:
int TICKS_PER_SECOND = 30;
int SKIP_TICKS = 1000 / TICKS_PER_SECOND;
int MAX_FRAMESKIP = 10;
long next_game_tick = GetTickCount();
int loops;
bool game_is_running = true;
while( game_is_running ) {
loops = 0;
while( GetTickCount() > next_game_tick && loops < MAX_FRAMESKIP) {
update_game();
next_game_tick += SKIP_TICKS;
loops++;
}
interpolation = float( GetTickCount() + SKIP_TICKS - next_game_tick )
/ float( SKIP_TICKS );
display_game( interpolation );
}
Thanks
You shouldn't use Canvas to draw fast sprites, especially if you're drawing a fullscreen image. Takes way too long, I tell you from experience. I believe Canvas is not hardware accelerated, which is the main reason you'll never get good performance out of it. Even simple sprites start to move slow when there are ~15 on screen. Switch to OpenGL, make an orthographic projection and for every Sprite make a textured quad. Believe me, I did it, and it's worth the effort.
EDIT: Actually, instead of a SurfaceView, the OpenGL way is to use a GLSurfaceView. You create your own class, derive from it, implement surfaceCreated, surfaceDestroyed and surfaceChanged, then you derive from Renderer too and connect both. Renderer handles an onDraw() function, which is what will render, GLSurfaceView manages how you will render (bit depth, render modes, etc.)
I'm making an android game and am currently not getting the performance I'd like. I have a game loop in its own thread which updates an object's position. The rendering thread will traverse these objects and draw them. The current behavior is what seems like choppy/uneven movement. What I cannot explain is that before I put the update logic in its own thread, I had it in the onDrawFrame method, right before the gl calls. In that case, the animation was perfectly smooth, it only becomes choppy/uneven specifically when I try to throttle my update loop via Thread.sleep. Even when I allow the update thread to go berserk (no sleep), the animation is smooth, only when Thread.sleep is involved does it affect the quality of the animation.
I've created a skeleton project to see if I could recreate the issue, below are the update loop and the onDrawFrame method in the renderer:
Update Loop
#Override
public void run()
{
while(gameOn)
{
long currentRun = SystemClock.uptimeMillis();
if(lastRun == 0)
{
lastRun = currentRun - 16;
}
long delta = currentRun - lastRun;
lastRun = currentRun;
posY += moveY*delta/20.0;
GlobalObjects.ypos = posY;
long rightNow = SystemClock.uptimeMillis();
if(rightNow - currentRun < 16)
{
try {
Thread.sleep(16 - (rightNow - currentRun));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
And here is my onDrawFrame method:
#Override
public void onDrawFrame(GL10 gl) {
gl.glClearColor(1f, 1f, 0, 0);
gl.glClear(GL10.GL_COLOR_BUFFER_BIT |
GL10.GL_DEPTH_BUFFER_BIT);
gl.glLoadIdentity();
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
gl.glTranslatef(transX, GlobalObjects.ypos, transZ);
//gl.glRotatef(45, 0, 0, 1);
//gl.glColor4f(0, 1, 0, 0);
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, vertexBuffer);
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, uvBuffer);
gl.glDrawElements(GL10.GL_TRIANGLES, drawOrder.length,
GL10.GL_UNSIGNED_SHORT, indiceBuffer);
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
gl.glDisableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
}
I've looked through replica island's source and he's doing his update logic in a separate thread, as well as throttling it with Thread.sleep, but his game looks very smooth. Does anyone have any ideas or has anyone experienced what I'm describing?
---EDIT: 1/25/13---
I've had some time to think and have smoothed out this game engine considerably. How I managed this might be blasphemous or insulting to actual game programmers, so please feel free to correct any of these ideas.
The basic idea is to keep a pattern of update, draw... update, draw... while keeping the time delta relatively the same (often out of your control).
My first course of action was to synchronize my renderer in such a way that it only drew after being notified it was allowed to do so. This looks something like this:
public void onDrawFrame(GL10 gl10) {
synchronized(drawLock)
{
while(!GlobalGameObjects.getInstance().isUpdateHappened())
{
try
{
Log.d("test1", "draw locking");
drawLock.wait();
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
When I finish my update logic, I call drawLock.notify(), releasing the rendering thread to draw what I just updated. The purpose of this is to help establish the pattern of update, draw... update, draw... etc.
Once I implemented that, it was considerably smoother, although I was still experiencing occasional jumps in movement. After some testing, I saw that I had multiple updates occurring between calls of ondrawFrame. This was causing one frame to show the result of two (or more) updates, a larger jump than normal.
What I did to resolve this was to cap the time delta to some value, say 18ms, between two onDrawFrame calls and store the extra time in a remainder. This remainder would be distributed to subsequent time deltas over the next few updates if they could handle it. This idea prevents all sudden long jumps, essentially smoothing a time spike out over multiple frames. Doing this gave me great results.
The downside to this approach is that for a little time, the position of objects will not be accurate with time, and will actually speed up to make up for that difference. But it's smoother and change in speed is not very noticeable.
Finally, I decided to rewrite my engine with the two above ideas in mind, rather than patching up the engine I had originally made. I made some optimizations for the thread synchronization that perhaps someone could comment on.
My current threads interact like this:
-Update thread updates the current buffer (double buffer system in order to update and draw simultaneously) and will then give this buffer to the renderer if the previous frame has been drawn.
-If the previous frame has not yet draw, or is drawing, the update thread will wait until the render thread notifies it that it has drawn.
-Render thread waits until notified by update thread that an update has occurred.
-When the render thread draws, it sets a "last drawn variable" indicating which of the two buffers it last drew and also notifies the update thread if it was waiting on the previous buffer to be drawn.
That may be a little convoluted, but what that's doing is allowing for the advantages of multithreading, in that it can perform the update for frame n while frame n-1 is drawing while also preventing multiple update iterations per frame if the renderer is taking a long time. To further explain, this multiple-update scenario is handled by the update thread locking if it detects that the lastDrawn buffer is equal to the one which was just updated. If they are equal, this indicates to the update thread that the frame before has not yet been drawn.
So far I'm getting good results. Let me know if anyone has any comments, would be happy to hear your thoughts on anything I'm doing, right or wrong.
Thanks
(The answer from Blackhex raised some interesting points, but I can't cram all this into a comment.)
Having two threads operating asynchronously is bound to lead to issues like this. Look at it this way: the event that drives animation is the hardware "vsync" signal, i.e. the point at which the Android surface compositor provides a new screen full of data to the display hardware. You want to have a new frame of data whenever vsync arrives. If you don't have new data, the game looks choppy. If you generated 3 frames of data in that period, two will be ignored, and you're just wasting battery life.
(Running a CPU full out may also cause the device to heat up, which can lead to thermal throttling, which slows everything in the system down... and can make your animation choppy.)
The easiest way to stay in sync with the display is to perform all of your state updates in onDrawFrame(). If it sometimes takes longer than one frame to perform your state updates and render the frame, then you're going to look bad, and need to modify your approach. Simply shifting all game state updates to a second core isn't going to help as much as you might like -- if core #1 is the renderer thread, and core #2 is the game state update thread, then core #1 is going to sit idle while core #2 updates the state, after which core #1 will resume to do the actual rendering while core #2 sits idle, and it's going to take just as long. To actually increase the amount of computation you can do per frame, you'd need to have two (or more) cores working simultaneously, which raises some interesting synchronization issues depending on how you define your division of labor (see http://developer.android.com/training/articles/smp.html if you want to go down that road).
Attempting to use Thread.sleep() to manage the frame rate generally ends badly. You can't know how long the period between vsync is, or how long until the next one arrives. It's different for every device, and on some devices it may be variable. You essentially end up with two clocks -- vsync and sleep -- beating against each other, and the result is choppy animation. On top of that, Thread.sleep() doesn't make any specific guarantees about accuracy or minimum sleep duration.
I haven't really gone through the Replica Island sources, but in GameRenderer.onDrawFrame() you can see the interaction between their game state thread (which creates a list of objects to draw) and the GL renderer thread (which just draws the list). In their model, the game state only updates as needed, and if nothing has changed it just re-draws the previous draw list. This model works well for an event-driven game, i.e. where the contents on screen update when something happens (you hit a key, a timer fires, etc). When an event occurs, they can do a minimal state update and adjust the draw list as appropriate.
Viewed another way, the render thread and the game state work in parallel because they're not rigidly tied together. The game state just runs around updating things as needed, and the render thread locks it down every vsync and draws whatever it finds. So long as neither side keeps anything locked up for too long, they don't visibly interfere. The only interesting shared state is the draw list, guarded with a mutex, so their multi-core issues are minimized.
For Android Breakout ( http://code.google.com/p/android-breakout/ ), the game has a ball bouncing around, in continuous motion. There we want to update our state as frequently as the display allows us to, so we drive the state change off of vsync, using a time delta from the previous frame to determine how far things have advanced. The per-frame computation is small, and the rendering is pretty trivial for a modern GL device, so it all fits easily in 1/60th of a second. If the display updated much faster (240Hz) we might occasionally drop frames (again, unlikely to be noticed) and we'd be burning 4x as much CPU on frame updates (which is unfortunate).
If for some reason one of these games missed a vsync, the player may or may not notice. The state advances by elapsed time, not a pre-set notion of a fixed-duration "frame", so e.g. the ball will either move 1 unit on each of two consecutive frames, or 2 units on one frame. Depending on the frame rate and the responsiveness of the display, this may not be visible. (This is a key design issue, and one that can mess with your head if you envisioned your game state in terms of "ticks".)
Both of these are valid approaches. The key is to draw the current state whenever onDrawFrame is called, and to update state as infrequently as possible.
Note for anyone else who happens to read this: don't use System.currentTimeMillis(). The example in the question used SystemClock.uptimeMillis(), which is based on the monotonic clock rather than wall-clock time. That, or System.nanoTime(), are better choices. (I'm on a minor crusade against currentTimeMillis, which on a mobile device could suddenly jump forward or backward.)
Update: I wrote an even longer answer to a similar question.
Update 2: I wrote an even longer longer answer about the general problem (see Appendix A).
One part of the problem may be caused by fact that Thread.sleep() is not accurate. Try to investigate what is the actual time of the sleep.
The most important thing that should make your animations smooth is that you should compute some interpolation factor, call it alpha, that linearly interpolates your animations in consecutive rendering thread calls between two consecutive animation update thread calls. In other words, if your update interval is high comparing to your framerate, not interpolating your animation update steps is like you'd be rendering at update interval framerate.
EDIT: As an example, this is how PlayN does it:
#Override
public void run() {
// The thread can be stopped between runs.
if (!running.get())
return;
int now = time();
float delta = now - lastTime;
if (delta > MAX_DELTA)
delta = MAX_DELTA;
lastTime = now;
if (updateRate == 0) {
platform.update(delta);
accum = 0;
} else {
accum += delta;
while (accum >= updateRate) {
platform.update(updateRate);
accum -= updateRate;
}
}
platform.graphics().paint(platform.game, (updateRate == 0) ? 0 : accum / updateRate);
if (LOG_FPS) {
totalTime += delta / 1000;
framesPainted++;
if (totalTime > 1) {
log().info("FPS: " + framesPainted / totalTime);
totalTime = framesPainted = 0;
}
}
}