Develop Reference

Performant 2D rendering with Android's Canvas API

Over the last month, I've taken a dive into trying my hand at a simple 2d side scroller using Android's default canvas APIs.
In short, my question boils down to "Is the Canvas API performant enough to pull 60fps with a simple side scroller with a multi-layer, parallaxing background?".
Now hear me out, because I've tried a ton of different approaches, and so far I've come up fairly empty handed on how to squeeze any more efficiency out of what I've attempted.
First, to address the easy problems:
I'm not allocating anything in the game loop
The Surface is Hardware Accelerated
Bitmaps are being loaded prior to starting the game, and drawn without scaling.
I don't believe I'm doing anything significant aside from the bitmap drawing in the game loop (mostly updating position variables and getting time)
The problems started, as you may have guessed, with the parallaxing BG. Seven layers in total, though I've been experimenting with as few as 3 while still unable to maintain 60fps. I'm clipping the parts that are overlapping to minimize overdraw, but I would guess I'm still drawing ~2-3x overdraw, all summed up.
As the game runs, I often start out at ~ 60fps, but about 30/40 seconds in, the fps drops to about 20. The time is random, and it seems to be a state of the phone, rather than a result of anything in the code (force killing the app & restarting causes the new app to start at ~20fps, but letting the phone sit for a while results in higher fps). My guess here is thermal throttling on the CPU...
Now I'm testing on a 5x, and naively thought these issues might disappear on a faster device (6P). Of course, due to the larger screen, the problems got worse, as it ran at ~15fps continuously.
A co-worker had suggested loading in small bitmaps and stretching them on draw, instead of scaling the bitmaps on load to the size they would appear on the screen. Implementing this by making each bitmap 1/3rd the size, and using the canvas.drawBitmap(bitmap, srcRect, destRect, Paint) method to compensate for size and scale yielded worse performance overall (though I'm sure it helped the memory footprint). I haven't tried the drawMesh method, but I imagined it wouldn't be more performant that the plain old drawBitmap.
In another attempt, I gave an array of ImageViews a shot, thinking the Android View class might be doing some magic I wasn't, but after an hour of fiddling with it, it didn't look any more promising.
Lastly, the GPU profiling on the device remains far beneath the green line, indicating 60fps, but the screen doesn't seem to reflect that. Not sure what's going on there.
Should I bite the bullet and switch over to OpenGL? I thought that Canvas would be up to the task of a side-scroller with a parallaxing background, but I'm either doin' it wrong™, or using the wrong tool for the job, it seems.
Any help or suggestions are sincerely appreciated, and apologies for the long-winded post.

Can Android phone running image processing app 24/7

I'm developing an image processing app on Android phones, which is expected to run 24/7. I've managed to do the following:
Use Camera2 interface to gain better fps.
Grab raw frames, using renderscript to convert to rgb and do image processing using opencv in a background service( no preview ). I got around 20fps after conversion to rgb at 1280x960 on LG G4.
So my questions are:
Is there anything else I need to optimize to minimize the memory and CPU usage?
Any chance that this application can run 24/7? Is delegating all the camera operations and processing to the background service sufficient to allow it run 24/7? When I leave it running, I can still feel the heat from the camera and its surrounding area.
Any suggestion would be appreciated. Thanks.
UPDATE 1
The app runs on LG G4 using Camera2 interface and do image processing in the background with the screen off, got too hot and the phone turned off itself after a few hours. What can I do to overcome this?

about the second question. I think the app can not run 24/7 because the phone will close itself because of the heat.

Before answering to your question i must say that i am also new to image processing with android (but not to image processing field).
For the question one:
May be yes. Because image processing tasks are much memory intensive you may need to optimize your app in order to avoid things like memory leak(even if the android runtime perform routine garbage collection) .
check the following links
link one
may be useful
when it comes to pixel level operation ( when you avoid using inbuilt functions of opencv or what ever the library you are using and do access and process pixels manually) it will be too slow. I am saying this based on my experience on my laptop. Hoping that you are using opencv for your app just take a look at the following opencv site ( it is for python but you can get the idea)
take an idea from this
and also this SO answer: SO answer
A tip i can remember: try to reduce Mat variable copying (copying Mat objects to other Mat objects) as much as you can
Question number two:
I will go with answer given by user7746903. This answer also linked with the memory that will be consumed by your background app. There will be much more memory intensive app running on background so it depends. Thank you.

For the first question:
I feel its worth mentioning that you should by pass java as much as possible. Ie. using Java as the interfacial layer, then using JNI C as the call loop.
eg:
Get texture from camera2 > supply texture to C function > call render script/ compute shaders from C and other processing functions > call java function to render to screen.
This speeds up CPU performance and reduces memory warnings (especially when rapid allocation and freeing of memory).

Will making too many transformations "onDrawFrame" create lag?

I am using the Rajawali framework to make OpenGL ES based live wallpapers. To achieve many of my animation effects I have created some functions that are called from the onDrawFrame method. These functions vary from simple x,y,z rotations to more complicated equations with conditional statements that simulate wind or other randomized motions. It works well currently, and it is highly responsive especially when sleeping and waking the device.
As my live wallpapers become more complex I am worried that my crude solution will eventually start causing performance issues. Is that the case?
Is there a better way to make these types of cyclical or repetitive changes, like maybe making background threads?

Try to move as much computing as posible into vertex and fragment shaders. It's easier to hit other limits than CPU limits. Nobody can answer your question easily without examples and without knowing in depth what are you trying to archive.
Best TIP: Measure often and early during development.
Hope that helps.

Rajawali onDrawFrame() method is called by a timer, which is set with the frameRate() method. This gives much smoother animation than estimating thread sleeps (an alternate solution which is not advised).
If you are at 100fps, this gives you 10ms in between frames to do your calculations - that's it, but that is a lot of time to a modern CPU.
Putting the GPU calculations in a separate thread is problematic since this is not thread safe. You could put any non-GPU calcs in a separate thread, but this could result in sync issues.
Shaders are compiled into native code, but really should be limited to bitwise (integer) operations required to render the actual pixels (always the bottleneck) after all the expensive 3D calcs are done.
For example, an anaglyph shader would take care of overlaying two stereoscopic frames.
Hope this helps!

Android drawBitmap Performance For Lots of Bitmaps?

I'm in the process of writing an Android game and I seem to be having performance issues with drawing to the Canvas. My game has multiple levels, and each has (obviously) a different number of objects in it.
The strange thing is that in one level, which contains 45 images, runs flawlessly (almost 60 fps). However, another level, which contains 81 images, barely runs at all (11 fps); it is pretty much unplayable. Does this seem odd to anybody besides me?
All of the images that I use are .png's and the only difference between the aforementioned levels is the number of images.
What's going on here? Can the Canvas simply not draw this many images each game loop? How would you guys recommend that I improve this performance?
Thanks in advance.

Seems strange to me as well. I am also developing a game, lots of levels, I can easily have a 100 game objects on screen, have not seen a similar problem.
Used properly, drawbitmap should be very fast indeed; it is little more than a copy command. I don't even draw circles natively; I have Bitmaps of pre-rendered circles.
However, the performance of Bitmaps in Android is very sensitive to how you do it. Creating Bitmaps can be very expensive, as Android can by default auto-scale the pngs which is CPU intensive. All this stuff needs to be done exactly once, outside of your rendering loop.
I suspect that you are looking in the wrong place. If you create and use the same sorts of images in the same sorts of ways, then doubling the number of screen images should not reduce performance by a a factor of over 4. At most it should be linear (a factor of 2).
My first suspicion would be that most of your CPU time is spent in collision detection. Unlike drawing bitmaps, this usually goes up as the square of the number of interacting objects, because every object has to be tested for collision against every other object. You doubled the number of game objects but your performance went down to a quarter, ie according to the square of the number of objects. If this is the case, don't despair; there are ways of doing collision detection which do not grow as the square of the number of objects.
In the mean time, I would do basic testing. What happens if you don't actually draw half the objects? Does the game run much faster? If not, its nothing to do with drawing.

I think this lecture will help you. Go to the 45 minute . There is a graph comparing the Canvas method and the OpenGl method. I think it is the answer.

I encountered a similar problem with performance - ie, level 1 ran great and level 2 didn't
Turned it wasn't the rendering that was a fault (at least not specifically). It was something else specific to the level logic that was causing a bottleneck.
Point is ... Traceview is your best friend.
The method profiling showed where the CPU was spending its time and why the glitch in the framerate was happening. (incidentally, the rendering cost was also higher in Level 2 but wasn't the bottleneck)

SurfaceView Fast Enough For Emulation?

For years now I've maintained a Tandy Color Computer Emulator applet on my home page. With the purchase of an Incredible I decided to do a port. Getting it going in Android didn't take long but I'm really surprised how slow it runs. You can literally see the pixels painting. I know there are other successful Android emulators so I must be doing something wrong.
My approach was to use a SurfaceView for rendering. There's a separate thread that runs a virtual 6809 CPU. Whenever that thread updates the emulated video memory, it calls SurfaceHolder.lockCanvas() with a Rect describing the part of the screen requiring a repaint. Then it calls the gfx routines with the resulting Canvas...this is where I did a repaint() in AWT/Swing. The gfx routines are smart enough to just render what's in the clipRect. Perhaps I'm still stuck in AWT but I can't think of any way to make this thing run at an acceptable speed. I tried to coalesce the gfx calls but that didn't work either. Any thoughts?

SurfaceView should be fast enough. All the drawing routines are smart enough to not do anything if outside of the clip region but you can cull the calls ahead of time if you want. It looks like you need to profile your app and see where you spend too much time.