Develop Reference

Avoid banding using Android's camera2 API

I'm trying to achieve the optimal settings for decoding barcodes using Android's tricky Camera2 API, using TEMPLATE_STILL_CAPTURE and CONTROL_SCENE_MODE_BARCODE WHILE playing with all the other settings like FPS, AF and AE.
So far however, I have been unable to remove the banding seen when reading barcodes from screens.
What would be the way to remove, or reduce, banding when using the Camera2 API taking pictures of screens?

You should set the anti-banding mode, see CONTROL_AE_ANTIBANDING_MODE

...
.setCaptureRequestOption(
CaptureRequest.CONTROL_AE_ANTIBANDING_MODE,
CameraMetadata.CONTROL_AE_ANTIBANDING_MODE_AUTO
)
.setCaptureRequestOption(
CaptureRequest.CONTROL_MODE,
CameraMetadata.CONTROL_MODE_AUTO
)
.setCaptureRequestOption(
CaptureRequest.CONTROL_SCENE_MODE,
CameraMetadata.CONTROL_SCENE_MODE_BARCODE
)
...
https://developer.android.com/reference/android/hardware/camera2/CaptureRequest#CONTROL_AE_ANTIBANDING_MODE

Depending on the technology used by the screen, the screen itself may be flickering at some rate. Unfortunately, the screen is also likely to be bright, meaning the camera has to reduce its exposure time so it doesn't overexpose.
A shorter exposure time makes it harder to apply anti-banding, since anti-banding normally involves having an exposure time that's a multiple of the flicker period. That is, if the flicker is at 100 hz (10 ms period), you need an exposure time that's a multiple of 10 ms. But if the screen is so bright that the camera needs a 5 ms exposure, then there's no way to cancel out the flicker.
In addition, most camera flicker detectors only handle 50 and 60-hz main power flicker rates. Increasingly they do more, since LED lighting has flicker rates that often unrelated to the power line frequencies, but if the device only handles 50 and 60hz, it probably can't compensate for most displays that flicker.
Besides turning on antibanding mode (which should be on by default anyway), you could try increasing exposure compensation to make the image brighter, which may give the device enough room to set an exposure time for flicker reduction. But beyond that there's not much you can do, unfortunately.

How to fix the frame rate of camera in Android phone

I want to fix the frame rate of camera preview in Android, i.e., 20fps, or 30 fps. However, we find the frame rate is unstable.
In the android document, it is said that the frame rate is fluctuated between the minimum frame rate and the maximum one which are defined in getSupportedPreviewFpsRange.
https://developer.android.com/reference/android/hardware/Camera.Parameters.html#getSupportedPreviewFpsRange%28%29
My questions are:
1) Which factors influence the frame rate? exposure time, white balance, frame resolution, background CPU loading, and etc.?
2) Is there any method to fix the frame rate by customised above factors?
3) In my project, higher frame rate is better. If the frame rate is unstable in the end. Can I increase the minimum frame rate? or fix the minimum frame rate?
4) It seems that the video taking is somewhat different with preview model, Can I fix the frame rate or minimum frame rate of video taking in Android?
Finally, we found that IOS can fix the frame rate using videoMinFrameDuration and
videoMaxFrameDuration.
Thanks.

First of all, please note that the camera API that you ask about was deprecated more than 3 years ago. The new camera2 API provides much more control over all aspects of capture, including frame rate.
Especially, if your goal is smooth video recording. Actually, the MediaRecorder performs its job decently on older devices, but I understand that this knowledge has little practical value if for some reason you cannot use the MediaRecorder.
Usually, the list of supported FPS ranges includes fixed ranges, e.g. 30 fps, intended exactly for video recording. Note that you are expected to choose a compliant (recommended) preview (video) resolution.
Two major factors cause frame rate variations within the declared range: exposure adjustments and focus adjustments. To achieve uniform rate, you should disable autofocus. If your camera supports exposure control, you should lock it, too. Refrain from using exotic "scenes" and "effects". SCENE_MODE_BARCODE and EFFECT_MONO don't seem to cause problems with frame rate. Whitebalance is OK, too.
There exist other factors that cause frame rate fluctuations that are completely under your control.
Make sure that your camera callbacks do not interfere with, and are not delayed by the Main (UI) thread. To achieve that, you must open the camera on a secondary HandlerThread. The new camera2 API makes thread management for camera callbacks easier.
Don't use setPreviewCallback() which automatically allocates pixel buffers for each frame. This is a significant burden for garbage collector, which may lock all threads once in a while for major cleanup. Instead, use setPreviewCallbackWithBuffer() and preallocate just enough pixel buffers to keep it always busy.
Don't perform heavy calculations in the context of your onPreviewFrame() callback. Pass all work to a different thread. Do your best to release the pixel buffer as early as possible.

Even for the old camera API, if the device lists a supported FPS range of (30, 30), then you should be able to select this range and get consistent, fixed video recording.
Unfortunately, some devices disregard your frame rate request once the scene conditions get too dark, and increase exposure times past 1/30s. For many applications, this is the preferable option, but such applications should simply be selecting a wider frame rate range like (15, 30).

Android OpenGL occasional stutter

I've written an OpenGL ES 2 app using NativeActivity that renders around 6 textured triangles that you can drag around using touch input. The app is rendering as fast as possible, calling glClear() before and eglSwapBuffers() after rendering. As expected the frame rate is capped to the display frequency of around 60 fps and the scene moves around smoothly. However, after some time (10 to 60 seconds) the movement starts to stutter even though the frame rate is still 60 fps. The device I'm using is OnePlus 3.
At first I suspected the culprit was battery saving mode, but investigation revealed no correlation between cpus' frequencies and the stutter.
I did notice something interesting though, the stutter starts when queueBuffer() execution time goes up.
This is the systrace before stutter:
This is the systrace after stutter:
This is the systrace during the transition from before to after:
How do I stop the app from stuttering?

I still don't know the exact cause of the stuttering, but I managed to find a work-around: Only render frames on Choreographer.postFrameCallback and set eglPresentationTimeANDROID to two vsyncs in the future. This avoids filling up the buffer queue, which seems to be one of the causes of the stuttering.
As a bonus it also improved the touch input latency a little.
More info here: Minimize Android GLSurfaceView lag

render camera in openles 2.0 with forced delay

Is it possible to render camera in openles 2.0 with forced delay? for example delay is 5 frames? For now i'm rendering output with surfacetexture and opengles-2.0.

If you're receiving 30 frames per second, and you want to introduce a constant delay of 5 frames, you need to do something with those 5 frames.
SurfaceTexture doesn't do (much) buffering. Because the producer and consumer endpoints are in the same process, it would be very easy to cause a deadlock by overrunning the consumer. So SurfaceTexture uses a BufferQueue in "async" mode, which means it drops frames if the consumer isn't ready for the next one.
So you'll need a way to do your own buffering, which means copying the data out of the "external" texture. One way to do this is to render the textures to a series of FBOs, from which you can render later. Be sure to keep an eye on the memory usage here -- a 1920x1080 32-bit ARGB image occupies about 8MB, so keeping 5 around increases your memory footprint by 40MB.
Some loss of color fidelity may result, as a YUV-to-RGB conversion will be involved, but if your eventual target is RGB then this shouldn't matter.
So it's possible, but it's not easy. What are you trying to accomplish?

How can I fix jumpy Android rendering performance due to GPU throttling? I don't need a high framerate, just a consistent one

I'm developing a GL live wallpaper that uses very little CPU and only modest GPU. On my older test phone, it can run at a full 58fps or so most of the time. But occasionally the effects ramp up, and then the render times jitter between 16ms and 50ms per frame. For example, it'll render several frames at 16ms, slide up to 50ms over a dozen frames or so, render several more frames at 50ms, then slide back down to 16ms and repeat. I discovered that if I set the CPU governor to "performance" (or "conservative", curiously enough) instead of the default "ondemand" it'll render with full effects at full speed. Alternatively, if I leave the governor alone and insert a busy loop in my code (increment a variable 100,000 times per frame) that bumps my CPU usage up enough to transition to a higher clock rate and render smoothly as well.
So it seems on this phone my app is bottlenecked by the GPU, but only when it throttles down. Now, I wouldn't mind if the GLSurfaceView rendered at a slower rate according to the GPU clock, but my problem here is that I'm getting the bursts of alternating high and low frame rates which makes my animation look fluid/frameskippy/fluid/frameskippy/etc. several times per second. It seems like the GPU clock is ramping up and down like crazy?
I got a visible improvement by using RENDERMODE_WHEN_DIRTY and calling requestRender() on a strictly timed thread, but the darn GPU keeps ramping up and down. Why won't it either render as fast as it can at the slower clock, or just jump to and STAY AT the higher clock?
The best solution I've come up with so far is using a sliding window to detect the average frame update time, then applying the difference from the target frame time until the two values converge. The time between render updates is slower but at least it's roughly constant. So that works in theory, but it takes several seconds to reach a steady state and it looks bad in the meantime.
I think a third option might be to cannibalize the GLSurfaceView source and make a custom version. From what I understand, the blocking GL calls are made in there, so it would be much easier for me to time render calls and react accordingly. I'm not very comfortable attempting that though because there's a lot of code in there that I'd have to spend a lot of time understanding before I could even begin to mess with it. Plus I'd then have to worry about how well version X of GLSurfaceView plays with any version Y of Android.
So, with all that said, do I have any other options here? Is there an easier fix to this?

try fixing the frame rate by pausing the thread (thread sleep) for the remaining time to reach a constant frame rate.