I am trying to write an application to trigger the Android camera at a fixed given time interval. I was testing it with TimerTask, however I read that I am not suppose to trigger the camera again until the JPEG is ready. Is there a method of triggering the camera at a fixed interval and letting the JPEG come when its ready and then trigger it again and let that next JPEG come when it's read, etc, without causing some sort of Heap Overflow? Is there a way to do this camera2?
Here are the relevant methods I have so far:
PictureCallback onPicTake=new PictureCallback() {
#Override
public void onPictureTaken ( byte[] bytes, Camera camera){
Log.d("data size",""+bytes.length);
Log.d("taken", "taken");
new SaveImageTask(getStorage()).execute(bytes);
resetCam();
}
};
Camera.ShutterCallback onShutter=new Camera.ShutterCallback()
{
#Override
public void onShutter () {
AudioManager mgr = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
mgr.playSoundEffect(AudioManager.FLAG_PLAY_SOUND);
}
};
private class CameraTrigger extends TimerTask{
public void run(){
mCamera.takePicture(onShutter, null, onPicTake);
}
}
preview.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View arg0) {
timer = new Timer();
timer.schedule(new CameraTrigger(), 0, 1000);
}
});
private void resetCam() {
mCamera.startPreview();
preview.setCamera(mCamera);
}
There is nothing terribly wrong in your code, as long as you know for sure that onPictureTaken() will not take more than 1000 ms.
One optimization that I would suggest is counterintuitively not to save the picture in a background task, but rather do it on the callback thread.
The reason is that the huge memory chunk of bytes cannot be easily garbage collected this way. From the point of view of JVM, the following pattern does not put a burden on garbage collector:
byte[] bytes = new byte[1Mb];
fill bytes with something
onPreviewFrame(bytes);
nobody needs bytes again
bytes memory is reclaimed
But if there are outstanding references to bytes, it may be hard for GC to decide, and you can see spikes of CPU usage, app not responding, and eventually, even TimeTask callbacks delayed.
Note that it is not healthy to use onPictureTaken() on the main (UI) thread. To keep the camera callbacks in background, you need to open the camera on a secondary Looper thread (see this example).
Related
The app I'm developing is a Flappy Bird clone.
I'm using a surfaceView object in which I have a gameThread and inside of its run method I draw the various components of the game on the canvas.
Everything runs smoothly as long as I just draw Rects to represent the objects, but as soon as I added the first Drawables i noticed a little bit of a loss in smoothness. If I try to draw the background as a Drawable the game suffers very significant frame rate loss.
What I tried:
Using png and all different kinds of bitmap as assets
Resizing the asset to fit the canvas perfectly, thus avoiding a rescale
None of this had any tangible effect.
Basically:
If I only use drawRect: 60fps
If I draw the back with drawRect and the other components with drawable.draw(canvas): 57fps
If I draw everything (background included) with drawable.draw(canvas): 15fps
Somewhat relevant code:
public class CannonView extends SurfaceView
implements SurfaceHolder.Callback {
private CannonThread cannonThread; // controls the game loop
private Drawable background;
// constructor
public CannonView(Context context, AttributeSet attrs) {
super(context, attrs); // call superclass constructor
getHolder().addCallback(this);
background= ResourcesCompat.getDrawable(getResources(), R.drawable.background, null);
}
public void newGame() {
background.setBounds(0,0, getScreenWidth(),getScreenHeight());
}
public void drawGameElements(Canvas canvas) {
background.draw(canvas);
}
public void stopGame() {
if (cannonThread != null)
cannonThread.setRunning(false); // tell thread to terminate
}
#Override
public void surfaceCreated(SurfaceHolder holder) {
if (!dialogIsDisplayed) {
newGame(); // set up and start a new game
cannonThread = new CannonThread(holder); // create thread
cannonThread.setRunning(true); // start game running
cannonThread.start(); // start the game loop thread
}
}
private class CannonThread extends Thread {
private SurfaceHolder surfaceHolder; // for manipulating canvas
private boolean threadIsRunning = true; // running by default
// initializes the surface holder
public CannonThread(SurfaceHolder holder) {
surfaceHolder = holder;
setName("CannonThread");
}
// changes running state
public void setRunning(boolean running) {
threadIsRunning = running;
}
// controls the game loop
#Override
public void run() {
Canvas canvas = null; // used for drawing
while (threadIsRunning) {
try {
// get Canvas for exclusive drawing from this thread
canvas = surfaceHolder.lockCanvas(null);
synchronized(surfaceHolder) {
drawGameElements(canvas);
}
}
finally {
if (canvas != null)
surfaceHolder.unlockCanvasAndPost(canvas);
}
}
}
}
}
It seems apparent that the dominant cause of the low frame rate is background.draw(). Switching to a Bitmap improves this somewhat, probably since it cached the output of draw(), and because it can be used with Canvas functions that are guaranteed not to need scaling (e.g., drawBitmap( Bitmap, float, float, Paint))
You also found that switching to RGB_565 as an intermediate format improves performance quite a bit, presumably because it throws away the alpha. (Otherwise, I would've expected this to be somewhat slower, b/c the format has to be converted back to RGBA_8888 as it's blitted into the SurfaceView.)
It's also apparent that Android won't let you go over 60fps. This is almost certainly because lockCanvas() takes part in a triple buffering scheme that throttles the drawing rate, to prevent you from submitting frames that could never be displayed (due to your device's fixed screen refresh rate of 60Hz).
This leaves the question of why you don't get a full 60fps, but something close to it. If drawGameElements() takes the same amount of time to run each time, and it's less than 16ms, then lockCanvas() should be throttling you, and no frames should ever get dropped (60fps continuously). It seems likely that there is a burble in the thread scheduler or something, and every so often, the CannonThread does not execute quickly enough to provide the frame before the triple-buffering scheme needs to page-flip. In this event, the frame must be delayed until the next screen refresh. You might try increasing CannonThread's thread priority, removing any extra processing in drawGameElements() that doesn't absolutely need to happen on CannonThread, or closing other apps running on your device.
As mentioned, OpenGL is the standard way of getting max sprite performance for games like these, because it is able to offload many operations to hardware. You may be approaching the performance limit of a drawBitmap()-based game.
I want to optimize my code. I can see in my sample camera app, I am creating thread for takePicture. Something like:
WAY 1
private void takePicture() {
mTakePictureThread = new Thread() {
run() {
camera.takePicture(cb, cb, ..);
}
}
mTakePictureThread.start();
}
Now I can do same thing with Handler too as below:
WAY 2
//consider mTakePictureThread is started in onCreate()
//and mTakePictureHandler is global variable
private void takePicture() {
mTakePictureHandler.sendMessage(1);
}
private class TakePictureThread extends Thread {
#override
public void run() {
Looper.prepare();
mTakePictureHandler = new Handler() {
public void handlerMessage(Message msg) {
int what = msg.what;
switch(what) {
case 1:
camera.takePicture(...);
break;
default:
break;
}
}
}
Looper.loop();
}
}
Considering takePicture is called many times. So in case 1, new thread will be created as many time as takePicture is called means every time new thread will be created.
But in second case, I can always hold one handler and call takePicture just by passing a message through handler.
So my query is, which one is better considering I am calling takePicture many time. In terms of performance and memory.
I have seen people using WAY 1 always(couldn't get satisfied reply why). So can anyone explain Pros and Cons of both approach and when should I follow which approach?
The second way queries your messages and will only use one thread to take the pictures. So if your camera.takePicture(..) is not a blocking call this will result in unneccessary waiting times for your threads.
The first way can handle different query counts in the same time if your camera can broadcast the actual image.
You can find a good explanation of loopers in the answer on this question looper purpose. A looper is better if you want one thread to handle messages in a sequential manner.
I'm working on a presentation app, which displays different images. There I wanted to to let the presentation slide through my List of images, video and pdf files, after a short amount of time.
I start my different views through intents, startActivityForResult(intent, RESULT_OK);
Starting videos and closing videos was not an issue. I used onPreparedListener and setOnCompletionListener and everything worked like a charm.
With pictures however, this was completely diffrent.
I created a new Thread in my ImageView and did put that thread to sleep(), after that I called the setresult() method and finish(). But instead of waiting, the picture wasn't shown at all and the presentation was stuck there, without setting the result and finishing the activity.
So I started searching for some explanation of time in android and found this explanation:
Explanation
I read through it and tried to get a good grasp on whats explained there. But the more I thought about it, the more I got insecure, which is the best way to implement the waiting behavior for my intended purpose.
So instead of some code, I am much more interested in, what you would advise me to use and why with a, if possible, detailed explanation.
elapsedRealtime()?
uptimeMillis()?
System.currentTimeMillis()?
From android docs:
• System.currentTimeMillis() is the standard "wall" clock (time and date) expressing milliseconds since the epoch. The wall clock can be set by the user or the phone network (see setCurrentTimeMillis(long)), so the time may jump backwards or forwards unpredictably. This clock should only be used when correspondence with real-world dates and times is important, such as in a calendar or alarm clock application. Interval or elapsed time measurements should use a different clock. If you are using System.currentTimeMillis(), consider listening to the ACTION_TIME_TICK, ACTION_TIME_CHANGED and ACTION_TIMEZONE_CHANGED Intent broadcasts to find out when the time changes.
• uptimeMillis() is counted in milliseconds since the system was booted. This clock stops when the system enters deep sleep (CPU off, display dark, device waiting for external input), but is not affected by clock scaling, idle, or other power saving mechanisms. This is the basis for most interval timing such as Thread.sleep(millls), Object.wait(millis), and System.nanoTime(). This clock is guaranteed to be monotonic, and is suitable for interval timing when the interval does not span device sleep. Most methods that accept a timestamp value currently expect the uptimeMillis() clock.
• elapsedRealtime() and elapsedRealtimeNanos() return the time since the system was booted, and include deep sleep. This clock is guaranteed to be monotonic, and continues to tick even when the CPU is in power saving modes, so is the recommend basis for general purpose interval timing.
If the time interval, you're going to measure, is relatively short, you can use pretty much any method which gives you correct time. I prefer currentTimeMillis(). In case the time interval is really long, the recommended method is to use elapsedRealtime().
Also, if you only want to do something with a delay, simply use: http://developer.android.com/reference/android/os/Handler.html#postDelayed(java.lang.Runnable, long) . It's simple and works great.
Simplest way to achieve that is CountDownTimer
private final class CountDownTimerImpl extends CountDownTimer {
//5 sec.
private static final long TIME_INTERVAL = 5000;
private final ImageView imageView;
private final List<Drawable> images;
public CountDownTimerImpl(ImageView imageView, List<Drawable> images) {
super(TIME_INTERVAL, TIME_INTERVAL);
this.imageView = imageView;
this.images = images;
//set first image from images array to imageView
imageView.setImageDrawable(images.get(0));
}
//this method is executed after TIME_INTERVAL (5 sec.)
public void onFinish() {
//remove drawable from imageView
imageView.setImageDrawable(null);
//remove this drawable from array
images.remove(0);
//if array is not empty start another count down
if (!images.isEmpty()) {
new CountDownTimerImpl(imageView, images).start();
}
}
public void onTick(long millisUntilFinished) {
//nothing to do here
}
}
You should start this CountDownTimer by:
new CountDownTimerImpl(imageView, images).start();
where images is of course an drawables array of your presentation images.
I have no time to test this solution but it should work - if not please leave a comment and I will update it later.
You can use TimerTask
int counter=0;
final Handler handler = new Handler();
Timer ourtimer = new Timer();
TimerTask timerTask = new TimerTask() {
public void run() {
handler.post(new Runnable() {
public void run() {
counter++;
//you can do stuffs here say like if (counter==15) { do something}
}
});
}};
ourtimer.schedule(timerTask, 0, 1000);
You can do this in a different way writing a callback module
Create a activity call it BaseActivity and let all you activities to extend it
Now declare a method call is void callback(){} keep the body empty
now in onCreate create a timer as above and call the callback function your code will look like
onCreate(){
final Handler handler = new Handler();
Timer callTimer = new Timer();
TimerTask timerTask = new TimerTask() {
public void run() {
handler.post(new Runnable() {
public void run() {
callback();
}
});
}};
callTimer.schedule(timerTask, 0, 1000);
}
Now in you activity override the callback method which will be called after the time you specified in timer,
Ex
Class a extends BaseActivity(){
#Override
onCreate(){
// playVideo
}
#Override
void onCallBack(){
//navigate to another activity
}
}
I am using OpenCV to attempt to do some live video processing. Since the processing is fairly heavy, it delays the output frames significantly, making the live stream look choppy.
I'd like to offload some of the processing into an AsyncTask. I've tried it and it actually makes the video much smoother. However, it ends up starting a large amount of Tasks at once, and then they will slowly start returning with some results.
Is there any way to slow this down, and wait for a result, either by using Synchronize statements, or some other method?
On each camera frame, I start one of these tasks. DoImgProcessing does the long processing and returns a string result.
private class LongOperation extends AsyncTask<Mat, Void, String> {
#Override
protected String doInBackground(Mat... params) {
Mat inputFrame = params[0];
cropToCenter(inputFrame);
return doImgProcessing(inputFrame);
}
#Override
protected void onPostExecute(String result) {
Log.d(TAG, "on post execute: "+result);
}
#Override
protected void onPreExecute() {
Log.d(TAG, "on pre execute");
}
}
public Mat onCameraFrame(Mat inputFrame) {
inputFrame.copyTo(mRgba);//this will be used for the live stream
LongOperation op = new LongOperation();
op.execute(inputFrame);
return mRgba;
}
I would do something like that :
// Example value for a timeout.
private static final long TIMEOUT = 1000L;
private BlockingQueue<Mat> frames = new LinkedBlockingQueue<Mat>();
Thread worker = new Thread() {
#Override
public void run() {
while (running) {
Mat inputFrame = frames.poll(TIMEOUT, TimeUnit.MILLISECONDS);
if (inputFrame == null) {
// timeout. Also, with a try {} catch block poll can be interrupted via Thread.interrupt() so not to wait for the timeout.
continue;
}
cropToCenter(inputFrame);
String result = doImgProcessing(inputFrame);
}
}
};
worker.start();
public Mat onCameraFrame(Mat inputFrame) {
inputFrame.copyTo(mRgba);//this will be used for the live stream
frames.put(inputFrame);
return mRgba;
}
The onCameraFrame puts the frame on the Queue, the worker Thread polls from the Queue.
This decorelate the reception and the treatment of the frame. You can monitor the growth of the Queue using frames.size().
This is a typical producer-consumer example.
If you're doing this on each frame, it sounds like you need a thread instead. An AsyncTask is for when you want to do a one-off activity on another thread. Here you want to do it repeatedly. Just create a thread, and when it finishes a frame have it post a message to a handler to run the post step on the UI thread. It can wait on a semaphore at the top of its loop for the next frame to be ready.
I'm implementing a SurfaceView subclass, where I run a separate thread to draw onto a SurfaceHolders Canvas.
I'm measuring time before and after call to lockCanvas(), and I'm getting from about 70ms to 100ms.
Does anyone could point me why i'm getting such high timings?
Here the relevant part of the code:
public class TestView extends SurfaceView implements SurfaceHolder.Callback {
....
boolean created;
public void surfaceChanged(SurfaceHolder holder, int format, int width,
int height) {
mThread = new DrawingThread(mHolder, true);
mThread.onWindowResize(width, height);
mThread.start();
}
public void surfaceCreated(SurfaceHolder holder) {
created = true;
}
public void surfaceDestroyed(SurfaceHolder holder) {
created = false;
}
class DrawingThread extends Thread {
public void run() {
while(created) {
Canvas canvas = null;
try {
long t0 = System.currentTimeMillis();
canvas = holder.lockCanvas(null);
long t1 = System.currentTimeMillis();
Log.i(TAG, "Timing: " + ( t1 - t0) );
} finally {
holder.unlockCanvasAndPost(canvas);
}
}
You're creating a thread every time the surface is changed. You should start your thread in surfaceCreated and kill it in surfaceDestroyed. surfaceChanged is for when the dimensions of your surface changes.
From SurfaceView.surfaceCreated docs:
This is called immediately after the surface is first created. Implementations of this should start up whatever rendering code they desire. Note that only one thread can ever draw into a Surface, so you should not draw into the Surface here if your normal rendering will be in another thread.
The multiple threads are probably getting you throttled. From SurfaceHolder.lockCanvas docs:
If you call this repeatedly when the Surface is not ready (before Callback.surfaceCreated or after Callback.surfaceDestroyed), your calls will be throttled to a slow rate in order to avoid consuming CPU.
However, I'm not convinced this is the only problem. Does surfaceChanged actually get called multiple times?
This is related to how lockCanvas is actually implemented in the android graphic framework.
You should probably already know that lockCanvas will return you an free piece of memory that you will be used to draw to. By free, it means this memory has not be used for composition and not for display. Internally, simply speaking, an SurfaceView is backed up by double buffer, one is for drawing , one is for composition/display. This double buffer is managed by BufferQueque. If composition/display is slow than drawing, we have to wait until we have free buffer available.
read this:
What does lockCanvas mean (elaborate)