I'm building an application on Android to take frames from the camera, process them, and then display the frame on a surfaceView, as well as drawing on the SurfaceView via the canvas and drawbitmap and all.
Just to check, is SurfaceView and Bitmaps and Canvases the best way to do it ? I'm after speed.
Assuming the answer to the above is Yes, the question would be: Where should I place the following function
camera_object.setPreviewCallback(new PreviewCallback()
public void onPreviewFrame(byte[] data, Camera camera){
should I place it in onCreate() or should I place it in surfaceCreated() or surfaceChanged() ?
I declared my mainactivity class as follows:
public class MainActivity extends Activity implements SurfaceHolder.Callback, Camera.PreviewCallback
{
and in that class Eclipse forces me to create an override function for onpreviewframe in the MainActivity class as follows
public void onPreviewFrame(byte[] data, Camera camera){
}
but it never gets called. Should I try to use this function ? is it better to use it ? or is it just an Eclipse thing ?
Please advise
Are you calling setPreviewDisplay(), startPreview() and setPreviewCallback(this) from the app? Without that you will not get any calls to onPreviewFrame(). In fact if you are using SurfaceView, then the callback preview buffers are a copy of the actual buffers that are being displayed on the screen. So if you want to display these copied buffers, you need to create a new view and overwrite it. This would be inefficient. I would suggest you use SurfaceTexture instead and use 'onFrameAvailable' callback to get the frames and then draw & display manually. An example of this can be found in the PanoramaActivity code of the default Android Camera App.
Without camera_object.setPreviewDisplay(surface_holder); you cannot receive camera callbacks; don't forget also
surface_view.setVisibility(View.VISIBLE);
surface_holder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
You can hide the camera preview under another view; on 3.0 and higher you can even push the surface out of the screen (display it below the bottom of the screen). I am not sure if the latter trick works on 2.3.6.
Related
My problem is this:
I have a seperate class which extends SurfaceView and my main activity has this view.
public class CameraPreview extends SurfaceView implements SurfaceHolder.Callback, Camera.PreviewCallback
I take camera preview's copy data from onPreviewFrame method, and save it to a Bitmap object, and this Bitmap object is used in onDraw method.
I set a flag to understand if the Bitmap is ready or not, onPreviewFrame makes the Bitmap ready, and onDraw takes this Bitmap and draws on the canvas.
But the problem is, I want to filter this Bitmap image, but even with no filter I get 5-10 fps with LG G4 device.
What should I do to improve performance? At this moment I haven't used any AsyncTask or Threads, but I'm not sure those alone will be enough.
Any suggestions?
Thank you.
I have an application that is using the Android port of GPUImage as the OpenGL Renderer and manager of several filters.
It currently does not have a video implementation, so I am trying to adapt the RecordFBOActivity from the Google grafika repository to work with the GPUImage architecture.
The base GPUImage class manages the GLContext and GLSurfaceView, and the GPUImageRenderer class implements the Renderer class.
This is the class where I am trying to adapt the RenderThread from the RecordFBOActivity of grafika. There are a few problems.
First, in the preparegl() method, I am passing a SurfaceTexture instead of a Surface, as GPUImage doesn't use the SurfaceHolder at all (I think I can implement it, but am trying not to change the base code too much, as i would like to push back my implementation to the aforementioned repo). I know that WindowSurface.java has an overloaded method to construct a WindowSurface from a SurfaceTexture as well as a Surface, but if I do this the mSurface iVar is always null, as I never have a surface to pass to it, which causes a NPE in the makeCurrent() method of recording.
Second, GPUImage attaches itself to a GLSurfaceView, not a SurfaceView like the grafika example uses, so I'm a little uncertain if there are any low level inconsistencies that may be causing conflicts for me...
Third, and I think this is the main issue, at least at the moment, is that I can't seem to reconcile the camera preview of GPUImage with the WindowSurface of grafika. If I comment out the prepareGl() method, the setUpSurfaceTexture() of GPUImage sets the preview texture of the camera from the SurfaceTexture that is created by glGenTextures() and the preview works fine.. as well as being attached to the filter render chain. However, if I try to call the prepareGL() method, and pass the exact same SurfaceTexture to the constructor of mWindowSurface, the camera service dies and i get a EGL_BAD_SURFACE error.
Long question, with a few moving parts, I know... Will attempt to edit/update as I can clarify issues and approaches to myself. But would love if anyone has any thoughts/interrogations... particularly #fadden :D
I was also trying to achieve the same thing and have tried what fadden has suggested. Tried to integrate CameraSurfaceRenderer functionality to GPUImageRenderer. The preview is fine but the recording is just a video with black frames. EGL14.eglGetCurrentContext() returns null for following call and my guess is if a new context is created it will not be same as what GPUImage might have
mVideoEncoder.startRecording(new TextureMovieEncoder.EncoderConfig(
mOutputFile, 640, 480, 1000000, EGL14.eglGetCurrentContext()));
#Jesses.co.tt were you able to achieve it?
(as I can't add comment it is added as an answer).
Environment:
Nexus 7 Jelly Bean 4.1.2
Problem:
I'm trying to make a Motion Detection application that works with RTSP using VideoView.
I wish that there was something like an onNewFrameListener
videoView.onNewFrame(Frame frame)
I've tried to get access to the raw frames of an RTSP stream via VideoView but couldn't find any support for that in the Android SDK.
I found out that VideoView encapsulates the Android's MediaPlayer class.
So i dived into the media_jni lib to try and find a way to access the raw frames, But couldn't find the byte buffer or whatever that represents a frame.
Question:
Anyone has an idea where or how can i find this buffer and get access to it ?
Or any other idea of implementing a Motion Detection over a VideoView ?
Even if it's sais that i need to recompile the AOSP.
You can extend the VideoView and override its draw(Canvas canvas) method.
Set your bitmap to the canvas received through draw.
Call super.draw() which will get the frame drawn onto your bitmap.
Access the frame pixels from the bitmap.
class MotionDetectorVideoView extends VideoView {
public Bitmap mFrameBitmap;
...
#Override
public void draw(Canvas canvas) {
// set your own member bitmap to canvas..
canvas.setBitmap(mFrameBitmap);
super.draw(canvas);
// do whatever you want with mFrameBitmap. It now contains the frame.
...
// Allocate `buffer` big enough to hold the whole frame.
mFrameBitmap.copyPixelsToBuffer(buffer);
...
}
}
I don't know whether this will work. Avoid doing heavy calculation in draw, start a thread there.
In your case I would use the Camera Preview instead the VideoView, if you are working with live motion, not recorded videos. You can use a Camera Preview Callback to catch everyframe captured by your camera. This callback implements :
onPreviewFrame(byte[] data, Camera camera)
Called as preview frames are displayed.
Which I think it could be useful for you.
http://developer.android.com/reference/android/hardware/Camera.PreviewCallback.html
Tell if that is what you are searching for.
Good luck.
I am creating an Android app to do some image processing techniques with the camera and it needs to be fast. This is the pseudo-code of how the entire system works:
1. loop while not finished
1.1 get image frame
1.2 process image for object detection
2. end loop
I actually have questions on the basics of the Camera class:
Is previewing the perceived image from the camera faster than no previews at all? The former means using SurfaceView to preview the image.
Let's say from the takePicture() method, can the image data array be obtained without the previews?
My real question is, what is the best way to obtain the image data (say, byte[] array) quickly and iteratively after processing the image (as stated on top)?
I planned to use takePicture() method to get the image data, but I need your opinion if this is the only way or if there other better ways.
You can setup a SurfaceView as the Camera's preview display and get the data of every preview frame using the PreviewCallback. This would be better than using takePicture if you don't need the high resolution that takePicture captures. In other words, if you want to capture images of lower quality at a faster rate, use PreviewCallback... if you want to capture images of higher quality at a very slow rate, use takePicture.
As for your questions, I don't think you can take pictures without using a preview display, but i could be wrong.
class MainActivity extends Activity implements Camera.PreviewCallback, SurfaceHolder.Callback {
...
public void surfaceCreated(SurfaceHolder holder) {
camera = Camera.open();
camera.setPreviewCallback(this);
...
}
public void onPreviewFrame(byte[] data, Camera camera) {
// image data contained in data... do as you wish
}
}
I am doing some image processing of previews captured by the camera. It is cpu consuming task and I have to stop preview to make it faster. Before new frame is being processed I am calling Camera.stopPreview() and after Camera.startPreview().
However, I would like to have last captured frame displayed on SurfaceView after stopping the preview. It works 'out of the box' on 2.3 devices, however, SurfaceView gets black after calling Camera.stopPreview() on older versions of SDK. Does anyone know what has changed and what to do?
Yes, this was an improvement of the 2.3.
I had this problem in 2.2 as well, there was no way to work on a preview image despite the fact this was theoretically possible according to the API. To solve this I had to actually take a picture using Camera.takePicture(null, null, Camera.PictureCallback myCallback) (see info here) and then to implement a callback to handle the taken picture. The instance of the class that implements this callback is actually the parameter to pass to Camera.takePicture() and the callback method itself looks like this:
public void onPictureTaken(byte[] JPEGData, Camera camera) {
final Bitmap bitmap = createBitmapFromView(JPEGData);
// do something with the Bitmap
}
Doing that way prevents the picture to be saved on the external storage with the regular pictures taken with the camera application. Should you need to serialize the Bitmap you'll have to do it explicitely. But it doesn't prevent the camera's trigger sound from being emitted.
Camera.takePicture() has to be called wile the preview is running. stopPreview() can be called right after.
One thing to be careful with /!\:
Camera.takePicture() is not reentrant (at all). The callback must have returned before any subsequent call of Camera.takePicture(). This was freezing my phone, I had to shutdown and restart it before it to be usable again. As the action was triggered by a button, on my side, I had to shield it with a boolean:
if (!mPictureTaken) {
mPictureTaken = true; // absolutely NOT reentrant. Any double click sticks the phone otherwise.
mCameraView.takePicture(callback);
}