If I use 'setPreviewDisplay' in the Camera, like the api below
public final void setPreviewDisplay (SurfaceHolder holder)
What kind of format will be passed to SurfaceHolder for display? Is it NY21 or YV12?
Thank you.
The exact format is platform-dependent, since the image frames sent to a Surface are not accessible to applications.
In fact, it may not be any of the formats listed in android.graphics.ImageFormat, since for efficiency, an entirely device-specific format may be in use, with different padding and stride requirements.
Do you have some use case in mind where knowing the format is important? Since the data is not accessible, I'm curious as to what that is.
Related
I am working with the camera2 API in android and am trying to understand this code I am using. Part of the code goes like this:
previewReader = ImageReader.newInstance(previewSize.getWidth(), previewSize.getHeight(),
ImageFormat.YUV_420_888, 4);
previewReader.setOnImageAvailableListener(imageListener, backgroundHandler);
// This adds another output surface but not sure where this surface comes from..
previewRequestBuilder.addTarget(previewReader.getSurface());
imageListener is an object from another class that implements android.media.ImageReader.OnImageAvailableListener and backgroundHandler is just a background thread. I am not including code for these two or previewRequestBuilder as they do not seem to be important for understanding my question.
I have searched extensively but it just seems like some magic happens and previewReader finds some surface somewhere, somehow. According to the documentation, what getSurface() does is to:
Get a Surface that can be used to produce Image for this ImageReader
Can anyone explain where it gets this?
That Surface belongs to the ImageReader; it was created in the native equivalent of the ImageReader's constructor, and is (effectively) an ImageReader private member, with a getter.
Here is the line in the native constructor that sets up the IGraphicBufferProducer (gbProducer), which is basically the native equivalent of a Surface.
Here is where you can see that the native code uses that same member to form the return value from getSurface()/nativeGetSurface() (you may have to trace through the code a bit, but it's all there).
So that's the literal answer to your question. But maybe you were asking because it isn't clear why the camera doesn't create the Surface, and force you to give it to the ImageReader, instead: A Surface is a complex object (actually, a buffer queue), and shouldn't be thought of as a simple, pre-allocated bitmap. At the time the capture takes place, the camera pipeline will communicate with its output Surfaces, and set up the correct dimensions and color planes and so forth. (Note that you can add multiple targets via addTarget(); the camera can use each of them.) All the camera needs to know is where it's going to send its output; it doesn't need to create the output Surface itself.
API level 21 introduced camera2, with it setRepeatingRequest and setRepeatingBurst. I have read the doc here, but still cannot catch the difference between the two. Any idea?
Well, you'll notice that the constructors for these two methods are slightly different. setRepeatingBurst's first argument is List<CaptureRequest>, and setRepeatingRequests's is just a CaptureRequest.
According to the docs,
setRepeatingBurst
With this method, the camera device will continually capture images, cycling through the settings in the provided list of CaptureRequests, at the maximum rate possible.
setRepeatingRequest
With this method, the camera device will continually capture images using the settings in the provided CaptureRequest, at the maximum rate possible.
So, setRepeatingBurst can be used to capture images with a list of different settings.
That's my best understanding, hope it helps!
Think of setRepeatingRequest as ONE CaptureRequest with one set of settings to continually capture images.
Where as in setRepeatingBurst there is a list CaptureRequest and each "CaptureRequest" has its own setting to continually capture images.
Conclusion: setRepeatingBurst call is like making multiple setRepeatingRequest calls in one call.
I am getting the TotalCaptureResults object from the camera, using the Camera2 API in Android. I am using a preview, not a single image. Is there a way to get bytes[] from TotalCaptureResults?
Thank you.
Short answer: no.
All CaptureResults objects contain only metadata about a frame capture, no actual pixel information. The associated pixel data are sent to wherever you designated as the target Surface in your CaptureRequest.Builder. So you need to check with whatever Surface you set up, such as an ImageReader which will give you access to an Image output from the camera, which will give you access to the bytes[].
this is regarding Android's Camera2 APIs. Since capture result and output frame are produced asynchronously, one could get capture result much before the actual frame. Is there a good way to associate produced frame with the corresponding capture result ?
Assuming you are talking about a frame that is sent to an ImageReader or SurfaceTexture upon capture (as in the ubiquitous camera2basic example), the trick is to compare unique timestamps identifying the images.
Save the TotalCaptureResult somewhere accessible when it is available in your CameraCaptureSession.CaptureCallback's onCaptureComplete(...) call.
Then, when the actual image is available via your ImageReader.OnAvailableListener or SurfaceTexture.OnFrameAvailableListener, get the image's timestamp:
Long imageTimestamp = Long.valueOf(reader.acquireNextImage().getTimestamp()); or
Long imageTimestamp = Long.valueOf(surfaceTexture.getTimestamp()), respectively.
Compare timestamps with: imageTimestamp.equals(totalCaptureResult.get(CaptureResult.SENSOR_TIMESTAMP));
Notes:
The timestamp may not be an actual true system timestamp for your device, but it is guaranteed to be unique and monotonically increasing, so it works as an ID.
If you are sending the image to a SurfaceHolder or something else instead, you're out of luck as only the pixel information gets sent, not the timestamp present in the Image object. I'm not sure about the other places you can send a frame, e.g., MediaRecorder or Allocation, but I think not.
You probably need to add each new TotalCaptureResult to a growing set as they are generated, and then compare an incoming image's timestamp against all of these, because of the asynchronous nature you noted. I'll let you figure out how to do that as you see fit.
I had to solve a similar situation (sync frames across surfaces); Sumner's solution (.getTimestamp() of the respective received Image object) did the trick for me for SurfaceTexture and ImageReader.
Just a quick note on other surfaces (which, as pointed out, don't give you an Image object): at least for MediaCodec, the BufferInfo object received by the onOutputBufferAvailable callback has a presentationTimeUs, which is "derived from the presentation timestamp passed in with the corresponding input buffer" and, at least for me, appears to match the timestamps from other surfaces. (Note the different unit though.)
I'd like to use MediaCodec to encode the data coming from the camera (reason: it's more low-level so hopefully faster than using MediaRecorder). Using Camera.PreviewCallBack, I capture the data from the camera into a byte-buffer, in order to pass it on to a MediaCodec object.
To do this, I need to fill in a MediaFormat-object, which would be fairly easy if I knew the MIME-code of the data coming from the camera. I can pick this format using setPreviewFormat() choosing one of the constants declared in te ImageFormat-class.
Hence my question: given the different options provided by the ImageFormat-class to set the camera preview-format, what are the corresponding MIME-type codes?
Thanks a lot in advance.
See example at https://gist.github.com/3990442. You should set MIME type of what you want to get out of encoder, i.e. "video/avc".