I use the new CameraX API for taking a picture like this :
imageButton.setOnClickListener{
val file = File(externalMediaDirs.first(),
"${System.currentTimeMillis()}.jpg")
imageCapture.takePicture(executor, object :
ImageCapture.OnImageCapturedListener() {
override fun onCaptureSuccess(
image: ImageProxy,
rotationDegrees: Int)
{
// DO I NEED TO USE 'image' TO ACCESS THE IMAGE DATA FOR MANIPULATION (e.g. image.planes)
}
override fun onError(
imageCaptureError: ImageCapture.ImageCaptureError,
message: String,
cause: Throwable?
) {
val msg = "Photo capture failed: $message"
Log.e("CameraXApp", msg, cause)
}
})
imageCapture.takePicture(file, executor,
object : ImageCapture.OnImageSavedListener {
override fun onError(
imageCaptureError: ImageCapture.ImageCaptureError,
message: String,
exc: Throwable?
) {
val msg = "Photo capture failed: $message"
Log.e("CameraXApp", msg, exc)
}
override fun onImageSaved(file: File) {
val msg = "Photo capture succeeded: ${file.absolutePath}"
Log.d("CameraXApp", msg)
}
}
}
I want to apply some image processing with Renderscript when the image is captured. But I dont know how to access the pixels of the image that is captured.
Can someone provide a solution ?
I tried it with image.planes within the onCaptureSuccess() callback (see comment)
I must admit that I am new to this and do not know really what planes are. Before this, I only worked with Bitmaps ( doing some image processing on Bitmaps with Renderscript). Is there a way to turn a frame/image into a Bitmap and apply some sort image processing on it "on the fly" before saving it as a file ? If yes, how ? The official CameraX guidelines are not very helpful when it comes to this.
This is probably very late answer but the answer is in ImageCapture callback.
You get an Image with
val image = imageProxy.image
Notice this will only work when you have images in YUV format, in order to configure your camera with that you can do it with
imageCapture = ImageCapture.Builder()
.setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
.setBufferFormat(ImageFormat.YUV_420_888)
...
.build()
Now can get image.planes
The default image format is JPEG
and you can access the buffer with
buffer: ByteBuffer = imageProxy.image.planes[0].getBuffer()
and if you plan on setting it to YUV
val image = imageProxy.image
val yBuffer = image.planes[0].buffer // Y
val uBuffer = image.planes[1].buffer // U
val vBuffer = image.planes[2].buffer // V
Related
I'm an android developer beginner and i try to use CameraX.
Is it possible to open the native camera and take a picture instead of build a page with preview and custom button to take a photo ?
I've read multiple article / tutorial but cannot find the solution.
Thanks for helping me
Using the example here, with a few tweaks - I was able to take a photo without any preview.
First we start the camera using ImageCapture use case:
private fun startCamera() {
val cameraProviderFuture = ProcessCameraProvider.getInstance(this)
cameraProviderFuture.addListener({
// Used to bind the lifecycle of cameras to the lifecycle owner
val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()
imageCapture = ImageCapture.Builder()
.build()
// Select back camera as a default
val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA
try {
// Unbind use cases before rebinding
cameraProvider.unbindAll()
// Bind use cases to camera
cameraProvider.bindToLifecycle(
this, cameraSelector, imageCapture)
} catch(exc: Exception) {
Log.e("Error", "Use case binding failed", exc)
}
}, ContextCompat.getMainExecutor(this))
}
Then we can take a photo by calling takePhoto method:
private fun takePhoto() {
// Get a stable reference of the modifiable image capture use case
val imageCapture = imageCapture ?: return
// Create time stamped name and MediaStore entry.
val contentValues = ContentValues().apply {
put(MediaStore.MediaColumns.DISPLAY_NAME, "sample_image")
put(MediaStore.MediaColumns.MIME_TYPE, "image/jpeg")
if(Build.VERSION.SDK_INT > Build.VERSION_CODES.P) {
put(MediaStore.Images.Media.RELATIVE_PATH, "Pictures/CameraX-Image")
}
}
// Create output options object which contains file + metadata
val outputOptions = ImageCapture.OutputFileOptions
.Builder(contentResolver,
MediaStore.Images.Media.EXTERNAL_CONTENT_URI,
contentValues)
.build()
// Set up image capture listener, which is triggered after photo has
// been taken
imageCapture.takePicture(
outputOptions,
ContextCompat.getMainExecutor(this),
object : ImageCapture.OnImageSavedCallback {
override fun onError(exc: ImageCaptureException) {
Log.e("Error", "Photo capture failed: ${exc.message}", exc)
}
override fun
onImageSaved(output: ImageCapture.OutputFileResults){
val msg = "Photo capture succeeded: ${output.savedUri}"
Toast.makeText(baseContext, msg, Toast.LENGTH_SHORT).show()
Log.d("Success", msg)
}
}
)
}
I've uploaded a working project to Github
I'm making a card holder app, the user is supposed to take a picture of his id, contacts, notes etc. so he can later use them digitally. Problem is how do I take a camera input and save it as an image inside the application so it stays there?
You can simply use the native Camera Application of your device to get the image and then save it to the device . Android Team has done much easy for developers to perform such task.
You need to make use of ActivityContracts and MediaStore to take the image and store it into your device respectively.
Step 1 :
First Generate a Uri for your Image , in the following manner
#RequiresApi(Build.VERSION_CODES.Q)
suspend fun createPhotoUri(source: Source): Uri? {
val imageCollection = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
MediaStore.Images.Media.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY)
} else {
MediaStore.Images.Media.EXTERNAL_CONTENT_URI
}
val dirDest = File(
Environment.DIRECTORY_PICTURES,
context.getString(R.string.app_name) + File.separator + "CAMERA"
)
val date = System.currentTimeMillis()
val fileName = "$date.jpg"
return withContext(Dispatchers.IO) {
val newImage = ContentValues().apply {
put(MediaStore.Images.Media.DISPLAY_NAME, fileName)
put(MediaStore.MediaColumns.RELATIVE_PATH, "$dirDest${File.separator}")
}
return#withContext context.contentResolver.insert(imageCollection, newImage)
}
}
Step 2:
Then when you want to capture the Image , then on the OnClickListener perform the following :
binding.takePictureButton.setOnClickListener {
viewLifecycleOwner.lifecycleScope.launch {
viewModel.createPhotoUri(Source.CAMERA)?.let { uri ->
actionTakePicture.launch(uri)
}
}
}
Step 3 :
The actionTakePicture ActivityContract is as follows :
private val actionTakePicture = registerForActivityResult(TakePicture()) { success ->
if (!success) {
Log.d(tag, "Image taken FAIL")
return#registerForActivityResult
}
Log.d(tag, "Image taken SUCCESS")
}
And you are done with capturing you Image and storing it .
Make sure you declare permission's before using the above code ,else it wont work .
The answer mentioned by #StefanoSansone can also be used . But the issue with that is you need to perfectly setup CameraX library and that might be tedious for your useCase . One should use library like CameraX if they want to have more control on Camera with other camera capabilities , when you application is more of a Camera Application . Else using the above method is perfectly fine . Saves one from tedious work .
If you are using Kotlin and Jetpack libraries, I suggest you to take a look to CameraX library.
You can use the takePicture method to take a photo with camera and save it in the storage.
A complete example can be found in the CameraX codelab
private fun takePhoto() {
// Get a stable reference of the modifiable image capture use case
val imageCapture = imageCapture ?: return
// Create time-stamped output file to hold the image
val photoFile = File(
outputDirectory,
SimpleDateFormat(FILENAME_FORMAT, Locale.US
).format(System.currentTimeMillis()) + ".jpg")
// Create output options object which contains file + metadata
val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile).build()
// Set up image capture listener, which is triggered after photo has
// been taken
imageCapture.takePicture(
outputOptions, ContextCompat.getMainExecutor(this), object : ImageCapture.OnImageSavedCallback {
override fun onError(exc: ImageCaptureException) {
Log.e(TAG, "Photo capture failed: ${exc.message}", exc)
}
override fun onImageSaved(output: ImageCapture.OutputFileResults) {
val savedUri = Uri.fromFile(photoFile)
val msg = "Photo capture succeeded: $savedUri"
Toast.makeText(baseContext, msg, Toast.LENGTH_SHORT).show()
Log.d(TAG, msg)
}
})
}
I had some difficulties last week with saving images.
I finally used sharedPreferences files and saved bitmap as text.
From what I've heard it's not a good practice, It's better to save in files and save the path.
However the code is very compact and it's working really well (in my case never have to load more than 4 pictures)
var bm= MediaStore.Images.Media.getBitmap(this.getContentResolver(), selectedImageUri)
bm.compress(Bitmap.CompressFormat.PNG, 100, baos) //bm is the bitmap object
val b = baos.toByteArray()
val encoded: String = Base64.encodeToString(b, Base64.DEFAULT)
editor.putString("backgroundBitmap",encoded)//put the bitmap as text in sharedPref files, use to back the bitmap in mainActivity
editor.commit()
I try to use new CameraX api and I got this error: When I capture the image this image stored with wrong rotation. For example I capture in portrait orientation but result image in landscape orientation.
There is my code:
private fun startCamera() {
val previewConfig = PreviewConfig.Builder().apply {
setTargetResolution(Size(textureView.width, textureView.height))
setTargetRotation(textureView.display.rotation)
}.build()
val imageCaptureConfig = ImageCaptureConfig.Builder().apply {
setCaptureMode(CaptureMode.MIN_LATENCY)
setTargetAspectRatio(RATIO_4_3)
setTargetRotation(textureView.display.rotation)
}.build()
imageCapture = ImageCapture(imageCaptureConfig)
val preview = Preview(previewConfig)
preview.setOnPreviewOutputUpdateListener { previewOutput ->
removeView(textureView)
addViewMatchParent(textureView, position = 0)
textureView.surfaceTexture = previewOutput.surfaceTexture
textureView.updateTransformForCameraFinderView()
}
(context as? LifecycleOwner)?.let { lifecycleOwner ->
CameraX.bindToLifecycle(lifecycleOwner, preview, imageCapture)
}
}
private fun capturePhoto() {
tempImageFile = generateTmpFile(false)
val executor = Executor { it.run() }
imageCapture.takePicture(tempImageFile!!, executor, object : OnImageSavedListener {
override fun onError(error: ImageCaptureError, message: String, exc: Throwable?) {
exc?.printStackTrace()
}
override fun onImageSaved(photoFile: File) {
post {
// load image into ImageView by Glide
showCapturedPhotoPreview(photoFile)
}
}
})
}
Please give me advise how can I fix it?
P.S. I tried to find solution so don't copy-paste first looking like something similar)
Update: I tried to do my CameraView like in this sample but in their case it works, in my - no)
Try it:
val imageCaptureConfig = ImageCaptureConfig.Builder().apply {
setCaptureMode(CaptureMode.MIN_LATENCY)
setTargetAspectRatio(RATIO_4_3)
// play with this line!
setTargetRotation(Surface.ROTATION_0)
setTargetRotation(textureView.display.rotation)
}.build()
I select line that fixes my problem again:
setTargetRotation(Surface.ROTATION_0)
I was working with CameraX and had hard time converting captured ImageProxy to Bitmap. After searching and trying, I formulated a solution. Later I found that it was not optimum so I changed the design. That forced me to drop hours of work.
Since I (or someone else) might need it in a future, I decided to post here as a question and post and answer to it for reference and scrutiny. Feel free to add better answer if you have one.
The relevant code is:
class ImagePickerActivity : AppCompatActivity() {
private var width = 325
private var height = 205
#RequiresApi(Build.VERSION_CODES.LOLLIPOP)
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_image_picker)
view_finder.post { startCamera() }
}
#RequiresApi(Build.VERSION_CODES.LOLLIPOP)
private fun startCamera() {
// Create configuration object for the viewfinder use case
val previewConfig = PreviewConfig.Builder().apply {
setTargetAspectRatio(Rational(1, 1))
//setTargetResolution(Size(width, height))
setLensFacing(CameraX.LensFacing.BACK)
setTargetAspectRatio(Rational(width, height))
}.build()
}
// Create configuration object for the image capture use case
val imageCaptureConfig = ImageCaptureConfig.Builder()
.apply {
setTargetAspectRatio(Rational(1, 1))
// We don't set a resolution for image capture instead, we
// select a capture mode which will infer the appropriate
// resolution based on aspect ration and requested mode
setCaptureMode(ImageCapture.CaptureMode.MIN_LATENCY)
}.build()
// Build the image capture use case and attach button click listener
val imageCapture = ImageCapture(imageCaptureConfig)
capture_button.setOnClickListener {
imageCapture.takePicture(object : ImageCapture.OnImageCapturedListener() {
override fun onCaptureSuccess(image: ImageProxy?, rotationDegrees: Int) {
//How do I get the bitmap here?
//imageView.setImageBitmap(someBitmap)
}
override fun onError(useCaseError: ImageCapture.UseCaseError?, message: String?, cause: Throwable?) {
val msg = "Photo capture failed: $message"
Toast.makeText(baseContext, msg, Toast.LENGTH_SHORT).show()
Log.e(localClassName, msg)
cause?.printStackTrace()
}
})
}
CameraX.bindToLifecycle(this, preview, imageCapture)
}
}
So the solution was to add extension method to Image and here is the code
class ImagePickerActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_image_picker)
}
private fun startCamera() {
val imageCapture = ImageCapture(imageCaptureConfig)
capture_button.setOnClickListener {
imageCapture.takePicture(object : ImageCapture.OnImageCapturedListener() {
override fun onCaptureSuccess(image: ImageProxy?, rotationDegrees: Int) {
imageView.setImageBitmap(image.image?.toBitmap())
}
//.....
})
}
}
}
fun Image.toBitmap(): Bitmap {
val buffer = planes[0].buffer
buffer.rewind()
val bytes = ByteArray(buffer.capacity())
buffer.get(bytes)
return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}
Slightly modified version. Using the inline function use on the Closable ImageProxy
imageCapture.takePicture(
object : ImageCapture.OnImageCapturedListener() {
override fun onCaptureSuccess(image: ImageProxy?, rotationDegrees: Int) {
image.use { image ->
val bitmap: Bitmap? = image?.let {
imageProxyToBitmap(it)
} ?: return
}
}
})
private fun imageProxyToBitmap(image: ImageProxy): Bitmap {
val buffer: ByteBuffer = image.planes[0].buffer
val bytes = ByteArray(buffer.remaining())
buffer.get(bytes)
return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}
Here is the safest approach, using MLKit's own implementation.
Tested and working on MLKit version 1.0.1
import com.google.mlkit.vision.common.internal.ImageConvertUtils;
Image mediaImage = imageProxy.getImage();
InputImage image = InputImage.fromMediaImage(mediaImage, imageProxy.getImageInfo().getRotationDegrees());
Bitmap bitmap = ImageConvertUtils.getInstance().getUpRightBitmap(image)
Java Implementation of Backbelt's Answer.
private Bitmap imageProxyToBitmap(ImageProxy image) {
ByteBuffer buffer = image.getPlanes()[0].getBuffer();
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);
return BitmapFactory.decodeByteArray(bytes,0,bytes.length,null);
}
There is second version of takePicture method at the moment (CameraX version 1.0.0-beta03). It provides several ways to persist image (OutputStream or maybe File can be useful in your case).
If you still want to convert ImageProxy to Bitmap here is my answer to similar question, which gives the correct implemetation of this conversion.
Please kindly take a look at this answer. All you need to apply it to your question is to get Image out of your ImageProxy
Image img = imaget.getImage();
I'm creating an app that takes pictures in .dng format in order to process them. I'm using the camera2 API. I was able to take pictures and save them into my phone, but in .jpg format. But when I change my code in order to save them with .dng extension, it compiles, show me the preview on my phone, but when the picture is taken, I get an error. The part of my code that takes and saves the picture is as follows.
val reader = ImageReader.newInstance(1280, 720, ImageFormat.RAW_SENSOR, 1)
val outputSurfaces = ArrayList<Surface>(2)
outputSurfaces.add(reader.surface)
outputSurfaces.add(Surface(previewTextureView.surfaceTexture))
val captureBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE)
captureBuilder.addTarget(reader.surface)
captureBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO)
val file = File("myPath/myImageName.dng")
var captureResult: CaptureResult? = null
And my listeners :
val readerListener = object : ImageReader.OnImageAvailableListener {
override fun onImageAvailable(reader: ImageReader) {
var image: Image? = null
var output: OutputStream? = null
val dngCreator = DngCreator(cameraManager.getCameraCharacteristics("0"), captureResult)
try {
image = reader.acquireLatestImage()
output = FileOutputStream(file)
dngCreator.writeImage(output, image)
} catch (e: FileNotFoundException) {
e.printStackTrace()
} catch (e: IOException) {
e.printStackTrace()
} finally {
output?.close()
image?.close()
}
}
}
reader.setOnImageAvailableListener(readerListener, backgroundHandler)
val captureListener = object : CameraCaptureSession.CaptureCallback() {
override fun onCaptureCompleted(session: CameraCaptureSession, request: CaptureRequest, result: TotalCaptureResult) {
captureResult = result
super.onCaptureCompleted(session, request, result)
}
}
And finally I capture the session with:
cameraDevice.createCaptureSession(outputSurfaces, object : CameraCaptureSession.StateCallback() {
override fun onConfigured(session: CameraCaptureSession) {
try {
session.capture(captureBuilder.build(), captureListener, backgroundHandler)
} catch (e: CameraAccessException) {
e.printStackTrace()
}
}
override fun onConfigureFailed(session: CameraCaptureSession) {}
}, backgroundHandler)
I'm having one warning and one error that I didn't have before, when I was saving the image as jpeg:
W/CameraDevice-JV-0: Stream configuration failed due to: createSurfaceFromGbp:1106: Camera 0: No supported stream configurations with format 0x20 defined, failed to create output stream
E/CameraCaptureSession: Session 1: Failed to create capture session; configuration failed
Things that I changed in order to save a dng file are :
I replaced ImageFormat.JPEG with ImageFormat.RAW_SENSOR
I changed the file extension from .jpg to .dng
Instead of using dngCreator.writeImage(output, image), I used :
val buffer = image!!.planes[0].buffer
val bytes = ByteArray(buffer.capacity())
buffer.get(bytes)
output.write()
Since there is not a lot of information about this subject, I'm not sure if my implementation is correct.
This a bit of an old post but for raw images you cannot set the resolution to an arbitrary value. Assuming your device can do raw_sensor reads it has to be set to the sensor size. You need to do something like this.
val largestRaw = Collections.max(Arrays.asList(*map.getOutputSizes(ImageFormat.RAW_SENSOR)), CompareSizesByArea())
rawImageReader = ImageReader.newInstance(largestRaw.width, largestRaw.height, ImageFormat.RAW_SENSOR, /*maxImages*/ 5).apply { setOnImageAvailableListener(onRawImageAvailableListener, backgroundHandler) }
Unfortunately in kotlin i am now encountering:
java.lang.IllegalArgumentException: Missing metadata fields for tag AsShotNeutral (c628)
The outdated java Camera2Raw sample listed above does work though.
After some research, I found a implementation in order to save an image that was taken with the Camera2API, in a .dng file :
if (mImage.format == ImageFormat.RAW_SENSOR) {
val dngCreator = DngCreator(mCharacteristics, mCaptureResult)
var output: FileOutputStream? = null
try {
output = FileOutputStream(mFile)
dngCreator.writeImage(output, mImage)
} catch (e: IOException) {
e.printStackTrace()
} finally {
mImage.close()
closeOutput(output)
}
}
Where :
mCharacteristics are CameraCharacteristics, ie the properties describing the CameraDevice
mCaptureResult is produced by the CameraDevice after processing the CaptureRequest
mImage is the image retrieved in the function dequeuAndSaveImage : image = reader.get()!!.acquireNextImage()
mFile is the File where the image will be saved, for example :
mFile = Environment
.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES),
"RAW_" + generateTimestamp()+ ".dng"
Maybe it will help somebody, but as #Alex Cohn said, it's recommended to begin with the official sample github.com/googlesamples/android-Camera2Raw. It's written in Java and not in Kotlin, but it's not that hard to transform it, if needed.