Develop Reference

DrawRect is not in the right position with the given coordinates

I was doing the simple feature that I am taking the photo with Camerax and uploading the photo to Imagga to do the face detection by using this API, and here is the API result [Face(confidence=100.0, coordinates=Coordinates(height=1243, width=1243, xmax=2660, xmin=1417, ymax=3359, ymin=2116), face_id=)]
And base on the API document, the Coordinates are parameters that I needed to input to drawRect(xmin, ymin, xmax, ymax, paint), but the rectangle is always in the wrong position on the screen.
I've tried total 64 combinations of those four points but non is correct.
What have I done wrong? here's my code
take and upload photo to Imagga:
override fun onImageSaved(output: ImageCapture.OutputFileResults) {
Log.e(TAG, "onImageSaved: " + output.savedUri)
savedUri = output.savedUri
val file = File(getPath(savedUri, applicationContext) ?: "")
val mFile = RequestBody.create(MediaType.parse("image/*"), file)
val fileToUpload = MultipartBody.Part.createFormData("image", file.name, mFile)
photoJob?.cancel()
photoJob = lifecycleScope.launch {
binding.isLoading = true
mainViewMode.startFaceDetection(fileToUpload)
}
}
fun getPath(contentUri: Uri?, context: Context): String? {
var res: String? = null
val proj = arrayOf(MediaStore.Images.Media.DATA)
val cursor: Cursor = context.contentResolver.query(contentUri!!, proj, null, null, null)!!
if (cursor.moveToFirst()) {
val index = cursor.getColumnIndexOrThrow(MediaStore.Images.Media.DATA)
res = cursor.getString(index)
}
cursor.close()
return res
}
receive face detection results from Imagga:
mainViewMode.apply {
faceResult.observe(this#MainActivity) {
showDetectionResult(it, savedUri)
}
}
private fun showDetectionResult(it: FaceDetectionResp, savedUri: Uri?) {
Log.e(TAG, "startFaceDetection: ${it.result.faces}")
val originBitmap = MediaStore.Images.Media.getBitmap(contentResolver, savedUri)
val bmp = originBitmap!!.copy(originBitmap.config, true)
val canvas = Canvas(bmp)
val paint = Paint().apply {
color = Color.GREEN
style = Paint.Style.STROKE
strokeWidth = 5f
}
val left = it.result.faces[0].coordinates.xmin.toFloat()
val top = it.result.faces[0].coordinates.ymin.toFloat()
val right = it.result.faces[0].coordinates.xmax.toFloat()
val bottom = it.result.faces[0].coordinates.ymax.toFloat()
// it did draw the rectangle but never in the right position that I expected
canvas.drawRect(left, top, right, bottom, paint)
//because I got landscape photo, so I rotate the bitmap to portrait
binding.ivFace.setImageBitmap(rotateImage(bmp, -90f))
}
fun rotateImage(source: Bitmap, angle: Float): Bitmap? {
val matrix = Matrix()
matrix.postRotate(angle)
return Bitmap.createBitmap(
source, 0, 0, source.width, source.height,
matrix, true
)
}
I've been stuck in this for a week, can anyone help me?

How to convert camera image to use it in tflite

I have an android app in Kotlin which depends on CameraX to analyze images and pass them to tensorflow lite image detection model ( not classification ). The problem is we get YUV or RGBA Output images and we need an argb bitmap to pass to tflite model to detect objects we have been struggling for weeks and we have a deadline soon help please.
ImageAnalysis.Analyzer {
private val supportedImages = listOf(ImageFormat.FLEX_RGBA_8888, ImageFormat.FLEX_RGB_888)
private var data = ""
#SuppressLint("UnsafeOptInUsageError")
override fun analyze(image: ImageProxy) {
if (image.format in supportedImages) {
val options = ObjectDetector.ObjectDetectorOptions.builder()
.setScoreThreshold(0.2F)
.setBaseOptions(BaseOptions.builder().build())
.build()
val objectDetector =
ObjectDetector.createFromFileAndOptions(
context, Model.createModel(context, "model.tflite").path, options
)
val bitmap = toBitmap(image)
val tfImage = TensorImage.fromBitmap(bitmap)
val results: List<Detection> = objectDetector.detect(tfImage)
for (result in results) {
val categories = result.categories
for (category in categories) {
val label = category.label
val boundingBox = result.boundingBox
data += "label : $label | bounding : $boundingBox"
}
}
listener(data)
image.close()
}
}
private fun toBitmap(image: ImageProxy): Bitmap {
val r = image.planes[0].buffer.int
val g = image.planes[1].buffer.int
val b = image.planes[2].buffer.int
val a = image.planes[3].buffer.int
var color = Color.argb(a, r, g, b)
val rect = Rect(0, 0, 1, 1)
color = ColorUtils.compositeColors(color, Color.WHITE)
val bitmap = Bitmap.createBitmap(rect.width(), rect.height(), Bitmap.Config.ARGB_8888)
val canvas = Canvas(bitmap)
val paint = Paint()
paint.color = color
canvas.drawRect(rect, paint)
savePic(bitmap = bitmap!!, display_name = Random(100000L).toString())
return bitmap
}
private fun savePic(bitmap: Bitmap, display_name: String): Boolean {
val mediaCollection = latest {
MediaStore.Images.Media.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY)
} ?: MediaStore.Images.Media.EXTERNAL_CONTENT_URI
val contentValues = ContentValues().apply {
put(MediaStore.Images.Media.DISPLAY_NAME, "${display_name}.jpg")
put(MediaStore.Images.Media.MIME_TYPE, "image/jpeg")
put(MediaStore.Images.Media.WIDTH, bitmap.width)
put(MediaStore.Images.Media.HEIGHT, bitmap.height)
}
return try {
context.contentResolver.insert(mediaCollection, contentValues)?.also { uri ->
context.contentResolver.openOutputStream(uri).use { outputStream ->
if (!bitmap.compress(Bitmap.CompressFormat.JPEG, 100, outputStream)) {
throw IOException("could not save bitmap!")
}
}
} ?: throw IOException("could not create media store entry")
return true
} catch (e: IOException) {
e.printStackTrace()
false
}
}
}```

Here is the function that converts YUV_420_888 imageproxy into argb format bitmap.
I know a late answer but providing for others.
fun imageToBitmap(image: Image): Bitmap {
val planes: Array<Image.Plane> = image.planes
val yBuffer: ByteBuffer = planes[0].buffer
val uBuffer: ByteBuffer = planes[1].buffer
val vBuffer: ByteBuffer = planes[2].buffer
val ySize = yBuffer.remaining()
val uSize = uBuffer.remaining()
val vSize = vBuffer.remaining()
val nv21 = ByteArray(ySize + uSize + vSize)
//U and V are swapped
yBuffer[nv21, 0, ySize]
vBuffer[nv21, ySize, vSize]
uBuffer[nv21, ySize + vSize, uSize]
val yuvImage = YuvImage(nv21, ImageFormat.NV21, image.width, image.height, null)
val out = ByteArrayOutputStream()
yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 75, out)
val imageBytes: ByteArray = out.toByteArray()
return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)
}
Provide imageproxy from cameraX analyser to this function and it will provide the bitmap.

How group of renderscript filters use for Adjustment with seekbar

I Want to like this with renderscript filters a example gif is given below
My problem
In my code my code all renderscript filters running well but problem is when i am changing filter then resest the bitmap then update the filter value with bitmap .
class Filters(private val context: Context) {
private val TAG = BrightnessFilter::class.java.simpleName
private var rs: RenderScript = RenderScript.create(context)
private var brightnessScript: ScriptC_brightness = ScriptC_brightness(rs)
private var contrastScript: ScriptC_contrast = ScriptC_contrast(rs)
private var saturationFilter: ScriptC_saturation = ScriptC_saturation(rs)
private var convolution: ScriptIntrinsicConvolve3x3 =
ScriptIntrinsicConvolve3x3.create(rs, Element.U8_4(rs))
private var vignetteFilter: ScriptC_vignette = ScriptC_vignette(rs)
fun setBrightnessX(values: Float, inputImage: Bitmap): Bitmap {
val outBitmap = Bitmap.createBitmap(inputImage)
val tempIn = Allocation.createFromBitmap(rs, inputImage)
val tempOut = Allocation.createFromBitmap(rs, outBitmap)
brightnessScript.invoke_setBright(values)
brightnessScript.forEach_brightness(tempIn, tempOut)
tempOut.copyTo(outBitmap)
return outBitmap
}
fun setContrastX(contrast: Float, inputImage: Bitmap): Bitmap {
val outBitmap = Bitmap.createBitmap(inputImage)
val tempIn = Allocation.createFromBitmap(rs, inputImage)
val tempOut = Allocation.createFromBitmap(rs, outBitmap)
contrastScript._contrast = contrast
contrastScript.forEach_contrastness(tempIn, tempOut)
tempOut.copyTo(outBitmap)
tempIn.destroy()
tempOut.destroy()
return outBitmap
}
fun setSaturationX(saturation: Float, inputImage: Bitmap): Bitmap {
val outBitmap = Bitmap.createBitmap(inputImage)
val tempIn = Allocation.createFromBitmap(rs, inputImage)
val tempOut = Allocation.createFromBitmap(rs, outBitmap)
saturationFilter._saturationValue = saturation
saturationFilter.forEach_saturation(tempIn, tempOut)
tempOut.copyTo(outBitmap)
tempIn.destroy()
tempOut.destroy()
return outBitmap
}
fun sharpenX(sharpen: Float, bitmap: Bitmap): Bitmap {
val matrix = floatArrayOf(
0f, -sharpen, 0f,
-sharpen, 1 + 4 * sharpen, -sharpen,
0f, -sharpen, 0f
)
return applySharpenConvolveX(bitmap, matrix)
}
private fun applySharpenConvolveX(inputImage: Bitmap, coefficients: FloatArray): Bitmap {
val outBitmap = Bitmap.createBitmap(inputImage)
val tempIn = Allocation.createFromBitmap(rs, inputImage)
val tempOut = Allocation.createFromBitmap(rs, outBitmap)
convolution.setInput(tempIn)
convolution.setCoefficients(coefficients)
convolution.forEach(tempOut)
tempOut.copyTo(outBitmap)
return outBitmap
}
fun vignetteX(values: Float, inputImage: Bitmap): Bitmap {
vignetteFilter.setScale(values)
return vignetteFilter.x(inputImage)
}
}
For Slider Value change currentFilter is called and update the output Bitmap with ImageView
private fun currentFilter(values: Float): Bitmap {
return when (_recentTab.value!!) {
FilterName.BRIGHTNESS -> _filter.value!!.setBrightnessX(values, mInputImage)
FilterName.CONTRAST -> _filter.value!!.setContrastX(values, mInputImage)
FilterName.SATURATION -> _filter.value!!.setSaturationX(values, mInputImage)
FilterName.SHARPEN -> _filter.value!!.sharpenX(values, mInputImage)
FilterName.VIGNETTE -> _filter.value!!.vignetteX(values, mInputImage)
}
}

how to select a portion of the image when using CameraX analyze

I'm using CameraX and then FirebaseVision to read some text from the image. when I'm analyzing the Image I want to select a portion of the image, not the entire Image, something like when you use a barcode scanner.
class Analyzer : ImageAnalysis.Analyzer {
override fun analyze(imageProxy: ImageProxy?, rotationDegrees: Int) {
// how to crop the image in here?
val image = imageProxy.image
val imageRotation = degreesToFirebaseRotation(degrees)
if (image != null) {
val visionImage = FirebaseVisionImage.fromMediaImage(image, imageRotation)
val textRecognizer = FirebaseVision.getInstance().onDeviceTextRecognizer
textRecognizer.processImage(visionImage)
}
}
I want to know, is there any way to crop the image?

Your problem is exactly what I have tackled 2 months ago...
object YuvNV21Util {
fun yuv420toNV21(image: Image): ByteArray {
val crop = image.cropRect
val format = image.format
val width = crop.width()
val height = crop.height()
val planes = image.planes
val data =
ByteArray(width * height * ImageFormat.getBitsPerPixel(format) / 8)
val rowData = ByteArray(planes[0].rowStride)
var channelOffset = 0
var outputStride = 1
for (i in planes.indices) {
when (i) {
0 -> {
channelOffset = 0
outputStride = 1
}
1 -> {
channelOffset = width * height + 1
outputStride = 2
}
2 -> {
channelOffset = width * height
outputStride = 2
}
}
val buffer = planes[i].buffer
val rowStride = planes[i].rowStride
val pixelStride = planes[i].pixelStride
val shift = if (i == 0) 0 else 1
val w = width shr shift
val h = height shr shift
buffer.position(rowStride * (crop.top shr shift) + pixelStride * (crop.left shr shift))
for (row in 0 until h) {
var length: Int
if (pixelStride == 1 && outputStride == 1) {
length = w
buffer[data, channelOffset, length]
channelOffset += length
} else {
length = (w - 1) * pixelStride + 1
buffer[rowData, 0, length]
for (col in 0 until w) {
data[channelOffset] = rowData[col * pixelStride]
channelOffset += outputStride
}
}
if (row < h - 1) {
buffer.position(buffer.position() + rowStride - length)
}
}
}
return data
}
}
then convert bytearray into bitmap
object BitmapUtil {
fun getBitmap(data: ByteArray, metadata: FrameMetadata): Bitmap {
val image = YuvImage(
data, ImageFormat.NV21, metadata.width, metadata.height, null
)
val stream = ByteArrayOutputStream()
image.compressToJpeg(
Rect(0, 0, metadata.width, metadata.height),
80,
stream
)
val bmp = BitmapFactory.decodeByteArray(stream.toByteArray(), 0, stream.size())
stream.close()
return rotateBitmap(bmp, metadata.rotation, false, false)
}
private fun rotateBitmap(
bitmap: Bitmap, rotationDegrees: Int, flipX: Boolean, flipY: Boolean
): Bitmap {
val matrix = Matrix()
// Rotate the image back to straight.
matrix.postRotate(rotationDegrees.toFloat())
// Mirror the image along the X or Y axis.
matrix.postScale(if (flipX) -1.0f else 1.0f, if (flipY) -1.0f else 1.0f)
val rotatedBitmap =
Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true)
// Recycle the old bitmap if it has changed.
if (rotatedBitmap != bitmap) {
bitmap.recycle()
}
return rotatedBitmap
}
}
Please have a look at my open source project https://github.com/minkiapps/Firebase-ML-Kit-Scanner-Demo, I build a demo app where portion of the image proxy is cropped before it is processed by ml kit.

Rotating images in Listview

I am trying to make a fake horizontal ListView. For that, I want to rotate the content of each cell in the adapter. It almost works:
class ImageAdapter(ctx: Context, viewResourceId: Int, imageResourceId: Int, strings: Array[String], icons: TypedArray, colorSelected: Int, colorNotSelected: Int, colorBorder: Int) extends ArrayAdapter[String](ctx, viewResourceId, strings) {
// Useful variables
private val mInflater: LayoutInflater = ctx.getSystemService(
Context.LAYOUT_INFLATER_SERVICE).asInstanceOf[LayoutInflater]
private val mStrings: Array[String] = strings
private val mIcons: TypedArray = icons
private val mViewResourceId: Int = viewResourceId
private var mSelected : Int = 0
def setSelected(s: Int) = mSelected = s
// Inherited from ArrayAdapter
override def getCount():Int = mStrings.length
override def getItem(position: Int): String = mStrings(position)
override def getItemId(position: Int): Long = position
// The rotation machinery
var rotation = 90
private var matrix = new Matrix()
val rect1 = new RectF(0, 0, 0, 0)
val rect2 = new RectF(0, 0, 0, 0)
val cst = Matrix.ScaleToFit.CENTER
// Getting the view.
override def getView(position: Int, convertView: View, parent: ViewGroup): View = {
var result = if(convertView == null) mInflater.inflate(mViewResourceId, null) else convertView
var iv: ImageView = result.findViewById(imageResourceId).asInstanceOf[ImageView]
val drawable = mIcons.getDrawable(position)
iv.setScaleType(ScaleType.MATRIX)
matrix.reset()
val height = drawable.getIntrinsicHeight()
val width = drawable.getIntrinsicWidth()
rotation match {
case 0 =>
rect1.set(0, 0, width, height)
rect2.set(0, 0, iv.getWidth(), iv.getHeight())
matrix.setRectToRect(rect1, rect2, cst)
case 90 =>
rect1.set(0, 0, width, height)
rect2.set(-iv.getHeight()/2, -iv.getWidth()/2, iv.getHeight()/2, iv.getWidth()/2)
matrix.setRectToRect(rect1, rect2, cst)
matrix.postRotate(rotation, 0, 0)
matrix.postTranslate(iv.getWidth()/2, iv.getHeight()/2)
...
case _ =>
}
iv.setImageMatrix(matrix)
iv.setImageDrawable(drawable)
iv.postInvalidate()
if(position == mSelected) {
result.setBackgroundColor(colorSelected)
} else {
result.setBackgroundColor(colorNotSelected)
}
result.setTag(mStrings(position))
result
}
}
The only problem is that it does not display the first picture. Instead, it displays this:
For the first list to appear, I need to click on one element of each list. The onclick method is the following:
listGraphismesView.setOnItemClickListener(new OnItemClickListener {
override def onItemClick(parent: AdapterView[_], view: View, position: Int, notUsedId: Long) = {
var graphismeName = view.getTag().asInstanceOf[String]
mAdapter.setSelected(position)
mAdapter.notifyDataSetChanged()
showGChallenge(position, graphismeName)
}
})
I tried then to put mAdapter.notifyDataSetChanged() in the onResume function of the fragment, but it does not update the list from the beginning. I still need to click on something for all images to appear.
Any idea why ?

I found out that iv.getWidth() and iv.getHeight() were both returning zero because the getview would be called by the onActivityCreated function. Putting notifyDataSetChanged() in the onResume function was too early too. So the drawable would be drawn with size 0, and reloaded when I could click on the button.
So to change that, I wrote the following piece of code, calling the resize function with a listener.
var iv: ImageView = result.findViewById(imageResourceId).asInstanceOf[ImageView]
val drawable = mIcons.getDrawable(position)
iv.setImageDrawable(drawable)
iv.setScaleType(ScaleType.MATRIX)
val vto = iv.getViewTreeObserver()
vto.addOnGlobalLayoutListener(new OnGlobalLayoutListener() {
private var matrix = new Matrix()
val rect1 = new RectF(0, 0, 0, 0)
val rect2 = new RectF(0, 0, 0, 0)
private var height = drawable.getIntrinsicHeight()
private var width = drawable.getIntrinsicWidth()
override def onGlobalLayout() {
val iv_width = iv.getWidth
val iv_height = iv.getHeight
matrix.reset()
rotation match {
case 0 =>
rect1.set(0, 0, width, height)
rect2.set(0, 0, iv_width, iv_height)
matrix.setRectToRect(rect1, rect2, cst)
case 90 =>
rect1.set(0, 0, width, height)
rect2.set(-iv_height/2, -iv_width/2, iv_height/2, iv_width/2)
matrix.setRectToRect(rect1, rect2, cst)
matrix.postRotate(rotation, 0, 0)
matrix.postTranslate(iv_width/2, iv_height/2)
...
case _ =>
}
iv.setImageMatrix(matrix)
}
})