Develop Reference

Extract a person from an image in Google ML Kit with Selfie Segmentation in Kotlin

I would like to remove a background from an image with a person in order to use this image in some other part of my android app. I have applied Google ML Kit - Selfie Segmentation and receive segmentation Mask as a result. What I want to do now is - save the image without a background to a device so I can then use it in other parts of my app.
Original image:
Image after Applied Selfie Segmentation:
My problem is I need to somehow apply values from segmentationMask (256 * 256) to the Bitmap where I will remove background and save new Bitmap (with values from segmentation mask on only those pixels that are currently blue) with pixels from segmentation mask. Would anyone be so king and point me in the direction? Should I try to achieve this with Matrix ?? Currently I am drawind this image on jetpack compose Canvas with drawPoints comand in DrawScope.

There is an enableRawSizeMask API on the SelfieSegmenterOptions.Builder. If you don't set it, the returned mask should have the same dimension as the input image.
Then a naive approach would be: Iterating through the result mask to determine whether a certain pixel should be kept, if not, call setPixel method on the original Bitmap to set the background pixel to transparent (or whatever color you want).
This is for sure not the most performant way, but maybe a good start.

Maybe you can try it,this is my way
private void initUI(){
ImageView imageView=findViewById(R.id.iv);
Paint paint=new Paint();
paint.setAntiAlias(true);
SelfieSegmenterOptions options=new SelfieSegmenterOptions.Builder()
.setDetectorMode(SelfieSegmenterOptions.STREAM_MODE)
//.enableRawSizeMask()
.build();
Segmenter segmenter = Segmentation.getClient(options);
Bitmap bitmap= BitmapFactory.decodeResource(getResources(), R.drawable.mv1);
InputImage image = InputImage.fromBitmap(bitmap, 0);
segmenter.process(image).addOnSuccessListener(segmentationMask -> {
ByteBuffer mask = segmentationMask.getBuffer();
int maskWidth = segmentationMask.getWidth();
int maskHeight = segmentationMask.getHeight();
Bitmap background=Bitmap.createBitmap(maskColorsFromByteBuffer(mask,maskWidth,maskHeight),maskWidth,maskHeight, Bitmap.Config.ARGB_8888);
//创建一个bitmap
Bitmap foreground=Bitmap.createBitmap(bitmap.getWidth(),bitmap.getHeight(), Bitmap.Config.ARGB_8888); //裁剪后的图像
Canvas canvas=new Canvas(foreground);
//绘制扣出来的背景
canvas.drawBitmap(background,0,0,paint);
//设置混合模式
paint.setXfermode(new PorterDuffXfermode(PorterDuff.Mode.SRC_OUT));
//绘制原图
canvas.drawBitmap(bitmap,0,0,paint);
imageView.setImageBitmap(foreground);
}).addOnFailureListener(e -> {
// Task failed with an exception
// ...
});
}
private int[] maskColorsFromByteBuffer(ByteBuffer byteBuffer,int maskWidth,int maskHeight) {
#ColorInt int[] colors = new int[maskWidth * maskHeight];
for (int i = 0; i < maskWidth * maskHeight; i++) {
float backgroundLikelihood = 1 - byteBuffer.getFloat();
if (backgroundLikelihood > 0.9) { //128
colors[i] = Color.argb(255, 255, 0, 255);
//colors[i] = Color.argb(0, 255, 0, 255);
} else if (backgroundLikelihood > 0.2) {
// Linear interpolation to make sure when backgroundLikelihood is 0.2, the alpha is 0 and
// when backgroundLikelihood is 0.9, the alpha is 128.
// +0.5 to round the float value to the nearest int.
int alpha = (int) (182.9 * backgroundLikelihood - 36.6 + 0.5);
colors[i] = Color.argb(255, 255, 0, 255);
}
}
return colors;
}

Can I render a texture at pixel resolution in Android?

I am trying to make a test image to check the tone curve on Android displays. The kotlin code below makes a Bitmap that fits the ImageView. The canvas is filled with the background colour. The central 50% is filled with alternate lightLine (white) and darkLine (black) 2-pixel wide horizontal stripes.
All the features are in the right places. The stripes should appear the same grey as the background. They sort-of do, but I think they are the wrong size, as though the BitmapShader was scaling from dp to px.
I could draw the stripes one at a time using myCanvas.drawRect(). If I am stuck, that is what I will do. But the BitmapShader would let me render general patterns if I could fix it.
class MainActivity : Activity() {
// Here are all the objects(instances)
// of classes that we need to do some drawing
lateinit var myImageView: ImageView
var background = Color.rgb(180, 180, 180)
var darkStripe = Color.rgb(0, 0, 0)
var lightStripe = Color.rgb(255, 255, 255)
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
// Initialize the ImageView
myImageView = findViewById(R.id.imageView)
myImageView.doOnLayout {
val myBitmap = Bitmap.createBitmap(
it.measuredWidth,
it.measuredHeight,
Bitmap.Config.ARGB_8888
)
val w = (it.measuredWidth/4).toFloat()
val h = (it.measuredHeight/4).toFloat()
val myCanvas = Canvas(myBitmap)
myCanvas.drawColor(background)
val tex = createBitmap(
intArrayOf(lightStripe, lightStripe, darkStripe, darkStripe),
0, 1, 1, 4, Bitmap.Config.ARGB_8888
)
val shader: Shader = BitmapShader(tex, Shader.TileMode.REPEAT, Shader.TileMode.REPEAT)
val myPaint = Paint()
myPaint.setAntiAlias(false)
myPaint.shader = shader
myCanvas.drawRect(w, h,w*3.0f,h*3.0f, myPaint)
myImageView.setImageBitmap(myBitmap)
}
}
(two hours later)
I replaced the BitmapShader with drawRect() calls like this...
myPaint.setAntiAlias(false)
myPaint.color = darkStripe
myCanvas.drawRect(xMin, yMin, xMax, yMax, myPaint)
myPaint.color = lightStripe
var y = yMin
while (y < yMax) {
myCanvas.drawRect(xMin, y, xMax, y+myStep, myPaint)
y += 2*myStep;
}
This still gave soft edges to my stripes. Maybe I was working in dp and not px. So I set the stripe width with....
myStep = resources.getDisplayMetrics().density
The stripes all look nice an uniform with hard edges. The step is 3.0, but if I look at the display through a microscope, I see its matrix has green dots on a square grid with alternate blue and red pixels at the square centres. The 3.0 step rectangle is four green dots high. I don't see how that translates into 3 of anything but it looks smooth. I expect the BitMap shader would have worked with a repeat of 3.

Android Canvas PorterDuff not masking correctly

What I'm trying to do
I'm trying to implement a view that can be masked. Let's say the user has an image like this:
which gets masked by a custom shape like this:
The resulting image should only let the shoe shine through and make the background transparent.
Implementation
I have tried to implement it this way:
Make all black pixels in the mask transparent
Override the View's draw function to draw the mask over the original image using an xfermode (PorterDuffXfermode) of DST_IN. (See Android docs for PorterDuff.Mode)
This works perfectly fine and gives me this image (green pixels mean transparent):
While this works perfectly fine, I couldn't implement a custom "drawing" functionality to let the user draw or erase the mask. I only succeeded in doing one or the other, but not both at the same time. This is what I achieved right now:
note that erasing the mask works as expected, but trying to extend the mask doesn't work and paints white pixels (instead of letting the original image (shoe) shine through).
This is the code I'm using right now:
override fun draw(baseCanvas: Canvas) {
super.draw(baseCanvas)
val image = imageBitmap
val mask = maskBitmap
val drawingBitmap = drawingBitmap
if (image != null && mask != null && drawingBitmap != null) {
run {
val canvas = Canvas(drawingBitmap)
// 1. Fill with white
canvas.drawColor(Color.WHITE)
// 2. Draw mask and only let non-transparent pixels through
val paint = Paint(Paint.ANTI_ALIAS_FLAG)
paint.xfermode = PorterDuffXfermode(PorterDuff.Mode.DST_IN)
canvas.drawBitmap(mask, 0f, 0f, paint)
// 3. Draw all lines point to point with white color and custom xfermode
paint.xfermode = null
paint.style = Paint.Style.STROKE
paint.color = Color.WHITE
paint.strokeWidth = penSize
paint.isDither = true
paint.strokeJoin = Paint.Join.ROUND
paint.strokeCap = Paint.Cap.ROUND
paint.pathEffect = CornerPathEffect(10f)
paint.isAntiAlias = true
lines.forEach { line ->
paint.xfermode = when (line.drawMode) {
DrawMode.ERASE -> PorterDuffXfermode(PorterDuff.Mode.DST_OUT)
DrawMode.DRAW -> PorterDuffXfermode(PorterDuff.Mode.SRC)
}
val path = Path().also { path ->
val points = line.points
val range = points.size - 1
for (i in 1..range) {
path.moveTo(points[i - 1].x, points[i - 1].y)
path.lineTo(points[i].x, points[i].y)
}
}
canvas.drawPath(path, paint)
}
}
run {
val canvas = Canvas(image)
val paint = Paint(Paint.ANTI_ALIAS_FLAG)
paint.xfermode = PorterDuffXfermode(PorterDuff.Mode.DST_ATOP)
canvas.drawBitmap(drawingBitmap, 0f, 0f, paint)
}
}
}
Important variables in scope:
imageBitmap: The shoe bitmap
maskBitmap: The bitmap that contains white colors for the original shape to shine through, transparent for everything that should be transparent
drawingBitmap: An empty bitmap with the same size as imageBitmap, I use this for drawing only and then draw that result onto the imageBitmap (using a Canvas, see second run block)
lines: The lines I want to draw. A line consists of a drawMode (draw or erase) and a list of all points I have tracked.
Weird Observation
The thing that's confusing me right now is that when I fill the canvas black before adding the image:
run {
val canvas = Canvas(image)
canvas.drawColor(Color.BLACK) // <-- ADD THIS HERE
val paint = Paint(Paint.ANTI_ALIAS_FLAG)
paint.xfermode = PorterDuffXfermode(PorterDuff.Mode.DST_ATOP)
canvas.drawBitmap(drawingBitmap, 0f, 0f, paint)
}
it looks like it's working fine:
Same result if I fill the canvas RED, it lets the Red color shine through instead of Black. Why does it work with the color, but not the original bitmap?
It even looks correct if inspected in the debugger:
Question
Does anyone here know why this does not work as I expect it to work? I tried to play around with all kinds of different PorterDuff modes, but couldn't manage to get it working smoothly.
Any help appreciated!

Okay I got it working. I'm not sure if there's a more efficient way to achieve this, but this is what I did:
Create a fourth Bitmap called imageDrawingBitmap.
Set imageDrawingBitmap to the actual ImageView
As in the code snippet above, first draw the mask, then paths ontop of that
Then create a Canvas for the imageDrawingBitmap
Draw original image in that Canvas
Draw mask (maskDrawingBitmap from step 3.) in that Canvas with PorterDuff DST_ATOP.
That seems to be working fine.

Filter a Bitmap to different colors using PorterDuffColorFilter

I have the following image:
Now I’d like to filter this image to different colors at runtime. E.g. I want to be able to generate a green version of it, where the lower part is of a darker green, and the top-left part is lighter green.
How can I do this? Is this even possible?
I have tried filtering the bitmap like as follows:
private static Bitmap applyColorFilter(Bitmap source, int color) {
Bitmap empty = Bitmap.createBitmap(source, 0, 0, source.getWidth(), source.getHeight());
Canvas c = new Canvas(empty);
Paint p = new Paint();
PorterDuffColorFilter pdcf = new PorterDuffColorFilter(color, PorterDuff.Mode.SRC_IN);
p.setColorFilter(pcdf);
c.drawBitmap(empty, 0, 0, p);
source.recycle();
return empty;
}
I have tried all of PorterDuff.Modes, but none of them really works. Sometimes (e.g. SRC_IN) I see the circle totally filled with the new color, without any shading. In other cases I see a square.
I have thought about generating a source image with no color itself, and to render its shading with an alpha channel. However, I’m not sure that would work. I want the resulting image to be fully opaque (apart from the outer shadow you can see above).

I have found a solution using ColorMatrixColorFilter.
// Take values from the destination color
int destA = (color >> 24);
int destR = (color >> 16) & 0xff;
int destG = (color >> 8) & 0xff;
int destB = color & 0xff;
// Give the same weight to source R,G,B, thus grasping the intensity
// -> the shading overlay
float mean = 0.33f;
ColorMatrixColorFilter cm = new ColorMatrixColorFilter(
new float[]{
mean,mean,mean,0,destR,
mean,mean,mean,0,destG,
mean,mean,mean,0,destB,
0,0,0,1,destA }
);
Paint p = new Paint();
p.setColorFilter(cm);
...
Inevitably, this slightly changes (brightens) the destination color, so I’ll wait some days for a better solution. For my input image, I have better results by lowering the mean value to approx. 0.2.
The same result could probably be obtained with a LightningColorFilter.

Android Color (setColor in Paint) Needs Negative Integer?

I'm simply trying to paint/draw to the Canvas in Android. However, when I set the color using hex values or using the setARGB method, it doesn't work. But when I use Color.x (eg, Color.GREEN) it works. Here's the code:
Bitmap image = Bitmap.createBitmap(bitmapWidth, bitmapHeight, Bitmap.Config.ARGB_8888);
Canvas canvas = new Canvas(image);
Paint paintBackground = new Paint();
int green = Color.argb(0, 0, 255, 0); // 65280 (Won't work)
green = 0x0000ff00; // 65280 (Won't work)
paintBackground.setARGB(0, 0, 255, 0);
green = paintBackground.getColor(); // 65280 (Won't work)
green = Color.GREEN; // -16711936 (Works!)
paintBackground.setColor(green);
green = paintBackground.getColor(); // -16711936
paintBackground.setStyle(Paint.Style.FILL);
canvas.drawRect(0, 0, bitmapWidth, bitmapHeight, paintBackground);
So basically Color.GREEN returns -16711936 - and this WORKS.
However, the hex value is 65280 - and this DOES NOT WORK. That is, it doesn't paint the green rectangle.
I need to use hex values because I need to set the color to 0x00ffff00 here and then later to a different hex value.
Does Android Color (setColor in Paint) Need a Negative Integer?

The problem is that 0x0000ff00 is not green, but fully transparent green. Fully opaque green would be 0xff00ff00 which is, as you have already noticed, -16711936. Similarly, when using setARGB you need to specify 255 for alpha for the color to be fully opaque.

Color holds 4 fields, alpha, red, green and blue. Whenever anything is mostly opaque it is negative. 50.2% transparent green is positive (0x7F00FF00/2,130,771,712) and 49.8% transparent green is negative (0x8000FF00/-2,147,418,368)

You can also call Color.rgb(0, 255, 0). With rgb() alpha is by default 255, fully opaque.