I am trying to find contours from the raw bytes received from onPreviewFrame.
These raw bytes received are not rotated when we do setDisplayOrientation(according to the android developer docs). It is becoming difficult to rotate the contours alone. How to rotate these bytes efficiently and then process it ? I am using openCV to find the contours.
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
previewSize = cameraConfigUtil.cameraInstance.getParameters().getPreviewSize();
Mat srcMat = new Mat(previewSize.height, previewSize.width, CvType.CV_8UC3);
srcMat.put(0, 0, data);
rect = ImageCorrection.getLargestContour(data, previewSize.height, previewSize.width);
android.graphics.Rect rectangle = new android.graphics.Rect();
rectangle.left = rect.x;
rectangle.top = rect.y;
rectangle.right = rect.x + rect.width;
rectangle.bottom = rect.y + rect.height;
mOverlay.clear();
BarcodeGraphic graphic = new BarcodeGraphic(mOverlay);
mOverlay.add(graphic);
graphic.updateItem(rectangle);
Imgproc.rectangle(srcMat, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(255, 0, 0, 255), 3);
Utils.matToBitmap(srcMat, bitmap);
}
Here rectangle gives the four points of the contour, I want this to be rotated according to the angle set in setDisplayOrientation.
Rotating around a point is a fairly standard geometric process. Cribbing from this answer:
The way to rotate by an arbitraty point is first substract the point
coordinates, do the rotation about the origin and then add the point
coordinates.
x2 = px + (x1-px)*cos(q)-(y1-py)*sin(q)
y2 = py + (x1-px)*sin(q)+(y1-py)*cos(q)
where px, py are the rotation point coordinates, and x1,y1 the
original 2D shape vertex and x2,y2 the rotated coordinates, and q the
angle in radians.
I would imagine that the most efficient way to do this would be to apply the transformation to the 4 vertices and then draw a new rectangle on the rotated canvas.
Related
Now I have a problem with NinePatchDrawable.
Matrix matrix = new Matrix();
matrix.postScale(1, -1);
....
NinePatchDrawable npd = (NinePatchDrawable) getResources().getDrawable(R.mipmap.bg_bubble);
Rect rect = new Rect((int) (curPointX + offsetX), (int) (curPointY + offsetY), (int) (curPointX + offsetX + bubbleImgWidth), (int) (curPointY + offsetY + bubbleImgHeight));
npd.setBounds(rect);
npd.draw(canvas);
the R.miamap.bg_bubble is a 9-Patch image, it should be added the matrix above.
Before, it isn't a 9-patch image, I used the code below
mBubbleBitmap = BitmapFactory.decodeResource(mContext.getResources(), R.mipmap.bg_bubble);
...
Matrix matrix = new Matrix();
matrix.postScale(1, -1);
...
Bitmap dstBmp = Bitmap.createBitmap(mBubbleBitmap, 0, 0, mBubbleBitmap.getWidth(), mBubbleBitmap.getHeight(), matrix, true);
But now, it is a 9-patch image, so I don't know how to do it. And I search the solution on Google, but with no result.
Don't do this .
Read this line
The NinePatch class permits drawing a bitmap in nine or more sections.
Essentially, it allows the creation of custom graphics that will scale
the way that you define, when content added within the image exceeds
the normal bounds of the graphic.
Oh,it seems like that I have done it in an opportunistic way. Exactly I want to rotate the 9-patch image, but the image also have different size because of the num of the content words. See the code below:
mBubbleBitmap = BitmapFactory.decodeResource(mContext.getResources(), R.mipmap.bg_bubble);
...
Matrix matrix = new Matrix();
matrix.postScale(1, -1);
// get the original bitmap with matrix
Bitmap dstBmp = Bitmap.createBitmap(mBubbleBitmap, 0, 0, mBubbleBitmap.getWidth(), mBubbleBitmap.getHeight(), matrix, true);
// translate it to a NinePatchDrawable drawable.
// Note: the third arg must be the original bitmap's chunk, because the dstBmp has no 9-patch chunk info.
NinePatchDrawable npd = new NinePatchDrawable(getResources(), dstBmp, mBubbleBitmap.getNinePatchChunk(), new Rect(), null);
// Draw it!
Rect rect = new Rect((int) (curPointX + offsetX), (int) (curPointY + offsetY), (int) (curPointX + offsetX + bubbleImgWidth), (int) (curPointY + offsetY + bubbleImgHeight));
npd.setBounds(rect);
npd.draw(canvas);
This is what I did. see the picture below:
enter image description here
What I say 'opportunistic' is that my 9-patch image's black points of left and top must be IN THE CENTER so that when I rotate it, the image will not be stretched in a wrong way !!! hah...
I have been battling with trying to draw a bitmap and then highlighting a region on it with a rectangle. Originally, I was drawing a bitmap with alpha black in paint to make image darker and then on top drawing original bitmap in a region creating effect of highlight. I discovered that largest slowdown was because of alpha in Paint. So I have reworked the code and ended up with following in my draw thread:
private synchronized void drawSquare(int xStart, int yStart, int xEnd, int yEnd) {
Canvas c = holder.lockCanvas();
if(c != null) {
// Draw the background picture on top with some changed alpha channel to blend
Paint paint = new Paint();
paint.setAntiAlias(true);
if(bg != null && cWidth > 0 && cHeight > 0) {
c.clipRect(xStart, yStart, xEnd, yEnd, Region.Op.DIFFERENCE);
c.drawBitmap(bg, gTransform, blackSqr); // Draw derker background
c.clipRect(xStart, yStart, xEnd, yEnd, Region.Op.REPLACE);
c.drawBitmap(bg, gTransform, paint); ///draw original in selection
c.clipRect(0, 0, cWidth, cHeight,Region.Op.REPLACE);
}
Matrix RTcorner = new Matrix();
RTcorner.setRotate(90);
RTcorner.postTranslate(xEnd + 13, yStart - 13);
Matrix RBcorner = new Matrix();
RBcorner.setRotate(180);
RBcorner.postTranslate(xEnd + 13, yEnd + 13);
Matrix LBcorner = new Matrix();
LBcorner.setRotate(270);
LBcorner.postTranslate(xStart - 13, yEnd + 13);
// Draw the fancy bounding box
c.drawRect(xStart, yStart, xEnd, yEnd, linePaintB);
// Draw corners for the fancy box
c.drawBitmap(corner, xStart - 13, yStart - 13, new Paint());
c.drawBitmap(corner, RBcorner, new Paint());
c.drawBitmap(corner, LBcorner, new Paint());
c.drawBitmap(corner, RTcorner, new Paint());
}
holder.unlockCanvasAndPost(c);
}
So this clips out my selection area, I draw with paint that has this code to make it darker.
blackSqr.setColorFilter(new LightingColorFilter(Color.rgb(100,100,100),0));
And in the area inside the clip I draw my original bitmap. It works. But I am not happy with response time. After profiling Bitmap is what takes the longest. I have scaled the bitmap to the size of the screen already so it's drawing 300x800-ish image. The biggest resource hog seems to be the Lighting effect. Because when I turn it off I get decent response time.
So I was wondering if I have missed anything to improve how quickly bitmap is drawn, maybe caching? Or am I just stuck with this because I want darker image and actually should rethink the "highlighting/selection" altogether? Why is is so expensive to draw a bitmap with alpha colour in 2D image?
if I understand what you want, you want a rectangle (with rounded corners) to highlight a part from another image.
if it is that, then I would use an image with the square wit draw9patch and use it as a floating view over the image view
RelativeLaoyut (Image container)
+- ImageView (your actual image)
+- view (it has the square as a background, and you only have to move it to the area you want to highlight)
I'm sorry, I'm not good explaining myself.
For anyone that is interested, perhaps facing similar problem. This solution applies to my particular situation, but I have a separate background bitmap with darkened pixels manually set using:
for(int i = 0; i < cWidth; i++){
for(int j = 0; j < cHeight; j++){
int c = bg2.getPixel(i, j);
float mult = 0.15f;
int r = (int) (Color.red(c) * mult);
int g = (int) (Color.green(c) * mult);
int b = (int) (Color.blue(c) * mult);
bg2.setPixel(i, j, Color.rgb(r, g, b));
}
}
Then use the bg2 to draw main part and the original (not darkened) for the clip rectangle of the selection. There is a bit of overhead for creating and maintaining the second bitmap but the draw speed and response time is quick and smooth in comparison to bitmaps with alpha.
I'm working on a word game and I was dynamically creating the textures for the letter tiles when the game loads, comprising of a background image and a font.
To do this I was drawing pixmaps onto pixmaps, this was all fine until I started working on scaling. The font scaling on the pixmaps was terrible, even with bilinear filtering turned on (left image below) even though my scaled fonts were looking pretty good elsewhere.
So I decided to get round this I'd use a frame buffer, render everything to that and then copy that out to a pixmap and create a texture from that. That way I could use the gpu filtering and it should look exactly the same as my other fonts, (middle image below) but it still didn't look quite as nice as the other fonts. A slight dark line round the outside, it looks like the alpha blending isn't working properly.
I then tried drawing straight over the tiles with the font at runtime to make sure it wasn't my imagination, and this definitely looks better with smooth blending into the image below (right image below), but this impacts my frame rate quite a lot.
So my question is, why is drawing to the frame buffer not producing the same result as when I draw to the screen? Code below.
Texture tx = Assets.loadTexture("bubbles/BubbleBlue.png");
tx.setFilter(TextureFilter.Linear, TextureFilter.Linear);
SpriteBatch sb = new SpriteBatch();
FrameBuffer fb = new FrameBuffer(Format.RGBA8888,
LayoutManager.getWidth(), LayoutManager.getHeight(), false);
fb.begin();
sb.begin();
sb.draw(tx, 0, 0, LetterGrid.blockWidth, LetterGrid.blockHeight);
Assets.candara80.font.getRegion().getTexture()
.setFilter(TextureFilter.Linear, TextureFilter.Linear);
Assets.candara80.setSize(0.15f);
TextBounds textBounds = Assets.candara80.getBounds(letter);
Assets.candara80.drawText(sb, letter,
(LetterGrid.blockWidth - textBounds.width) / 2,
(LetterGrid.blockHeight + textBounds.height) / 2);
sb.end();
Pixmap pm = ScreenUtils.getFrameBufferPixmap(0, 0,
(int) LetterGrid.blockWidth, (int) LetterGrid.blockHeight);
Pixmap flipped = flipPixmap(pm);
result = new Texture(flipped);
fb.end();
pm.dispose();
flipped.dispose();
tx.dispose();
fb.dispose();
sb.dispose();
set PROJECTION is the problem.
EXAMPLE
public Texture texture(Color fg_color, Color bg_color)
{
Pixmap pm = render( fg_color, bg_color );
texture = new Texture(pm);//***here's your new dynamic texture***
disposables.add(texture);//store the texture
}
//---------------------------
public Pixmap render(Color fg_color, Color bg_color)
{
int width = Gdx.graphics.getWidth();
int height = Gdx.graphics.getHeight();
SpriteBatch spriteBatch = new SpriteBatch();
m_fbo = new FrameBuffer(Format.RGB565, (int)(width * m_fboScaler), (int)(height * m_fboScaler), false);
m_fbo.begin();
Gdx.gl.glClearColor(bg_color.r, bg_color.g, bg_color.b, bg_color.a);
Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT);
/**set PROJECTION**/
Matrix4 normalProjection = new Matrix4().setToOrtho2D(0, 0, Gdx.graphics.getWidth(), Gdx.graphics.getHeight());
spriteBatch.setProjectionMatrix(normalProjection);
spriteBatch.begin();
spriteBatch.setColor(fg_color);
//do some drawing ***here's where you draw your dynamic texture***
...
spriteBatch.end();//finish write to buffer
pm = ScreenUtils.getFrameBufferPixmap(0, 0, (int) width, (int) height);//write frame buffer to Pixmap
m_fbo.end();
// pm.dispose();
// flipped.dispose();
// tx.dispose();
m_fbo.dispose();
m_fbo = null;
spriteBatch.dispose();
// return texture;
return pm;
}
I am working on opencv in android and i want to change eye pupil color through Hue channel and i achieve this already but the problem is that the region i detected is in rectangle but i want this region circular as eye pupil is circular region. kindly help how i achieve this.
private Mat get_template(CascadeClassifier clasificator, Rect area,int size){
Mat template = new Mat();
Mat mROI = mGray.submat(area);
MatOfRect eyes = new MatOfRect();
Point iris = new Point();
Rect eye_template = new Rect();
clasificator.detectMultiScale(mROI, eyes, 1.15, 2,Objdetect.CASCADE_FIND_BIGGEST_OBJECT|Objdetect.CASCADE_SCALE_IMAGE, new Size(30,30),new Size());
Rect[] eyesArray = eyes.toArray();
for (int i = 0; i < eyesArray.length; i++){
Rect e = eyesArray[i];
e.x = area.x + e.x;
e.y = area.y + e.y;
Rect eye_only_rectangle = new Rect((int)e.tl().x,(int)( e.tl().y + e.height*0.4),(int)e.width,(int)(e.height*0.6));
mROI = mGray.submat(eye_only_rectangle);
Mat vyrez = mRgba.submat(eye_only_rectangle);
Core.MinMaxLocResult mmG = Core.minMaxLoc(mROI);
Core.circle(vyrez, mmG.minLoc,2, new Scalar(255, 255, 255, 255),2);
iris.x = mmG.minLoc.x + eye_only_rectangle.x;
iris.y = mmG.minLoc.y + eye_only_rectangle.y;
eye_template = new Rect((int)iris.x-size/2,(int)iris.y-size/2 ,size,size);
Core.rectangle(mRgba,eye_template.tl(),eye_template.br(),new Scalar(255, 0, 0, 255), 2);
template = (mGray.submat(eye_template)).clone();
return template;
}
return template;
}
Some potential solutions:
the simplest, although it might not be very robust is to calculate the inscribed circle (the circle bound by the rectangle) and change it's color - if your pupil detection is very accurate this solution may work fine.
a more robust solution would be to detect the area of the pupil based on color or gradient (edge detection)
I am developing application in which I have to detect rectangular object and draw outline I am using Open cv android library....
I succesfully detect Circle and draw outline inside image but repeatedly fail to detect Square or rectangle and draw....Here is my code to for circle..
Bitmap imageBmp = BitmapFactory.decodeResource(MainActivityPDF.this.getResources(),R.drawable.loadingplashscreen);
Mat imgSource = new Mat(), imgCirclesOut = new Mat();
Utils.bitmapToMat(imageBmp , imgSource);
//grey opencv
Imgproc.cvtColor(imgSource, imgSource, Imgproc.COLOR_BGR2GRAY);
Imgproc.GaussianBlur( imgSource, imgSource, new Size(9, 9), 2, 2 );
Imgproc.HoughCircles( imgSource, imgCirclesOut, Imgproc.CV_HOUGH_GRADIENT, 1, imgSource.rows()/8, 200, 100, 0, 0 );
float circle[] = new float[3];
for (int i = 0; i < imgCirclesOut.cols(); i++)
{
imgCirclesOut.get(0, i, circle);
org.opencv.core.Point center = new org.opencv.core.Point();
center.x = circle[0];
center.y = circle[1];
Core.circle(imgSource, center, (int) circle[2], new Scalar(255,0,0,255), 4);
}
Bitmap bmp = Bitmap.createBitmap(imageBmp.getWidth(), imageBmp.getHeight(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgSource, bmp);
ImageView frame = (ImageView) findViewById(R.id.imageView1);
//Bitmap bmp = Bitmap.createBitmap(100, 100, Bitmap.Config.ARGB_8888);
frame.setImageBitmap(bmp);
any help for detect square/rectangle for android ......I am wondering from 2 days ..every example are in either C++ or in C++ and I can't get through that languages...
Thanks.
There are many ways of detecting a rectangle using opencv, the most appropriate way of doing this is by finding the contours after applying Canny Edge Detection.
Steps are as follows :-
1.Convert the image to MAT
Grayscale the image
3.Apply Gausian Blur
4.Apply Morphology for filling the holes if any
5.Apply Canny Detection
6.Find Contours of the image
7.Find the largest contour of the rest
8.Draw the largest contour.
Code is as follows -
1.Convert the image to MAT
Utils.bitmapToMat(image,src)
Grayscale the image
val gray = Mat(src.rows(), src.cols(), src.type())
Imgproc.cvtColor(src, gray, Imgproc.COLOR_BGR2GRAY)
3.Apply Gausian Blur
Imgproc.GaussianBlur(gray, gray, Size(5.0, 5.0), 0.0)
4.Apply Morphology for filling the holes if any and also dilate the image
val kernel = Imgproc.getStructuringElement(
Imgproc.MORPH_ELLIPSE, Size(
5.0,
5.0
)
)
Imgproc.morphologyEx(
gray,
gray,
Imgproc.MORPH_CLOSE,
kernel
) // fill holes
Imgproc.morphologyEx(
gray,
gray,
Imgproc.MORPH_OPEN,
kernel
) //remove noise
Imgproc.dilate(gray, gray, kernel)
5.Apply Canny Detection
val edges = Mat(src.rows(), src.cols(), src.type())
Imgproc.Canny(gray, edges, 75.0, 200.0)
6.Find Contours of the image
val contours = ArrayList<MatOfPoint>()
val hierarchy = Mat()
Imgproc.findContours(
edges, contours, hierarchy, Imgproc.RETR_LIST,
Imgproc.CHAIN_APPROX_SIMPLE
)
7.Find the largest contour of the rest
public int findLargestContour(ArrayList<MatOfPoint> contours) {
double maxVal = 0;
int maxValIdx = 0;
for (int contourIdx = 0; contourIdx < contours.size(); contourIdx++) {
double contourArea = Imgproc.contourArea(contours.get(contourIdx));
if (maxVal < contourArea) {
maxVal = contourArea;
maxValIdx = contourIdx;
}
}
return maxValIdx;
}
8.Draw the largest contour which is the rectangle
Imgproc.drawContours(src, contours, idx, Scalar(0.0, 255.0, 0.0), 3)
There you go you have found the rectangle .
If any error persist in getting the process .Try resizing the source Image to half of its height and width.
Have a look at the below link for proper Java code of the above explained
https://github.com/dhananjay-91/DetectRectangle
Also,
https://github.com/aashari/android-opencv-rectangle-detector
You are on the right way by using the Houghtransformation. Instead of using Houghcircles you have to use Houghlines and check the obtained lines for intersections. If you really have to find rectangles (and not 4 edged polygones) - you should look for lines with the same angle(+- a small offset) and if you found at least a pair of these lines you have to look for lines that lay perpendicular to this, find a pair as well and check for intersections. It should not be a big deal using vectors(endpoint - startpoint) and lines to perform the angle and intersection tests.