Converting Camera Coordinates to Custom View Coordinates - android

I am trying to make a simple face detection app consisting of a SurfaceView (essentially a camera preview) and a custom View (for drawing purposes) stacked on top. The two views are essentially the same size, stacked on one another in a RelativeLayout. When a person's face is detected, I want to draw a white rectangle on the custom View around their face.
The Camera.Face.rect object returns the face bound coordinates using the coordinate system explained here and the custom View uses the coordinate system described in the answer to this question. Some sort of conversion is needed before I can use it to draw on the canvas.
Therefore, I wrote an additional method ScaleFacetoView() in my custom view class (below) I redraw the custom view every time a face is detected by overriding the OnFaceDetection() method. The result is the white box appears correctly when a face is in the center. The problem I noticed is that it does not correct track my face when it moves to other parts of the screen.
Namely, if I move my face:
Up - the box goes left
Down - the box goes right
Right - the box goes upwards
Left - the box goes down
I seem to have incorrectly mapped the values when scaling the coordinates. Android docs provide this method of converting using a matrix, but it is rather confusing and I have no idea what it is doing. Can anyone provide some code on the correct way of converting Camera.Face coordinates to View coordinates?
Here's the code for my ScaleFacetoView() method.
public void ScaleFacetoView(Face[] data, int width, int height, TextView a){
//Extract data from the face object and accounts for the 1000 value offset
mLeft = data[0].rect.left + 1000;
mRight = data[0].rect.right + 1000;
mTop = data[0].rect.top + 1000;
mBottom = data[0].rect.bottom + 1000;
//Compute the scale factors
float xScaleFactor = 1;
float yScaleFactor = 1;
if (height > width){
xScaleFactor = (float) width/2000.0f;
yScaleFactor = (float) height/2000.0f;
}
else if (height < width){
xScaleFactor = (float) height/2000.0f;
yScaleFactor = (float) width/2000.0f;
}
//Scale the face parameters
mLeft = mLeft * xScaleFactor; //X-coordinate
mRight = mRight * xScaleFactor; //X-coordinate
mTop = mTop * yScaleFactor; //Y-coordinate
mBottom = mBottom * yScaleFactor; //Y-coordinate
}
As mentioned above, I call the custom view like so:
#Override
public void onFaceDetection(Face[] arg0, Camera arg1) {
if(arg0.length == 1){
//Get aspect ratio of the screen
View parent = (View) mRectangleView.getParent();
int width = parent.getWidth();
int height = parent.getHeight();
//Modify xy values in the view object
mRectangleView.ScaleFacetoView(arg0, width, height);
mRectangleView.setInvalidate();
//Toast.makeText( cc ,"Redrew the face.", Toast.LENGTH_SHORT).show();
mRectangleView.setVisibility(View.VISIBLE);
//rest of code

Using the explanation Kenny gave I manage to do the following.
This example works using the front facing camera.
RectF rectF = new RectF(face.rect);
Matrix matrix = new Matrix();
matrix.setScale(1, 1);
matrix.postScale(view.getWidth() / 2000f, view.getHeight() / 2000f);
matrix.postTranslate(view.getWidth() / 2f, view.getHeight() / 2f);
matrix.mapRect(rectF);
The returned Rectangle by the matrix has all the right coordinates to draw into the canvas.
If you are using the back camera I think is just a matter of changing the scale to:
matrix.setScale(-1, 1);
But I haven't tried that.

The Camera.Face class returns the face bound coordinates using the image frame that the phone would save into its internal storage, rather than using the image displayed in the Camera Preview. In my case, the images were saved in a different manner from the camera, resulting in a incorrect mapping. I had to manually account for the discrepancy by taking the coordinates, rotating it counter clockwise 90 degrees and flipping it on the y-axis prior to scaling it to the canvas used for the custom view.
EDIT:
It would also appear that you can't change the way the face bound coordinates are returned by modifying the camera capture orientation using the Camera.Parameters.setRotation(int) method either.

Related

Add animation to drawLine and drawBitmap

I have a custom view that extends relative layout. In the dispatchDraw method I draw lines and images inside a for loop because i have different start and end points for multiple lines.
I would like to know the easiest way to draw lines with animations from point a to point b in a slow way for example, this animation must be set only for a line that I decide and not for all the others.
I would also like to draw images with animations, I don't want to move the image but I would like to draw the image a little bigger first and then reduce it to the right size, or draw the image from top to bottom. Again this animation must be set only for a image that I decide and not for all the others.
#Override
protected void dispatchDraw(Canvas canvas) {
int width = getWidth();
int height = getHeight();
super.dispatchDraw(canvas);
size = width / (lines);
insideMargin = size / margin;
vMargin = (height - lines * size) / 2f;
canvas.translate(0,vMargin);
for(int x=0; x<lines;x++){
for(int y=0;y<lines;y++){
if(/*a condintion is true*/) {
//draw line with animation, how to do it?
}else{
canvas.drawLine(x * size,
y * size,
(x + 1) * size,
y* size,
paint
);
}
if(/*a condition is true*/) {
//draw bitmap with animation, how to do it?
}else
canvas.drawBitmap(image, src, dest, null);
}
}
}
//i have on touch event that will call invalidate
I think you should use ValueAnimator for your purpose. As I understood, you want to animate values once a touch event occurs. The value returned by the animator must be used for all objects so the animated value should be general as possible. I recommend to use 0 to 1 as float value. You can use this value for the animation. For the bitmap animation, you can define target rectangle start and end size then you can find appropriate size by facilitating the animated fraction value. For the line case, you can use the slope for the animation. I think you already have the line's start and end point so you can use slope and fraction to calculate animated end point.

Use x/y coordinates to set bounds of a moving particle

I have two bitmaps that I draw onto the center of a canvas:
One is only a background, it's a spirit level in top view which doesnt move. The second one is a bitmap looking like a "air bubble". When the user tilts the phone, the sensors read the tilt and the air bubble moves according to the sensor values along the x-axis. However, I need to make sure that the air bubble doesnt move too far, e.g out of the background-bitmap.
So I tried to which x coordinate the bubble can travel to,
before I have to set xPos = xPos -1 using trial and error
This works fine on my device.
To clarify: On my phone, the air bubble could move to the coordinate x = 50 from the middle of the screen. This would be the point, where the bitmap is at the very left of the background spirit level.
On a larger phone, the position x = 50 is too far left, and therefore looking like the air bubble travelled out of the water level.
Now I've tried following:
I calculated the area in % in which the air bubble can move. Let's say that
is 70% of the entire width of the bitmap. So I tried to calculate the two x boundary values:
leftBoundary = XmiddlePoint - (backgroundBitmap.getWidth() * 0.35);
rightBoundary = XmiddlePoint + (backgroundBitmap.getWidth() * 0.35);
...which doesnt work when testing with different screen sizes :(
Is it possible to compensate for different screen sizes and densities using absolute coordinates or do I have to rethink my idea?
If you need any information that I forgot about, please let me know. If this question has already been answered, I would appreciate a link :) Thanks in advance!
Edit:
I load my bitmaps like this:
private Bitmap backgroundBitmap;
private static final int BITMAP_WIDTH = 1898;
private static final int BITMAP_HEIGHT = 438;
public class SimulationView extends View implements SensorEventListener{
public SimulationView(Context context){
Bitmap map = BitmapFactory.decodeResource(getResources, R.mipmap.backgroundImage);
backgroundBitmap = Bitmap.createScaledBitmap(map, BITMAP_WIDTH, BITMAP_HEIGHT, true;
}
and draw it like this:
protected void onDraw(Canvas canvas){
canvas.drawBitmap(backgroundBitmap, XmiddlePoint - BITMAP_WIDTH / 2, YmiddlePont - BITMAP_HEIGHT / 2, null);
}
backgroundBitmap.getWidth() and getHeight() prints out the correct sizes.
Calculating like mentioned above would return following boundaries:
DisplayMetrics displayMetrics = new DisplayMetrics();
((Activity) getContext()).getWindowManager().getDefaultDisplay().getMetrics(displayMetrics);
int width = displayMetrics.widthPixels;
//which prints out width = 2392
xMiddlePoint = width / 2;
// = 1196
leftBoundary = xMiddlePoint - (BITMAP.getWidth()* 0.35);
// = 531,7
However, when I use trial and error, the right x coordinate seems to be at around 700.
I've come across a great explanation on how to fix my issue here.
As user AgentKnopf explained, you have to scale coordinates or bitmaps like this:
X = (targetScreenWidth / defaultScreenWidth) * defaultXCoordinate
Y = (targetScreenHeight / defaultScreenHeight) * defaultYCoordinate
which, in my case, translates to:
int defaultScreenWidth = 1920;
int defaultXCoordinate = 333;
DisplayMetrics displayMetrics = new DisplayMetrics();
((Activity) getContext()).getWindowManager().getDefaultDisplay().getMetrics(displayMetrics);
displayWidth = displayMetrics.widthPixels;
leftBoundary = (displayWidth / defaultScreenWidth) * defaultXCoordinates

What do I do about drawText() and drawRect() drawing in two different locations when given the same coordinates?

I'm trying to draw a text to the screen and then have a rectangle in the same spot to account for knowing when the user clicks within it (and clicks the text). Problem is when I put the text in one spot on the screen and the rectangle in the same spot, it apparently isn't in the same spot. Is there some setting I need to set somewhere?
private final String[] options = {"Start", "High Scores", "Help", "Quit"};
public void draw(final Canvas g)
{
for (int i = 0; i < options.length; i++)
{
width = HORIZONTALOFFSET * 3 + spaceInvadersTitle.getWidth();
height = GamePanel.getScreenHeight() / 4 - (75 * 2 + TEXT_SIZE) / 2 + i * 75;
rect[i] = new Rect(width, height, width + 325, height + TEXT_SIZE);
g.drawRect(rect[i], paint);
g.drawText(options[i], width, height, paint);
}
}
FYI "width" and "height" are more like x- and y-coords. The horizontal alignment is not the problem - just the vertical. If you'll notice from the code, I'm setting the starting x- and y-coordinates for the drawText and drawRect the same exact position, but that's not how they're showing on the screen. Instead it seems the drawText is using that position as a lowerleft anchorpoint or something like that. Is that how it works? If so, how do I change that?
Also, if you have any suggestions on how to approach listening for when the user clicks on a drawn text, I'm all ears. This is the easiest way I could think of, and how I do it in regular desktop Java.

Draw a segmented circle in Android: OpenGL vs Cavans?

I need to draw something like this:
I was hoping that this guy posted some code of how he drew his segmented circle to begin with, but alas he didn't.
I also need to know which segment is where after interaction with the wheel - for instance if the wheel is rotated, I need to know where the original segments are after the rotation action.
Two questions:
Do I draw this segmented circle (with varying colours and content placed on the segment) with OpenGL or using Android Canvas?
Using either of the options, how do I register which segment is where?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
EDIT:
Ok, so I've figured out how to draw the segmented circle using Canvas (I'll post the code as an answer). And I'm sure I'll figure out how to rotate the circle soon. But I'm still unsure how I'll recognize a separate segment of the drawn wheel after the rotation action.
Because, what I'm thinking of doing is drawing the segmented circle with these wedges, and the sort of handling the entire Canvas as an ImageView when I want to rotate it as if it's spinning. But when the spinning stops, how do I differentiate between the original segments drawn on the Canvas?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I've read about how to draw a segment on its own (here also), OpenGL, Canvas and even drawing shapes and layering them, but I've yet to see someone explaining how to recognize the separate segments.
Can drawBitmap() or createBitmap() perhaps be used?
If I go with OpenGL, I'll probably be able to rotate the segmented wheel using OpenGL's rotation, right?
I've also read that OpenGL might be too powerful for what I'd like to do, so should I rather consider "the graphic components of a game library built on top of OpenGL"?
This kind of answers my first question above - how to draw the segmented circle using Android Canvas:
Using the code found here, I do this in the onDraw function:
// Starting values
private int startAngle = 0;
private int numberOfSegments = 11;
private int sweepAngle = 360 / numberOfSegments;
#Override
protected void onDraw(Canvas canvas) {
setUpPaint();
setUpDrawingArea();
colours = getColours();
Log.d(TAG, "Draw the segmented circle");
for (int i = 0; i < numberOfSegments; i++) {
// pick a colour that is not the previous colour
paint.setColor(colours.get(pickRandomColour()));
// Draw arc
canvas.drawArc(rectF, startAngle, sweepAngle, true, paint);
// Set variable values
startAngle -= sweepAngle;
}
}
This is how I set up the drawing area based on the device's screen size:
private void setUpDrawingArea() {
Log.d(TAG, "Set up drawing area.");
// First get the screen dimensions
Point size = new Point();
Display display = DrawArcActivity.this.getWindowManager().getDefaultDisplay();
display.getSize(size);
int width = size.x;
int height = size.y;
Log.d(TAG, "Screen size = "+width+" x "+height);
// Set up the padding
int paddingLeft = (int) DrawArcActivity.this.getResources().getDimension(R.dimen.padding_large);
int paddingTop = (int) DrawArcActivity.this.getResources().getDimension(R.dimen.padding_large);
int paddingRight = (int) DrawArcActivity.this.getResources().getDimension(R.dimen.padding_large);
int paddingBottom = (int) DrawArcActivity.this.getResources().getDimension(R.dimen.padding_large);
// Then get the left, top, right and bottom Xs and Ys for the rectangle we're going to draw in
int left = 0 + paddingLeft;
int top = 0 + paddingTop;
int right = width - paddingRight;
int bottom = width - paddingBottom;
Log.d(TAG, "Rectangle placement -> left = "+left+", top = "+top+", right = "+right+", bottom = "+bottom);
rectF = new RectF(left, top, right, bottom);
}
That (and the other functions which are pretty straight forward, so I'm not going to paste the code here) draws this:
The segments are different colours with every run.

Mapping A "Touch Region" to a Bitmap

I am trying to gain some more familiarity with the Android SurfaceView class, and in doing so am attempting to create a simple application that allows a user to move a Bitmap around the screen. The troublesome part of this implementation is that I am also including the functionality that the user may drag the image again after it has been placed. In order to do this, I am mapping the bitmap to a simple set of coordinates that define the Bitmap's current location. The region I am mapping the image to, however, does not match up with the image.
The Problem
After placing an image on the SurfaceView using canvas.drawBitmap(), and recording the coordinates of the placed image, the mapping system that I have set up misinterprets the Bitmap's coordinates somehow and does not display correctly. As you can see in this image, I have simply used canvas.drawLine() to draw lines representing the space of my touch region, and the image is always off and to the right:
The Code
Here, I shall provide the relevant code excerpts to help answer my question.
CustomSurface.java
This method encapsulates the drawing of the objects onto the canvas. The comments clarify each element:
public void onDraw(Canvas c){
//Simple black paint
Paint paint = new Paint();
//Draw a white background
c.drawColor(Color.WHITE);
//Draw the bitmap at the coordinates
c.drawBitmap(g.getResource(), g.getCenterX(), g.getCenterY(), null);
//Draws the actual surface that is receiving touch input
c.drawLine(g.left, g.top, g.right, g.top, paint);
c.drawLine(g.right, g.top, g.right, g.bottom, paint);
c.drawLine(g.right, g.bottom, g.left, g.bottom, paint);
c.drawLine(g.left, g.bottom, g.left, g.top, paint);
}
This method encapsulates how I capture touch events:
public boolean onTouchEvent(MotionEvent e){
switch(e.getAction()){
case MotionEvent.ACTION_DOWN:{
if(g.contains((int) e.getX(), (int) e.getY()))
item_selected = true;
break;
}
case MotionEvent.ACTION_MOVE:{
if(item_selected)
g.move((int) e.getX(), (int) e.getY());
break;
}
case MotionEvent.ACTION_UP:{
item_selected = false;
break;
}
default:{
//Do nothing
break;
}
}
return true;
}
Graphic.java
This method is used to construct the Graphic:
//Initializes the graphic assuming the coordinate is in the upper left corner
public Graphic(Bitmap image, int start_x, int start_y){
resource = image;
left = start_x;
top = start_y;
right = start_x + image.getWidth();
bottom = start_y + image.getHeight();
}
This method detects if a user is clicking inside the image:
public boolean contains(int x, int y){
if(x >= left && x <= right){
if(y >= top && y <= bottom){
return true;
}
}
return false;
}
This method is used to move the graphic:
public void move(int x, int y){
left = x;
top = y;
right = x + resource.getWidth();
bottom = y + resource.getHeight();
}
I also have 2 methods that determine the center of the region (used for redrawing):
public int getCenterX(){
return (right - left) / 2 + left;
}
public int getCenterY(){
return (bottom - top) / 2 + top;
}
Any help would be greatly appreciated, I feel as though many other StackOverflow users could really benefit from a solution to this issue.
There's a very nice and thorough explanation of touch/multitouch/gestures on Android Developers blog, that includes free and open source code example at google code.
Please, take a look. If you don't need gestures -- just skip that part, read about touch events only.
This issue ended up being much simpler than I had thought, and after some tweaking I realized that this was an issue of image width compensation.
This line in the above code is where the error stems from:
c.drawBitmap(g.getResource(), g.getCenterX(), g.getCenterY(), null);
As you can tell, I manipulated the coordinates from within the Graphic class to produce the center of the bitmap, and then called canvas.drawBitmap() assuming that it would draw from the center outward.
Obviously, this would not work because the canvas always drops from the top left of an image downwards and to the right, so the solution was simple.
The Solution
Create the touch region with regards to the touch location, but draw it relative to a distance equal to the image width subtracted from the center location in the x and y directions. I basically changed the architecture of the Graphic class to implement a getDrawX() and getDrawY() method that would return the modified x and y coordinates of where it should be drawn in order to have the center_x and center_y values (determined in the constructor) actually appear to be at the center of the region.
It all comes down to the fact that in an attempt to compensate for the way the canvas draws bitmaps, I unfortunately incorporated some bad behaviors and in the end had to handle the offset in a completely different way.

Categories

Resources