How can i handle the rotation issue with Preview & FaceDetection - android

i got some issue nagging me for quite some time now.
I got my custom Camera-App that shows a live Preview. Aswell i am using the FaceDetection for getting a better focus on peoples faces. When i check my taken Pictures in Gallery i can see the Rectangle correctly. The next step is to make the FaceDetection-Rectangle visible in the Live-Preview. So i decided to use a canvas that gets the coordinates from the Preview-Rectangle and transform them to coordinates that can be used by the canvas.
My problem is that i had to rotate the Preview by 90degrees that it shows the Preview correctly. So when i also rotate the View of the canvas before i draw it, the rectangle shows correctly and moves the right axis aswell. BUT the rectangle can move out of screen on left and right, and only uses about half of the available height. I assume that the rotating causes the trouble, but i can't manage to put things right. Someone got an idea?
Screenshot (i added purple lines to show the top/bottom-parts that cant be reached by red rectangle):
Preview:
mCamera = Camera.open();
mCamera.getParameters();
mCamera.setDisplayOrientation(90);
mCamera.setFaceDetectionListener(new FaceDetectionListener() {
#Override
public void onFaceDetection(Face[] faces, Camera camera) {
if(mDraw != null) {
mDraw.update(f.rect, f);
}
}
}
}
});
mCamera.startFaceDetection();
}
private DrawOnTop mDraw = null;
public void setDrawOnTop(DrawOnTop d) {
this.mDraw = d;
}
DrawOnTop:
public DrawOnTop(Context context) {
super(context);
myColor = new Paint();
myColor.setStyle(Paint.Style.STROKE);
myColor.setColor(Color.RED);
}
#Override
protected void onDraw(Canvas canvas) {
rect.set((((rect.left+1000)*1000) / WIDTH_DIVISOR),(((rect.top+1000)*1000) / HEIGHT_DIVISOR),(((rect.right+1000)*1000) / WIDTH_DIVISOR),(((rect.bottom+1000)*1000) / HEIGHT_DIVISOR ));
setRotation(90);
canvas.drawRect(rect, myColor);
}
public void update(Rect rect, Face face) {
this.invalidate();
this.rect = rect;
this.face = face;
}
----------------------------------------------------------------------------------------
EDIT:
i came to the conclusion that this is a rare but known bug and that there is so far no other solution but forcing the application to landscape-mode. Works ok, but dimensions look a bit stretched or clinched depending on which perspective the user is operating.

EDIT: I misread the question and talked about the wrong rectangle. This is what i meant:
Basically, you just need to scale the purple rectangle. Find ut where it is defined, then put it onto a canvas and do the following:
float screenWidth = /*get the width of your screen here*/;
float xScale = rect.width / screenWidth;
float screenHeight = /*get the height of your screen here*/;
float yScale = rect.height / screenWidth;
canvas.setScaleX(xScale);
canvas.setScaleY(yScale);
This way, the coordinates will be translated properly.
SECOND EDIT (in response to your comment): You can also do this with views, if you like.
Have fun.

Related

View gets cropped above 10.000 pixel width

UPDATE:
Works on Emulator with Android Oreo (8.X). I have the possibility to do changes right to the android sources, so it would also help if someone knews what in the android sources I have to change or update to get this working (so I don't really need an Android 7 workaround for this. An update to Android 8 though is not possible.)
I'm having a SurfaceView inside a FrameLayout. The SurfaceView usually displays a video, for example purposes I'm actually drawing an image.
The problem is, if I'm setting the size of the FrameLayout (or the SurfaceView) above 10.000 pixels in width, it gets cropped on the left side.
Tested on Android 7.1.1 (on a device and Emulator: Android TV (1080p) API 25
public class TestActivity extends Activity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
//add surfaceview
TestSurfaceView testSurfaceView = new TestSurfaceView(this);
setContentView(testSurfaceView);
//resize
FrameLayout.LayoutParams bgParams = (FrameLayout.LayoutParams) testSurfaceView.getLayoutParams();
//testcase full hd - working
bgParams.width = 1920;
//testcase 2 - working - each segment is 330px as 9900 / 1920 * 64 (default segment width) == 330
//bgParams.width = 9900;
//testcase 3 - working
//bgParams.width = 10000;
//testcase 4 - not working - each segment is 335px which is correct but first cell gets cropped by exactly 50px to the left
//bgParams.width = 10050;
bgParams.height = (int)Math.floor(bgParams.width * (9d/16d)); //16:9
testSurfaceView.setX(0); //doesnt help
/*
Also the position counts into the 10000px limitation as you can see on following testcases
*/
/*
bgParams.width = 9900;
bgParams.height = (int)Math.floor(bgParams.width * (9d/16d)); //16:9
//works - as 9900 + 90 < 10000
testSurfaceView.setX(90);
//doesnt work, crops 50px to the left - 9900 + 150 -> 10050
testSurfaceView.setX(150);
*/
}
}
public class TestSurfaceView extends SurfaceView implements SurfaceHolder.Callback
{
public TestSurfaceView(TestActivity context) {
super(context);
SurfaceHolder holder = this.getHolder();
holder.addCallback(this);
}
#Override
public void surfaceCreated(SurfaceHolder surfaceHolder)
{
//load bitmap from file
BitmapFactory.Options options = new BitmapFactory.Options();
options.inPreferredConfig = Bitmap.Config.ARGB_8888;
Bitmap bitmap = BitmapFactory.decodeFile(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS).getAbsolutePath() + "/testimg.png", options);
Canvas c = surfaceHolder.lockCanvas();
Rect rect = new Rect();
rect.set(0,0, 1920, 1080); //image size
Rect destRect = new Rect();
destRect.set(0, 0, this.getWidth(), this.getHeight());
//draw the image on the surface
c.drawBitmap(bitmap, rect, destRect, null);
surfaceHolder.unlockCanvasAndPost(c);
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
}
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
}
}
styles.xml - for the theme
<resources>
<!-- Base application theme. -->
<style name="AppTheme" parent="android:Theme.Material.Light.NoActionBar.Fullscreen">
<!-- Customize your theme here. -->
</style>
</resources>
The code above produces following output, which is correct.
If I change the bgParams.width to 9600 - it scales up correctly and still displays starting from the left edge of the image:
But if I change the code to e.g. 10050, the image gets cropped by 50 pixels to the left.
If I set:
destRect.set(50, 0, this.getWidth(), this.getHeight());
It gets displayed correctly, but as I can't do that for the MediaPlayer and it's super weird, I'm trying to find a good solution.
I also tried setting the sizes directly on the SurfaceView and instead of changing the LayoutParams, I tried setting scaleX and scaleY of the Framelayout but ended up with the same results.
(btw. opengl max texture size is about 16000px - setting it above the ~16000px results in a black screen and an exception, so that is not the cause of the problem)
Update:
Posted all sources. Anyway here is the complete android studio project:
WeTransfer project download link

Converting Camera Coordinates to Custom View Coordinates

I am trying to make a simple face detection app consisting of a SurfaceView (essentially a camera preview) and a custom View (for drawing purposes) stacked on top. The two views are essentially the same size, stacked on one another in a RelativeLayout. When a person's face is detected, I want to draw a white rectangle on the custom View around their face.
The Camera.Face.rect object returns the face bound coordinates using the coordinate system explained here and the custom View uses the coordinate system described in the answer to this question. Some sort of conversion is needed before I can use it to draw on the canvas.
Therefore, I wrote an additional method ScaleFacetoView() in my custom view class (below) I redraw the custom view every time a face is detected by overriding the OnFaceDetection() method. The result is the white box appears correctly when a face is in the center. The problem I noticed is that it does not correct track my face when it moves to other parts of the screen.
Namely, if I move my face:
Up - the box goes left
Down - the box goes right
Right - the box goes upwards
Left - the box goes down
I seem to have incorrectly mapped the values when scaling the coordinates. Android docs provide this method of converting using a matrix, but it is rather confusing and I have no idea what it is doing. Can anyone provide some code on the correct way of converting Camera.Face coordinates to View coordinates?
Here's the code for my ScaleFacetoView() method.
public void ScaleFacetoView(Face[] data, int width, int height, TextView a){
//Extract data from the face object and accounts for the 1000 value offset
mLeft = data[0].rect.left + 1000;
mRight = data[0].rect.right + 1000;
mTop = data[0].rect.top + 1000;
mBottom = data[0].rect.bottom + 1000;
//Compute the scale factors
float xScaleFactor = 1;
float yScaleFactor = 1;
if (height > width){
xScaleFactor = (float) width/2000.0f;
yScaleFactor = (float) height/2000.0f;
}
else if (height < width){
xScaleFactor = (float) height/2000.0f;
yScaleFactor = (float) width/2000.0f;
}
//Scale the face parameters
mLeft = mLeft * xScaleFactor; //X-coordinate
mRight = mRight * xScaleFactor; //X-coordinate
mTop = mTop * yScaleFactor; //Y-coordinate
mBottom = mBottom * yScaleFactor; //Y-coordinate
}
As mentioned above, I call the custom view like so:
#Override
public void onFaceDetection(Face[] arg0, Camera arg1) {
if(arg0.length == 1){
//Get aspect ratio of the screen
View parent = (View) mRectangleView.getParent();
int width = parent.getWidth();
int height = parent.getHeight();
//Modify xy values in the view object
mRectangleView.ScaleFacetoView(arg0, width, height);
mRectangleView.setInvalidate();
//Toast.makeText( cc ,"Redrew the face.", Toast.LENGTH_SHORT).show();
mRectangleView.setVisibility(View.VISIBLE);
//rest of code
Using the explanation Kenny gave I manage to do the following.
This example works using the front facing camera.
RectF rectF = new RectF(face.rect);
Matrix matrix = new Matrix();
matrix.setScale(1, 1);
matrix.postScale(view.getWidth() / 2000f, view.getHeight() / 2000f);
matrix.postTranslate(view.getWidth() / 2f, view.getHeight() / 2f);
matrix.mapRect(rectF);
The returned Rectangle by the matrix has all the right coordinates to draw into the canvas.
If you are using the back camera I think is just a matter of changing the scale to:
matrix.setScale(-1, 1);
But I haven't tried that.
The Camera.Face class returns the face bound coordinates using the image frame that the phone would save into its internal storage, rather than using the image displayed in the Camera Preview. In my case, the images were saved in a different manner from the camera, resulting in a incorrect mapping. I had to manually account for the discrepancy by taking the coordinates, rotating it counter clockwise 90 degrees and flipping it on the y-axis prior to scaling it to the canvas used for the custom view.
EDIT:
It would also appear that you can't change the way the face bound coordinates are returned by modifying the camera capture orientation using the Camera.Parameters.setRotation(int) method either.

9patch looks pixeled when setSize is called too many times

I have a sprite that is supposed to act like a loadbar. I have tried this by using an example image that has been created like a 9patch-type (http://cdn.dibbus.com/wp-content/uploads/2011/03/btn_black.9.png). It seems okay in the start, but as the width of the sprite increases the sprite starts to look pixeled. Anyone know what the problem could be, or have any solution? The code is shown below.
public Sprite loaded;
public void init()
{
atlas = new TextureAtlas(Gdx.files.
internal("data/misc/menu_button.pack"));
loaded = atlas.createSprite("loadbar");
loaded.setPosition((Misc.WIDTH/2) - unloaded.getWidth()/2,
Misc.HEIGTH - unloaded.getHeight());
}
public void draw_load_bar() //render function
{
if(loaded.getWidth() < 600)
{
loaded.setSize(loaded.getWidth()+ 0.5f, loaded.getHeight());
}
loaded.draw(batch);
}
Dont use a Sprite to stretch it. I'd recommend a real Ninepatch from libgdx for it.
public NinePatch loaded;
private float posX, posY, width, height;
public void init()
{
loaded = new NinePatch(the Texture here,10, 10, 10, 10); //bounds outside
//set right pos here...
}
public void draw_load_bar(SpriteBatch batch) //render function
{
if(loaded.getWidth() < 600)
{
//update the size of it here (guess pos is static)
width++;
}
//need to parse the batch and the right sizes.
loaded.draw(batch, posx, posy, width, height);
}
after that you can handle it like a Sprite or Texture but it does stratch right without issues. If you want to the full Picture to be Stretched simply do net set any bounds at the creation new NinePatch(texture, 0,0,0,0)

Bitmap Matrix postRotate changes X and Y axis direction

I'm trying to rotate my Bitmap using a readymade solution I found somewhere. The code is below:
public void onDraw(Canvas canvas) {
float x = ship.Position.left;
float y = ship.Position.top;
canvas.drawBitmap(ship.ship, x,y,null);
invalidate();
}
However, when I do it, the X and Y axii change their direction - if I increase the Y the image goes towards the top of the screen, not towards the bottom. Same happens to X if I rotate by 90 degrees.
I need to rotate it but without changing the Y and X axii directions.
Even rotated, I still want the Bitmap to go towards the bottom if I increase Y and to the right if I increase the X.
public void update()
{
if(!moving)
{
fall();
}
else //moving
{
move();
faceDirection();
}
Position.top += Speed;
}
private void move() {
if(Speed < MAXSPEED)
Speed -= 0.5f;
}
private void fall() {
if(Speed > MAXSPEED*-1)
Speed += 0.2f;
}
private void faceDirection() {
double OldDiretion = Direction;
Direction = DirectionHelper.FaceObject(Position, ClickedDiretion);
if (Direction != OldDiretion)
{
Matrix matrix = new Matrix();
matrix.postRotate((float)Direction);
ship = Bitmap.createBitmap(ship, 0, 0, ship.getWidth(),ship.getHeight(), matrix, false);
}
I tried the code above, but it's still changing the Y direction, It's going to bottom of the BitMap, not bottom of the screen.
Here is the project: https://docs.google.com/file/d/0B8V9oTk0eiOKOUZJMWtsSmUtV3M/edit?usp=sharing
You should first rotate, than translate:
matrix.postTranslate(x, y);
matrix.postRotate(degree);
alternative would be to try to use preRotate() instead of postRotate().
I also strongly recommend to translate/rotate the original while drawing. So your createBitmap() call shouldn't modify the orientation. At least not when you change it dynamically on user interaction. Otherwise you would create a lot of bitmaps to represent rotations over and over again which would impact the performance.
The problem is that you don't actually rotate the bitmap - you just draw it rotated. So, the next time you redraw it, you first push it towards the bottom or right by incrementing x/y and then rotate it.
You have to actually rotate the bitmap itself. You could use the following code:
ship.ship = Bitmap.createBitmap(ship.ship, 0, 0, ship.ship.getWidth(), ship.ship.getHeight(), matrix, false);
Here you create a new rotated bitmap and set your reference to point to it.
Note! You must do this only once! So you can't do it in the onDraw method, since then it will get rotated every time it's redrawn. You have to do it somewhere else and then draw it as usual in onDraw (without the matrix rotations).

Draw Circle is inaccurate

In my app, when the user touches the screen, a circle is drawn, and if a circle has already been drawn at point a, then another circle cannot be drawn there.
when I use canvas.drawCircle(...), it's innacurate. if I tap close to the top of the screen, then it is just very minutely missing where I tapped it (slightly to the left or right). The farther I go down the farther up my circle is from where I touched. If I'm touching the left side of the screen, the circle goes to the right of the touch point, and if I touch the right side of the screen, the circle is on the left.
Here's my code:
public void onCreate(...){
super.onCreate();
setcontentView(...);
drawnX = new ArrayList<Float>();
drawnY = new ArrayList<Float>();
layout = (ImageView)findViewById(R.id.layout);
layout.setOnTouchListener(new View.OnTouchListener() {
#Override
public boolean onTouch(View view, MotionEvent me) {
newY = me.getY();
newX = me.getX();
int action = me.getAction();
if (action==0){
Log.v(tag, "New Touch");
Log.i(tag, String.valueOf("Action " + action +
" happened at X,Y: " + "("+newX+","+newY + ")"));
getDistance(newX, newY);
}
return true;
}
});
#Override
public void onResume(){
super.onResume();
display = getWindowManager().getDefaultDisplay();
screenHeight = display.getHeight();
screenWidth = display.getWidth();
bitmap = Bitmap.createBitmap(screenWidth, screenHeight, Bitmap.Config.ARGB_8888);
canvas = new Canvas(bitmap);
paint = new Paint();
circleRadius = screenHeight/50;
}
public void getDistance(float xNew, float yNew){
// here I compare the new touch x and y values with the x and y values saved
// in the Array Lists I made above. If distance is > circleRadius, then:
drawPoint(xNew, yNew)
}
public void drawPoint(final float x, final float y){
minDistance = Float.NaN;
new Thread(new Runnable(){
#Override
public void run(){
paint.setColor(Color.RED);
canvas.drawCircle(x, y, circleRadius, paint);
layout.post(new Runnable(){
#Override
public void run(){
layout.setImageDrawable(new BitmapDrawable(bitmap));
//I have also tried setImageBitmap and setBackgroundDrawable
}
});
}
}).start();
}
Int the process of all of this, I also do some logging in the drawPoint, and it shows that the x and y coordinates match that which are gotten in the onTouch.
I've also tried maknig it a LinearLayout, but that had the same effect.
And, the bitmap size matches the screen size and canvas size. also, for testing I did
canvas.drawARGB(...)
and it covered the whole screen, so I know it's not just that the bitmap is not stretching to the bottom. The layout's height is less than the height of everything else, and the width of the layout is the same as everything else, but when I use layout.getHeight and layout.getWidth in my onResume, they always return 0, so I can't really use that.
I really have no clue what's causing it. I also tried it on two emulators and got the same response. Any help would be greatly appreciated.
Also, if I tap on the drawn circle (toward the bottom of the screen), another circle will then be drawn above that one, but if I tap where I previously tapped, then a new circle will not be drawn, which really is what's suppose to happen. the circle is just not showing on the screen correctly.
I found the answer.
Oddly enough, even though the width of the bitmap matched the width of the layout, when I did a little more testing, I found the dimensions didn't actually fit together on the left and right either.
I wasn't using the layout.getWidth() and layout.getHeight() due to it returning 0 in onResume()
But, I thought the different height from display.getHeight() very well may be causing the problem, so I did essentially:
new Handler().postDelayed(getLayoutSizeRunnable, 5000);
where getLayoutSizeRunnable was essentially:
screenWidth = layout.getWidth();
screenHeight = layout.getHeight();
Then everything worked perfectly.
And for the usage in my app, I'll use an if statement so the first time a touch point is made, the layout size will be calculated, and the bitmap will be created based on it.

Categories

Resources