I know there is some post talking about this topic but I could not find my answer.
I want to calibrate my android camera without chessboard for 3d reconstruction, so I need my intrinsic and extrinsic parameters.
My first goal is to extract the 3D real system to be able to put some 3d Model on screen.
My step :
From a picture of a building I extract 4 points that represent the real 3D system
/!\ this step require camera calibration /!\
Convert them to 3d Point (solvePnP for exemple)
Then from my 3D Axis I create a OpenGL projection and modelview matrix
My main problem is that I want to avoid a calibration step, so how can calibrate without chessboard? I have some data from android such as focal length. I can guess that the projection center is the center of my camera picture.
Any idea or advice? or other way to do it ?
here is nochess calibation of qtcalib.
This scheme is recomended when you need obtain a camera calibration
from a image that don't have calibration chessboard. In this case, you
can approximate the camera calibration if you know 4 points in the
image forming a flat rectangle in real world. Is important to remark
that the aproximated calibration depends on the 4 selected points and
the values that you will set for the dimensions of the rectangle
Related
I am very new in terms of Augmented Reality with some experience on OpenGL. I wanna draw something such as a triangle with OpenGL ES(Android or iOS, doesn't matter for the platform). The background needs to be captured by mobile's camera. Basically the result likes the Pokemon Go, via the camera you got the real world as the background and a Pokemon inserts into the real world.
So, where should I begin?
One approach.
Capture the image and load into a 2D texture map.
Render a quadrilateral with this 2D texture. The quadrilateral will need to be far enough from your virtual camera so that it forms a background (you will want to enable depth testing). The quadrilateral will have to be large enough to cover the background (which depends on both the distance from the camera and your perspective field of view).
Now render your scene (triangles).
The figure below shows the view frustum in the xz-plane once you have things in view coordinates. N and F are the distances to the near and far clipping planes and θ is the vertical field of view -- let a = aspect ratio = w/h of your image (which should match the aspect ratio of your viewport). H is the height of the quad you wish to render and Q is the distance to the quad from the camera. The height of the quad should then be H = 2*Qtan(θ/2). The width of your quad is W = aH.
The distances from the camera of the objects you wish to be in the foreground should be between N and Q.
I am assuming you know how to set of the view matrix (via the "look-at transform") to position of your camera, and set the projection matrix to specify the perspective projection. Also assuming many other things (like how to load a texture map, draw a filled quad with texture coordinates, enable depth testing, etc...). If you need more details let me know.
If you wish to embed the object within the scene that will require some computer vision techniques (ascertaining depth via stereo image pairs). This is a non-trivial problem.
I was inspired by Pokemon GO and wanted to make a simple prototype for learning purposes. I am a total beginner in image processing.
I did a little research on the subject. Here is what I came up with. In order to place any 3D model in the real world, I must know the orientation. Say if I am placing a cube on a table:
1) I need to know the angles $\theta$, $\phi$ and $\alpha$ where $\theta$ is the rotation along the global UP vector, $\phi$ is the rotation along the camera's FORWARD vector and $\alpha$ is rotation along camera's RIGHT vector.
2) Then I have to multiply these three rotation matrices with the object's transform using these Euler angles.
3) The object's position would be at the center point of the surface for protyping.
4) I can find the distance of the surface using android's camera's inbuilt distance estimation using focal length. Then I can scale the object accordingly.
Is there any more straight forward way to do this using OpenCV or am I going in the right track?
In OpenCV I use the camera to capture a scene containing two squares a and b, both at the same distance from the camera, whose known real sizes are, say, 10cm and 30cm respectively. I find the pixel widths of each square, which let's say are 25 and 40 pixels (to get the 'pixel-width' OpenCV detects the squares as cv::Rect objects and I read their width field).
Now I remove square a from the scene and change the distance from the camera to square b. The program gets the width of square b now, which let's say is 80. Is there an equation, using the configuration of the camera (resolution, dpi?) which I can use to work out what the corresponding pixel width of square a would be if it were placed back in the scene at the same distance as square b?
The math you need for your problem can be found in chapter 9 of "Multiple View Geometry in Computer Vision", which happens to be freely available online: https://www.robots.ox.ac.uk/~vgg/hzbook/hzbook2/HZepipolar.pdf.
The short answer to your problem is:
No not in this exact format. Given you are working in a 3D world, you have one degree of freedom left. As a result you need to get more information in order to eliminate this degree of freedom (e.g. by knowing the depth and/or the relation of the two squares with respect to each other, the movement of the camera...). This mainly depends on your specific situation. Anyhow, reading and understanding chapter 9 of the book should help you out here.
PS: to me it seems like your problem fits into the broader category of "baseline matching" problems. Reading around about this, in addition to epipolar geometry and the fundamental matrix, might help you out.
Since you write of "squares" with just a "width" in the image (as opposed to "trapezoids" with some wonky vertex coordinates) I assume that you are considering an ideal pinhole camera and ignoring any perspective distortion/foreshortening - i.e. there is no lens distortion and your planar objects are exactly parallel to the image/sensor plane.
Then it is a very simple 2D projective geometry problem, and no separate knowledge of the camera geometry is needed. Just write down the projection equations in the first situation: you have 4 unknowns (the camera focal length, the common depth of the squares, the horizontal positions of their left sides (say), and 4 equations (the projections of each of the left and right sides of the squares). Solve the system and keep the focal length and the relative distance between the squares. Do the same in the second image, but now with known focal length, and compute the new depth and horizontal location of square b. Then add the previously computed relative distance to find where square a would be.
In order to understand the transformations performed by the camera to project the 3D world in the 2D image you need to know its calibration parameters. These are basically divided into two sets:
Intrensic parameters: These are fixed parameters that are specific for each camera. They are normally represented by a Matrix called k.
Extrensic parameters: These depend on the camera position in the 3D world. Normally they are represented by two matrices: R and T where the first one represents the rotation and the second one represents the translation
In order to calibrate a camera your need some pattern (basically a set of 3D points which coordinates are known). There are several examples for this in OpenCV library which provides support to perform the camera calibration:
http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html
Once you have your camera calibrated you can transform from 3D to 2D easily by the following equation:
Pimage = K · R · T · P3D
So it will not only depend on the position of the camera but it depends on all the calibration parameters. The following presentation go through the camera calibration details and the different steps and equations that are used during the 3D <-> Image transformations.
https://www.cs.umd.edu/class/fall2013/cmsc426/lectures/camera-calibration.pdf
With this in mind you can project whatever 3D point to the image and get its coordinate on it. The reverse transformation is not unique since going back from 2D to 3D will give you a line instead of a unique point.
I am developing an AR application on Android with Eclipse. I have the 4 corner coordinates of the marker which a virtual object must augment on it. My question is how can I call an OpenCV calibration function to calculate the
camera's extrinsic parameters including rotation vector and
translation vector? I would really appreciate it if you answer me quickly :-)
I am new in OpenCV.
I found http://docs.opencv.org/java/org/opencv/calib3d/Calib3d.html but I don't know how to use these functions or where should I start.
If I understand you well, you have 4 points in the image and you know their world coordinates (you have 4 2D-3D correspondences) to find the rotation matrix and translation vector (known as the camera pose), you can use the solvePnP function.
This function takes as inputs the 3D coordinates, 2D coordinates, the camera matrix (focal distance and center of projection) and the distortion coefficients, which you should have obtained by an intrinsic calibration process. The output is the rotation and translation vectors. The obtained rotation vector can be transformed in to a rotation matrix with the function Rodrigues.
There is also a version of solvePnP that uses a RANSAC algorithm to discard outliers.
You can find here a document with some theory on pose estimation.
Edit
Obtaining camera intrinsic parameters is known as camera calibration, you can find instructions on how to do so in this OpenCV document. There is also an older tutorial with source code here.
I am trying to make a AR app for android in witch the user points the camera to a square marker and a 3d model should show on top of the marker.
I am using OpenCV to get the rotation and translation of the marker but..
To get these 2 matrices, I use solvePnP for which I have to provide a camera matrix and a
distortion matrix which (from my understanding) are specific for each camera type.
Unfortunately, this is a huge drawback for me since the app should be supported by most Android devices and I also can't ask the user to run the camera calibration procedure (functions provided by openCV).
So the question: is there a way to eliminate the camera and distortion matrices? Or, is there another way to calculate the marker 3D position relative to the device?
I tried QCAR and Unity AR but (since the 3D models are downloaded form a server and are changing constantly), I was forced to go with OpenCV.
Any help will be really appreciated.
Thanks.
Bad news for you. The answer is no.
You can't know anything about your image if you don't know anything about your camera. The camera is described by the two matrices: camera mat and distortions mat.
But.. along with the bad news, there are some good news. You can do something, maybe.
Distortion matrix can be ignored in many AR applications. You just don't care about the small distortions.
And the camera matrix can be constructed if you know camera's field of view.
Camera matrix:
w/(2*tan(camera.fovX())), 0, w/2, 0,
0, h/(2*tan(camera.fovY())), h/2, 0,
0, 0, 1, 0
On Android there's an API for retrieving the aperture: getHorizontalAperture() and getVerticalAperture(). Aperture is field of view.
However, on many phones, the returned value isn't correct, because the manufacturers did not care to set it right. It might return bogus angles like 10 or 180 degrees. There's no good way to fix this apart from testing individual phone models.
Good luck!