Camera Calibration and 3D Reconstruction

https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html

The functions in this section use a so-called pinhole camera model. In this model, a scene view is formed by projecting 3D points into the image plane using a perspective transformation.

$s \; m' = A [R|t] M'$

$s \vecthree{u}{v}{1} = \vecthreethree{f_x}{0}{c_x}{0}{f_y}{c_y}{0}{0}{1}
\begin{bmatrix}
r_{11} & r_{12} & r_{13} & t_1 \\
r_{21} & r_{22} & r_{23} & t_2 \\
r_{31} & r_{32} & r_{33} & t_3
\end{bmatrix}
\begin{bmatrix}
X \\
Y \\
Z \\
1
\end{bmatrix}$

where:

$(X, Y, Z)$

are the coordinates of a 3D point in the world coordinate space

$(u, v)$ are the coordinates of the projection point in pixels

$A$ is a camera matrix, or a matrix of intrinsic parameters

$(cx, cy)$ is a principal point that is usually at the image center

$fx, fy$ are the focal lengths expressed in pixel units.

Thus, if an image from the camera is scaled by a factor, all of these parameters should be scaled (multiplied/divided, respectively) by the same factor. The matrix of intrinsic parameters does not depend on the scene viewed. So, once estimated, it can be re-used as long as the focal length is fixed (in case of zoom lens). The joint rotation-translation matrix $[R|t]$

is called a matrix of extrinsic parameters. It is used to describe the camera motion around a static scene, or vice versa, rigid motion of an object in front of a still camera. That is, $[R|t]$ translates coordinates of a point $(X, Y, Z)$ to a coordinate system, fixed with respect to the camera. The transformation above is equivalent to the following (when $z \ne 0$ ):

$\begin{array}{l}
\vecthree{x}{y}{z} = R \vecthree{X}{Y}{Z} + t \\
x' = x/z \\
y' = y/z \\
u = f_x*x' + c_x \\
v = f_y*y' + c_y
\end{array}$

The following figure illustrates the pinhole camera model.

../../../_images/pinhole_camera_model.png

Real lenses usually have some distortion, mostly radial distortion and slight tangential distortion. So, the above model is extended as:

$\begin{array}{l} \vecthree{x}{y}{z} = R \vecthree{X}{Y}{Z} + t \\ x' = x/z \\ y' = y/z \\ x'' = x' \frac{1 + k_1 r^2 + k_2 r^4 + k_3 r^6}{1 + k_4 r^2 + k_5 r^4 + k_6 r^6} + 2 p_1 x' y' + p_2(r^2 + 2 x'^2) \\ y'' = y' \frac{1 + k_1 r^2 + k_2 r^4 + k_3 r^6}{1 + k_4 r^2 + k_5 r^4 + k_6 r^6} + p_1 (r^2 + 2 y'^2) + 2 p_2 x' y' \\ \text{where} \quad r^2 = x'^2 + y'^2 \\ u = f_x*x'' + c_x \\ v = f_y*y'' + c_y \end{array}$

$k_1$ , $k_2$ , $k_3$ , $k_4$ , $k_5$ , and $k_6$ are radial distortion coefficients. $p_1$ and $p_2$ are tangential distortion coefficients. Higher-order coefficients are not considered in OpenCV.

The next figure shows two common types of radial distortion: barrel distortion (typically $k_1 > 0$ and pincushion distortion (typically $k_1 < 0$ ).

../../../_images/distortion_examples.png

In the functions below the coefficients are passed or returned as

$(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6]])$

vector. That is, if the vector contains four elements, it means that $k_3=0$ . The distortion coefficients do not depend on the scene viewed. Thus, they also belong to the intrinsic camera parameters. And they remain the same regardless of the captured image resolution. If, for example, a camera has been calibrated on images of 320 x 240 resolution, absolutely the same distortion coefficients can be used for 640 x 480 images from the same camera while $f_x$ , $f_y$ , $c_x$ , and $c_y$ need to be scaled appropriately.

The functions below use the above model to do the following:

Project 3D points to the image plane given intrinsic and extrinsic parameters.

Compute extrinsic parameters given intrinsic parameters, a few 3D points, and their projections.

Estimate intrinsic and extrinsic camera parameters from several views of a known calibration pattern (every view is described by several 3D-2D point correspondences).

Estimate the relative position and orientation of the stereo camera “heads” and compute the rectification transformation that makes the camera optical axes parallel.

Note

A calibration sample for 3 cameras in horizontal position can be found at opencv_source_code/samples/cpp/3calibration.cpp
A calibration sample based on a sequence of images can be found at opencv_source_code/samples/cpp/calibration.cpp
A calibration sample in order to do 3D reconstruction can be found at opencv_source_code/samples/cpp/build3dmodel.cpp
A calibration sample of an artificially generated camera and chessboard patterns can be found at opencv_source_code/samples/cpp/calibration_artificial.cpp
A calibration example on stereo calibration can be found at opencv_source_code/samples/cpp/stereo_calib.cpp
A calibration example on stereo matching can be found at opencv_source_code/samples/cpp/stereo_match.cpp
(Python) A camera calibration sample can be found at opencv_source_code/samples/python2/calibrate.py

calibrateCamera

Finds the camera intrinsic and extrinsic parameters from several views of a calibration pattern.

C++: double calibrateCamera(InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints, Size imageSize, InputOutputArray cameraMatrix, InputOutputArray distCoeffs, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs, int flags=0, TermCriteria criteria=TermCriteria( TermCriteria::COUNT+TermCriteria::EPS, 30, DBL_EPSILON) )

Python: cv2.calibrateCamera(objectPoints, imagePoints, imageSize[, cameraMatrix[, distCoeffs[, rvecs[, tvecs[, flags[, criteria]]]]]]) → retval, cameraMatrix, distCoeffs, rvecs, tvecs

C: double cvCalibrateCamera2(const CvMat* object_points, const CvMat* image_points, const CvMat* point_counts, CvSize image_size, CvMat* camera_matrix, CvMat* distortion_coeffs, CvMat* rotation_vectors=NULL, CvMat* translation_vectors=NULL, int flags=0, CvTermCriteria term_crit=cvTermCriteria( CV_TERMCRIT_ITER+CV_TERMCRIT_EPS,30,DBL_EPSILON) )

Python: cv.CalibrateCamera2(objectPoints, imagePoints, pointCounts, imageSize, cameraMatrix, distCoeffs, rvecs, tvecs, flags=0) → None

Parameters:

Parameters:	objectPoints – In the new interface it is a vector of vectors of calibration pattern points in the calibration pattern coordinate space (e.g. std::vector<std::vector<cv::Vec3f>>). The outer vector contains as many elements as the number of the pattern views. If the same calibration pattern is shown in each view and it is fully visible, all the vectors will be the same. Although, it is possible to use partially occluded patterns, or even different patterns in different views. Then, the vectors will be different. The points are 3D, but since they are in a pattern coordinate system, then, if the rig is planar, it may make sense to put the model to a XY coordinate plane so that Z-coordinate of each input object point is 0. In the old interface all the vectors of object points from different views are concatenated together. imagePoints – In the new interface it is a vector of vectors of the projections of calibration pattern points (e.g. std::vector<std::vector<cv::Vec2f>>). `imagePoints.size()` and `objectPoints.size()` and `imagePoints[i].size()` must be equal to `objectPoints[i].size()` for each `i`. In the old interface all the vectors of object points from different views are concatenated together. point_counts – In the old interface this is a vector of integers, containing as many elements, as the number of views of the calibration pattern. Each element is the number of points in each view. Usually, all the elements are the same and equal to the number of feature points on the calibration pattern. imageSize – Size of the image used only to initialize the intrinsic camera matrix. cameraMatrix – Output 3x3 floating-point camera matrix $A = \vecthreethree{f_x}{0}{c_x}{0}{f_y}{c_y}{0}{0}{1}$ . If `CV_CALIB_USE_INTRINSIC_GUESS` and/or `CV_CALIB_FIX_ASPECT_RATIO` are specified, some or all of `fx, fy, cx, cy` must be initialized before calling the function. distCoeffs – Output vector of distortion coefficients $(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6]])$ of 4, 5, or 8 elements. rvecs – Output vector of rotation vectors (see `Rodrigues()` ) estimated for each pattern view (e.g. std::vector<cv::Mat>>). That is, each k-th rotation vector together with the corresponding k-th translation vector (see the next output parameter description) brings the calibration pattern from the model coordinate space (in which object points are specified) to the world coordinate space, that is, a real position of the calibration pattern in the k-th pattern view (k=0.. M -1). tvecs – Output vector of translation vectors estimated for each pattern view. flags – Different flags that may be zero or a combination of the following values: CV_CALIB_USE_INTRINSIC_GUESS `cameraMatrix` contains valid initial values of `fx, fy, cx, cy` that are optimized further. Otherwise, `(cx, cy)` is initially set to the image center ( `imageSize` is used), and focal distances are computed in a least-squares fashion. Note, that if intrinsic parameters are known, there is no need to use this function just to estimate extrinsic parameters. Use `solvePnP()` instead. CV_CALIB_FIX_PRINCIPAL_POINT The principal point is not changed during the global optimization. It stays at the center or at a different location specified when `CV_CALIB_USE_INTRINSIC_GUESS` is set too. CV_CALIB_FIX_ASPECT_RATIO The functions considers only `fy` as a free parameter. The ratio `fx/fy` stays the same as in the input `cameraMatrix` . When `CV_CALIB_USE_INTRINSIC_GUESS` is not set, the actual input values of `fx` and `fy` are ignored, only their ratio is computed and used further. CV_CALIB_ZERO_TANGENT_DIST Tangential distortion coefficients $(p_1, p_2)$ are set to zeros and stay zero. CV_CALIB_FIX_K1,...,CV_CALIB_FIX_K6 The corresponding radial distortion coefficient is not changed during the optimization. If `CV_CALIB_USE_INTRINSIC_GUESS` is set, the coefficient from the supplied `distCoeffs` matrix is used. Otherwise, it is set to 0. CV_CALIB_RATIONAL_MODEL Coefficients k4, k5, and k6 are enabled. To provide the backward compatibility, this extra flag should be explicitly specified to make the calibration function use the rational model and return 8 coefficients. If the flag is not set, the function computes and returns only 5 distortion coefficients. criteria – Termination criteria for the iterative optimization algorithm. term_crit – same as `criteria`.

objectPoints –
In the new interface it is a vector of vectors of calibration pattern points in the calibration pattern coordinate space (e.g. std::vector<std::vector<cv::Vec3f>>). The outer vector contains as many elements as the number of the pattern views. If the same calibration pattern is shown in each view and it is fully visible, all the vectors will be the same. Although, it is possible to use partially occluded patterns, or even different patterns in different views. Then, the vectors will be different. The points are 3D, but since they are in a pattern coordinate system, then, if the rig is planar, it may make sense to put the model to a XY coordinate plane so that Z-coordinate of each input object point is 0.

In the old interface all the vectors of object points from different views are concatenated together.
imagePoints –
In the new interface it is a vector of vectors of the projections of calibration pattern points (e.g. std::vector<std::vector<cv::Vec2f>>). imagePoints.size() and objectPoints.size() and imagePoints[i].size() must be equal to objectPoints[i].size() for each i.

In the old interface all the vectors of object points from different views are concatenated together.
point_counts – In the old interface this is a vector of integers, containing as many elements, as the number of views of the calibration pattern. Each element is the number of points in each view. Usually, all the elements are the same and equal to the number of feature points on the calibration pattern.
imageSize – Size of the image used only to initialize the intrinsic camera matrix.
cameraMatrix – Output 3x3 floating-point camera matrix $A = \vecthreethree{f_x}{0}{c_x}{0}{f_y}{c_y}{0}{0}{1}$ . If CV_CALIB_USE_INTRINSIC_GUESS and/or CV_CALIB_FIX_ASPECT_RATIO are specified, some or all of fx, fy, cx, cy must be initialized before calling the function.
distCoeffs – Output vector of distortion coefficients $(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6]])$ of 4, 5, or 8 elements.
rvecs – Output vector of rotation vectors (see Rodrigues() ) estimated for each pattern view (e.g. std::vector<cv::Mat>>). That is, each k-th rotation vector together with the corresponding k-th translation vector (see the next output parameter description) brings the calibration pattern from the model coordinate space (in which object points are specified) to the world coordinate space, that is, a real position of the calibration pattern in the k-th pattern view (k=0.. M -1).
tvecs – Output vector of translation vectors estimated for each pattern view.
flags –
Different flags that may be zero or a combination of the following values:
- CV_CALIB_USE_INTRINSIC_GUESS cameraMatrix contains valid initial values of fx, fy, cx, cy that are optimized further. Otherwise, (cx, cy) is initially set to the image center ( imageSize is used), and focal distances are computed in a least-squares fashion. Note, that if intrinsic parameters are known, there is no need to use this function just to estimate extrinsic parameters. Use solvePnP() instead.
- CV_CALIB_FIX_PRINCIPAL_POINT The principal point is not changed during the global optimization. It stays at the center or at a different location specified when CV_CALIB_USE_INTRINSIC_GUESS is set too.
- CV_CALIB_FIX_ASPECT_RATIO The functions considers only fy as a free parameter. The ratio fx/fy stays the same as in the input cameraMatrix . When CV_CALIB_USE_INTRINSIC_GUESS is not set, the actual input values of fx and fy are ignored, only their ratio is computed and used further.
- CV_CALIB_ZERO_TANGENT_DIST Tangential distortion coefficients $(p_1, p_2)$ are set to zeros and stay zero.
- CV_CALIB_FIX_K1,...,CV_CALIB_FIX_K6 The corresponding radial distortion coefficient is not changed during the optimization. If CV_CALIB_USE_INTRINSIC_GUESS is set, the coefficient from the supplied distCoeffs matrix is used. Otherwise, it is set to 0.
- CV_CALIB_RATIONAL_MODEL Coefficients k4, k5, and k6 are enabled. To provide the backward compatibility, this extra flag should be explicitly specified to make the calibration function use the rational model and return 8 coefficients. If the flag is not set, the function computes and returns only 5 distortion coefficients.
criteria – Termination criteria for the iterative optimization algorithm.
term_crit – same as criteria.

The function estimates the intrinsic camera parameters and extrinsic parameters for each of the views. The algorithm is based on [Zhang2000] and [BouguetMCT]. The coordinates of 3D object points and their corresponding 2D projections in each view must be specified. That may be achieved by using an object with a known geometry and easily detectable feature points. Such an object is called a calibration rig or calibration pattern, and OpenCV has built-in support for a chessboard as a calibration rig (see findChessboardCorners() ). Currently, initialization of intrinsic parameters (when CV_CALIB_USE_INTRINSIC_GUESS is not set) is only implemented for planar calibration patterns (where Z-coordinates of the object points must be all zeros). 3D calibration rigs can also be used as long as initial cameraMatrix is provided.

The algorithm performs the following steps:

Compute the initial intrinsic parameters (the option only available for planar calibration patterns) or read them from the input parameters. The distortion coefficients are all set to zeros initially unless some of CV_CALIB_FIX_K? are specified.
Estimate the initial camera pose as if the intrinsic parameters have been already known. This is done using solvePnP() .
Run the global Levenberg-Marquardt optimization algorithm to minimize the reprojection error, that is, the total sum of squared distances between the observed feature points imagePoints and the projected (using the current estimates for camera parameters and the poses) object points objectPoints. See projectPoints() for details.

The function returns the final re-projection error.

Note

If you use a non-square (=non-NxN) grid and findChessboardCorners() for calibration, and calibrateCamera returns bad values (zero distortion coefficients, an image center very far from (w/2-0.5,h/2-0.5), and/or large differences between $f_x$ and $f_y$ (ratios of 10:1 or more)), then you have probably used patternSize=cvSize(rows,cols) instead of using patternSize=cvSize(cols,rows) in findChessboardCorners() .

calibrationMatrixValues

Computes useful camera characteristics from the camera matrix.

C++: void calibrationMatrixValues(InputArray cameraMatrix, Size imageSize, double apertureWidth, double apertureHeight, double& fovx, double& fovy, double& focalLength, Point2d& principalPoint, double& aspectRatio)

Python: cv2.calibrationMatrixValues(cameraMatrix, imageSize, apertureWidth, apertureHeight) → fovx, fovy, focalLength, principalPoint, aspectRatio

Parameters:

Parameters:	cameraMatrix – Input camera matrix that can be estimated by `calibrateCamera()` or `stereoCalibrate()` . imageSize – Input image size in pixels. apertureWidth – Physical width in mm of the sensor. apertureHeight – Physical height in mm of the sensor. fovx – Output field of view in degrees along the horizontal sensor axis. fovy – Output field of view in degrees along the vertical sensor axis. focalLength – Focal length of the lens in mm. principalPoint – Principal point in mm. aspectRatio – $f_y/f_x$

cameraMatrix – Input camera matrix that can be estimated by calibrateCamera() or stereoCalibrate() .
imageSize – Input image size in pixels.
apertureWidth – Physical width in mm of the sensor.
apertureHeight – Physical height in mm of the sensor.
fovx – Output field of view in degrees along the horizontal sensor axis.
fovy – Output field of view in degrees along the vertical sensor axis.
focalLength – Focal length of the lens in mm.
principalPoint – Principal point in mm.
aspectRatio – $f_y/f_x$

The function computes various useful camera characteristics from the previously estimated camera matrix.

Note

Do keep in mind that the unity measure ‘mm’ stands for whatever unit of measure one chooses for the chessboard pitch (it can thus be any value).

composeRT

Combines two rotation-and-shift transformations.

C++: void composeRT(InputArray rvec1, InputArray tvec1, InputArray rvec2, InputArray tvec2, OutputArray rvec3, OutputArray tvec3, OutputArray dr3dr1=noArray(), OutputArray dr3dt1=noArray(), OutputArray dr3dr2=noArray(), OutputArray dr3dt2=noArray(), OutputArray dt3dr1=noArray(), OutputArray dt3dt1=noArray(), OutputArray dt3dr2=noArray(), OutputArray dt3dt2=noArray() )

Python: cv2.composeRT(rvec1, tvec1, rvec2, tvec2[, rvec3[, tvec3[, dr3dr1[, dr3dt1[, dr3dr2[, dr3dt2[, dt3dr1[, dt3dt1[, dt3dr2[, dt3dt2]]]]]]]]]]) → rvec3, tvec3, dr3dr1, dr3dt1, dr3dr2, dr3dt2, dt3dr1, dt3dt1, dt3dr2, dt3dt2

Parameters:

Parameters:	rvec1 – First rotation vector. tvec1 – First translation vector. rvec2 – Second rotation vector. tvec2 – Second translation vector. rvec3 – Output rotation vector of the superposition. tvec3 – Output translation vector of the superposition. dd – Optional output derivatives of `rvec3` or `tvec3` with regard to `rvec1`, `rvec2`, `tvec1` and `tvec2`, respectively.

rvec1 – First rotation vector.
tvec1 – First translation vector.
rvec2 – Second rotation vector.
tvec2 – Second translation vector.
rvec3 – Output rotation vector of the superposition.
tvec3 – Output translation vector of the superposition.
d*d* – Optional output derivatives of rvec3 or tvec3 with regard to rvec1, rvec2, tvec1 and tvec2, respectively.

The functions compute:

$\begin{array}{l} \texttt{rvec3} = \mathrm{rodrigues} ^{-1} \left ( \mathrm{rodrigues} ( \texttt{rvec2} ) \cdot \mathrm{rodrigues} ( \texttt{rvec1} ) \right ) \\ \texttt{tvec3} = \mathrm{rodrigues} ( \texttt{rvec2} ) \cdot \texttt{tvec1} + \texttt{tvec2} \end{array} ,$

where $\mathrm{rodrigues}$ denotes a rotation vector to a rotation matrix transformation, and $\mathrm{rodrigues}^{-1}$ denotes the inverse transformation. See Rodrigues() for details.

Also, the functions can compute the derivatives of the output vectors with regards to the input vectors (see matMulDeriv() ). The functions are used inside stereoCalibrate() but can also be used in your own code where Levenberg-Marquardt or another gradient-based solver is used to optimize a function that contains a matrix multiplication.

computeCorrespondEpilines

For points in an image of a stereo pair, computes the corresponding epilines in the other image.

C++: void computeCorrespondEpilines(InputArray points, int whichImage, InputArray F, OutputArray lines)

C: void cvComputeCorrespondEpilines(const CvMat* points, int which_image, const CvMat* fundamental_matrix, CvMat* correspondent_lines)

Python: cv.ComputeCorrespondEpilines(points, whichImage, F, lines) → None

Parameters:

Parameters:	points – Input points. $N \times 1$ or $1 \times N$ matrix of type `CV_32FC2` or `vector<Point2f>` . whichImage – Index of the image (1 or 2) that contains the `points` . F – Fundamental matrix that can be estimated using `findFundamentalMat()` or `stereoRectify()` . lines – Output vector of the epipolar lines corresponding to the points in the other image. Each line $ax + by + c=0$ is encoded by 3 numbers $(a, b, c)$ .

points – Input points. $N \times 1$ or $1 \times N$ matrix of type CV_32FC2 or vector<Point2f> .
whichImage – Index of the image (1 or 2) that contains the points .
F – Fundamental matrix that can be estimated using findFundamentalMat() or stereoRectify() .
lines – Output vector of the epipolar lines corresponding to the points in the other image. Each line $ax + by + c=0$ is encoded by 3 numbers $(a, b, c)$ .

For every point in one of the two images of a stereo pair, the function finds the equation of the corresponding epipolar line in the other image.

From the fundamental matrix definition (see findFundamentalMat() ), line $l^{(2)}_i$ in the second image for the point $p^{(1)}_i$ in the first image (when whichImage=1 ) is computed as:

$l^{(2)}_i = F p^{(1)}_i$

And vice versa, when whichImage=2, $l^{(1)}_i$ is computed from $p^{(2)}_i$ as:

$l^{(1)}_i = F^T p^{(2)}_i$

Line coefficients are defined up to a scale. They are normalized so that $a_i^2+b_i^2=1$ .

convertPointsToHomogeneous

Converts points from Euclidean to homogeneous space.

C++: void convertPointsToHomogeneous(InputArray src, OutputArray dst)

Python: cv2.convertPointsToHomogeneous(src[, dst]) → dst

Parameters:	src – Input vector of `N`-dimensional points. dst – Output vector of `N+1`-dimensional points.

The function converts points from Euclidean to homogeneous space by appending 1’s to the tuple of point coordinates. That is, each point (x1, x2, ..., xn) is converted to (x1, x2, ..., xn, 1).

convertPointsFromHomogeneous

Converts points from homogeneous to Euclidean space.

C++: void convertPointsFromHomogeneous(InputArray src, OutputArray dst)

Python: cv2.convertPointsFromHomogeneous(src[, dst]) → dst

Parameters:	src – Input vector of `N`-dimensional points. dst – Output vector of `N-1`-dimensional points.

The function converts points homogeneous to Euclidean space using perspective projection. That is, each point (x1, x2, ... x(n-1), xn) is converted to (x1/xn, x2/xn, ..., x(n-1)/xn). When xn=0, the output point coordinates will be (0,0,0,...).

convertPointsHomogeneous

Converts points to/from homogeneous coordinates.

C++: void convertPointsHomogeneous(InputArray src, OutputArray dst)

C: void cvConvertPointsHomogeneous(const CvMat* src, CvMat* dst)

Python: cv.ConvertPointsHomogeneous(src, dst) → None

Parameters:	src – Input array or vector of 2D, 3D, or 4D points. dst – Output vector of 2D, 3D, or 4D points.

The function converts 2D or 3D points from/to homogeneous coordinates by calling either convertPointsToHomogeneous() or convertPointsFromHomogeneous().

Note

The function is obsolete. Use one of the previous two functions instead.

correctMatches

Refines coordinates of corresponding points.

C++: void correctMatches(InputArray F, InputArray points1, InputArray points2, OutputArray newPoints1, OutputArray newPoints2)

Python: cv2.correctMatches(F, points1, points2[, newPoints1[, newPoints2]]) → newPoints1, newPoints2

C: void cvCorrectMatches(CvMat* F, CvMat* points1, CvMat* points2, CvMat* new_points1, CvMat* new_points2)

Parameters:	F – 3x3 fundamental matrix. points1 – 1xN array containing the first set of points. points2 – 1xN array containing the second set of points. newPoints1 – The optimized points1. newPoints2 – The optimized points2.

The function implements the Optimal Triangulation Method (see Multiple View Geometry for details). For each given point correspondence points1[i] <-> points2[i], and a fundamental matrix F, it computes the corrected correspondences newPoints1[i] <-> newPoints2[i] that minimize the geometric error $d(points1[i], newPoints1[i])^2 + d(points2[i],newPoints2[i])^2$ (where $d(a,b)$ is the geometric distance between points $a$ and $b$ ) subject to the epipolar constraint $newPoints2^T * F * newPoints1 = 0$ .

decomposeProjectionMatrix

Decomposes a projection matrix into a rotation matrix and a camera matrix.

C++: void decomposeProjectionMatrix(InputArray projMatrix, OutputArray cameraMatrix, OutputArray rotMatrix, OutputArray transVect, OutputArray rotMatrixX=noArray(), OutputArray rotMatrixY=noArray(), OutputArray rotMatrixZ=noArray(), OutputArray eulerAngles=noArray() )

Python: cv2.decomposeProjectionMatrix(projMatrix[, cameraMatrix[, rotMatrix[, transVect[, rotMatrixX[, rotMatrixY[, rotMatrixZ[, eulerAngles]]]]]]]) → cameraMatrix, rotMatrix, transVect, rotMatrixX, rotMatrixY, rotMatrixZ, eulerAngles

C: void cvDecomposeProjectionMatrix(const CvMat* projMatr, CvMat* calibMatr, CvMat* rotMatr, CvMat* posVect, CvMat* rotMatrX=NULL, CvMat* rotMatrY=NULL, CvMat* rotMatrZ=NULL, CvPoint3D64f* eulerAngles=NULL )

Python: cv.DecomposeProjectionMatrix(projMatrix, cameraMatrix, rotMatrix, transVect, rotMatrX=None, rotMatrY=None, rotMatrZ=None) → eulerAngles

Parameters:	projMatrix – 3x4 input projection matrix P. cameraMatrix – Output 3x3 camera matrix K. rotMatrix – Output 3x3 external rotation matrix R. transVect – Output 4x1 translation vector T. rotMatrX – Optional 3x3 rotation matrix around x-axis. rotMatrY – Optional 3x3 rotation matrix around y-axis. rotMatrZ – Optional 3x3 rotation matrix around z-axis. eulerAngles – Optional three-element vector containing three Euler angles of rotation in degrees.

Parameters:

projMatrix – 3x4 input projection matrix P.
cameraMatrix – Output 3x3 camera matrix K.
rotMatrix – Output 3x3 external rotation matrix R.
transVect – Output 4x1 translation vector T.
rotMatrX – Optional 3x3 rotation matrix around x-axis.
rotMatrY – Optional 3x3 rotation matrix around y-axis.
rotMatrZ – Optional 3x3 rotation matrix around z-axis.
eulerAngles – Optional three-element vector containing three Euler angles of rotation in degrees.

The function computes a decomposition of a projection matrix into a calibration and a rotation matrix and the position of a camera.

It optionally returns three rotation matrices, one for each axis, and three Euler angles that could be used in OpenGL. Note, there is always more than one sequence of rotations about the three principle axes that results in the same orientation of an object, eg. see [Slabaugh]. Returned tree rotation matrices and corresponding three Euler angules are only one of the possible solutions.

The function is based on RQDecomp3x3() .

drawChessboardCorners

Renders the detected chessboard corners.

C++: void drawChessboardCorners(InputOutputArray image, Size patternSize, InputArray corners, bool patternWasFound)

Python: cv2.drawChessboardCorners(image, patternSize, corners, patternWasFound) → None

C: void cvDrawChessboardCorners(CvArr* image, CvSize pattern_size, CvPoint2D32f* corners, int count, int pattern_was_found)

Python: cv.DrawChessboardCorners(image, patternSize, corners, patternWasFound) → None

Parameters:

Parameters:	image – Destination image. It must be an 8-bit color image. patternSize – Number of inner corners per a chessboard row and column `(patternSize = cv::Size(points_per_row,points_per_column))`. corners – Array of detected corners, the output of `findChessboardCorners`. patternWasFound – Parameter indicating whether the complete board was found or not. The return value of `findChessboardCorners()` should be passed here.

image – Destination image. It must be an 8-bit color image.
patternSize – Number of inner corners per a chessboard row and column (patternSize = cv::Size(points_per_row,points_per_column)).
corners – Array of detected corners, the output of findChessboardCorners.
patternWasFound – Parameter indicating whether the complete board was found or not. The return value of findChessboardCorners() should be passed here.

The function draws individual chessboard corners detected either as red circles if the board was not found, or as colored corners connected with lines if the board was found.

findChessboardCorners

Finds the positions of internal corners of the chessboard.

C++: bool findChessboardCorners(InputArray image, Size patternSize, OutputArray corners, int flags=CALIB_CB_ADAPTIVE_THRESH+CALIB_CB_NORMALIZE_IMAGE )

Python: cv2.findChessboardCorners(image, patternSize[, corners[, flags]]) → retval, corners

C: int cvFindChessboardCorners(const void* image, CvSize pattern_size, CvPoint2D32f* corners, int* corner_count=NULL, int flags=CV_CALIB_CB_ADAPTIVE_THRESH+CV_CALIB_CB_NORMALIZE_IMAGE )

Python: cv.FindChessboardCorners(image, patternSize, flags=CV_CALIB_CB_ADAPTIVE_THRESH) → corners

Parameters:

Parameters:	image – Source chessboard view. It must be an 8-bit grayscale or color image. patternSize – Number of inner corners per a chessboard row and column `( patternSize = cvSize(points_per_row,points_per_colum) = cvSize(columns,rows) )`. corners – Output array of detected corners. flags – Various operation flags that can be zero or a combination of the following values: CALIB_CB_ADAPTIVE_THRESH Use adaptive thresholding to convert the image to black and white, rather than a fixed threshold level (computed from the average image brightness). CALIB_CB_NORMALIZE_IMAGE Normalize the image gamma with `equalizeHist()` before applying fixed or adaptive thresholding. CALIB_CB_FILTER_QUADS Use additional criteria (like contour area, perimeter, square-like shape) to filter out false quads extracted at the contour retrieval stage. CALIB_CB_FAST_CHECK Run a fast check on the image that looks for chessboard corners, and shortcut the call if none is found. This can drastically speed up the call in the degenerate condition when no chessboard is observed.

image – Source chessboard view. It must be an 8-bit grayscale or color image.
patternSize – Number of inner corners per a chessboard row and column ( patternSize = cvSize(points_per_row,points_per_colum) = cvSize(columns,rows) ).
corners – Output array of detected corners.
flags –
Various operation flags that can be zero or a combination of the following values:
- CALIB_CB_ADAPTIVE_THRESH Use adaptive thresholding to convert the image to black and white, rather than a fixed threshold level (computed from the average image brightness).
- CALIB_CB_NORMALIZE_IMAGE Normalize the image gamma with equalizeHist() before applying fixed or adaptive thresholding.
- CALIB_CB_FILTER_QUADS Use additional criteria (like contour area, perimeter, square-like shape) to filter out false quads extracted at the contour retrieval stage.
- CALIB_CB_FAST_CHECK Run a fast check on the image that looks for chessboard corners, and shortcut the call if none is found. This can drastically speed up the call in the degenerate condition when no chessboard is observed.

The function attempts to determine whether the input image is a view of the chessboard pattern and locate the internal chessboard corners. The function returns a non-zero value if all of the corners are found and they are placed in a certain order (row by row, left to right in every row). Otherwise, if the function fails to find all the corners or reorder them, it returns 0. For example, a regular chessboard has 8 x 8 squares and 7 x 7 internal corners, that is, points where the black squares touch each other. The detected coordinates are approximate, and to determine their positions more accurately, the function calls cornerSubPix(). You also may use the function cornerSubPix() with different parameters if returned coordinates are not accurate enough.

Sample usage of detecting and drawing chessboard corners:

Size patternsize(8,6); //interior number of corners
Mat gray = ....; //source image
vector<Point2f> corners; //this will be filled by the detected corners

//CALIB_CB_FAST_CHECK saves a lot of time on images
//that do not contain any chessboard corners
bool patternfound = findChessboardCorners(gray, patternsize, corners,
        CALIB_CB_ADAPTIVE_THRESH + CALIB_CB_NORMALIZE_IMAGE
        + CALIB_CB_FAST_CHECK);

if(patternfound)
  cornerSubPix(gray, corners, Size(11, 11), Size(-1, -1),
    TermCriteria(CV_TERMCRIT_EPS + CV_TERMCRIT_ITER, 30, 0.1));

drawChessboardCorners(img, patternsize, Mat(corners), patternfound);

Note

The function requires white space (like a square-thick border, the wider the better) around the board to make the detection more robust in various environments. Otherwise, if there is no border and the background is dark, the outer black squares cannot be segmented properly and so the square grouping and ordering algorithm fails.

findCirclesGrid

Finds centers in the grid of circles.

C++: bool findCirclesGrid(InputArray image, Size patternSize, OutputArray centers, int flags=CALIB_CB_SYMMETRIC_GRID, const Ptr<FeatureDetector>& blobDetector=new SimpleBlobDetector() )

Python: cv2.findCirclesGridDefault(image, patternSize[, centers[, flags]]) → retval, centers

Parameters:

Parameters:	image – grid view of input circles; it must be an 8-bit grayscale or color image. patternSize – number of circles per row and column `( patternSize = Size(points_per_row, points_per_colum) )`. centers – output array of detected centers. flags – various operation flags that can be one of the following values: CALIB_CB_SYMMETRIC_GRID uses symmetric pattern of circles. CALIB_CB_ASYMMETRIC_GRID uses asymmetric pattern of circles. CALIB_CB_CLUSTERING uses a special algorithm for grid detection. It is more robust to perspective distortions but much more sensitive to background clutter. blobDetector – feature detector that finds blobs like dark circles on light background.

image – grid view of input circles; it must be an 8-bit grayscale or color image.
patternSize – number of circles per row and column ( patternSize = Size(points_per_row, points_per_colum) ).
centers – output array of detected centers.
flags –
various operation flags that can be one of the following values:
- CALIB_CB_SYMMETRIC_GRID uses symmetric pattern of circles.
- CALIB_CB_ASYMMETRIC_GRID uses asymmetric pattern of circles.
- CALIB_CB_CLUSTERING uses a special algorithm for grid detection. It is more robust to perspective distortions but much more sensitive to background clutter.
blobDetector – feature detector that finds blobs like dark circles on light background.

The function attempts to determine whether the input image contains a grid of circles. If it is, the function locates centers of the circles. The function returns a non-zero value if all of the centers have been found and they have been placed in a certain order (row by row, left to right in every row). Otherwise, if the function fails to find all the corners or reorder them, it returns 0.

Sample usage of detecting and drawing the centers of circles:

Size patternsize(7,7); //number of centers
Mat gray = ....; //source image
vector<Point2f> centers; //this will be filled by the detected centers

bool patternfound = findCirclesGrid(gray, patternsize, centers);

drawChessboardCorners(img, patternsize, Mat(centers), patternfound);

Note

The function requires white space (like a square-thick border, the wider the better) around the board to make the detection more robust in various environments.

solvePnP

Finds an object pose from 3D-2D point correspondences.

C++: bool solvePnP(InputArray objectPoints, InputArray imagePoints, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess=false, int flags=ITERATIVE )

Python: cv2.solvePnP(objectPoints, imagePoints, cameraMatrix, distCoeffs[, rvec[, tvec[, useExtrinsicGuess[, flags]]]]) → retval, rvec, tvec

C: void cvFindExtrinsicCameraParams2(const CvMat* object_points, const CvMat* image_points, const CvMat* camera_matrix, const CvMat* distortion_coeffs, CvMat* rotation_vector, CvMat* translation_vector, int use_extrinsic_guess=0 )

Python: cv.FindExtrinsicCameraParams2(objectPoints, imagePoints, cameraMatrix, distCoeffs, rvec, tvec, useExtrinsicGuess=0) → None

Parameters:

Parameters:	objectPoints – Array of object points in the object coordinate space, 3xN/Nx3 1-channel or 1xN/Nx1 3-channel, where N is the number of points. `vector<Point3f>` can be also passed here. imagePoints – Array of corresponding image points, 2xN/Nx2 1-channel or 1xN/Nx1 2-channel, where N is the number of points. `vector<Point2f>` can be also passed here. cameraMatrix – Input camera matrix $A = \vecthreethree{fx}{0}{cx}{0}{fy}{cy}{0}{0}{1}$ . distCoeffs – Input vector of distortion coefficients $(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6]])$ of 4, 5, or 8 elements. If the vector is NULL/empty, the zero distortion coefficients are assumed. rvec – Output rotation vector (see `Rodrigues()` ) that, together with `tvec` , brings points from the model coordinate system to the camera coordinate system. tvec – Output translation vector. useExtrinsicGuess – If true (1), the function uses the provided `rvec` and `tvec` values as initial approximations of the rotation and translation vectors, respectively, and further optimizes them. flags – Method for solving a PnP problem: CV_ITERATIVE Iterative method is based on Levenberg-Marquardt optimization. In this case the function finds such a pose that minimizes reprojection error, that is the sum of squared distances between the observed projections `imagePoints` and the projected (using `projectPoints()` ) `objectPoints` . CV_P3P Method is based on the paper of X.S. Gao, X.-R. Hou, J. Tang, H.-F. Chang “Complete Solution Classification for the Perspective-Three-Point Problem”. In this case the function requires exactly four object and image points. CV_EPNP Method has been introduced by F.Moreno-Noguer, V.Lepetit and P.Fua in the paper “EPnP: Efficient Perspective-n-Point Camera Pose Estimation”.

objectPoints – Array of object points in the object coordinate space, 3xN/Nx3 1-channel or 1xN/Nx1 3-channel, where N is the number of points. vector<Point3f> can be also passed here.
imagePoints – Array of corresponding image points, 2xN/Nx2 1-channel or 1xN/Nx1 2-channel, where N is the number of points. vector<Point2f> can be also passed here.
cameraMatrix – Input camera matrix $A = \vecthreethree{fx}{0}{cx}{0}{fy}{cy}{0}{0}{1}$ .
distCoeffs – Input vector of distortion coefficients $(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6]])$ of 4, 5, or 8 elements. If the vector is NULL/empty, the zero distortion coefficients are assumed.
rvec – Output rotation vector (see Rodrigues() ) that, together with tvec , brings points from the model coordinate system to the camera coordinate system.
tvec – Output translation vector.
useExtrinsicGuess – If true (1), the function uses the provided rvec and tvec values as initial approximations of the rotation and translation vectors, respectively, and further optimizes them.
flags –
Method for solving a PnP problem:
- CV_ITERATIVE Iterative method is based on Levenberg-Marquardt optimization. In this case the function finds such a pose that minimizes reprojection error, that is the sum of squared distances between the observed projections imagePoints and the projected (using projectPoints() ) objectPoints .
- CV_P3P Method is based on the paper of X.S. Gao, X.-R. Hou, J. Tang, H.-F. Chang “Complete Solution Classification for the Perspective-Three-Point Problem”. In this case the function requires exactly four object and image points.
- CV_EPNP Method has been introduced by F.Moreno-Noguer, V.Lepetit and P.Fua in the paper “EPnP: Efficient Perspective-n-Point Camera Pose Estimation”.

The function estimates the object pose given a set of object points, their corresponding image projections, as well as the camera matrix and the distortion coefficients.

Note

An example of how to use solvePNP for planar augmented reality can be found at opencv_source_code/samples/python2/plane_ar.py

solvePnPRansac

Finds an object pose from 3D-2D point correspondences using the RANSAC scheme.

C++: void solvePnPRansac(InputArray objectPoints, InputArray imagePoints, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess=false, int iterationsCount=100, float reprojectionError=8.0, int minInliersCount=100, OutputArray inliers=noArray(), int flags=ITERATIVE )

Python: cv2.solvePnPRansac(objectPoints, imagePoints, cameraMatrix, distCoeffs[, rvec[, tvec[, useExtrinsicGuess[, iterationsCount[, reprojectionError[, minInliersCount[, inliers[, flags]]]]]]]]) → rvec, tvec, inliers

Parameters:

Parameters:	objectPoints – Array of object points in the object coordinate space, 3xN/Nx3 1-channel or 1xN/Nx1 3-channel, where N is the number of points. `vector<Point3f>` can be also passed here. imagePoints – Array of corresponding image points, 2xN/Nx2 1-channel or 1xN/Nx1 2-channel, where N is the number of points. `vector<Point2f>` can be also passed here. cameraMatrix – Input camera matrix $A = \vecthreethree{fx}{0}{cx}{0}{fy}{cy}{0}{0}{1}$ . distCoeffs – Input vector of distortion coefficients $(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6]])$ of 4, 5, or 8 elements. If the vector is NULL/empty, the zero distortion coefficients are assumed. rvec – Output rotation vector (see `Rodrigues()` ) that, together with `tvec` , brings points from the model coordinate system to the camera coordinate system. tvec – Output translation vector. useExtrinsicGuess – If true (1), the function uses the provided `rvec` and `tvec` values as initial approximations of the rotation and translation vectors, respectively, and further optimizes them. iterationsCount – Number of iterations. reprojectionError – Inlier threshold value used by the RANSAC procedure. The parameter value is the maximum allowed distance between the observed and computed point projections to consider it an inlier. minInliersCount – Number of inliers. If the algorithm at some stage finds more inliers than `minInliersCount` , it finishes. inliers – Output vector that contains indices of inliers in `objectPoints` and `imagePoints` . flags – Method for solving a PnP problem (see `solvePnP()` ).

objectPoints – Array of object points in the object coordinate space, 3xN/Nx3 1-channel or 1xN/Nx1 3-channel, where N is the number of points. vector<Point3f> can be also passed here.
imagePoints – Array of corresponding image points, 2xN/Nx2 1-channel or 1xN/Nx1 2-channel, where N is the number of points. vector<Point2f> can be also passed here.
cameraMatrix – Input camera matrix $A = \vecthreethree{fx}{0}{cx}{0}{fy}{cy}{0}{0}{1}$ .
distCoeffs – Input vector of distortion coefficients $(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6]])$ of 4, 5, or 8 elements. If the vector is NULL/empty, the zero distortion coefficients are assumed.
rvec – Output rotation vector (see Rodrigues() ) that, together with tvec , brings points from the model coordinate system to the camera coordinate system.
tvec – Output translation vector.
useExtrinsicGuess – If true (1), the function uses the provided rvec and tvec values as initial approximations of the rotation and translation vectors, respectively, and further optimizes them.
iterationsCount – Number of iterations.
reprojectionError – Inlier threshold value used by the RANSAC procedure. The parameter value is the maximum allowed distance between the observed and computed point projections to consider it an inlier.
minInliersCount – Number of inliers. If the algorithm at some stage finds more inliers than minInliersCount , it finishes.
inliers – Output vector that contains indices of inliers in objectPoints and imagePoints .
flags – Method for solving a PnP problem (see solvePnP() ).

The function estimates an object pose given a set of object points, their corresponding image projections, as well as the camera matrix and the distortion coefficients. This function finds such a pose that minimizes reprojection error, that is, the sum of squared distances between the observed projections imagePoints and the projected (using projectPoints() ) objectPoints. The use of RANSAC makes the function resistant to outliers. The function is parallelized with the TBB library.

findFundamentalMat

Calculates a fundamental matrix from the corresponding points in two images.

C++: Mat findFundamentalMat(InputArray points1, InputArray points2, int method=FM_RANSAC, double param1=3., double param2=0.99, OutputArray mask=noArray() )

Python:


              
           
              
              
            
            相關推薦
			   
            
            
            
 

    

    
    Camera Calibration and 3D Reconstruction
       
 
 https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html 
 The functions in this section use a so-called pinhole  

  
 

    

    
    讀real time localization and 3D reconstruction筆記
       
 
        首先，我們提出了從視訊流中估計相機運動和三維結構的完整方法使用區域性束調整來解釋我們的增量方法：我們建議僅使用限制在最後一個攝像機和這些攝像機觀察到的3D點的一組引數來優化3D結構的末端。在第二部分中，我們提供實際資料的實驗和結果，並與GPS定位 

  
 

    

    
    影象檢索與三維重建：From Single Image Query to Detailed 3D reconstruction
      
                先前基於影象檢索的三維重建方式由於只關注尺度和外表相似的影象而往往會導致細節缺失問題，為解決該問題，本文結合了基於運動的重建(SfM, Structure-from-Motion)和多尺度場景下的影象檢索進行三維重建。文中對重建系統和檢索系統建立的連線，使得能夠根據當前的3D 

  
 

    

    
    相機標定（Camera calibration）及實踐
      
                



在機器視覺領域，相機的標定是一個關鍵的環節，它決定了機器視覺系統能否有效的定位，能否有效的計算目標物。相機的標定基本上可以分為兩種，第一種是相機的自標定；第二種是依賴於標定參照物的標定方法。前者是相機拍攝周圍物體，通過數字影象處理的方法和相關的幾何計算得到相機引數，但 

  
 

    

    
    ros學習之camera calibration 單目攝像頭標定
      
							
							
							環境：ubuntu16.04    ros版本 kinetic
標定筆記本單目攝像頭
需要的準備：1、標定圖：下面這張列印到a4紙上。

2、攝像機驅動，我使用的是usb_cam，下載地址：https://github.com/ros-drivers/usb_c 

  
 

    

    
    Advanced Imaging and Image Analysis Services: Digital Pathology, Machine Learning and 3D Cell Culture Models
      Three major opportunities for improvement in early-stage in vitro and animal model studies are to improve the predictive capability of in vitro models them 

  
 

    

    
    單目相機標定(Camera Calibration)
      
							
							
							為什麼需要相機標定

通過image formation pipeline, 我們可以將已知世界座標系上的點對映在畫素座標系上，那麼如何預測影象上的點在3d世界中的位置呢？

第一步便是相機標定。相機標定的工作是消除鏡頭造成的影象變形(distortion)。
 

  
 

    

    
    論文閱讀：Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis
      
							
							
							
  這篇論文是在2017年3月22日發表在CVPR上的，作者在這篇論文中提出了一個叫做深度從粗糙到精細化的多工卷積神經網路（Deep MANTA），該模型可以用於對一張圖片中的車輛進行多工的分析。該網路同時執行的多工包括：車輛檢測、部件定位、可見性描述和三維形 

  
 

    

    
    Camera Calibration 相機標定：原理簡介（五）
      
							
							
							5 基於2D標定物的標定方法

基於2D標定物的標定方法，原理與基於3D標定物相同，只是通過相機對一個平面進行成像，就可得到相機的標定引數，由於標定物為平面，本身所具有的約束條機，相對後者標定更為簡單。經典演算法為Z. Zhang(PAMI, 2000) A F 

  
 

    

    
    攝像機標定01_面陣攝像機（Camera Calibration）
      
							
							
							面陣攝像機的攝像機模型

機器視覺應用中常用的兩種鏡頭：普通鏡頭和遠心鏡頭。 
使用普通鏡頭時，世界座標系到影象座標系的投影為透視投影。普通鏡頭與針孔攝像機實現的投影方式相同，因此將攝像機與普通鏡頭的組合成為針孔攝像機模型。 
使用遠心鏡頭時，世界座標系到影象座 

  
 

    

    
    無人駕駛-鐳射雷達與相機聯合校準（Lidar Camera Calibration）
      
                1.鐳射雷達與攝像頭效能對比

在無人駕駛環境感知裝置中，鐳射雷達和攝像頭分別有各自的優缺點。
攝像頭的優點是成本低廉，用攝像頭做演算法開發的人員也比較多，技術相對比較成熟。攝像頭的劣勢，第一，獲取準確三維資訊非常難（單目攝像頭幾乎不可能，也有人提出雙目或三目攝像頭去做）；另 

  
 

    

    
    攝像機標定03_標定過程（Camera Calibration）
      
							
							
							攝像機標定過程

為了進行攝像機標定，必須已知世界座標系中足夠多三維空間點的座標，找到這些點在影象中的投影的二維影象座標，並建立對應關係。

平面標定板的幾大優勢：非常易於操作；可以製作的非常精確；可以方便用在背光照明的應用上，只需要使用透明材料製作放置標誌點的 

  
 

    

    
    ROS 教程之 vision: 攝像頭標定camera calibration
      
								
								            
						
                
        在上一個ROS教程視覺文章中，我們使用usb_cam包讀入併發布了影象訊息，但是影象沒有被標定，因此存在畸變。ROS官方提供了用於單目或者雙目標定的camera_calibration 

  
 

    

    
    【學習ARToolkit小記之三】 攝像頭標定（Camera Calibration）
      
                


本部分內容只用於個人的學習記錄與記錄，詳細內容請參考官方網站教程。
一、開發環境
1、作業系統：Windows 7（64位）
2、程式設計環境：Microsoft Visual Studio 2010
3、ARToolkit版本：ARToolKit-2.72.1

二、 

  
 

    

    
    Camera calibration(相機校正)
      
                
以前DIP課程有做過Camera calibration，這次因為用Gopro做Visual SLAM，所以又要撿一下校正的過程。
主要還是張正友的方法。

OpenCV：
用OpenCV自帶的Sample：路徑C:\opencv\sources\samples\cpp\t 

  
 

    

    
    6D姿態估計從0單排——看論文的小雞篇——Learning Descriptors for Object Recognition and 3D Pose Estimation
      iss   dem   vol   nodes   rim   ext   obj   tor   triplet   這篇文章和前一篇把神經網絡主要集中於descriptor的對比方面不一樣的是，這一篇中CNN還用來區分不同的物體類別和同類別不同姿態從而確保不同類間距較大同類間距較小（但是足以區分pose） 

  
 

    

    
    3D人臉重建：《Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network》
       
 
  
  
 《Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network》 
 目錄 
 
 
   
    
    0 前言 
    1 概述 
    2 相關工作
       

  
 

    

    
    3D【9】人臉重建:Joint3D Face Reconstruction and Dense Alignment with Position Map Regression
      
							
							
							該論文同時預測人臉特徵點和3D mesh的頂點座標x，y，z。人臉特徵點沒什麼好說的，現在已經有很多成功的案例了。主要關注該論文是如何預測頂點座標的。其實大概想想，就能夠大致確定直接預測頂點座標是有不少困難的。首先是，你不能把所有的頂點拉成一個向量，用全連線層預 

  
 

    

    
    3D Slicer Programmatically Set Slice Offset and Intersections 用代碼修改Slicer中的切片偏移和交叉點顯示
      cnblogs   們的   int   mod   call   eof   col   all   mha    
在3D Slicer中，當我們導入.mha/.mhd等格式的volume文件後，那麽我們就可以在Axial, Sagittal, Coronal三個方向來觀察我們的MRI或者CT的圖像了。 

  
 

    

    
    3D Face Reconstruction
      ima   per   inf   VR   blank   wid   lan   direct   rect    
方法1 
Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regressio