Evaluation of epipole estimation methods with/without rank-2 constraint across algebraic/geometric error functions

The best method for estimating the fundamental matrix and/or the epipole over a given set of point correspondences between two images is a nonlinear minimization, which searches a rank-2 fundamental matrix that minimizes the geometric error cost function. When convenience is preferred to accuracy, we often use a linear approximation method, which searches a rank-3 matrix that minimizes the algebraic error. Although it has been reported that the algebraic error causes very poor results, it is currently thought that the relatively inaccurate results of a linear estimation method are a consequence of neglecting the rank-2 constraint, and not a result of exploiting the algebraic error. However, the reason has not been analyzed fully. In the present paper, we analyze the effects of the cost function selection and the rank-2 constraint based on covariance matrix analyses and show theoretically and experimentally that it is more important to enforce the rank-2 constraint than to minimize the geometric cost function.