Evaluation of close-range stereo matching algorithms using stereoscopic measurements

The performance of binocular stereo reconstruction is highly dependent on the quality of the stereo matching result. In order to evaluate the performance of different stereo matchers, several quality metrics have been developed based on quantifying error statistics with respect to a set of independent measurements usually referred to as ground truth data. However, such data are frequently not available, particularly in practical applications or planetary data processing. To address this, we propose a ground truth independent evaluation protocol based on manual measurements. A stereo visualization tool has been specifically developed to evaluate the quality of the computed correspondences. We compare the quality of disparity maps calculated from three stereo matching algorithms, developed based on a variation of GOTCHA, which has been used in planetary robotic rover image reconstruction at UCL-MSSL (Otto and Chau, 1989). From our evaluation tests with the images pairs from Mars Exploration Rover (MER) Pancam and the field data collected in PRoViScout 2012, it has been found that all three processing pipelines used in our test (NASA-JPL, JR, UCL-MSSL) trade off matching accuracy and completeness differently. NASA-JPL’s stereo pipeline produces the most accurate but less complete disparity map, whilst JR’s pipeline performs best in terms of the reconstruction completeness.