MARS: parallelism based metrically accurate 3D reconstruction system in real-time

Due to the increasing application demands, lightweight device-based 3D recovery draws many attentions from a wide group of researchers in both academic and industrial fields. The current 3D reconstruction solutions are commonly achieved either using depth data or RGB data. The depth data usually come from a deliberately designed hardware for specific tasks, while the RGB data-based solutions only employ a single RGB camera with vision-based computing algorithms. Limitations are expected from both. Depth sensors are commonly either bulky or relatively expensive compared to RGB cameras, thus of less flexibility. Normal RGB cameras usually have better mobility but less accuracy in 3D sensing than depth sensors. Recently, machine learning based depth estimation has also been presented. However, its accuracy is still limited. To improve the flexibility of the 3D reconstruction system without loss in accuracy, this paper presents a solution of unconstrained Metrically Accurate 3D Reconstruction System (MARS) for 3D sensing based on a consumer-grade camera. With a simple initialization from a depth map, the system can achieve incremental 3D reconstruction with a stable metric scale. Experiments are conducted using both real-world data and public datasets. Competitive results are obtained using the proposed system compared with several existing methods.