CalibRCNN: Calibrating Camera and LiDAR by recurrent convolutional neural network and geometric constraints
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
In this paper, we present Calibration Recurrent Convolutional Neural Network (CalibRCNN) to infer a 6 degrees of freedom (DOF) rigid body transformation between 3D LiDAR and 2D camera. Different from the existing methods, our 3D-2D CalibRCNN not only uses the LSTM network to extract the temporal features between 3D point clouds and RGB images of consecutive frames, but also uses the geometric loss and photometric loss obtained by the interframe constraint to refine the calibration accuracy of the predicted transformation parameters. The CalibRCNN aims at inferring the correspondence between projected depth image and RGB image to learn the underlying geometry of 2D-3D calibration. Thus, the proposed calibration model achieves a good generalization ability to adapt to unknown initial calibration error ranges, and other 3D LiDAR and 2D camera pairs with different intrinsic parameters from the training dataset. Extensive experiments have demonstrated that our CalibRCNN can achieve state-of-the-art accuracy by comparison with other CNN based methods.
|Title of host publication||2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2020|
|Number of pages||6|
|Publication status||Published - 10 Feb 2021|
|Event||2020 IEEE/RSJ International Conference on Intelligent Robots and Systems - Las Vegas, United States|
Duration: 24 Oct 2020 → 24 Jan 2021
|Name||IEEE IROS Proceedings Series|
|Conference||2020 IEEE/RSJ International Conference on Intelligent Robots and Systems|
|Abbreviated title||IROS 2020|
|Period||24/10/20 → 24/01/21|
- CalibRCNN post-print
Rights statement: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript (Post-print), 1.56 MB, PDF document