CalibRCNN: Calibrating Camera and LiDAR by recurrent convolutional neural network and geometric constraints

Jieying Shi, Ziheng Zhu, Jianhua Zhang, Ruyu Liu, Zhenhua Wang, Shengyong Chen, Honghai Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

221 Downloads (Pure)

Abstract

In this paper, we present Calibration Recurrent Convolutional Neural Network (CalibRCNN) to infer a 6 degrees of freedom (DOF) rigid body transformation between 3D LiDAR and 2D camera. Different from the existing methods, our 3D-2D CalibRCNN not only uses the LSTM network to extract the temporal features between 3D point clouds and RGB images of consecutive frames, but also uses the geometric loss and photometric loss obtained by the interframe constraint to refine the calibration accuracy of the predicted transformation parameters. The CalibRCNN aims at inferring the correspondence between projected depth image and RGB image to learn the underlying geometry of 2D-3D calibration. Thus, the proposed calibration model achieves a good generalization ability to adapt to unknown initial calibration error ranges, and other 3D LiDAR and 2D camera pairs with different intrinsic parameters from the training dataset. Extensive experiments have demonstrated that our CalibRCNN can achieve state-of-the-art accuracy by comparison with other CNN based methods.

Original languageEnglish
Title of host publication2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2020
PublisherIEEE
Pages10197-10202
Number of pages6
ISBN (Electronic)9781728162126
ISBN (Print)9781728162133
DOIs
Publication statusPublished - 10 Feb 2021
Event2020 IEEE/RSJ International Conference on Intelligent Robots and Systems - Las Vegas, United States
Duration: 24 Oct 202024 Jan 2021

Publication series

NameIEEE IROS Proceedings Series
PublisherIEEE
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Conference

Conference2020 IEEE/RSJ International Conference on Intelligent Robots and Systems
Abbreviated titleIROS 2020
CountryUnited States
CityLas Vegas
Period24/10/2024/01/21

Fingerprint

Dive into the research topics of 'CalibRCNN: Calibrating Camera and LiDAR by recurrent convolutional neural network and geometric constraints'. Together they form a unique fingerprint.

Cite this