TY - GEN
T1 - CalibRCNN
T2 - 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems
AU - Shi, Jieying
AU - Zhu, Ziheng
AU - Zhang, Jianhua
AU - Liu, Ruyu
AU - Wang, Zhenhua
AU - Chen, Shengyong
AU - Liu, Honghai
N1 - Funding Information:
This work was partially supported by National Key R&D Program of China (2018YFB1305200). This publication was partially funded by the National Natural Science Foundation of China (61876167 and 61802348) and the Natural Science Foundation of Zhejiang Province (LY20F030017).
Publisher Copyright:
© 2020 IEEE.
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2021/2/10
Y1 - 2021/2/10
N2 - In this paper, we present Calibration Recurrent Convolutional Neural Network (CalibRCNN) to infer a 6 degrees of freedom (DOF) rigid body transformation between 3D LiDAR and 2D camera. Different from the existing methods, our 3D-2D CalibRCNN not only uses the LSTM network to extract the temporal features between 3D point clouds and RGB images of consecutive frames, but also uses the geometric loss and photometric loss obtained by the interframe constraint to refine the calibration accuracy of the predicted transformation parameters. The CalibRCNN aims at inferring the correspondence between projected depth image and RGB image to learn the underlying geometry of 2D-3D calibration. Thus, the proposed calibration model achieves a good generalization ability to adapt to unknown initial calibration error ranges, and other 3D LiDAR and 2D camera pairs with different intrinsic parameters from the training dataset. Extensive experiments have demonstrated that our CalibRCNN can achieve state-of-the-art accuracy by comparison with other CNN based methods.
AB - In this paper, we present Calibration Recurrent Convolutional Neural Network (CalibRCNN) to infer a 6 degrees of freedom (DOF) rigid body transformation between 3D LiDAR and 2D camera. Different from the existing methods, our 3D-2D CalibRCNN not only uses the LSTM network to extract the temporal features between 3D point clouds and RGB images of consecutive frames, but also uses the geometric loss and photometric loss obtained by the interframe constraint to refine the calibration accuracy of the predicted transformation parameters. The CalibRCNN aims at inferring the correspondence between projected depth image and RGB image to learn the underlying geometry of 2D-3D calibration. Thus, the proposed calibration model achieves a good generalization ability to adapt to unknown initial calibration error ranges, and other 3D LiDAR and 2D camera pairs with different intrinsic parameters from the training dataset. Extensive experiments have demonstrated that our CalibRCNN can achieve state-of-the-art accuracy by comparison with other CNN based methods.
UR - http://www.scopus.com/inward/record.url?scp=85102413165&partnerID=8YFLogxK
U2 - 10.1109/IROS45743.2020.9341147
DO - 10.1109/IROS45743.2020.9341147
M3 - Conference contribution
AN - SCOPUS:85102413165
SN - 9781728162133
T3 - IEEE IROS Proceedings Series
SP - 10197
EP - 10202
BT - 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2020
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 24 October 2020 through 24 January 2021
ER -