Fusion of camera-based vessel detection and AIS for maritime surveillance

Camera-based vessel identification is one of the essential maritime surveillance measures. The existing deep learning (DL)-based vessel classification methods have challenges in achieving the detailed vessel information (e.g., vessel identity) under poor visual conditions. In some applications, they cannot meet the safe scheduling and abnormal activity identification purposes in the camera surveillance system. This paper proposed a fusion framework, which takes the visual detection and the automatic identification system (AIS) as inputs to obtain accurate and robust vessel identification to address these challenges. Visual detection uses an innovative method to estimate the distance of the identified vessel target from the camera and the azimuth angle relative to the camera based on the images from the monocular camera. The distance and azimuth angle were further fused with AIS data to obtain the fused output. The test results based on 3976 images show that the mean absolute error of the visual-based distance estimation method is 0.72km. The success rate of visual detection and AIS data association is 75.7%.