| 摘要: |
| 针对毫米波雷达和视觉传感器融合算法在特征融合层面缺乏有效监督的问题，提出了一种引入激光雷达监督的多模态融合三维目标检测算法（Radar and Camera Fusion Based on Lidar Supervision，LRCFusion）。该算法首先分别提取视觉传感器、激光雷达和毫米波雷达各自的数据特征；接着使用知识蒸馏的方法，利用激光雷达特征作为教师模型监督毫米波雷达特征，以提升毫米波雷达特征的表达水平；然后引入注意力机制实现毫米波雷达和视觉特征融合，并采用基于点云的三维物体检测方法对融合的特征进行目标检测和3D锚框预测；最后，使用预测的3D锚框更新融合前的3D参考点。与基线算法进行比较，所提算法的平均精度提高1.2%，归一化检测得分提高1%。 |
| 关键词: 毫米波雷达 目标检测 特征级监督 激光雷达 视觉传感器 |
| DOI：10.20079/j.issn.1001-893x.240105002 |
|
| 基金项目:国家科技部重点研发专项(2017YFE0135700)；深圳市科技创新委员会项目（JCYJ20210324120002007）；广东省科技厅先进智能感知技术重点实验室科技计划项目（2019B121203006） |
|
| Feature Level Supervised Millimeter Wave Radar and Vision Fusion for Target Detection |
| HUANG Xiaohong,HE Qing,TIAN Ziran |
| (1.College of Artificial Intelligence,North China University of Technology,Tangshan 063000,China;2.Key Laboratory of Industrial Intelligent Perception of Hebei Province,Tangshan 063000,China) |
| Abstract: |
| To address the issue of lacking effective supervision in the feature fusion stage of millimeter-wave(MMW) radar and vision sensor fusion algorithms,a multimodal fusion-based 3D object detection algorithm called LRCFusion(Radar and Camera Fusion Based on LiDAR Supervision) is proposed.The algorithm first extracts data features separately from the vision sensor,LiDAR,and MMW radar.Then,with the method of knowledge distillation,it uses LiDAR features as the teacher model to supervise MMW radar features,in order to improve the expression level of millimeter wave radar features.Subsequently,an attention mechanism is introduced to fuse the MMW radar and vision features,and a point cloud-based 3D object detection method is employed to detect objects and predict 3D anchor boxes based on the fused features.Finally,the predicted 3D anchor boxes are used to update the 3D reference points before fusion.Compared with baseline algorithms,the LRCFusion algorithm achieves a 1.2% increase in average precision and a 1% improvement in normalized detection score. |
| Key words: millimeter wave radar target detection feature level supervision LiDAR vision sensor |
