| 摘要: |
| 针对毫米波雷达和相机融合进行三维目标检测时毫米波雷达点云特征提取不充分、雷达点云投影到2D图像平面时丢失空间信息造成空间关系捕捉不足，导致目标漏检和错检的问题，提出了一种基于残差多层感知机(Residual Multilayer Perceptron，ResMLP)的雷达相机融合三维目标检测方法(ResMLP-Fusion)。首先，利用图像特征提取器提取多视角图像的特征；其次，利用雷达点云的位置信息和雷达截面信息将点云聚类后输入到设计的ResMLP网络中提取雷达点云特征，用于后续与图像特征融合；然后，将雷达特征和图像特征利用可变形注意力机制进行采样融合之后再更新查询，最后输入到解码器进行解码，对目标进行三维边界框预测和类别预测，并对预测结果进行迭代，以改进预测结果的准确性，实现具有准确性和鲁棒性的三维目标检测。在nuScenes数据集上对所提方法进行评估，其平均精度达到了0.404，nuScenes检测得分达到了0.518。 |
| 关键词: 三维目标检测 毫米波雷达；视觉传感器；传感器融合；残差多层感知机 |
| DOI：10.20079/j.issn.1001-893x.240628002 |
|
| 基金项目: |
|
| Radar-Camera Fusion for 3D Object Detection with Application of Residual Multilayer Perceptron |
| HUANG Xiaohong,TIAN Ziran,XU Kunqiang |
| (1.College of Artificial Intelligence,North China University of Science and Technology,Tangshan 063210,China;2.Key Laboratory of Industrial Intelligent Perception of Hebei Province,Tangshan 063000,China) |
| Abstract: |
| To address the issue of insufficient feature extraction of millimeter-wave(MMW) radar point cloud and insufficient capturing of spatial relationship caused by the loss of spatial information when MMW radar point cloud is projected to two-dimensional(2D) image plane,which leads to object omission and misdetection,a radar-camera fusion three-dimensional(3D) object detection method called Residual Multilayer Perceptron Fusion(ResMLP-Fusion) is proposed.Firstly,the features of the multi-view image are extracted using an image feature extractor.Secondly,the radar point cloud is clustered by using the radar point cloud搒 positional information and the radar cross section information and then input into the designed ResMLP network to extract the point cloud features.Subsequently,the radar features and image features are sampled and fused using the deformable attention mechanism before updating the query,which is finally input to the decoder for decoding to perform 3D bounding box prediction and category prediction of the object,and the prediction results are iterated to improve the accuracy of the prediction results to realize 3D object detection with accuracy and robustness.The proposed method is evaluated on the nuScenes dataset,and the experimental results show that it achieves a mean average precision of 0.404 and a nuScenes detection score of 0.518. |
| Key words: 3D object detection millimeter-wave radar vision sensor sensor fusion residual multilayer perceptron |
