论文标题
Minkowski跟踪器：用于关节对象检测和跟踪的稀疏时空R-CNN
Minkowski Tracker: A Sparse Spatio-Temporal R-CNN for Joint Object Detection and Tracking
论文作者
论文摘要
多任务学习的最新研究揭示了解决单个神经网络中相关问题的好处。 3D对象检测和多对象跟踪（MOT）是两个严重的交织问题，可以预测并关联跨时间的对象实例位置。但是，3D MOT中的大多数先前作品都将检测器视为先前的分离管道，不一致地将检测器的输出作为跟踪器的输入。在这项工作中，我们提出了Minkowski Tracker，这是一种稀疏时空的R-CNN，可以共同解决对象检测和跟踪。受基于区域的CNN（R-CNN）的启发，我们建议将跟踪作为对象检测器R-CNN的第二阶段，该跟踪可以预测轨道的分配概率。首先，Minkowski Tracker将4D点云作为输入，以通过4D稀疏卷积编码器网络生成时空鸟的视图（BEV）特征映射。然后，我们提出的TrackAlign聚集了BEV功能的轨道区域（ROI）功能。最后，Minkowski Tracker根据ROI功能预测的检测到追踪匹配概率更新了跟踪和置信度得分。我们在大规模实验中表明，我们方法的总体性能增长是由于四个因素：1。4D编码器的时间推理提高了检测性能2。对象检测的多任务学习和MOT互相增强3。检测到跟踪的匹配得分匹配得分学习模型，以增强型号的指数4。检测到检测得分4。取得匹配匹配匹配匹配匹配匹配匹配的评分。结果，Minkowski Tracker在没有手工设计的运动模型的情况下实现了Nuscenes数据集跟踪任务上的最新性能。
Recent research in multi-task learning reveals the benefit of solving related problems in a single neural network. 3D object detection and multi-object tracking (MOT) are two heavily intertwined problems predicting and associating an object instance location across time. However, most previous works in 3D MOT treat the detector as a preceding separated pipeline, disjointly taking the output of the detector as an input to the tracker. In this work, we present Minkowski Tracker, a sparse spatio-temporal R-CNN that jointly solves object detection and tracking. Inspired by region-based CNN (R-CNN), we propose to solve tracking as a second stage of the object detector R-CNN that predicts assignment probability to tracks. First, Minkowski Tracker takes 4D point clouds as input to generate a spatio-temporal Bird's-eye-view (BEV) feature map through a 4D sparse convolutional encoder network. Then, our proposed TrackAlign aggregates the track region-of-interest (ROI) features from the BEV features. Finally, Minkowski Tracker updates the track and its confidence score based on the detection-to-track match probability predicted from the ROI features. We show in large-scale experiments that the overall performance gain of our method is due to four factors: 1. The temporal reasoning of the 4D encoder improves the detection performance 2. The multi-task learning of object detection and MOT jointly enhances each other 3. The detection-to-track match score learns implicit motion model to enhance track assignment 4. The detection-to-track match score improves the quality of the track confidence score. As a result, Minkowski Tracker achieved the state-of-the-art performance on Nuscenes dataset tracking task without hand-designed motion models.
