论文标题
学会从多个视图纠正3D重建
Learning to Correct 3D Reconstructions from Multiple Views
论文作者
论文摘要
本文是关于降低事后建立良好大规模3D重建的成本。我们提供了现有重建的2D视图，并培训了卷积神经网络（CNN），该卷积神经网络（CNN）可以完善逆深度以匹配更高质量的重建。由于我们正确正确的视图是从相同的重建中呈现的，因此它们共享相同的几何形状，因此重叠的视图相互补充。我们通过两种方式利用这一点。首先，我们在训练期间施加损失，该损失指导对邻近观点的预测具有相同的几何形状，并已被证明可以提高性能。其次，与以前的工作相比，该工作独立纠正每个视图，我们还共同对相邻观点的集进行了预测。这是通过视图之间的翘曲特征图来实现的，从而绕过内存密集型3D计算。我们观察到特征映射中的特征是观点依赖性的，并提出了一种通过从视图之间的相对姿势产生的动态过滤器来转换特征的方法。在我们的实验中，我们表明，最后一步对于成功融合视图之间的特征图是必要的。
This paper is about reducing the cost of building good large-scale 3D reconstructions post-hoc. We render 2D views of an existing reconstruction and train a convolutional neural network (CNN) that refines inverse-depth to match a higher-quality reconstruction. Since the views that we correct are rendered from the same reconstruction, they share the same geometry, so overlapping views complement each other. We take advantage of that in two ways. Firstly, we impose a loss during training which guides predictions on neighbouring views to have the same geometry and has been shown to improve performance. Secondly, in contrast to previous work, which corrects each view independently, we also make predictions on sets of neighbouring views jointly. This is achieved by warping feature maps between views and thus bypassing memory-intensive 3D computation. We make the observation that features in the feature maps are viewpoint-dependent, and propose a method for transforming features with dynamic filters generated by a multi-layer perceptron from the relative poses between views. In our experiments we show that this last step is necessary for successfully fusing feature maps between views.
