论文标题
在动态场景中用于新型视图和时间综合的便携式多座相机
A Portable Multiscopic Camera for Novel View and Time Synthesis in Dynamic Scenes
论文作者
论文摘要
我们提出了一种便携式多型相机系统，该系统具有专用模型，用于动态场景中的新型视图和时间综合。我们的目标是使用我们的便携式多座相机从任何角度从任何角度渲染高质量的图像。为了获得这种新颖的视图和时间综合，我们开发了一个配备了五个相机的物理多型摄像头，以在时间和空间域中训练神经辐射场（NERF），以进行动态场景。我们的模型将6D坐标（3D空间位置，1D时间坐标和2D观看方向）映射到观看依赖性且随时间变化的发射辐射和体积密度。使用音量渲染用于在指定的相机姿势和时间上渲染光真实的图像。为了提高物理相机的鲁棒性，我们提出了一个摄像机参数优化模块和一个时间框架插值模块，以促进跨时间的信息传播。我们对现实世界和合成数据集进行了实验以评估我们的系统，结果表明，我们的方法在定性和定量上都优于替代解决方案。我们的代码和数据集可从https://yuenfuilau.github.io获得。
We present a portable multiscopic camera system with a dedicated model for novel view and time synthesis in dynamic scenes. Our goal is to render high-quality images for a dynamic scene from any viewpoint at any time using our portable multiscopic camera. To achieve such novel view and time synthesis, we develop a physical multiscopic camera equipped with five cameras to train a neural radiance field (NeRF) in both time and spatial domains for dynamic scenes. Our model maps a 6D coordinate (3D spatial position, 1D temporal coordinate, and 2D viewing direction) to view-dependent and time-varying emitted radiance and volume density. Volume rendering is applied to render a photo-realistic image at a specified camera pose and time. To improve the robustness of our physical camera, we propose a camera parameter optimization module and a temporal frame interpolation module to promote information propagation across time. We conduct experiments on both real-world and synthetic datasets to evaluate our system, and the results show that our approach outperforms alternative solutions qualitatively and quantitatively. Our code and dataset are available at https://yuenfuilau.github.io.
