论文标题
NeurAllift-360：将野外2D照片提升到具有360°视图的3D对象
NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views
论文作者
论文摘要
虚拟现实和增强现实（XR）带来了对3D内容的需求不断增长。但是，创建高质量的3D内容需要人类专家必须做的乏味的工作。在这项工作中，我们研究了将单个图像提升为3D对象的挑战性任务，并首次展示了具有与给定参考图像良好相对应的360°视图的合理的3D对象的能力。通过对参考图像进行调节，我们的模型可以满足综合图像对象的新颖观点的永恒好奇心。我们的技术阐明了为3D艺术家和XR设计师提供工作流程的有希望的方向。我们提出了一个被称为Neurallift-360的新型框架，该框架利用了深度感知的神经辐射表示（NERF），并学会了通过降级扩散模型来制作的场景。通过引入排名损失，我们的Neurallift-360可以在野外进行粗略的深度估计。在提供连贯的指导之前，我们还为扩散采用了夹子引导的采样策略。广泛的实验表明，我们的神经360显着优于现有的最新基准。项目页面：https：//vita-group.github.io/neurallift-360/
Virtual reality and augmented reality (XR) bring increasing demand for 3D content. However, creating high-quality 3D content requires tedious work that a human expert must do. In this work, we study the challenging task of lifting a single image to a 3D object and, for the first time, demonstrate the ability to generate a plausible 3D object with 360° views that correspond well with the given reference image. By conditioning on the reference image, our model can fulfill the everlasting curiosity for synthesizing novel views of objects from images. Our technique sheds light on a promising direction of easing the workflows for 3D artists and XR designers. We propose a novel framework, dubbed NeuralLift-360, that utilizes a depth-aware neural radiance representation (NeRF) and learns to craft the scene guided by denoising diffusion models. By introducing a ranking loss, our NeuralLift-360 can be guided with rough depth estimation in the wild. We also adopt a CLIP-guided sampling strategy for the diffusion prior to provide coherent guidance. Extensive experiments demonstrate that our NeuralLift-360 significantly outperforms existing state-of-the-art baselines. Project page: https://vita-group.github.io/NeuralLift-360/
