论文标题
野外的照片现实360头化身
Photo-realistic 360 Head Avatars in the Wild
论文作者
论文摘要
提供沉浸式的3D经验进行人类交流，需要一种获得360度的人类照相化身的方法。为了使所有人都可以访问这些体验，只有商品硬件（例如手机摄像机）才能捕获化身创建所需的数据。为了使化身从任何角度实际渲染，我们需要从各个角度训练图像和相机姿势。但是，我们不能依靠所有图像的前景或背景中都有可追踪功能，以估算姿势，尤其是从头部的侧面或背面使用。为了克服这一点，我们提出了一个新颖的地标探测器，该探测器对合成数据进行了培训，以估算相机的姿势，该摄像头来自360度的人头的手机视频，用于在多阶段优化过程中使用，从而创建照片真实的头像。我们使用合成数据执行验证实验，并在360度化身中展示了通过手机视频训练的360度化身。
Delivering immersive, 3D experiences for human communication requires a method to obtain 360 degree photo-realistic avatars of humans. To make these experiences accessible to all, only commodity hardware, like mobile phone cameras, should be necessary to capture the data needed for avatar creation. For avatars to be rendered realistically from any viewpoint, we require training images and camera poses from all angles. However, we cannot rely on there being trackable features in the foreground or background of all images for use in estimating poses, especially from the side or back of the head. To overcome this, we propose a novel landmark detector trained on synthetic data to estimate camera poses from 360 degree mobile phone videos of a human head for use in a multi-stage optimization process which creates a photo-realistic avatar. We perform validation experiments with synthetic data and showcase our method on 360 degree avatars trained from mobile phone videos.
