论文标题
深度素描引导的卡通视频
Deep Sketch-guided Cartoon Video Inbetweening
论文作者
论文摘要
我们提出了一个新颖的框架，以通过从用户草图引导的动画运动时从两个输入键框中获取颜色信息来制作卡通视频。提出方法的关键思想是估计草图和卡通视频帧之间密集的跨域对应关系，并采用带有遮挡估算的混合模块来合成由草图指导的中间框架。之后，将配备已建立对应的输入帧和合成框架送入任意时间帧插值管道中，以生成和完善框架之间的额外框架。最后，采用了保留时间一致性的模块。与常见的框架插值方法相比，我们的方法可以以相对较大的运动来解决框架，并且具有灵活性，使用户可以通过编辑草图指南来控制生成的视频序列。通过明确考虑框架和草图之间的对应关系，我们可以比其他图像合成方法获得更高的质量结果。我们的结果表明，我们的系统可以很好地推广到不同的电影框架，从而获得了比现有解决方案更好的结果。
We propose a novel framework to produce cartoon videos by fetching the color information from two input keyframes while following the animated motion guided by a user sketch. The key idea of the proposed approach is to estimate the dense cross-domain correspondence between the sketch and cartoon video frames, and employ a blending module with occlusion estimation to synthesize the middle frame guided by the sketch. After that, the input frames and the synthetic frame equipped with established correspondence are fed into an arbitrary-time frame interpolation pipeline to generate and refine additional inbetween frames. Finally, a module to preserve temporal consistency is employed. Compared to common frame interpolation methods, our approach can address frames with relatively large motion and also has the flexibility to enable users to control the generated video sequences by editing the sketch guidance. By explicitly considering the correspondence between frames and the sketch, we can achieve higher quality results than other image synthesis methods. Our results show that our system generalizes well to different movie frames, achieving better results than existing solutions.
