论文标题
高保真图像用gan倒置插入
High-Fidelity Image Inpainting with GAN Inversion
论文作者
论文摘要
图像介入寻求一种语义一致的方法，以根据其未掩盖的内容来恢复损坏的图像。以前的方法通常将训练有素的甘恩重复使用，然后在产生逼真的斑块中，以用于缺少gan倒置的孔。然而，这些算法中对硬约束的无知可能会产生gan倒置和图像插入之间的差距。在解决这个问题的情况下，在本文中，我们设计了一个新颖的GAN反转模型，用于图像插入，称为Interverfill，主要由带有预热模块的编码器和具有F＆W+潜在空间的GAN GENELATOR组成。在编码器中，预处理网络利用多尺度结构将更具歧视性语义编码为样式向量。为了弥合GAN倒置和图像插入之间的缝隙，提出了F＆W+潜在空间，以消除明显的颜色差异和语义不一致。为了重建忠实和逼真的图像，一个简单而有效的软上升平均潜在模块旨在捕获更多样化的内域模式，从而综合了用于大型腐败的高保真质地。在包括Ploce2，Celeba-HQ，Metfaces和Scenery在内的四个具有挑战性的数据集上进行的全面实验表明，我们的Intervill在定性和定量上胜过高级方法，并支持室外图像的完成。
Image inpainting seeks a semantically consistent way to recover the corrupted image in the light of its unmasked content. Previous approaches usually reuse the well-trained GAN as effective prior to generate realistic patches for missing holes with GAN inversion. Nevertheless, the ignorance of a hard constraint in these algorithms may yield the gap between GAN inversion and image inpainting. Addressing this problem, in this paper, we devise a novel GAN inversion model for image inpainting, dubbed InvertFill, mainly consisting of an encoder with a pre-modulation module and a GAN generator with F&W+ latent space. Within the encoder, the pre-modulation network leverages multi-scale structures to encode more discriminative semantics into style vectors. In order to bridge the gap between GAN inversion and image inpainting, F&W+ latent space is proposed to eliminate glaring color discrepancy and semantic inconsistency. To reconstruct faithful and photorealistic images, a simple yet effective Soft-update Mean Latent module is designed to capture more diverse in-domain patterns that synthesize high-fidelity textures for large corruptions. Comprehensive experiments on four challenging datasets, including Places2, CelebA-HQ, MetFaces, and Scenery, demonstrate that our InvertFill outperforms the advanced approaches qualitatively and quantitatively and supports the completion of out-of-domain images well.
