论文标题
DES3：使用VIT相似性自适应注意力驱动的自我和柔和的阴影去除
DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similarity
论文作者
论文摘要
删除缺乏单个图像中缺乏明确边界的柔软和自我阴影仍然具有挑战性。自我阴影是在对象本身上施放的阴影。大多数现有的方法都依赖于二进制阴影面具，而无需考虑柔软和自我阴影的模棱两可的边界。在本文中，我们提出了DES3，这种方法可以根据适应性注意力和VIT相似性去除硬，柔软和自我阴影。我们新颖的VIT相似性损失利用从预先训练的视觉变压器中提取的功能。这种损失有助于指导反向采样到恢复场景结构。我们的自适应注意力能够将阴影区域与基础物体以及阴影区域与铸造阴影的对象区分开。该功能使DES3即使对象被阴影部分遮住，也可以更好地恢复对象的结构。与在训练阶段依赖约束的现有方法不同，我们在抽样阶段结合了VIT相似性。我们的方法的表现优于SRD，AISTD，LRSS，USR和UIUC数据集上的最新方法，可以强烈地删除硬，柔软和自我阴影。具体而言，我们的方法比LRSS数据集上整个图像的16 \％优于SOTA方法。我们的数据和代码可在：\ url {https://github.com/jinyeying/des3_deshadow}中获得
Removing soft and self shadows that lack clear boundaries from a single image is still challenging. Self shadows are shadows that are cast on the object itself. Most existing methods rely on binary shadow masks, without considering the ambiguous boundaries of soft and self shadows. In this paper, we present DeS3, a method that removes hard, soft and self shadows based on adaptive attention and ViT similarity. Our novel ViT similarity loss utilizes features extracted from a pre-trained Vision Transformer. This loss helps guide the reverse sampling towards recovering scene structures. Our adaptive attention is able to differentiate shadow regions from the underlying objects, as well as shadow regions from the object casting the shadow. This capability enables DeS3 to better recover the structures of objects even when they are partially occluded by shadows. Different from existing methods that rely on constraints during the training phase, we incorporate the ViT similarity during the sampling stage. Our method outperforms state-of-the-art methods on the SRD, AISTD, LRSS, USR and UIUC datasets, removing hard, soft, and self shadows robustly. Specifically, our method outperforms the SOTA method by 16\% of the RMSE of the whole image on the LRSS dataset. Our data and code is available at: \url{https://github.com/jinyeying/DeS3_Deshadow}
