论文标题

无监督的卫星图像中语义概念的发现,具有基于样式的小波驱动的生成模型

Unsupervised Discovery of Semantic Concepts in Satellite Imagery with Style-based Wavelet-driven Generative Models

论文作者

Kostagiolas, Nikos, Nicolaou, Mihalis A., Panagakis, Yannis

论文摘要

近年来,在生成的对抗网络(GAN)领域已取得了很大的进步,尤其是在基于样式的架构的出现,这些架构解决了许多关键的缺点 - 无论是在建模能力和网络解释性方面。尽管有这些改进,但在卫星图像领域中采用这种方法并不直接。生成任务中使用的典型视觉数据集经过良好结合和注释,并且具有有限的可变性。相比之下,卫星图像表现出很大的空间和光谱变异性,广泛的高频细节的存在,而注释卫星图像的繁琐本质会导致注释稀缺性 - 进一步激励了无监督学习的发展。从这个角度来看,我们介绍了第一个基于预训练的样式和小波的GAN模型,该模型可以很容易地在各种环境和条件下综合了一系列现实的卫星图像,同时还可以保留高频信息。此外,我们表明,通过分析网络的中间激活,人们可以发现许多可解释的语义方向,这些方向促进了卫星图像的指导综合,而无需使用任何形式的监督。通过一组定性和定量实验,我们证明了我们框架的功效,这是在适合下游任务(例如,数据增强),合成成像质量以及对看不见数据集的概括能力方面的功效。

In recent years, considerable advancements have been made in the area of Generative Adversarial Networks (GANs), particularly with the advent of style-based architectures that address many key shortcomings - both in terms of modeling capabilities and network interpretability. Despite these improvements, the adoption of such approaches in the domain of satellite imagery is not straightforward. Typical vision datasets used in generative tasks are well-aligned and annotated, and exhibit limited variability. In contrast, satellite imagery exhibits great spatial and spectral variability, wide presence of fine, high-frequency details, while the tedious nature of annotating satellite imagery leads to annotation scarcity - further motivating developments in unsupervised learning. In this light, we present the first pre-trained style- and wavelet-based GAN model that can readily synthesize a wide gamut of realistic satellite images in a variety of settings and conditions - while also preserving high-frequency information. Furthermore, we show that by analyzing the intermediate activations of our network, one can discover a multitude of interpretable semantic directions that facilitate the guided synthesis of satellite images in terms of high-level concepts (e.g., urbanization) without using any form of supervision. Via a set of qualitative and quantitative experiments we demonstrate the efficacy of our framework, in terms of suitability for downstream tasks (e.g., data augmentation), quality of synthetic imagery, as well as generalization capabilities to unseen datasets.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源