论文标题
Victor：与变压器和特定于时尚的对比预训练的视觉不相容性检测
VICTOR: Visual Incompatibility Detection with Transformers and Fashion-specific contrastive pre-training
论文作者
论文摘要
为了使时尚服装在美学上令人愉悦，构成它们的服装需要在视觉方面（例如样式，类别和颜色）兼容。以前的作品将视觉兼容性定义为二进制分类任务，而在服装中的项目被认为是完全兼容或完全不兼容的。但是，这不适用于用户创建自己的服装的服装制造商应用程序，并且需要知道哪些特定项目可能与其余的服装不相容。为了解决这个问题，我们提出了针对两个任务进行了优化的视觉不兼容变压器（Victor）：1）总体兼容性作为回归和2）检测不匹配项目并利用时尚特定于时尚的对比性语言图像预训练，用于微调计算机视觉神经网络在时尚成像上的微调计算机视觉神经网络。我们基于Polyvore服装基准测试，以产生部分不匹配的服装，创建一个新的数据集，称为Polyvore-Misfits，该数据集用于训练Victor。一系列消融和比较分析表明，所提出的架构可以竞争，甚至超过Polyvore数据集上的最新目前，同时将实例的浮动操作减少88％，从而在高性能和效率之间达到平衡。我们在https://github.com/stevejpapad/visual-compatibility-transformer上发布代码
For fashion outfits to be considered aesthetically pleasing, the garments that constitute them need to be compatible in terms of visual aspects, such as style, category and color. Previous works have defined visual compatibility as a binary classification task with items in a garment being considered as fully compatible or fully incompatible. However, this is not applicable to Outfit Maker applications where users create their own outfits and need to know which specific items may be incompatible with the rest of the outfit. To address this, we propose the Visual InCompatibility TransfORmer (VICTOR) that is optimized for two tasks: 1) overall compatibility as regression and 2) the detection of mismatching items and utilize fashion-specific contrastive language-image pre-training for fine tuning computer vision neural networks on fashion imagery. We build upon the Polyvore outfit benchmark to generate partially mismatching outfits, creating a new dataset termed Polyvore-MISFITs, that is used to train VICTOR. A series of ablation and comparative analyses show that the proposed architecture can compete and even surpass the current state-of-the-art on Polyvore datasets while reducing the instance-wise floating operations by 88%, striking a balance between high performance and efficiency. We release our code at https://github.com/stevejpapad/Visual-InCompatibility-Transformer
