论文标题
视觉语义AI中低调的证据
Evidence for Hypodescent in Visual Semantic AI
论文作者
论文摘要
我们针对低调或单势规则的规则检查了最先进的多模式“视觉语义”模型剪辑（“对比语言图像”），在这些规则中，多种族的人更有可能将与少数派或不利的种族或种族群体相比，将种族或种族标签分配给相比，比相比或等同的多数群体。基于心理学研究的面部变形实验表明，在1,000个变形的图像的中间点，剪辑将69.7％的黑白女性图像与白色文本标签上的黑白女性形象相关，同样会在Latina（75.8％）和亚洲（89.1％）的女性（89.1％）的女性中，以示为中等的女性 - 低调。此外，对模型中潜在余弦相似性的评估表明，与白人的关联与“人”的关联相关，皮尔逊的Rho在21,000图像的变体系列中高达0.82，表明白人对应于夹子中人的默认表示。最后，我们表明，图像的刻板印象 - 一致性愉悦的关联与剪辑中的黑色文本标签有关，Pearson的Rho = 0.48，用于21,000黑白多种族多种族男性图像，而黑白多种多种女性图像的Rho = 0.41。使用从美国网站（Wikipedia）收集的数据收集的英语文本进行了培训，我们的发现表明，剪辑嵌入了美国种族等级的价值观，反映了人类思想中存在的隐性和明确信念。我们将这些发现在低调的历史和心理学中与这些发现相关。总体而言，数据表明，使用自然语言监督的AI将学习反映种族层次结构的偏见。
We examine the state-of-the-art multimodal "visual semantic" model CLIP ("Contrastive Language Image Pretraining") for the rule of hypodescent, or one-drop rule, whereby multiracial people are more likely to be assigned a racial or ethnic label corresponding to a minority or disadvantaged racial or ethnic group than to the equivalent majority or advantaged group. A face morphing experiment grounded in psychological research demonstrating hypodescent indicates that, at the midway point of 1,000 series of morphed images, CLIP associates 69.7% of Black-White female images with a Black text label over a White text label, and similarly prefers Latina (75.8%) and Asian (89.1%) text labels at the midway point for Latina-White female and Asian-White female morphs, reflecting hypodescent. Additionally, assessment of the underlying cosine similarities in the model reveals that association with White is correlated with association with "person," with Pearson's rho as high as 0.82 over a 21,000-image morph series, indicating that a White person corresponds to the default representation of a person in CLIP. Finally, we show that the stereotype-congruent pleasantness association of an image correlates with association with the Black text label in CLIP, with Pearson's rho = 0.48 for 21,000 Black-White multiracial male images, and rho = 0.41 for Black-White multiracial female images. CLIP is trained on English-language text gathered using data collected from an American website (Wikipedia), and our findings demonstrate that CLIP embeds the values of American racial hierarchy, reflecting the implicit and explicit beliefs that are present in human minds. We contextualize these findings within the history and psychology of hypodescent. Overall, the data suggests that AI supervised using natural language will, unless checked, learn biases that reflect racial hierarchies.
