论文标题
一个多功能框架，用于评估群体公平和相关性方面的排名列表
A Versatile Framework for Evaluating Ranked Lists in terms of Group Fairness and Relevance
论文作者
论文摘要
我们提出了一个简单且通用的框架，用于根据群体公平和相关性评估排名列表，其中组（即可能的属性值）可以是象征性的，也可以是词立。首先，我们证明，如果属性集为二进制，我们的框架可以轻松量化每个排名列表的整体极性。其次，通过利用现有的多元化搜索测试收集并将每个意图视为属性价值，我们证明我们的框架可以处理软组成员资格，并且我们的团体公平度量与Adhoc IR和此设置下的多元化IR措施高度相关。第三，我们演示了我们的框架如何基于多个属性集量化交叉组的公平性。我们还表明，应仔细选择用于比较属性值所达到的目标和目标分布的相似性函数。
We present a simple and versatile framework for evaluating ranked lists in terms of group fairness and relevance, where the groups (i.e., possible attribute values) can be either nominal or ordinal in nature. First, we demonstrate that, if the attribute set is binary, our framework can easily quantify the overall polarity of each ranked list. Second, by utilising an existing diversified search test collection and treating each intent as an attribute value, we demonstrate that our framework can handle soft group membership, and that our group fairness measures are highly correlated with both adhoc IR and diversified IR measures under this setting. Third, we demonstrate how our framework can quantify intersectional group fairness based on multiple attribute sets. We also show that the similarity function for comparing the achieved and target distributions over the attribute values should be chosen carefully.
