论文标题
funck：信息渠道和瓶颈不变代表学习
FUNCK: Information Funnels and Bottlenecks for Invariant Representation Learning
论文作者
论文摘要
对下游任务仍然有用的学习不变表示仍然是机器学习的关键挑战。我们研究了一组相关的信息渠道和瓶颈问题，这些问题声称从数据中学习不变表示。我们还向这个信息理论目标家族提出了一个新元素：带有附带信息的条件隐私渠道，我们在完全和半监督的设置中进行了调查。鉴于总体上棘手的目标，我们使用由神经网络对摊销的变异推理进行了可拖动近似，并研究了这些目标的内在权衡。我们从经验上描述了所提出的方法，并表明有了一些标签，可以学习公平的分类器并生成有用的表示形式，大约是不需要的变化来源。此外，当数据稀缺时，我们还提供有关这些方法在现实世界中具有普通表格数据集的适用性的见解。
Learning invariant representations that remain useful for a downstream task is still a key challenge in machine learning. We investigate a set of related information funnels and bottleneck problems that claim to learn invariant representations from the data. We also propose a new element to this family of information-theoretic objectives: The Conditional Privacy Funnel with Side Information, which we investigate in fully and semi-supervised settings. Given the generally intractable objectives, we derive tractable approximations using amortized variational inference parameterized by neural networks and study the intrinsic trade-offs of these objectives. We describe empirically the proposed approach and show that with a few labels it is possible to learn fair classifiers and generate useful representations approximately invariant to unwanted sources of variation. Furthermore, we provide insights about the applicability of these methods in real-world scenarios with ordinary tabular datasets when the data is scarce.
