论文标题
为什么曝光偏见很重要：模仿学习观点，即语言生成中错误积累的观点
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation
论文作者
论文摘要
当前的语言生成模型遭受重复，不一致和幻觉等问题。一个经常重复的假设是，生成模型的这种脆弱性是由训练和生成过程不匹配引起的，也称为暴露偏见。在本文中，我们通过从模仿学习的角度分析暴露偏见来验证这一假设。我们表明，暴露偏见会导致错误的积累，分析为什么困惑无法捕获这种积累，并从经验上表明这种积累会导致发电质量差。可以在https://github.com/kushalarora/quantifying_exposure_bias上获得重现这些实验的源代码。
Current language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis is that this brittleness of generation models is caused by the training and the generation procedure mismatch, also referred to as exposure bias. In this paper, we verify this hypothesis by analyzing exposure bias from an imitation learning perspective. We show that exposure bias leads to an accumulation of errors, analyze why perplexity fails to capture this accumulation, and empirically show that this accumulation results in poor generation quality. Source code to reproduce these experiments is available at https://github.com/kushalarora/quantifying_exposure_bias
