论文标题
祖先到凝结的转移不是在公园里散步
Ancestor-to-Creole Transfer is Not a Walk in the Park
论文作者
论文摘要
我们旨在学习不容易获得大量数据的克里奥尔语言的语言模型，因此探索了祖先语言的潜在转移（“祖先转移假设”）。我们发现标准转移方法不会促进祖先转移。令人惊讶的是，与其他非凝聚语言不同，克里奥尔语出现了一种非常独特的两相模式：随着我们的训练损失高原，语言模型开始过分贴在其源语言上，因此克里奥尔语的困惑下降了。我们探索这个压缩阶段是否可以导致实际上有用的语言模型（“祖先瓶颈假说”），但也伪造了这一点。此外，我们表明，即使在随机，无关的语言训练时，克里奥尔人甚至表现出这种两相模式。因此，克里奥尔人似乎是类型的异常值，我们推测两个观察结果之间是否存在联系。
We aim to learn language models for Creole languages for which large volumes of data are not readily available, and therefore explore the potential transfer from ancestor languages (the 'Ancestry Transfer Hypothesis'). We find that standard transfer methods do not facilitate ancestry transfer. Surprisingly, different from other non-Creole languages, a very distinct two-phase pattern emerges for Creoles: As our training losses plateau, and language models begin to overfit on their source languages, perplexity on the Creoles drop. We explore if this compression phase can lead to practically useful language models (the 'Ancestry Bottleneck Hypothesis'), but also falsify this. Moreover, we show that Creoles even exhibit this two-phase pattern even when training on random, unrelated languages. Thus Creoles seem to be typological outliers and we speculate whether there is a link between the two observations.
