论文标题
测量各个领域因素在自我监督的预训练中的影响
Measuring the Impact of Individual Domain Factors in Self-Supervised Pre-Training
论文作者
论文摘要
人类语音数据包括丰富的领域因素，例如口音，句法和语义多样性或声学环境。先前的工作探讨了域不匹配在整体培训和微调之间自动语音识别中的影响，但并未剖析单个因素的贡献。在本文中，我们提出了一项对照研究，以更好地了解此类因素对预训练的表现对自动语音识别的影响。为此，我们在修改的自然语音或合成的音频上预先训练模型，并修改了单个域因子，然后在微调后测量性能。结果表明，语音域因子在预训练期间起着重要作用，而语法和句法因素的重要性远不那么重要。据我们所知，这是第一项更好地了解自我监管的言语预训练中预训练的域特征的研究。
Human speech data comprises a rich set of domain factors such as accent, syntactic and semantic variety, or acoustic environment. Previous work explores the effect of domain mismatch in automatic speech recognition between pre-training and fine-tuning as a whole but does not dissect the contribution of individual factors. In this paper, we present a controlled study to better understand the effect of such factors on the performance of pre-trained representations on automatic speech recognition. To do so, we pre-train models either on modified natural speech or synthesized audio, with a single domain factor modified, and then measure performance after fine-tuning. Results show that phonetic domain factors play an important role during pre-training while grammatical and syntactic factors are far less important. To our knowledge, this is the first study to better understand the domain characteristics of pre-trained sets in self-supervised pre-training for speech.
