论文标题
重新审查选区解析从预训练的语言模型中提取的实际有效性
Revisiting the Practical Effectiveness of Constituency Parse Extraction from Pre-trained Language Models
论文作者
论文摘要
从预训练的语言模型（CPE-PLM）中提取的选区解析是最近的范式，它试图诱导只依靠预训练的语言模型的内部知识来诱导选区解析树。尽管从类似于文本学习的角度来看，它不需要特定于任务的微调，但这种方法的实际有效性仍然不清楚，只是它可以作为调查语言模型的内部工作的探测。在这项工作中，我们在数学上重新制定了CPE-PLM，并提出了为其量身定制的两种先进的合奏方法，表明新的解析范式可以通过引入一套使用我们的技术组合的异质性PLM来与常见的无监督解析器竞争。此外，我们探讨了一些CPE-PLM生成的树木实际上有用的情况。具体来说，我们表明CPE-PLM在几次设置中比典型的监督解析器更有效。
Constituency Parse Extraction from Pre-trained Language Models (CPE-PLM) is a recent paradigm that attempts to induce constituency parse trees relying only on the internal knowledge of pre-trained language models. While attractive in the perspective that similar to in-context learning, it does not require task-specific fine-tuning, the practical effectiveness of such an approach still remains unclear, except that it can function as a probe for investigating language models' inner workings. In this work, we mathematically reformulate CPE-PLM and propose two advanced ensemble methods tailored for it, demonstrating that the new parsing paradigm can be competitive with common unsupervised parsers by introducing a set of heterogeneous PLMs combined using our techniques. Furthermore, we explore some scenarios where the trees generated by CPE-PLM are practically useful. Specifically, we show that CPE-PLM is more effective than typical supervised parsers in few-shot settings.
