论文标题
基于示例的超网络用于分布概括
Example-based Hypernetworks for Out-of-Distribution Generalization
论文作者
论文摘要
随着自然语言处理（NLP）算法不断实现新的里程碑，分布式概括仍然是一个重大挑战。本文介绍了陌生域的多源改编的问题：我们利用了从多个源域标记的数据来推广到训练中未知目标域。我们的创新框架采用了基于示例的超网络改编：T5编码器最初从输入示例生成唯一的签名，将其嵌入源域的语义空间中。此签名随后被超网络用于生成任务分类器的权重。我们在29个适应方案中评估了两个任务的方法 - 情感分类和自然语言推论 - 在该方案中，它超过了已建立的算法。在高级版本中，签名还丰富了输入示例的表示。我们还将我们的固定体系结构与少数弹药的GPT-3进行了比较，证明了其在基本用例中的有效性。据我们所知，这标志着超网络对未知域的适应性的首次应用。
As Natural Language Processing (NLP) algorithms continually achieve new milestones, out-of-distribution generalization remains a significant challenge. This paper addresses the issue of multi-source adaptation for unfamiliar domains: We leverage labeled data from multiple source domains to generalize to unknown target domains at training. Our innovative framework employs example-based Hypernetwork adaptation: a T5 encoder-decoder initially generates a unique signature from an input example, embedding it within the source domains' semantic space. This signature is subsequently utilized by a Hypernetwork to generate the task classifier's weights. We evaluated our method across two tasks - sentiment classification and natural language inference - in 29 adaptation scenarios, where it outpaced established algorithms. In an advanced version, the signature also enriches the input example's representation. We also compare our finetuned architecture to few-shot GPT-3, demonstrating its effectiveness in essential use cases. To our knowledge, this marks the first application of Hypernetworks to the adaptation for unknown domains.
