论文标题
使用复杂加权的多发表自动机代表无序的数据
Representing Unordered Data Using Complex-Weighted Multiset Automata
论文作者
论文摘要
在多个字段的许多设置中都会出现无序的，可变大小的输入。近年来，设置和多组导向的神经网络处理这种输入的能力一直是很多工作的重点。我们建议使用复杂加权的多磁力自动机代表多组，并展示如何将某些现有神经体系结构的多键表示形式视为我们的特殊情况。也就是说，（1）我们为变压器模型使用正弦函数的位置表示新的理论和直观理由，并且（2）我们扩展了DeepSets模型以使用复杂数字，使其能够在其任务之一的扩展上优于现有模型。
Unordered, variable-sized inputs arise in many settings across multiple fields. The ability for set- and multiset-oriented neural networks to handle this type of input has been the focus of much work in recent years. We propose to represent multisets using complex-weighted multiset automata and show how the multiset representations of certain existing neural architectures can be viewed as special cases of ours. Namely, (1) we provide a new theoretical and intuitive justification for the Transformer model's representation of positions using sinusoidal functions, and (2) we extend the DeepSets model to use complex numbers, enabling it to outperform the existing model on an extension of one of their tasks.
