论文标题
带有树序的变压器，用于神经程序生成
Transformer with Tree-order Encoding for Neural Program Generation
论文作者
论文摘要
尽管大量的语义解析方法采用了RNN架构来生成代码生成任务，但很少有尝试调查变形金刚在此任务中的适用性。事实证明，包括基础编程语言语法的层次结构信息对代码生成有效。由于变压器的位置编码只能以平坦序列表示位置，因此我们扩展了编码方案，以允许注意机制也可以通过输入中的层次位置参加。此外，我们已经实现了基于限制性语法图模型的解码器，以提高生成准确性并确保生成的代码的形成良好。虽然我们没有超过最新技术的状态，但我们的发现表明，使用基于树的位置编码与共享的自然语言子词汇结合使用，可以改善对顺序位置编码的发电性能。
While a considerable amount of semantic parsing approaches have employed RNN architectures for code generation tasks, there have been only few attempts to investigate the applicability of Transformers for this task. Including hierarchical information of the underlying programming language syntax has proven to be effective for code generation. Since the positional encoding of the Transformer can only represent positions in a flat sequence, we have extended the encoding scheme to allow the attention mechanism to also attend over hierarchical positions in the input. Furthermore, we have realized a decoder based on a restrictive grammar graph model to improve the generation accuracy and ensure the well-formedness of the generated code. While we did not surpass the state of the art, our findings suggest that employing a tree-based positional encoding in combination with a shared natural-language subword vocabulary improves generation performance over sequential positional encodings.
