论文标题
CERES：半结构化会话数据的图形条件变压器进行预处理
CERES: Pretraining of Graph-Conditioned Transformer for Semi-Structured Session Data
论文作者
论文摘要
用户会议每天都会授权许多搜索和推荐任务。此类会话数据是半结构的，该数据编码查询和产品之间的异质关系，每个项目都由非结构化文本描述。尽管在文本或图表的自学学习学习方面取得了最新进展，但缺乏自我监督的学习模型，这些模型可以有效地捕获半结构性会话的项目内语义和项目间相互作用。为了填补这一空白，我们提出了CERES，这是一种基于图的变压器模型，用于半结构化会话数据。 Ceres学习的表示形式可以通过（1）（1）一份图形的掩盖语言预处理任务来捕获Intera Inter-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-In-Ins语义； （2）将图形的变压器体系结构传播到项目级表示形式。我们使用约4.68亿个亚马逊会议预处理谷神星，发现Ceres在三个会话搜索和实体链接任务中的强大预读基线的表现最多高9％。
User sessions empower many search and recommendation tasks on a daily basis. Such session data are semi-structured, which encode heterogeneous relations between queries and products, and each item is described by the unstructured text. Despite recent advances in self-supervised learning for text or graphs, there lack of self-supervised learning models that can effectively capture both intra-item semantics and inter-item interactions for semi-structured sessions. To fill this gap, we propose CERES, a graph-based transformer model for semi-structured session data. CERES learns representations that capture both inter- and intra-item semantics with (1) a graph-conditioned masked language pretraining task that jointly learns from item text and item-item relations; and (2) a graph-conditioned transformer architecture that propagates inter-item contexts to item-level representations. We pretrained CERES using ~468 million Amazon sessions and find that CERES outperforms strong pretraining baselines by up to 9% in three session search and entity linking tasks.
