论文标题
素描 - 伯特：学习草图双向编码器代表来自变形金刚的素描的学习
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt
论文作者
论文摘要
先前对草图的研究经常以像素格式和杠杆化的CNN模型考虑草图中的草图。从根本上讲，草图被存储为一系列数据点，矢量格式表示，而不是像素的光真实图像。 Sketchrnn研究了长期记忆网络（LSTM）的矢量格式草图的生成神经表示。不幸的是，Sketchrnn学到的表示形式主要用于生成任务，而不是识别和检索草图的其他任务。为此，我们受到了最近的BERT模型的启发，我们提出了一种学习素描双向编码器表示的模型（Sketch-bert）。我们将BERT推广到草图域，新颖的组件和预训练算法，包括新设计的草图嵌入网络，以及对草图Gestalt的自欺欺人学习。特别是，在训练前任务中，我们提出了一种新颖的草图格斯塔尔特模型（SGM），以帮助培训素描 - bert。在实验上，我们证明了Sketch-Bert的学说表示可以帮助和改善草图识别，草图检索和草图Gestalt的下游任务的性能。
Previous researches of sketches often considered sketches in pixel format and leveraged CNN based models in the sketch understanding. Fundamentally, a sketch is stored as a sequence of data points, a vector format representation, rather than the photo-realistic image of pixels. SketchRNN studied a generative neural representation for sketches of vector format by Long Short Term Memory networks (LSTM). Unfortunately, the representation learned by SketchRNN is primarily for the generation tasks, rather than the other tasks of recognition and retrieval of sketches. To this end and inspired by the recent BERT model, we present a model of learning Sketch Bidirectional Encoder Representation from Transformer (Sketch-BERT). We generalize BERT to sketch domain, with the novel proposed components and pre-training algorithms, including the newly designed sketch embedding networks, and the self-supervised learning of sketch gestalt. Particularly, towards the pre-training task, we present a novel Sketch Gestalt Model (SGM) to help train the Sketch-BERT. Experimentally, we show that the learned representation of Sketch-BERT can help and improve the performance of the downstream tasks of sketch recognition, sketch retrieval, and sketch gestalt.
