论文标题
暂时指导音乐到体现的一代
Temporally Guided Music-to-Body-Movement Generation
论文作者
论文摘要
本文提出了一种神经网络模型，以从音乐音频产生虚拟小提琴手的3-D骨架运动。从传统的复发性神经网络模型中改进了以前的作品中生成二维骨骼数据的改进，该模型结合了编码器decoder架构，以及自我发挥作用的机制，以模拟身体运动序列中复杂的动态。为了促进自我注意力模型的优化，使用BEAT跟踪来确定训练示例的有效尺寸和边界。解码器伴随着精炼的网络和鞠躬攻击推理机制，以强调右手行为和鞠躬攻击时机。客观和主观评估都表明，所提出的模型的表现优于最新方法。据我们所知，这项工作代表了考虑音乐身体运动中的关键特征的首次尝试产生3D小提琴家的身体运动。
This paper presents a neural network model to generate virtual violinist's 3-D skeleton movements from music audio. Improved from the conventional recurrent neural network models for generating 2-D skeleton data in previous works, the proposed model incorporates an encoder-decoder architecture, as well as the self-attention mechanism to model the complicated dynamics in body movement sequences. To facilitate the optimization of self-attention model, beat tracking is applied to determine effective sizes and boundaries of the training examples. The decoder is accompanied with a refining network and a bowing attack inference mechanism to emphasize the right-hand behavior and bowing attack timing. Both objective and subjective evaluations reveal that the proposed model outperforms the state-of-the-art methods. To the best of our knowledge, this work represents the first attempt to generate 3-D violinists' body movements considering key features in musical body movement.
