论文标题
HSMF-NET：沉浸式触觉和点击式360度视频的语义视口预测
HSMF-Net: Semantic Viewport Prediction for Immersive Telepresence and On-Demand 360-degree Video
论文作者
论文摘要
在将偏远环境中现实的存在感介导给当地的人类用户时，会妨碍沉浸式触觉系统的接受。用户的自我运动和视觉反应之间的分歧引起了运动病的出现。视口或头部运动（HM）预测技术在补偿用户和远程站点之间明显延迟方面起着关键作用。我们提出了一个基于深度学习的视口预测范式，该范式将过去的HM轨迹与场景语义融合在一起，以晚融合方式。实际HM概况用于评估所提出的方法。获得的平均补偿率高达99.99％，显然优于最先进的薪酬。提出了一个按需360度视频流框架，以证明其一般有效性。所提出的方法提高了感知到的视频质量，同时需要显着降低传输速率。
The acceptance of immersive telepresence systems is impeded by the latency that is present when mediating the realistic feeling of presence in a remote environment to a local human user. A disagreement between the user's ego-motion and the visual response provokes the emergence of motion sickness. Viewport or head motion (HM) prediction techniques play a key role in compensating the noticeable delay between the user and the remote site. We present a deep learning-based viewport prediction paradigm that fuses past HM trajectories with scene semantics in a late-fusion manner. Real HM profiles are used to evaluate the proposed approach. A mean compensation rate as high as 99.99% is obtained, clearly outperforming the state-of-the-art. An on-demand 360-degree video streaming framework is presented to prove its general validity. The proposed approach increases the perceived video quality while requiring a significantly lower transmission rate.
