论文标题
实时学生跟踪数字木偶的单眼视频
Real-time Pupil Tracking from Monocular Video for Digital Puppetry
论文作者
论文摘要
我们提出了一种简单的实时方法，用于从移动设备上实时视频进行学生跟踪。我们的方法扩展了具有两个新组件的最先进的面部网格探测器：一个微型神经网络，可预测学生在2D中的位置，并基于基于位移的瞳孔混合形状系数估计。我们的技术可用于准确控制虚拟木偶的学生运动，并为其提供活力和能量。所提出的方法在现代手机上的运行量超过50 fps，并在任何实时木偶管道中都可以使用。
We present a simple, real-time approach for pupil tracking from live video on mobile devices. Our method extends a state-of-the-art face mesh detector with two new components: a tiny neural network that predicts positions of the pupils in 2D, and a displacement-based estimation of the pupil blend shape coefficients. Our technique can be used to accurately control the pupil movements of a virtual puppet, and lends liveliness and energy to it. The proposed approach runs at over 50 FPS on modern phones, and enables its usage in any real-time puppeteering pipeline.
