论文标题
Frankmocap：快速单眼3D手和身体运动捕获通过回归和整合
FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integration
论文作者
论文摘要
尽管人类运动的基本细微差别通常是作为身体运动和手势的组合传达的，但现有的单眼运动捕获方法主要集中在任何一个身体运动捕获上，仅忽略手部零件或仅在不考虑身体运动的情况下捕获手动运动。在本文中，我们提出了Frankmocap，这是一种运动捕获系统，可以通过更快的速度（9.5 fps）估算3D手和身体运动，并且比以前的工作更好。我们的方法是接近实时的（9.5 fps），并产生3D主体和手动捕获输出作为统一的参数模型结构。我们的方法旨在通过具有挑战性的野外视频来同时捕获3D身体和手动运动。要构建Frankmocap，我们通过采用整个身体参数模型（SMPL-X）的手部来构建最新的单眼3D“手”运动捕获方法。我们的3D手运动捕获输出可以有效地集成到单眼运动捕获输出中，从而产生整个身体运动导致统一的偏头模型结构。我们在公共基准测试中演示了手运动捕获系统的最新性能，并展示了我们整个身体运动捕获的高质量，导致各种挑战性的现实场景，包括现场演示场景。
Although the essential nuance of human motion is often conveyed as a combination of body movements and hand gestures, the existing monocular motion capture approaches mostly focus on either body motion capture only ignoring hand parts or hand motion capture only without considering body motion. In this paper, we present FrankMocap, a motion capture system that can estimate both 3D hand and body motion from in-the-wild monocular inputs with faster speed (9.5 fps) and better accuracy than previous work. Our method works in near real-time (9.5 fps) and produces 3D body and hand motion capture outputs as a unified parametric model structure. Our method aims to capture 3D body and hand motion simultaneously from challenging in-the-wild monocular videos. To construct FrankMocap, we build the state-of-the-art monocular 3D "hand" motion capture method by taking the hand part of the whole body parametric model (SMPL-X). Our 3D hand motion capture output can be efficiently integrated to monocular body motion capture output, producing whole body motion results in a unified parrametric model structure. We demonstrate the state-of-the-art performance of our hand motion capture system in public benchmarks, and show the high quality of our whole body motion capture result in various challenging real-world scenes, including a live demo scenario.
