论文标题
行为：跟踪人类对象互动的数据集和方法
BEHAVE: Dataset and Method for Tracking Human Object Interactions
论文作者
论文摘要
建模人类与自然环境中对象之间的相互作用对于许多应用，包括游戏，虚拟和混合现实以及人类行为分析和人类机器人协作至关重要。这种具有挑战性的操作场景需要对大量对象，场景和人类行为进行概括。不幸的是，没有这样的数据集。此外，需要在不同的自然环境中获取这些数据，这排除了4D扫描仪和基于标记的捕获系统。我们展示了数据集，该数据集是具有多视图RGBD帧的第一个完整的人类对象交互数据集以及相应的3D SMPL和对象以及它们之间的带注释的触点。我们在5个位置记录了大约15K帧，其中8个受试者与20个常见对象进行了广泛的相互作用。我们使用这些数据来学习一个模型，该模型可以通过易于使用的便携式多相机设置在自然环境中共同跟踪人和物体。我们的主要见解是预测人类和对象到统计机构模型的对应关系，以在相互作用期间获得人类对象的接触。我们的方法不仅可以记录和跟踪人类和对象，还可以在3D中以表面接触为模型。我们的代码和数据可以在以下网址找到：http：//virtualhumans.mpi-inf.mpg.de/behave
Modelling interactions between humans and objects in natural environments is central to many applications including gaming, virtual and mixed reality, as well as human behavior analysis and human-robot collaboration. This challenging operation scenario requires generalization to vast number of objects, scenes, and human actions. Unfortunately, there exist no such dataset. Moreover, this data needs to be acquired in diverse natural environments, which rules out 4D scanners and marker based capture systems. We present BEHAVE dataset, the first full body human- object interaction dataset with multi-view RGBD frames and corresponding 3D SMPL and object fits along with the annotated contacts between them. We record around 15k frames at 5 locations with 8 subjects performing a wide range of interactions with 20 common objects. We use this data to learn a model that can jointly track humans and objects in natural environments with an easy-to-use portable multi-camera setup. Our key insight is to predict correspondences from the human and the object to a statistical body model to obtain human-object contacts during interactions. Our approach can record and track not just the humans and objects but also their interactions, modeled as surface contacts, in 3D. Our code and data can be found at: http://virtualhumans.mpi-inf.mpg.de/behave
