论文标题
视觉和IMU的移动机器人手工武器远程操作系统
A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU
论文作者
论文摘要
在本文中，我们提出了一个多模式移动遥控系统，该系统由基于新型的基于视觉的手姿势回归网络（Transteleop）和一种基于IMU的ARM跟踪方法组成。 Transteleop通过低成本深度摄像头观察人的手，不仅生成关节角度，还可以通过图像到图像的翻译过程生成配对机器人手摆姿势的深度图像。基于按键的重建损失探讨了人与机器人手之间的外观和解剖结构的相似之处，并丰富了重建图像的局部特征。可穿戴的相机支架可以同时进行手工控制，并促进整个远程操作系统的活动能力。测试数据集和各种复杂的操作任务上的网络评估结果超出了简单的选择操作，这表明了我们多模式遥控系统的效率和稳定性。
In this paper, we present a multimodal mobile teleoperation system that consists of a novel vision-based hand pose regression network (Transteleop) and an IMU-based arm tracking method. Transteleop observes the human hand through a low-cost depth camera and generates not only joint angles but also depth images of paired robot hand poses through an image-to-image translation process. A keypoint-based reconstruction loss explores the resemblance in appearance and anatomy between human and robotic hands and enriches the local features of reconstructed images. A wearable camera holder enables simultaneous hand-arm control and facilitates the mobility of the whole teleoperation system. Network evaluation results on a test dataset and a variety of complex manipulation tasks that go beyond simple pick-and-place operations show the efficiency and stability of our multimodal teleoperation system.
