论文标题
从RGBD视频推断出明确的刚体动力学
Inferring Articulated Rigid Body Dynamics from RGBD Video
论文作者
论文摘要
能够重现从光相互作用到接触力学的物理现象，模拟器在越来越多的应用程序域变得越来越有用，而现实世界中的相互作用或很难获得标记的数据。尽管最近取得了进展，但仍需要大量的人力努力来配置模拟器以准确地再现现实世界的行为。我们引入了一条管道，将反向渲染与可区分的模拟相结合，以从深度或RGB视频中创建数字化的真实世界铰接机制。我们的方法会自动发现关节类型并估算其运动学参数，而整体机制的动态特性则可以调整以获得物理准确的模拟。正如我们在模拟系统上证明的那样，在我们派生的仿真传输中优化的控制策略成功地回到了原始系统。此外，我们的方法准确地重建了通过机器人操纵的铰接机制的运动学树，以及现实世界中耦合的摆机制的高度非线性动力学。 网站：https：//Eric-heiden.github.io/video2sim
Being able to reproduce physical phenomena ranging from light interaction to contact mechanics, simulators are becoming increasingly useful in more and more application domains where real-world interaction or labeled data are difficult to obtain. Despite recent progress, significant human effort is needed to configure simulators to accurately reproduce real-world behavior. We introduce a pipeline that combines inverse rendering with differentiable simulation to create digital twins of real-world articulated mechanisms from depth or RGB videos. Our approach automatically discovers joint types and estimates their kinematic parameters, while the dynamic properties of the overall mechanism are tuned to attain physically accurate simulations. Control policies optimized in our derived simulation transfer successfully back to the original system, as we demonstrate on a simulated system. Further, our approach accurately reconstructs the kinematic tree of an articulated mechanism being manipulated by a robot, and highly nonlinear dynamics of a real-world coupled pendulum mechanism. Website: https://eric-heiden.github.io/video2sim
