论文标题
通过深入学习的时间分割手术子任务，并使用多个数据源
Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources
论文作者
论文摘要
机器人辅助手术中的许多任务（RAS）可以由有限状态机器（FSM）表示，其中每个状态代表动作（例如拾起针）或观察（例如出血）。朝着此类手术任务自动化的关键步骤是对当前手术场景的时间感知，这需要对FSMS中的状态进行实时估计。这项工作的目的是根据执行的动作或事件随着任务的进行估算手术任务的当前状态。我们提出了Fusion-KVE，这是一个统一的手术状态估计模型，该模型结合了包括运动学，视觉和系统事件在内的多个数据源。此外，我们研究具有不同代表性特征或粒度水平的分割状态中不同状态估计模型的优势和劣势。我们评估了我们的模型关于JHU-ISI的手势和技能评估工作集（Jigsaws），以及使用DA Vinci XI Surgical System创建的涉及机器人内部超声（RIAS）成像的更复杂的数据集。我们的模型可实现卓越的状态估计精度高达89.4％，从而提高了拼图缝合数据集和我们的RIOS数据集中最先进的手术状态估计模型。
Many tasks in robot-assisted surgeries (RAS) can be represented by finite-state machines (FSMs), where each state represents either an action (such as picking up a needle) or an observation (such as bleeding). A crucial step towards the automation of such surgical tasks is the temporal perception of the current surgical scene, which requires a real-time estimation of the states in the FSMs. The objective of this work is to estimate the current state of the surgical task based on the actions performed or events occurred as the task progresses. We propose Fusion-KVE, a unified surgical state estimation model that incorporates multiple data sources including the Kinematics, Vision, and system Events. Additionally, we examine the strengths and weaknesses of different state estimation models in segmenting states with different representative features or levels of granularity. We evaluate our model on the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), as well as a more complex dataset involving robotic intra-operative ultrasound (RIOUS) imaging, created using the da Vinci Xi surgical system. Our model achieves a superior frame-wise state estimation accuracy up to 89.4%, which improves the state-of-the-art surgical state estimation models in both JIGSAWS suturing dataset and our RIOUS dataset.
