论文标题
胸部X光片和放射学报告的关节建模用于肺水肿评估
Joint Modeling of Chest Radiographs and Radiology Reports for Pulmonary Edema Assessment
论文作者
论文摘要
我们提出并展示了一种新型的机器学习算法，该算法评估了胸部X光片的肺水肿严重程度。尽管存在大量的胸部X光片和自由文本放射学报告的大量公开数据集，但只能从放射学报告中提取有限的数值水肿严重性标签。这是学习图像分类的此类模型的重大挑战。为了利用放射学报告中存在的丰富信息，我们开发了一种神经网络模型，该模型在图像和自由文本上都受过训练，以评估推理时胸部X光片的肺水肿严重程度。我们的实验结果表明，与仅在图像上训练的监督模型相比，联合图像文本表示学习可以提高肺水肿评估的性能。我们还展示了文本用联合模型解释图像分类的使用。据我们所知，我们的方法是第一个利用自由文本放射学报告来改善此应用程序中图像模型的性能。我们的代码可从https://github.com/rayruizhiliao/joint_chestxray获得。
We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports exist, only limited numerical edema severity labels can be extracted from radiology reports. This is a significant challenge in learning such models for image classification. To take advantage of the rich information present in the radiology reports, we develop a neural network model that is trained on both images and free-text to assess pulmonary edema severity from chest radiographs at inference time. Our experimental results suggest that the joint image-text representation learning improves the performance of pulmonary edema assessment compared to a supervised model trained on images only. We also show the use of the text for explaining the image classification by the joint model. To the best of our knowledge, our approach is the first to leverage free-text radiology reports for improving the image model performance in this application. Our code is available at https://github.com/RayRuizhiLiao/joint_chestxray.
