论文标题
基于模型的ROC（MROC）曲线：检查病例混合和模型校准对ROC图的影响
Model-based ROC (mROC) curve: examining the effect of case-mix and model calibration on the ROC plot
论文作者
论文摘要
风险预测模型的性能通常以歧视和校准来表征。接收器操作特性（ROC）曲线被广泛用于评估模型歧视。在评估新样本中风险预测模型的性能时，ROC曲线的形状受案例混合和假定模型的影响。此外，与歧视相比，评估校准尚未受到相同水平的关注。用于模型校准的常用方法涉及平滑或分组的主观规范。利用熟悉的ROC框架，我们引入了基于模型的ROC（MROC）曲线，以评估新样本中预指定模型的校准。 MROC曲线是ROC曲线，如果在样品中校准预先指定的模型，则应观察到。如果在该样品中校准模型，我们显示样品渐近收敛的经验ROC和MROC曲线。因此，MROC曲线可用于视觉评估病例混合和模型错误校准的效果。此外，我们提出了一个新的统计测试，以进行校准，该测试不需要任何平滑或分组。模拟支持测试的充分性。一个案例研究使这些事态发展置于实际情况下。我们得出的结论是，可以轻松地构建MROC并用于评估Case-Mix和模型校准对ROC图的影响，从而在评估风险预测模型的评估中增加了ROC曲线分析的实用性。提供了建议的方法的R代码（https://github.com/msadatsafavi/mroc/）。
The performance of risk prediction models is often characterized in terms of discrimination and calibration. The Receiver Operating Characteristic (ROC) curve is widely used for evaluating model discrimination. When evaluating the performance of a risk prediction model in a new sample, the shape of the ROC curve is affected by both case-mix and the postulated model. Further, compared to discrimination, evaluating calibration has not received the same level of attention. Commonly used methods for model calibration involve subjective specification of smoothing or grouping. Leveraging the familiar ROC framework, we introduce the model-based ROC (mROC) curve to assess the calibration of a pre-specified model in a new sample. mROC curve is the ROC curve that should be observed if a pre-specified model is calibrated in the sample. We show the empirical ROC and mROC curves for a sample converge asymptotically if the model is calibrated in that sample. As a consequence, the mROC curve can be used to assess visually the effect of case-mix and model mis-calibration. Further, we propose a novel statistical test for calibration that does not require any smoothing or grouping. Simulations support the adequacy of the test. A case study puts these developments in a practical context. We conclude that mROC can easily be constructed and used to evaluate the effect of case-mix and model calibration on the ROC plot, thus adding to the utility of ROC curve analysis in the evaluation of risk prediction models. R code for the proposed methodology is provided (https://github.com/msadatsafavi/mROC/).
