# 校准曲线

Reporting on calibration performance is recommended by the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) guidelines for prediction modeling studies

Clinical predictive model performance is commonly published based on discrimination measures, but use of models for individualized predictions requires adequate model calibration

in contrast to discrimination, which refers to the ability of a model to rank patients according to risk, calibration refers to the agreement between the estimated and the “true” risk of an outcome

When predictive models are built based on a population that differs from the population in which they will be used, blind application of these models could result in large “residuals” (ie, a large difference between a model’s estimate and the true outcome) because of factors that are difficult to account for.

Calibration plot is a visual tool to assess the agreement between predictions and observations in different percentiles (mostly deciles) of the predicted values.

The distance between the predicted outcome and actual outcome is central to quantify overall model performance from a statistical modeler’s perspective 32. The distance is Y −Ŷ for continuous outcomes. For binary outcomes, with Y defined 0 – 1, Ŷ is equal to the predicted probability p, and for survival outcomes it is the predicted event probability at a given time (or as a function of time)

Yingxiang Huang, Wentao Li, Fima Macheret, Rodney A Gabriel, Lucila Ohno-Machado, A tutorial on calibration measurements and calibration models for clinical prediction models, Journal of the American Medical Informatics Association, Volume 27, Issue 4, April 2020, Pages 621–633, https://doi.org/10.1093/jamia/ocz228

# DCA曲线原理介绍

In brief, decision curve analysis calculates a clinical “net benefit” for one or more prediction models or diagnostic tests in comparison to default strategies of treating all or no patients.

In the case of diagnosis, the income is true positives (e.g., finding a cancer) and the expenditure is false positives (e.g., unnecessary biopsies), with the “exchange rate” being the number of false positives that are worth one true positive. The exchange rate will depend on the relative seriousness of the intervention and outcome. For instance, we will be willing to conduct more unnecessary biopsies to find one cancer if the biopsy procedure is safe vs. dangerous or the cancer is aggressive vs. more indolent. The exchange rate is calculated, as explained above, from the threshold probability. Another analogy is with net health benefit or net monetary benefit, which both depend on the willingness to pay threshold in their exchange of benefits in terms of health and costs

threshold probabilities, defined as the minimum probability of disease at which further intervention would be warranted, as net benefit = sensitivity × prevalence – (1 – specificity) × (1 – prevalence) × w where w is the odds at the threshold probability

DCA曲线的横轴是阈概率，纵轴是净获益，可以看到阈概率在DCA曲线上展示的都是比较小的，刚刚给大家举的例子着实有点极端了。再回顾一下净获益，刚刚写了获益的意思是正确识别阳性，但是模型其实还有损失的嘛（就是错误的识别了阳性），因为对于阳性我们就得进行干预，但是干预有没有益处，得比较两个东西——就是真阳性的获益和假阳性的损失，两个做差就叫做净获益。

“intervention for all”这条线和“intervention for none”这条线还有一个交点，就是说在某个阈概率水平下，对于阳性病人采取全干预和全不干预的净获益都是一样的，就像刚刚给大家写的例子反过来：如果干预一个真阳性病人的获益是9，干预一个假阳性病人的损失是1，此时阈概率应该为1/（9+1）=0.1，就是说在这种情况下阈概率为0.1的时候两条线就会相交。

Vickers, A.J., van Calster, B. & Steyerberg, E.W. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res 3, 18 (2019). https://doi.org/10.1186/s41512-019-0064-7

# 如何读DCA曲线

Hence, we can conclude that, except for a small range of low preferences, intervening on (i.e., biopsying) patients on the basis of the prediction model leads to higher benefit than the alternative strategies of biopsying all patients, biopsying no patients, or only biopsy those patients who are positive on the diagnostic test. For the prostate biopsy study, the conclusion is that using the model to determine whether patients should have a biopsy would lead to improved clinical outcome.

# 小结

原文作者：Codewar
原文地址: https://www.cnblogs.com/Codewar/p/16376455.html
本文转自网络文章，转载此文章仅为分享知识，如有侵权，请联系博主进行删除。