Decision Curve Analysis

Decision curve analysis graph

Decision curve analysis is a simple method for evaluating prediction models, diagnostic tests, and molecular markers.

The method was first published as:

Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making. 2006 Nov-Dec;26(6):565-74.

A subsequent discussion paper gives some further details about the method:

Steyerberg EW, Vickers AJ. Decision curve analysis: a discussion. Medical Decision Making. 2008 Jan-Feb;28(1):146-9.

A paper describing various extensions to decision curve analysis, such as application to survival time data, is:

Vickers AJ, Cronin AM, Elkin EB, Gonen M. BMC Medical Informatics and Decision Making. 2008 Nov 26;8(1):53. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers.

For more on the theoretical background of decision curve analysis, see:

Vickers AJ. Decision analysis for the evaluation of diagnostic tests, prediction models and molecular markers. Am Stat. 2008;62(4):314-320.

Baker SG, Cook NR, Vickers A, Kramer BS. Using relative utility curves to evaluate risk prediction. J R Stat Soc Ser A Stat Soc. 2009 Oct 1;172(4):729-748.

Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010 Jan;21(1):128-38.

Vickers AJ, Cronin AM. Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol. 2010 Feb;37(1):31-8.

Two papers that extend decision curve analysis to issues of treatment response are:

Vickers AJ, Kattan MW, Daniel S. Method for evaluating prediction models that apply the results of randomized trials to individual patients. Trials. 2007;8:14

Vickers AJ, Kramer BS, Baker SG. Selecting patients for randomized trials: a systematic approach based on risk group. Trials. 2006;7:30.

Editorials recommending decision curve analysis have been published in several major journals.

Kerr KF, Brown MD, Zhu K, Janes H. Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use. J Clin Oncol. 2016 Jul 20;34(21):2534-40. doi: 10.1200/JCO.2015.65.5654.

Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016 Jan 25;352:i6. doi: 10.1136/bmj.i6.

Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. 2015 Jan 27;313(4):409-10. doi: 10.1001/jama.2015.37.

Localio AR, Goodman S. Beyond the usual prediction accuracy metrics: reporting results for clinical decision making. Ann Intern Med. 2012 Aug 21;157(4):294-5. doi: 10.7326/0003-4819-157-4-201208210-00014.

Statistical Code for Running Decision Curve Analysis

Download Stata Code

Download R code

Download SAS Code

Tutorials

A tutorial for running decision curve analysis using Stata, R and SAS takes the reader step by step through doing a basic decision curve analysis, including formatting the graph, interpreting the results and saving the output (download PDF ). Also included are instructions on: showing net reduction in interventions; evaluation of joint or conditional tests; evaluation of published models; application to censored data, including competing risk.

The zip file contains the data sets, Stata code, and R code used in the tutorial (download Decision Curve Analysis tutorial.zip). The example data sets used are provided as text, CSV, Stata, and R format.

Project Members