A note on the evaluation of novel biomarkers: do not rely on integrated discrimination improvement and net reclassification index
Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt
Standard
A note on the evaluation of novel biomarkers : do not rely on integrated discrimination improvement and net reclassification index. / Hilden, Jørgen; Gerds, Thomas A.
I: Statistics in Medicine, Bind 33, Nr. 19, 30.08.2014, s. 3405-14.Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - A note on the evaluation of novel biomarkers
T2 - do not rely on integrated discrimination improvement and net reclassification index
AU - Hilden, Jørgen
AU - Gerds, Thomas A
N1 - Copyright © 2013 John Wiley & Sons, Ltd.
PY - 2014/8/30
Y1 - 2014/8/30
N2 - The 'integrated discrimination improvement' (IDI) and the 'net reclassification index' (NRI) are statistics proposed as measures of the incremental prognostic impact that a new biomarker will have when added to an existing prediction model for a binary outcome. By design, both measures were meant to be intuitively appropriate, and the IDI and NRI formulae do look intuitively plausible. Both have become increasingly popular. We shall argue, however, that their use is not always safe. If IDI and NRI are used to measure gain in prediction performance, then poorly calibrated models may appear advantageous, and in a simulation study, even the model that actually generates the data (and hence is the best possible model) can be improved on without adding measured information. We illustrate these shortcomings in actual cancer data as well as by Monte Carlo simulations. In these examples, we contrast IDI and NRI with the area under ROC and the Brier score. Unlike IDI and NRI, these traditional measures have the characteristic that prognostic performance cannot be accidentally or deliberately inflated.
AB - The 'integrated discrimination improvement' (IDI) and the 'net reclassification index' (NRI) are statistics proposed as measures of the incremental prognostic impact that a new biomarker will have when added to an existing prediction model for a binary outcome. By design, both measures were meant to be intuitively appropriate, and the IDI and NRI formulae do look intuitively plausible. Both have become increasingly popular. We shall argue, however, that their use is not always safe. If IDI and NRI are used to measure gain in prediction performance, then poorly calibrated models may appear advantageous, and in a simulation study, even the model that actually generates the data (and hence is the best possible model) can be improved on without adding measured information. We illustrate these shortcomings in actual cancer data as well as by Monte Carlo simulations. In these examples, we contrast IDI and NRI with the area under ROC and the Brier score. Unlike IDI and NRI, these traditional measures have the characteristic that prognostic performance cannot be accidentally or deliberately inflated.
U2 - 10.1002/sim.5804
DO - 10.1002/sim.5804
M3 - Journal article
C2 - 23553436
VL - 33
SP - 3405
EP - 3414
JO - Statistics in Medicine
JF - Statistics in Medicine
SN - 0277-6715
IS - 19
ER -
ID: 134781513