• Qini curve
• impact
• policy
• population metrics
• risk model

• Algorithm fairness
• Clinical decision-making
• Equal opportunity
• High-risk identification
• Risk distribution

• R01 CA236468 NCI NIH HHS
• R01 HL138306 NHLBI NIH HHS
• R01 LM014401 NLM NIH HHS

• adult brain injury
• biomarkers
• epidemiology
• traumatic brain injury

• K23 NS110828 NINDS NIH HHS

• Calibration
• Discrimination
• Sample size
• Simulation

• Humans
• Sample Size
• Risk Assessment/methods
• Risk Assessment/statistics & numerical data
• Models, Statistical
• Computer Simulation
• Algorithms

• MC_UU_00004/09 Medical Research Council
• MC_UU_12023/29 Medical Research Council
• MR/P015190/1 Medical Research Council
• BRC-1215-20014 NIHR Cambridge Biomedical Research Centre

• Humans
• Breast Neoplasms
• Female
• Risk Assessment
• Prognosis
• Early Detection of Cancer
• Risk Factors

• Alzheimer’s disease
• benefit-risk
• clinical utility
• decision theory
• diagnostic yield table
• loss function
• relative net-benefit

• Male
• Humans
• Alzheimer Disease/diagnosis
• Cognitive Dysfunction/diagnosis
• Sensitivity and Specificity
• Stroke/diagnosis
• Diagnostic Tests, Routine
• Neuropsychological Tests

• Analytical goals
• Clustering
• Exploratory data analysis
• High-dimensional data
• Initial data analysis
• Multiple testing
• Omics data
• Prediction
• STRATOS initiative

• SA580/10-1 Deutsche Forschungsgemeinschaft
• BigInsight Norsk Institutt for Vannforskning
• project no 237718 Norsk Institutt for Vannforskning
• PRIN 2017 Italian Ministry of Education
• prot. 20178S4EK9_004 Italian Ministry of Education
• DFG Deutsche Forschungsgemeinschaft
• BO3139 Deutsche Forschungsgemeinschaft
• BMBF 01IS18036A Bundesministerium für Bildung und Forschung

• Canada
• Indigenous
• childhood maltreatment
• colonization
• sexual abuse

• n/a Canadian Psychological Association Criminal Justice Section

• Bayesian algorithm
• biomarker
• personalized medicine

• Humans
• Carcinoma, Non-Small-Cell Lung/drug therapy
• Carcinoma, Non-Small-Cell Lung/genetics
• Lung Neoplasms/drug therapy
• Lung Neoplasms/genetics
• Protein-Tyrosine Kinases/therapeutic use
• Bayes Theorem
• Proto-Oncogene Proteins
• Biomarkers/analysis
• Receptor Protein-Tyrosine Kinases/therapeutic use
• Molecular Targeted Therapy/methods

• Calibration
• Clinical usefulness
• Discrimination
• Impact assessment
• Model evaluation
• Model updating
• Prediction model
• Validation

• Acute Kidney Injury/blood
• Acute Kidney Injury/epidemiology
• Acute Kidney Injury/etiology
• Acute Kidney Injury/therapy
• Area Under Curve
• Cardiac Surgical Procedures/adverse effects
• Humans
• Models, Statistical
• Postoperative Complications/blood
• Postoperative Complications/epidemiology
• Postoperative Complications/etiology
• Predictive Value of Tests
• ROC Curve
• Renal Dialysis
• Retrospective Studies
• Risk Assessment/methods

• U01 DK129980 NIDDK NIH HHS

In research, policy, and practice, continuous variables are often categorized. Statisticians have generally advised against categorization for many reasons, such as loss of information and precision as well as distortion of estimated statistics. Here, a different kind of problem with categorization is considered: the idea that, for a given continuous variable, there is a unique set of cut points that is the objectively correct or best categorization. It is shown that this is unlikely to be the case because categorized variables typically exist in webs of statistical relationships with other variables. The choice of cut points for a categorized variable can influence the values of many statistics relating that variable to others. This essay explores the substantive trade‐offs that can arise between different possible cut points to categorize a continuous variable, making it difficult to say that any particular categorization is objectively best. Limitations of different approaches to selecting cut points are discussed. Contextual trade‐offs may often be an argument against categorization. At the very least, such trade‐offs mean that research inferences, or decisions about policy or practice, that involve categorized variables should be framed and acted upon with flexibility and humility.

In practical settings, the choice of cut points for categorizing a continuous variable is likely to entail trade‐offs across multiple statistical relationships between the categorized variable and other variables. These trade‐offs mean that no single categorization is objectively best or correct.

• data analysis
• statistical data interpretation
• statistics
• translational medical research
• translational medical science

• AUC
• BRCA1
• BRCA2
• Decision Curves
• Diagnostic Testing
• Incremental Net Benefit
• Net Benefit
• ROC
• Screening
• Youden’s index
• cost effectiveness
• decision analysis
• risk prediction

• U01 DK094157 NIDDK NIH HHS
• U01 DK094176 NIDDK NIH HHS
• Z99 CA999999 Intramural NIH HHS

Risk-prediction models for health outcomes are used in practice as part of clinical decision-making, and it is essential that their performance be externally validated. An important aspect in the design of a validation study is choosing an adequate sample size. In this paper, we investigate the sample size requirements for validation studies with binary outcomes to estimate measures of predictive performance (C-statistic for discrimination and calibration slope and calibration in the large). We aim for sufficient precision in the estimated measures. In addition, we investigate the sample size to achieve sufficient power to detect a difference from a target value. Under normality assumptions on the distribution of the linear predictor, we obtain simple estimators for sample size calculations based on the measures above. Simulation studies show that the estimators perform well for common values of the C-statistic and outcome prevalence when the linear predictor is marginally Normal. Their performance deteriorates only slightly when the normality assumptions are violated. We also propose estimators which do not require normality assumptions but require specification of the marginal distribution of the linear predictor and require the use of numerical integration. These estimators were also seen to perform very well under marginal normality. Our sample size equations require a specified standard error (SE) and the anticipated C-statistic and outcome prevalence. The sample size requirement varies according to the prognostic strength of the model, outcome prevalence, choice of the performance measure and study objective. For example, to achieve an SE < 0.025 for the C-statistic, 60–170 events are required if the true C-statistic and outcome prevalence are between 0.64–0.85 and 0.05–0.3, respectively. For the calibration slope and calibration in the large, achieving SE < 0.15 would require 40–280 and 50–100 events, respectively. Our estimators may also be used for survival outcomes when the proportion of censored observations is high.

• C-statistic
• Sample size calculation
• calibration
• discrimination
• prediction model

The Gail model is the most well-known tool for breast cancer risk assessment worldwide. Although it was validated in various Western populations, inconsistent results were reported from Asian populations. We used data from a large case-control study and evaluated the discriminatory accuracy of the Gail model for breast cancer risk assessment among the Iranian female population.

We used data from 942 breast cancer patients and 975 healthy controls at the Cancer Institute of Iran, Tehran, Iran, in 2016. We refitted the Gail model to our case-control data (the IR-Gail model). We compared the discriminatory power of the IR-Gail with the original Gail model, using ROC curve analyses and estimation of the area under the ROC curve (AUC).

Except for the history of biopsies that showed an extremely high relative risk (OR=9.1), the observed ORs were similar to the estimates observed in Gail’s study. Incidence rates of breast cancer were extremely lower in Iran than in the USA, leading to a lower average absolute risk among the Iranian population (2.78, ±SD 2.45). The AUC was significantly improved after refitting the model, but it remained modest (0.636 vs. 0.627, ΔAUC = 0.009, bootstrapped P=0.008). We reported that the cut-point of 1.67 suggested in the Gail study did not discriminate between breast cancer patients and controls among the Iranian female population.

Although the coefficients from the local study improved the discriminatory accuracy of the model, it remained modest. Cohort studies are warranted to evaluate the validity of the model for Iranian women.

• Breast neoplasms
• Logistic models
• Models, Statistical
• Risk assessment

Risk prediction models have been developed in many contexts to classify individuals according to a single outcome, such as risk of a disease. Emerging “-omic” biomarkers provide panels of features that can simultaneously predict multiple outcomes from a single biological sample, creating issues of multiplicity reminiscent of exploratory hypothesis testing. Here I propose definitions of some basic criteria for evaluating prediction models of multiple outcomes. I define calibration in the multivariate setting and then distinguish between outcome-wise and individual-wise prediction, and within the latter between joint and panel-wise prediction. I give examples such as screening and early detection in which different senses of prediction may be more appropriate. In each case I propose definitions of sensitivity, specificity, concordance, positive and negative predictive value and relative utility. I link the definitions through a multivariate probit model, showing that the accuracy of a multivariate prediction model can be summarised by its covariance with a liability vector. I illustrate the concepts on a biomarker panel for early detection of eight cancers, and on polygenic risk scores for six common diseases.

• Risk prediction
• biomarkers
• multiplicity
• multivariate analysis
• polygenic risk score
• screening

• Biomarkers
• Humans
• Neoplasms
• Predictive Value of Tests

• area under the curve
• cancer risk model
• model discrimination
• partial risk score
• two-phase studies

• Aged
• Female
• Humans
• Lung Neoplasms/epidemiology
• Male
• Middle Aged
• Risk Assessment
• Risk Factors

• Z99 CA999999 Intramural NIH HHS

• Adult
• Aged
• Aged, 80 and over
• Biomarkers, Tumor/blood
• Biomarkers, Tumor/genetics
• Case-Control Studies
• Female
• Genetic Predisposition to Disease
• Germ-Line Mutation
• Humans
• Male
• Middle Aged
• Models, Statistical
• Pancreatic Neoplasms/blood
• Pancreatic Neoplasms/epidemiology
• Pancreatic Neoplasms/genetics
• Polymorphism, Single Nucleotide
• Prospective Studies
• Risk Assessment/methods
• Risk Assessment/statistics & numerical data
• Risk Factors
• United States/epidemiology

• N01WH42129 WHI NIH HHS
• U01 CA167552 NCI NIH HHS
• N01WH42115 WHI NIH HHS
• N01 WH042116 WHI NIH HHS
• N01WH42131 NHLBI NIH HHS
• N01WH32115 WHI NIH HHS
• N01WH32106 WHI NIH HHS
• N01WH42119 WHI NIH HHS
• N01 WH42124 NIH HHS
• UM1 CA186107 NCI NIH HHS
• R01 HL026490 NHLBI NIH HHS
• N01WH42107 WHI NIH HHS
• N01WH32119 WHI NIH HHS
• N01WH32111 WHI NIH HHS
• N01WH32100 WHI NIH HHS
• N01WH42125 WHI NIH HHS
• N01WH42124 WHI NIH HHS
• N01 WH32115 WHI NIH HHS
• N01WH32109 WHI NIH HHS
• N01WH32122 WHI NIH HHS
• N01WH42120 WHI NIH HHS
• N01WH42111 NHLBI NIH HHS
• N01WH44221 WHI NIH HHS
• U01 CA210171 NCI NIH HHS
• N01WH42132 WHI NIH HHS
• N01WH24152 WHI NIH HHS
• P01 CA087969 NCI NIH HHS
• N01WH32102 WHI NIH HHS
• P50 CA062924 NCI NIH HHS
• N01WH42123 WHI NIH HHS
• N01 WH042108-011 WHI NIH HHS
• N01WH42109 WHI NIH HHS
• R01 HL034595 NHLBI NIH HHS
• N01 WH042112 WHI NIH HHS
• N01WH42130 NHLBI NIH HHS
• N01WH42121 WHI NIH HHS
• UM1 CA167552 NCI NIH HHS
• N01WH32113 WHI NIH HHS
• N01WH42118 WHI NIH HHS
• N01WH32101 WHI NIH HHS
• N01WH42113 WHI NIH HHS
• R01 CA049449 NCI NIH HHS
• R01 CA040360 NCI NIH HHS
• N01WH32118 NHLBI NIH HHS
• N01WH42126 NHLBI NIH HHS
• N01WH42117 WHI NIH HHS
• R01 CA097193 NCI NIH HHS
• N01WH32108 WHI NIH HHS
• R01 CA034944 NCI NIH HHS
• N01WH22110 WHI NIH HHS
• N01 WH42114 WHI NIH HHS
• N01WH42108 WHI NIH HHS
• N01WH32105 NHLBI NIH HHS
• N01WH42122 WHI NIH HHS
• N01WH42110 WHI NIH HHS
• N01WH32112 WHI NIH HHS

• Breast cancer
• IBIS model
• Tyrer–Cuzick model
• breast density
• calibration
• risk assessment

• N01 CN067009 NCI NIH HHS
• P01 CA154292 NCI NIH HHS
• HHSN261201300012I NCI NIH HHS
• N01PC35142 NCI NIH HHS
• R50 CA211115 NCI NIH HHS
• HHSN261201000029C NCI NIH HHS
• A16891 Cancer Research UK
• 16891 Cancer Research UK
• N01 CN005230 NCI NIH HHS
• HHSN261201100031C NCI NIH HHS

Frailty is associated with higher mortality in individuals at high cardiovascular disease (CVD) risk. We hypothesize that frailty is a more important prognostic factor than CVD risk factors and aim to determine the prognostic value of a cumulative deficit frailty index in patients with or at high risk for CVD.

We conducted an individual‐level pooled analysis of participants with or at risk for CVD, recruited in 14 multicenter clinical trials. The cumulative deficit index was calculated as the proportion of 26 deficits exhibited. Individuals were categorized as nonfrail, prefrail, or frail if they had indexes of ≤0.1, >0.1 to 0.21, or >0.21, respectively. CVD risk was assessed using the Framingham score. Outcomes included CVD event (new or recurrent myocardial infarction, stroke, or heart failure) and mortality. We studied 154 696 patients (mean age, 70.8 years; 63% men) with median follow‐up of 3.2 years. There were 17 535 CVD events and 15 067 deaths. The frail group (n=13 872) had higher risk of a CVD event (incidence rate ratio, 1.97; 95% CI, 1.85–2.08), all‐cause mortality (hazard ratio, 1.91; 95% CI, 1.79–2.03), and CVD mortality (hazard ratio, 1.91; 95% CI, 1.77–2.05) than the nonfrail group (n=101 343). Associations remained unchanged after adjusting for CVD risk factors. The index statistically outperformed the Framingham score in its ability to discriminate CVD events (C‐statistic, 0.60 [95% CI, 0.60–0.61] versus 0.58 [95% CI, 0.57–0.58], respectively; P<0.001).

In individuals with or at high risk of developing CVD, the cumulative deficit index is associated with increased CVD events and mortality, independent of CVD risk factors, and adds incremental prognostic value.

• cardiovascular outcomes
• deficit accumulation
• frailty
• mortality

• High-dimensional data
• U-statistics
• genetic risk prediction
• nicotine dependence

• Biomarkers/analysis
• Genetic Predisposition to Disease
• Genetic Variation
• Genome-Wide Association Study
• Genotype
• Humans
• Models, Genetic
• Models, Statistical
• Predictive Value of Tests
• Risk Assessment
• Tobacco Use Disorder/genetics

• K01 HL140333 NHLBI NIH HHS
• R01 DA043501 NIDA NIH HHS
• R01 LM012848 NLM NIH HHS
• R03 HD092854 NICHD NIH HHS

• Clinical risk prediction model
• Data science
• Decision support techniques
• Diagnosis
• Prognosis
• Risk
• Threshold

Dynamic risk models, which incorporate disease-free survival and repeated measurements over time, might yield more accurate predictions of future health status compared to static models. The objective of this study was to develop and apply a dynamic prediction model to estimate the risk of developing type 2 diabetes mellitus.

Both a static prediction model and a dynamic landmark model were used to provide predictions of a 2-year horizon time for diabetes-free survival, updated at 1, 2, and 3 years post-baseline i.e., predicting diabetes-free survival to 2 years and predicting diabetes-free survival to 3 years, 4 years, and 5 years post-baseline, given the patient already survived past 1 year, 2 years, and 3 years post-baseline, respectively. Prediction accuracy was evaluated at each time point using robust non-parametric procedures. Data from 2057 participants of the Diabetes Prevention Program (DPP) study (1027 in metformin arm, 1030 in placebo arm) were analyzed.

The dynamic landmark model demonstrated good prediction accuracy with area under curve (AUC) estimates ranging from 0.645 to 0.752 and Brier Score estimates ranging from 0.088 to 0.135. Relative to a static risk model, the dynamic landmark model did not significantly differ in terms of AUC but had significantly lower (i.e., better) Brier Score estimates for predictions at 1, 2, and 3 years (e.g. 0.167 versus 0.099; difference − 0.068 95% CI − 0.083 to − 0.053, at 3 years in placebo group) post-baseline.

Dynamic prediction models based on longitudinal, repeated risk factor measurements have the potential to improve the accuracy of future health status predictions.

The online version of this article (10.1186/s12874-019-0812-y) contains supplementary material, which is available to authorized users.

• Diabetes
• Prediction
• Statistical methods

• bias
• case-control study
• cost function
• cross-sectional data
• model assessment
• net benefit
• risk model

• Biometry/methods
• Calibration
• Decision Making
• Risk

• Z99 CA999999 Intramural NIH HHS
• ZIA CP010188 Intramural NIH HHS

• Cancer screening
• ROC analysis
• Risk assessment
• Stratified screening

• AUC
• BRCA1
• BRCA2
• ROC
• decision curve
• diagnostic testing
• mean risk stratification
• net benefit
• risk prediction

• Censoring
• Competing risks
• Model validation
• Prediction modeling
• Residual plot
• Survival analysis

• Algorithms
• Calibration
• Computer Simulation
• Data Analysis
• Dementia/diagnosis
• Dementia/mortality
• Dementia/therapy
• Humans
• Kidney Transplantation/methods
• Kidney Transplantation/mortality
• Models, Statistical
• Predictive Value of Tests
• Sensitivity and Specificity
• Survival Analysis

• censored event time
• positive predictive value
• precision-recall curve
• risk prediction
• screening
• time-dependent prediction accuracy

• Biometry
• Cancer Survivors
• Computer Simulation
• Heart Failure/complications
• Humans
• Predictive Value of Tests
• Risk Assessment/methods
• Risk Factors
• Statistics, Nonparametric
• Time Factors

• U24 CA055727 NCI NIH HHS
• CIHR

• AUC
• ROC curve
• decision analysis
• net benefit
• relative utility
• risk prediction

• Growth curve
• Statistical interaction

• GENERAL MEDICINE (see Internal Medicine)

• Acute Kidney Injury/diagnosis
• Acute Kidney Injury/mortality
• Acute Kidney Injury/therapy
• Adolescent
• Adult
• Aged
• Aged, 80 and over
• Area Under Curve
• Creatinine/blood
• Decision Support Techniques
• Emergency Service, Hospital/statistics & numerical data
• Female
• Humans
• Kidney Function Tests
• Length of Stay/statistics & numerical data
• Logistic Models
• Male
• Middle Aged
• Predictive Value of Tests
• ROC Curve
• Retrospective Studies
• Risk Assessment/methods
• Risk Factors
• Severity of Illness Index
• Time Factors
• United Kingdom
• Young Adult

• Association
• Causal Effect
• Copula
• Cross-ratio
• Dependence
• Diagnostics

• R01 CA129102 NCI NIH HHS

• Clinical utility
• cost/benefit ratio
• decision theory
• diagnostic yield
• relative net benefit
• risk threshold
• weighted accuracy

• benefit-risk
• diagnostic yield
• diagnostics
• pragmatism

• MATS-1
• Static-99
• prediction
• risk assessment
• sex offenders

• Adult
• Age Factors
• Aged
• Criminals
• Humans
• Middle Aged
• Recurrence
• Risk Assessment
• Secondary Prevention
• Sex Offenses/prevention & control
• Surveys and Questionnaires

• cardiovascular diseases
• diabetes mellitus, type 1
• models, cardiovascular
• risk factors
• survival analysis

• Asymmetric link
• Generalized t-distribution
• Macrosomia
• Prediction
• Skewed multivariate random effects model
• Ultrasound measurement

• Breast/physiopathology
• Breast Neoplasms/diagnosis
• Breast Neoplasms/diagnostic imaging
• Breast Neoplasms/physiopathology
• Early Detection of Cancer/methods
• Female
• Humans
• Mammography/methods
• Patient Advocacy
• Radiographic Image Interpretation, Computer-Assisted/methods

• Decision analysis
• predictive marker
• principal stratification
• prognostic marker
• randomized trial
• relative utility curve
• subgroup
• treatment selection marker

• Algorithms
• Biomarkers/analysis
• Data Interpretation, Statistical
• Forecasting
• Humans
• Outcome Assessment, Health Care/methods

• Z99 CA999999 Intramural NIH HHS