Deze pagina lezen in het Nederlands.

Author: dr. M.D.J. Wolvers
Co-author: Jeroen Hoogland en Nan van Geloven
Authorship

This page was translated from the Dutch-language original using AI and has not been fully reviewed. Where the two differ, the Dutch version takes precedence.

• What is multivariable regression?
• What can multivariable regression be used for?
  • Description
  • Prediction
  • Explanation
    • Experimental research
      • Multivariable regression for efficiency
    • Observational research
      • Multivariable regression to correct for confounding
      • Table 2 fallacy
  • Note: explanation is often the actual purpose!
• Frequently asked questions
  • What is overfitting and why is it a problem?
  • How do I choose which variables to include in my multivariable model?
  • I used backwards selection - how do I interpret my results?
  • What is the best way to select my variables, through manual selection or an automatic procedure?
  • How do I report the results of my multivariable model?
  • How do I ensure a clear and user-friendly presentation of my multivariable model?
  • How do I use univariable regression to select factors for a multivariable regression?
  • How do I choose which variables to include in my model?
  • There is overlap between two of our predictors - can they be included together in a multivariable model?
  • How should I correct for baseline values?
• References
  • Additional resources
• Further reading

Multivariable regression is a form of regression in which several variables are simultaneously related to an outcome measure. These variables are also referred to as covariates, predictors, independent variables, or factors. Most multivariable regression models in the clinical literature belong to the family of generalized linear models. Besides linear regression, this also includes logistic and multinomial logistic regression. In addition, Cox regression is a widely used form of multivariable regression. This page focuses on the general principles of regression with multiple covariates and assumes a dataset with independent observations.

A statistical analysis can broadly serve three purposes [1,2,3]: description, prediction or explanation. Each purpose requires a different use of multivariable regression. Below, common applications and pitfalls of multivariable regression are discussed for each purpose.

The first purpose is description: using data to provide a quantitative summary of certain characteristics of the world [4]. No claims or suggestions are made about the cause of the difference or association found.

An example of descriptive analyses is Table 1 in many medical scientific articles, which presents demographic and clinical characteristics of the sample. Sometimes the research question itself is also purely descriptive, for example: how often does <this specific comorbidity> occur in <this specific population>.

However, multivariable regression very rarely has a place in descriptive medical research. This is because a purely associative analysis does not require correction for confounding [4].

Multivariable methods that you could consider for descriptive research purposes are principal component analysis or cluster analysis [5]. These techniques are useful when you are interested in the relationships within a group of variables, without an outcome measure being defined.

The second purpose is prediction; also referred to as predictive or prognostic research. The goal here is to predict an outcome as accurately as possible, for example to develop a model for predicting 5-year survival following a diagnosis of breast cancer [6].

Usually such an outcome lies in the future, but sometimes, as is often the case in diagnostic research, this outcome is measured at (approximately) the same time. This happens, for example, when the outcome is too invasive, expensive, or impractical to measure on a large scale [5].

Multivariable regression is a useful and widely used tool in this type of research. In this form of research, predictors are chosen based on expertise, but it can be useful to use statistical techniques to select predictors or competing models. More on that below.

Contrary to what is often thought, a statistical model does not need to resemble 'the real world' very closely [1] in order to produce accurate predictions. Moreover, the individual parameters of the model have no meaningful interpretation (see also: Table 2 fallacy); in predictive research these are subordinate to the extent to which the final model is able to generate accurate predictions for (new) cases.

For a valuable, clinically usable prediction model, building and reporting it is therefore not enough on its own. You also want to analyze, at minimum, how well the model is able to predict new cases. This process is called validation.

The book Clinical Prediction Models by Steyerberg, or the methodological series mentioned above, are useful starting points if you are interested in conducting prognostic research. The TRIPOD guideline is also relevant to mention in this context.

The third and final research purpose is explanation. In a more statistical context, the terms causal inference or counterfactual prediction are also used for this. This purpose can be pursued in very different ways.

The most commonly used form of explanatory research is experimental research, with the randomized controlled trial (rct) as the guiding example. In a perfectly conducted RCT, where randomization allows you to plausibly attribute any differences solely to the allocation/intervention, the use of multivariable analysis is not necessary to obtain a valid result.

In experimental research, however, it can indeed be useful or necessary to use multivariable regression. For example, when stratified randomization has been used, it is desirable to include the stratification factor as a covariate in the primary analysis EMA guideline on covariate adjustment. The selection of covariates then follows, in principle, from the study design. In that case there is forced entry of these covariates. When covariates are used for efficiency, data-driven or automatic selection of covariates or competing models is strongly discouraged.

It is not always ethical or feasible to answer causal questions using the gold standard, RCTs. This can also be achieved with observational research, albeit with limitations, namely a much larger set of (unverifiable) assumptions. A textbook example of such a question is "does smoking cause lung cancer?" In observational research, the research question and/or the estimand is often not clearly formulated [3]. Luijken et al. [7] discuss, using pharmacological research as an example, why formulating an estimand can be difficult but is essential.

To arrive at a concrete research question and estimand, it can be useful to formulate a hypothetical target trial [8,9]. From there, it is often possible to reason about which factors - given the practical and ethical restrictions - pose a relevant obstacle to reliably estimating the effect/estimand of interest.

A multivariable regression model is a tool that can be used to correct for confounding and selection bias. There are several ways in which this can be achieved, such as

1. including a sufficient adjustment set as covariates in a multivariable regression model [10].
2. or, in multiple steps, using inverse propensity score weighting (IPSW) or inverse probability of treatment weights (IPTW). In this approach, a multivariable regression model is used to assign a weight to observations (usually: patients) in the final analysis [11]. In both scenarios, an approach is preferable in which, in advance, a statistical analysis plan has established and justified for each research question how each variable is used to produce as relevant an estimate of the estimand as possible. It is advisable to use directed acyclic graphs (DAGs) when determining the set of confounders. This introduction in R and this free online course are potentially useful starting points.

Another point of attention when using multivariable models in this context is the so-called Table 2 fallacy [12]: presenting the estimated regression coefficients of a multivariable model with the implication that these are practical, clinically interpretable quantities. It may be the case that (part of) the modeled associations happen, purely by chance, to be adequately corrected for confounding and are therefore quantifications of (relevant) causal effects, but this cannot simply be expected: in principle, none of the coefficients have a causal interpretation, with the sole exception of the coefficient belonging to a carefully considered causal estimand.

Research questions that are actually about cause-and-effect relationships are regularly presented as a descriptive or predictive research question by avoiding causal terms [4]. This happens in observational research, but also in post-hoc analyses or secondary research questions of experimental research. Consider phrasings such as: "what are the most important predictors of ..." ([3]), while in principle the search is for factors that (given their causal effect on a relevant outcome measure) are worth intervening on. By avoiding causal terminology, sight is lost of what is actually needed to achieve the (underlying, causal) goal. This makes it difficult to evaluate the methods and conclusions [4].

Overfitting is a phenomenon in which noise in the training data is modeled and thus attributed to the covariates. As a result, the model loses generalizability. This happens when too many parameters are estimated relative to the number of data points used to calculate them. The estimated betas are larger than in the population, and the p-values and standard errors are underestimated. This effect worsens when stepwise selection methods are used. Babyak gives a very readable explanation of this problem.

Overfitting is a potential problem for all applications of multivariable regression, especially in the case of biomedical/clinical data, which is often sparse. This should therefore be taken into account in the design of the study.

For an explanatory research question, the selection of covariates is a methodological issue that must already be resolved at the design stage. The alternative, data-driven selection, could after all cause (part of) the - hopefully carefully selected [10] - minimal sufficient adjustment set to be selected out, substantially changing (and biasing) the interpretation of the association found. Overfitting can then only be prevented by taking this into account when determining the sample size.

In predictive research, data-driven selection of variables is indeed an acceptable way to prevent overfitting. A more modern data-driven selection of predictors in the case of predictive research is regularization [13,14]. In this approach, more complex models are numerically penalized when fitting the model, which results in a more useful, better generalizable model. But note: data-driven selection and inference (i.e., using or interpreting standard errors and p-values) generally do not go well together [15]. Moreover, the typical application of "traditional" stepwise regression methods has numerous problems of its own [15,16].

I have 90 patients for whom I collected baseline parameters (9 lab parameters and 13 clinical parameters). Using a non-parametric test, I found that 1 lab parameter and several clinical parameters differ between the two outcome groups (=diagnosis yes/no after 2 years). Now I wanted to see which parameters are the best predictors of outcome. I performed backward logistic regression with the parameters that had previously shown a significant difference, but I'm not sure whether this is the best choice. Perhaps it should be done differently, or more parameters should be included, or parameters should be combined?

Your question deserves some sharpening; roughly speaking, it comes down to the following trade-off: Do you want to build a model that allows you to predict as accurately as possible, in new patients, whether they will receive a diagnosis after 2 years, or do you want to examine, for each of the measured baseline parameters, whether it explains whether people have a diagnosis after 2 years?

I am conducting research into which variables influence a tumor biopsy outcome (diagnosis yes or no). For this, I took a number of parameters (e.g., tumor size, tumor location, the degree to which the tumor protrudes, etc.) that I expect to have an effect. I ran a backward multiple regression on this and do get results, but I'm not quite sure how to interpret them.

If I understand correctly, you are using observational data to answer a series of explanatory research questions. Data-driven selection is in principle not suited to this; in that case, you want to determine, yourself, for each of the parameters under investigation, what its minimal sufficient adjustment set looks like.

There are various methods for performing a multivariable regression. I want to use a backwards selection strategy. What is advisable? A manual procedure with enter selection, where you manually remove the variable with the highest p-value until a set of variables remains that are all significant, or an automatic backward procedure?

That depends entirely on the purpose you have in mind. Keep in mind that data-driven selection methods generally do not go well together with inference, in other words that after such a procedure you can do little with p-values and standard errors.

For a questionnaire study on quality of life, I performed linear regression with forward selection in the analysis. In the final model there are variables with a non-significant p-value. If you make a table of the variables in the final model, would you sort them by the size of the beta, by significance of the p-value, or by the amount of variance explained? Or would you simply take over the SPSS output?

P-values and standard errors are - bluntly put - worthless after such an automatic selection procedure [15]. If you still want to report the betas: there are no fixed rules for the order in which you place variables in a table. There could, for example, be a hierarchy among the variables, or a chronological order (for instance, age and sex first, and only later the influence of smoking or similar). It is standard practice, however, to show all variables of your final model (including the non-significant ones), so that a reader can see alongside which other predictors the variables were assessed.

This article by Sullivan et al. [17] may help you with that. It shows how you can translate the results of your regression model into risk scores.

We are analyzing the differences in outcome (success rate) between two operations. We first tested a number of factors individually for their effect on success rate. We then carried the significant factors forward into a multivariable logistic regression. Is this a good methodology?

With this kind of 'univariable preselection', it is methodologically preferable not only to carry forward variables that are significant in the univariable analysis, but to use a more lenient p-value threshold (e.g., variables with a univariable p<0.20 or even p<0.50 (Steyerberg, according to Babyak)).

I have a data file with a great many variables. I am looking for as simple a predictive model as possible (and therefore easy to apply). How do I choose which variables to include in my model?

Steyerberg has written a very useful and open-access guide for building predictive models.

We are looking for predictive factors for the occurrence of hepatitis A. We tested several factors univariately (age, sex, comorbidity, etc.). In the univariate analysis, we found that "male" vs. "female" was positively associated with the outcome measure and that "comorbidity" was negatively associated (tested with a chi-square test). The group "without comorbidity" contains many men, and the group "with comorbidity" contains many women, so there is some overlap between the groups. Is it possible to include both predictive factors in a multivariable logistic regression, or should we first test the degree of overlap, since otherwise the model might become unstable?

Your research question may need some sharpening. See this section. The following answer assumes that the aim is to build a prognostic model for the occurrence of hepatitis A.

If the overlap between two predictors is high, then, as you say, the model will indeed become unstable. This is called the problem of collinearity. When the model becomes unstable, you will see this in very large standard errors of the regression coefficients.

We are conducting research into predictors of physical functioning after total hip replacement. Our outcome measure of physical functioning, HOOS (Hip disability and Osteoarthritis Outcome Score), expresses function on a scale of 0-100, measured at baseline and after 2 years. The following regressions were performed with age as predictor and HOOS as dependent variable.

• 1: outcome HOOS absolute outcome at 2 years, predictors age + baseline HOOS, p-value for age: 0.503
• 2: outcome HOOS difference score between 2 years and baseline, predictors age + baseline HOOS, p-value for age: 0.503
• 3. outcome HOOS difference score between 2 years and baseline, predictor age (uncorrected), p-value for age: 0.048

As you can see, there is a difference in significance depending on the correction for baseline. Which outcome should I use?

Your research question may need some sharpening. See this section.

• The use of the term predictors seems to indicate that the goal is to predict physical functioning after total hip replacement as accurately as possible.
• The term 'correction', however, and the interpretation of p-values, points to an 'explanatory' research question. The answer below is therefore based on this latter interpretation of your research goal.

A likely explanation for your results is that the correlation between age and HOOS score at baseline causes the apparent significance in model 3. In general, it is advisable to include the baseline score as a covariate in the model; see also the EMA Guideline on adjustment for baseline covariates, section 4.2.6, which also states the expected agreement between models 1 and 2.

1. Shmueli, G. (2010). To explain or to predict? Statistical Science, 25(3), 289–310. DOI:10.1214/10-STS330
2. Miguel A. Hernán, John Hsu & Brian Healy (2019) A Second Chance to Get Causal Inference Right: A Classification of Data Science Tasks, CHANCE, 32:1, 42-49, DOI:10.1080/09332480.2019.1579578
3. Carlin, J. B., & Moreno-Betancur, M. (2024). On the uses and abuses of regression models: A call for reform of statistical practice and teaching. arXiv. DOI:10.48550/arXiv.2309.06668
4. Hernán MA. The C-Word: Scientific Euphemisms Do Not Improve Causal Inference From Observational Data. Am J Public Health. 2018 May;108(5):616-619. DOI: 10.2105/AJPH.2018.304337 | PMID: 29565659
5. Hamaker EL, Mulder JD, and van IJzendoorn MH. Description, prediction and causation: Methodological challenges of studying child and adolescent development. Dev Cogn Neurosci. 2020 Dec;46:100867. DOI: 10.1016/j.dcn.2020.100867 | PMID: 33186867
6. Example from: Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., & Altman, D. G. (2013). Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research. PLOS Medicine, 10(2), e1001381. DOI:10.1371/journal.pmed.1001381
7. Luijken K, van Eekelen R, Gardarsdottir H, Groenwold RHH, and van Geloven N. Tell me what you want, what you really really want: Estimands in observational pharmacoepidemiologic comparative effectiveness and safety studies. Pharmacoepidemiol Drug Saf. 2023 Aug;32(8):863-872. DOI: 10.1002/pds.5620 | PMID: 36946319
8. Hernán MA and Robins JM. Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available. Am J Epidemiol. 2016 Apr 15;183(8):758-64. DOI: 10.1093/aje/kwv254 | PMID: 26994063
9. Goetghebeur, E., le Cessie, S., De Stavola, B., Moodie, E. E. M., & Waernbaum, I. (2020). Formulating causal questions and principled statistical answers. Statistics in Medicine, 39(30), 4922–4948. DOI:10.1002/sim.8741
10. Greenland S, Pearl J, and Robins JM. Causal diagrams for epidemiologic research. Epidemiology. 1999 Jan;10(1):37-48. PMID: 9888278
11. Chesnaye NC, Stel VS, Tripepi G, Dekker FW, Fu EL, Zoccali C, and Jager KJ. An introduction to inverse probability of treatment weighting in observational research. Clin Kidney J. 2022 Jan;15(1):14-20. DOI: 10.1093/ckj/sfab158 | PMID: 35035932
12. Daniel Westreich, Sander Greenland, The Table 2 Fallacy: Presenting and Interpreting Confounder and Modifier Coefficients, American Journal of Epidemiology, Volume 177, Issue 4, 15 February 2013, Pages 292–298, DOI:10.1093/aje/kws412
13. McNeish, D. (2015). Using Lasso for Predictor Selection and to Assuage Overfitting: A Method Long Overlooked in Behavioral Sciences. Multivariate Behavioral Research, 50(5), 471–484. DOI:10.1080/00273171.2015.1036965
14. Friedrich S, Groll A, Ickstadt K, Kneib T, Pauly M, Rahnenführer J, and Friede T. Regularization approaches in clinical biostatistics: A review of methods and their applications. Stat Methods Med Res. 2023 Feb;32(2):425-440. DOI: 10.1177/09622802221133557 | PMID: 36384320
15. Babyak MA. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med. 2004 May-Jun;66(3):411-21. DOI: 10.1097/01.psy.0000127692.23278.a9 | PMID: 15184705
16. Livingston E, Cao J, Dimick JB. Tread Carefully With Stepwise Regression. Arch Surg. 2010;145(11):1039–1040. doi:10.1001/archsurg.2010.240
17. Sullivan LM, Massaro JM, and D'Agostino RB Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat Med. 2004 May 30;23(10):1631-60. DOI: 10.1002/sim.1742 | PMID: 15122742
18. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996 Dec;49(12):1373-9.
19. Regression Modeling Strategies with Applications to Linear Models, Logistic Regression and Survival Analysis. Frank E. Harrell, Jun, Springer-Verlag, New York, 2001. No. of pages: 568. ISBN 0-387-95232-2.

• Guideline on adjustment for baseline covariates in clinical trials
• Guideline on the investigation of subgroups in confirmatory clinical trials
• Everything You Always Wanted to Know About Evaluating Prediction Models (But Were Too Afraid to Ask) Andrew J. Vickers and Angel M. Cronin, Urology Volume 76, Issue 6, December 2010, Pages 1298-1301.

The models discussed on this page assume that the observations are independent, meaning that the measurements for each observational unit (usually: patient) do not correlate with measurements of the other observational units in the sample. Extensions of multivariable regression are available for data that does not meet this condition. The pages repeated measures or mixed effects models may help you further in that case.

---

Finished reading? You can go to the Overview of all topics, or to Choosing a test for help picking a suitable test or analysis. Want to learn more about biostatistics? Then follow the AMC e-learning Practical Biostatistics. Did you find something on these pages that isn't correct? Does a link not work? Or would you like to contribute to the wiki? Then get in touch with us.

This biostatistics wiki is an initiative of the former statistics helpdesk of Amsterdam UMC, location AMC. Amsterdam UMC staff can request statistical support via intranet.