Deze pagina lezen in het Nederlands.

Author: dr. ir. N van Geloven
Authorship

This page was translated from the Dutch-language original using AI and has not been fully reviewed. Where the two differ, the Dutch version takes precedence.

• How do I interpret a different outcome in a univariable and multivariable model?
• How do I choose which variables to include in my multivariable model?
• I used backward selection - how do I interpret my results?
• What is the best way to select my variables, via manual selection or with an automated procedure?
• How do I report the results of my multivariable model?
• How do I ensure a clear and user-friendly presentation of my multivariable model?
• How do I use univariable regression to select factors for a multivariable regression?
• When is it necessary to include age and sex as covariates in my analyses?
• How do I choose which variables to include in my model?
• There is overlap between two of our predictors — are they allowed to be included together in a multivariable model?
• How should I adjust for baseline values?
• References
• Additional resources

In a multivariable regression, multiple predictors are related to the outcome simultaneously. A multivariable regression model is often used to adjust for possible confounders.

For one of my studies I performed a logistic regression analysis. My outcome is a dichotomous variable: 1 is improvement, 0 is no improvement. I then want to know whether age and certain genetic abnormalities have an influence on that outcome or not. If I add only age to my model, it comes out as very, very significant. If I then also add the genetic variables, the significance of age as a predictor disappears (p=0.054) and only the genetic variables are significant predictors. My question now is, how should I interpret this?

If a variable in your multivariable model is not significant, it means that it does not have a significant association with the outcome, taking into account the effect of the other variables that are currently in your model (i.e., conditional on those variables). In your case, it appears that the effect of age, adjusted for the other covariates, does not have a significant effect (at a significance level of 5%) on the outcome. This can be influenced by two things. Either the other covariates absorb the effect of age, or there is not enough data to identify the effect of age as significant. Even if the conditional association (adjusted, in a multivariable model) is just as large as the unconditional one (univariable model), the precision of the conditional model is often still lower (wider confidence intervals), because multiple associations are being estimated within one model, leaving less power to detect effects. By looking not only at the p-value but also at the confidence interval, you become somewhat less dependent on the hard (but arbitrary) 5% cutoff.

I have 90 patients for whom I collected baseline parameters (9 lab parameters and 13 clinical parameters). Using a non-parametric test I found that 1 lab parameter and several clinical parameters differ between the two outcome groups (= diagnosis yes/no after 2 years). Now I wanted to look at which parameters are the best predictors of outcome. I performed backward logistic regression with the parameters that had previously shown a significant difference, but I am not sure whether this is the best choice. Perhaps it should be done differently, or more parameters should be included, or parameters should be combined?

In your case, where you have 90 patients and want to test the effect of 21 (9+13) variables, you do not have enough patients to test all these variables in 1 multivariable model (which would of course be ideal). A choice therefore has to be made as to which variables will be included in a multivariable model. There is no consensus on the best way to select variables, but the choice is always made based on two criteria:

• 1. Which variables do I expect to have an effect / which variables do I consider relevant?
• 2. For which variables does the data indicate that an effect might be present (which ones does the model identify as significant)? What you have done is a kind of univariable preselection of variables, where you first test the variables individually and include those that come out significant in a multivariable model. Although methodological drawbacks can be cited for this approach, it is a widely used one. With this, you are trying to answer question 2. However, you describe that you carried out this preselection based on a non-parametric test, and thus not based on the same (logistic?) model as the one you used for the multivariable analysis. Using the same model would make the analysis more consistent. Furthermore, it is important not to be too strict with rejection during the preselection. A p-value of 0.10 or 0.20 could be used for this purpose. Your final conclusions will be based on the multivariable model, where you can apply the usual strict p-value. Also take another critical look at the variables in light of question 1. Finally, combining variables is also a good way to reduce the number of variables to be tested.

I am working on a study into which variables influence a tumor biopsy outcome (diagnosis yes or no). For this I have taken a number of parameters (e.g. tumor size, location of the tumor, the extent to which the tumor protrudes, etc.) that I expect to have an influence. I ran a backward multiple regression on these and do get results from it, but I'm not entirely sure how to interpret them.

The output shows which factors in the final model are associated with the outcome. In the output you see the p-value and a regression coefficient B with standard error (SE (B)). This coefficient indicates how large the effect is. If you stepwise remove variables from your model that do not contribute significantly to it (as you have done), you end up with the variables that are significantly associated with your outcome, while adjusting for the other variables in your model. If you wanted to go a step further, you could also choose to do a forward selection and see whether the same variables are then associated with your outcome.

There are various methods for performing a multivariable regression. I want to use a backward selection strategy. What is recommended? A manual procedure with enter selection, where you manually remove the variable with the highest p-value until a set of variables remains that are all significant, or an automatic backward procedure?

I recommend the manual way, because it keeps you in control of the steps yourself, lets you get to know your data, and allows you to intervene where needed. In general you don't want important selections to be made by a 'machine' — you want to be able to notice where variables considered highly relevant are dropped, and also where there might be collinearity, etc.

The only reason to go for the automatic option is that, after excluding a variable, the set of previously dropped predictors is often re-offered one by one to the smaller model to see whether they now do contribute significantly. Of course you can also do this manually, but that is quite a lot of work. A good combination is to do it manually first and then run the automatic procedure afterwards as a check.

For a questionnaire study regarding quality of life, I performed linear regression with forward selection in the analysis. In the final model there are variables with a non-significant p-value. If you make a table of the variables in the final model, would you sort them by size of the beta, by significance of the p-value, or by amount of explained variance? Or would you simply take over the SPSS output?

There are no fixed rules for the order in which you place variables in a table. There could, for example, be a hierarchy among the variables, or a chronological order (for example age and sex first, and only later the influence of smoking or similar). It is, however, standard practice to show all the variables from your final model (including the non-significant ones), so that a reader can see alongside which other predictors the variables were evaluated.

This article by Sullivan et al. [1] might help you with this. It shows how you can translate the results of your regression model into risk scores.

We are analyzing differences in outcome (success rate) between two operations. We first tested a number of factors individually for their influence on success rate. We then carried the significant factors forward into a multivariable logistic regression. Is this a good methodology?

For this kind of 'univariable preselection', it is methodologically preferable not to carry forward only the variables that are significant in the univariable analysis, but to use a more lenient p-value threshold for this (e.g. variables with univariable p<0.20 or p<0.30).

Among other things, I am performing an independent sample t-test to examine the difference in a dependent variable (continuous) between two groups, and Pearson Chi-square analyses to examine the difference in a dependent variable (dichotomous) between two groups.

I did check whether the two groups differ significantly in terms of age and sex. This is not the case, and for this reason I did not include age and sex in my further analyses. Is this sufficiently justified, or should you also look at the relationship between sex/age and the dependent variable?

The principles for inclusion are as follows:

1. If there is a difference in age/sex between the two groups & age and sex do have an influence on the outcome, then you must adjust for it, because otherwise it can lead to bias.
2. If there is a difference in age/sex between the two groups & age and sex have no influence on the outcome, then including the variables will not lead to different results, and you might just as well not do it.
3. If there is no difference in age/sex between the two groups & age and sex have no influence on the outcome, then including the variables will not lead to different results, and you might just as well not do it.
4. If there is no difference in age/sex between the two groups & age and sex do have an influence on the outcome, then including the variables will not lead to different estimates for the group variable, but a stronger model can be obtained by taking the extra factors into account. The significance level of the group variable can, however, change as a result, so you would preferably still want to include the variables in the model.

Whether the variables differ across the group variable is best examined in your own data (this need not be a significant difference, so a difference can occur fairly quickly; in the case of a randomized group allocation you can usually assume there is no difference). Whether there is reason to assume that age and sex influence the dependent variable(s) is best judged based on (external) knowledge. If you have good reasons to assume that this is the case, then you could include them in a multivariable model regardless. It could indeed be that this influence is present, but that you simply do not happen to find it in your own data.

A prerequisite for including extra covariates is that you have enough data to include such an extra term in the model (in terms of power). If you do not have enough room in the data, you would be less 'keen' to include the variables age and sex. You could then do a 'preselection' and look in your own data to see whether there is reason to include them. For example by again looking at the influence in your own data:

1. check whether they differ across the 2 groups (as you have done)
2. check whether they have a (univariable) influence on the outcome. See also the previous question about preselection.

I have a dataset with a large number of variables. I am looking for as simple as possible a predictive model (and therefore easy to apply). How do I choose which variables to include in my model?

There are many different ways to build a multivariable model. Here we give an example of a simple procedure. This procedure is applicable to various forms of regression analysis.

• Step 1. Perform a series of regression analyses, including one explanatory variable at a time. For each analysis, note the p-value of the explanatory variable in the regression analysis.
• Step 2. Take all variables that had a p-value smaller than 0.3 in step 1. Perform a regression analysis with all these variables together.
• Step 3. Remove the variable from step 2 with the largest p-value and rerun the regression analysis.
• Step 4. Repeat step 3 until all variables remaining in the model have a p-value smaller than 0.05.
• Step 5. Once all variables have a p-value smaller than 0.05, you have your final model. You can describe this model further in your manuscript.

We are looking for predictive factors for the occurrence of hepatitis A. We tested several factors univariably (age, sex, comorbidity, etc.). In the univariable analysis we found that "male" vs. "female" was positively associated with the outcome, and that "comorbidity" was negatively associated (tested with a Chi-square test). The group "without comorbidity" contains many men, and the group "with comorbidity" contains many women, so there is some overlap between the groups. Is it possible to include both predictive factors in a multivariable logistic regression, or should we first test the degree of overlap, for example, because otherwise the model would become unstable?

The degree of overlap between the two predictors indeed should not be too high, otherwise, as you say, the model will become unstable. This is called the problem of collinearity. It is a good idea to look at this first, although there are no hard cutoffs for when the overlap is too large. One rule of thumb sometimes used is that the correlation coefficient between the two predictors should not be higher than 0.8. When the model becomes unstable, you will see this from very large standard errors of the regression coefficients. In SPSS you can also conveniently use the 'Collinearity Diagnostics' option, found under 'Statistics' in the linear regression procedure. You then proceed as if you want to perform a linear regression, request these diagnostics, and then only look at that part of the output to assess whether the predictors do not have too much overlap.

We are conducting a study into the predictors of physical functioning after total hip replacement. Our outcome measure of physical functioning, HOOS (Hip disability and Osteoarthritis Outcome Score), expresses function on a scale of 0-100, measured at baseline and after 2 years. The following regressions, with age as predictor and HOOS as the dependent variable, were performed.

• 1: outcome HOOS absolute value at 2 years, predictors age + baseline HOOS, p-value for age: 0.503
• 2: outcome HOOS change score between 2 years and baseline, predictors age + baseline HOOS, p-value for age: 0.503
• 3. outcome HOOS change score between 2 years and baseline, predictor age (unadjusted), p-value for age: 0.048

As you can see, there is a difference in significance depending on the adjustment for baseline. Which outcome should I use?

A likely explanation for your results is that the correlation between age and the HOOS score at baseline is causing the apparent significance in model 3. In general it is advisable to include the baseline score as a covariate in the model; see also the EMA Guideline on adjustment for baseline covariates (draft April 2013), paragraph 4.2.6, which also mentions the expected agreement between model 1 and 2.

1. Sullivan LM, Massaro JM, and D'Agostino RB Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat Med. 2004 May 30;23(10):1631-60. DOI: 10.1002/sim.1742 | PMID: 15122742
2. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996 Dec;49(12):1373-9.
3. Regression Modeling Strategies with Applications to Linear Models, Logistic Regression and Survival Analysis. Frank E. Harrell, Jun, Springer-Verlag, New York, 2001. No. of pages: 568. ISBN 0-387-95232-2.

• Guideline on adjustment for baseline covariates in clinical trials

• Guideline on the investigation of subgroups in confirmatory clinical trials

• What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models, Babyak M.A., Psychosom Med. 2004 May-Jun;66(3):411-21

• Everything You Always Wanted to Know About Evaluating Prediction Models (But Were Too Afraid to Ask) Andrew J. Vickers and Angel M. Cronin, Urology Volume 76, Issue 6, December 2010, Pages 1298-1301.

---

Finished reading? You can go to the Overview of all topics, or to Choosing a test for help picking a suitable test or analysis. Want to learn more about biostatistics? Then follow the AMC e-learning Practical Biostatistics. Did you find something on these pages that isn't correct? Does a link not work? Or would you like to contribute to the wiki? Then get in touch with us.

This biostatistics wiki is an initiative of the former statistics helpdesk of Amsterdam UMC, location AMC. Amsterdam UMC staff can request statistical support via intranet.