Logistic regression EN - pdmoerland/wiki-biostatistiek-bilingual GitHub Wiki

Logistic regression

Deze pagina lezen in het Nederlands.

Author: prof dr R.J. de Haan
Co-author: prof dr J.W.R. Twisk
Authorship

This page was translated from the Dutch-language original using AI and has not been fully reviewed. Where the two differ, the Dutch version takes precedence.

Contents

When do I use a logistic regression model?

With logistic regression you can relate a dichotomous outcome variable (dead versus alive, symptoms present or absent, etc.) to one or more predictors. The basic idea behind logistic regression is that you transform the outcome variable in such a way that a form of linear regression becomes possible. The logistic regression coefficients of the independent variables in the model can then be converted into odds ratios.

Unlike linear regression, logistic regression does not require many statistical conditions to be met. For example, no assumptions need to be made about the distribution of the outcome variable. The predictors (or explanatory variables) can be either discrete or continuous (just as with linear regression). However, the observations must be independent of each other. This means that logistic regression is not suitable for analyzing research data in which patients have been measured repeatedly over time (other techniques exist for repeated measures).

How does logistic regression work?

In a logistic regression model it is not the dichotomous outcome itself that is modeled, but the probability of that outcome. Because a probability ranges from a minimum of 0 to a maximum of 1, and in linear regression the outcome variable must be continuous, we cannot use the probability directly as the outcome variable. We can, however, make use of the relative probability: the odds. After all, the odds can be regarded as a continuous variable ranging from 0 to infinity. Statistically, one further step is still needed. The odds range from 0 to infinity; values below 0 are not possible. The outcome variable used is therefore not the odds itself, but the natural logarithm of the odds. This ln(odds), also called the logit, can take on any value between minus infinity and infinity. In the logistic regression equation

$\ln(odds)=a+b_1X_1+b_2X_2+\cdots+b_kX_k$

the natural logarithm of the odds is the outcome variable, $a$ is the constant, $X_i, (i = 1,2, .., k)$ are the explanatory variables or covariates, and $b_1, b_2, \ldots b_k$ form the logistic regression coefficients.

How do I interpret my SPSS output when using a simple (univariable) logistic model?

(The example below is taken from the book "Inleiding in de toegepaste biostatistiek" by J.W.R Twisk, Elsevier gezondheidszorg, Maarssen, 2007).

In a case-control study, the relationship between smoking and having a myocardial infarction was investigated in 100 patients and 100 control subjects. The results are shown in the following 2x2 table.

patient control total
smoker 39 14 53
non-smoker 61 86 147
total 100 100 200

The odds ratio (OR) for smoking compared to not smoking with respect to having a myocardial infarction can be calculated as follows:

$OR = \frac{a/b}{c/d}=\frac{a*d}{b*c} = \frac{39*86}{14*61} = 3.93.$

The 95% confidence interval ranges from 1.70 to 7.86. There is a significant association, because the value corresponding to the null hypothesis (no association: OR = 1) lies outside the confidence interval.

We will now repeat this analysis, but this time using logistic regression. If, in the screens of the Binary Logistic module in SPSS, we enter the variable 'type of subject' as the dependent variable (0 = control, 1 = patient) and the variable 'smoking' (0 = no, 1 = yes) as the independent variable, then SPSS will, among other output, generate the final table "Variables in the Equation".

Variables in the Equation
  B S.E. Wald df Sig. Exp(B) 95.0% C.I.for Exp(B)
Lower Upper
Step 1 smoking 1.368 .354 14.960 1 .000 3.927 1.694 7.855
Constant -.343 .167 4.210 1 .040 .709    

The logistic regression coefficient for smoking has the value 1.368. The Wald test indicates that there is a significant association (p < 0.001) between smoking and having an infarction. Because the logistic regression coefficient equals the natural logarithm of the OR, we can take the exponent of the regression coefficient (exp(B)), which effectively 'removes' the natural logarithm and leaves the OR. The OR for having a myocardial infarction for smokers compared to non-smokers is thus $e^{1.368}$ = 3.927 (95% CI: 1.694 - 7.855).

How do I interpret my SPSS output when using a multiple logistic model?

In this example, we will use multiple (or multivariable) logistic regression to include several independent variables (covariates) in the model at the same time. Suppose that in a prospective cohort study of 115 elderly patients admitted to hospital, we want to predict the probability of delirium occurring (0 = no, 1 = yes) during their admission. We do this using the following patient characteristics: cognitive functioning prior to admission (0 = no impairment, 1 = impairment), ADL level prior to admission (0 = not limited, 1 = limited), and age (continuous variable). Note that the estimated OR for a continuous variable, in this case age, must be interpreted as the OR for developing delirium associated with a one-unit difference in age (here expressed in years). Suppose, for example, that the variable 'age' has an OR of 1.028 (as in the example); this means that the odds of developing delirium for a person who is one year older are 1.028 times as high as the odds for a person who is one year younger.

SPSS shows the following results in the final table "Variables in the Equation":

Variables in the Equation
  B S.E. Wald df Sig. Exp(B) 95.0% C.I.for Exp(B)
Lower Upper
Step 1 Cognition 1.800 .607 8.794 1 .003 6.047 1.841 19.863
ADL 1.237 .508 5.928 1 .015 3.444 1.273 9.321
Age .028 .032 .761 1 .383 1.028 .966 1.094
Constant -5.034 2.527 3.970 1 .046 .007    

Cognitive functioning (p = 0.003) and ADL level (p = 0.015) prove to be independent predictors of delirium occurring during hospital admission. Cognition shows an OR of 6.047. In other words, adjusted for ADL level and age, the odds of developing delirium in patients with cognitive impairment are about 6 times as high as the odds for patients without cognitive impairment. Taking the effects of cognition and age into account, patients with functional limitations also appear to have a higher probability of developing delirium (OR = 3.444). Adjusted for the two indicators of functional health (cognition and ADL), however, the patient's age does not appear to be significantly associated with developing delirium (OR = 1.028; p = 0.383).

How do I interpret the output of a continuous predictor in a logistic model?

One of the risk factors in our model is the total duration of anesthesia in minutes. In the model this gives a B value of 0.005 and an odds ratio of 1.005. This gives a p-value that is highly significant (0.001), but I don't quite understand how to interpret that odds ratio. What I think is that it represents the increase in the probability of postoperative nausea per minute of anesthesia duration, but I'm not entirely sure.

Your per-minute interpretation is correct. To get a more meaningful OR, you could first divide the duration variable in the dataset by, for example, 10, and then run the model. The OR is then given per 10-minute increase in anesthesia duration. As long as you do not round the values, nothing else about the model changes.

How do I interpret the output of a categorical predictor in a logistic model?

I am running a logistic regression with psychological complaints (yes/no) as the outcome and HIV as a predictor; this is a predictor with 3 options, namely HIV 1 = positive, 2 = negative, 3 = unknown (no HIV test performed). I find it difficult to understand what the odds ratio means in this case. Do people with HIV, people without HIV, or people who don't know their status have a higher risk of psychological complaints? Below is the relevant output.

Variables in the Equation
  B S.E. Wald df Sig. Exp(B) 95.0% C.I.for Exp(B)
Lower Upper
Step 1 HIV 1.591 2 .451
HIV(1) 0.361 0.425 0.722 1 .396 1.435 0.624 3.300
HIV(2) 0.087 0.431 0.041 1 .840 1.091 0.469 2.539
Constant -0.375 0.392 0.915 1 .339 .688    

To interpret the odds ratio for categorical variables, it is important to know which coding SPSS uses. SPSS chooses a reference category for the categorical variables. It tells you which one in the table 'Categorical Variables Codings':

Categorical Variables Codings
Frequency Parameter coding
(1) (2)
HIV Positive 147 1.000 .000
Negative 126 .000 1.000
unknown 27 .000 .000

The category that has only 0s in its row is the reference category. In your case this is 'unknown'. The other categories are numbered by the '1' in a particular column. For example, HIV-Positive is coded with a (1) and HIV-Negative with a (2). Further on in the output tables, you will encounter this coding again, in the 'variables in the equation' table.

Here you can see that the HIV-positive group (HIV(1)) has 1.4 times higher odds of complaints than the people with unknown HIV status. Likewise, you see that the HIV-negative group has 1.1 times higher odds of complaints compared to people with unknown HIV status. To obtain robust odds ratios (with a small confidence interval), it is advisable to use a reference category of reasonable size. The 'unknown' category is in fact the smallest, so it is not a very useful reference category. The reference category can be chosen in the 'Categorical' screen (choice of 'first' or 'last' — don't forget to click 'change'). By coding the outcome categories differently, you can make any desired category the 'first' or 'last'.

Incidentally, the variable HIV does not appear to contribute significantly in this model (the overall test across all three categories gives p=0.451), so you cannot speak with statistical certainty about a higher or lower probability of complaints here.

How do I determine whether to include age in the model as a continuous predictor or divided into categories?

For age, I was unsure whether to leave it as a continuous value or divide it into (4) age categories. Can you help me with this?

Statistically, it is cleaner to include age as a continuous predictor and assume a linear relationship, because then you only have to estimate 1 parameter, leaving more room in the model for estimating other effects. That single parameter represents the extra risk per year, and you then assume that the effect of one extra year does not depend on the starting age. For example, age 21 compared to age 20 then gives the same amount of 'extra risk' as age 76 compared to age 75. To check whether that assumption is reasonable for your data, you could first analyze the data divided into (4, or possibly more — you do need a reasonable number of people per category) categories. You can then see whether the risk estimate indeed indeed increases 'linearly'. For example:

category OR
20-30 ref
30-40 1.10
40-50 1.21
50-60 1.46

increases in a clearly linear fashion (every additional 10 years the OR is multiplied by 1.1), whereas a result such as

category OR
20-30 ref
30-40 1.20
40-50 0.63
50-60 1.01

does not increase linearly at all: 30-40 gives more risk than 20-30, and 40-50 gives less risk than 20-30. In this second example, including age as a continuous predictor does not fit well.

Should I include my ordinal predictor as a categorical variable in the logistic model?

I am currently running a logistic regression analysis. My predictors are ordinal variables (score 0, 1, 3 or 5). Should I enter these into the logistic regression model as categorical variables or not?

If you do not enter the variable as a categorical variable, a linear increasing or decreasing effect is assumed. That means that 1 odds ratio is estimated, indicating how much extra probability of the outcome a 1-point increase in the score gives. A predictor score of 3 versus a score of 1 then gives the same extra probability of the outcome as a predictor score of 5 versus 3. A score of 1 versus 0 gives half of that extra probability (all measured on the log-odds scale). If you do enter the predictor as categorical, then 3 odds ratios are estimated. The model can then freely estimate all kinds of effects. For example, it could turn out that categories 0 and 5 give a high probability of the outcome and categories 1 and 3 a low one. The direction of the effect therefore does not need to be consistently increasing or decreasing. Entering it as categorical imposes fewer restrictions and is therefore advisable. However, estimating all those separate effects requires more parameters (3 instead of 1), which could lead to less efficient estimates (less power). One piece of advice could be to first look at the results using a categorical predictor. If the odds ratios found increase or decrease consistently, you could consider entering the predictor as numeric after all, provided this is also consistent with your hypothesis.

What do the regression coefficients in a logistic model mean?

I am investigating the influence of the patient's age on whether or not an examination is performed. dependent = examination yes/no (dichotomous) independent = age (numeric) I used logistic regression here, and at first it looked very interesting: Exp(B) is 0.951. I thought this meant that with every additional year of age you had a 5% lower probability of getting the examination. But later I saw that both the a (-0.05) and the b (-0.0556) were negative. Now I can no longer interpret it — would that mean that for every age the probability of getting the examination is less than 0? Have I overlooked something?

The first interpretation is the correct one. An OR of 0.951 does indeed mean that with every additional year of age, the probability (strictly speaking, the odds) of getting the examination decreases by ~5%. The a is usually not important. Within the model it represents the intercept (i.e. the expected value at an age of 0). The a is expressed on the logit scale. The proportion corresponding to an a of -0.05 can be calculated from this as follows:

$\ln(odds)= \ln{\frac{p}{1-p}} = -0.05$

$p = \frac{e^{-0.05}}{1+e^{-0.05}} = 0.49.$

So you can see that for children aged 0, an estimated proportion of 0.49 is predicted for examination = yes. Often a patient with covariate value 0 is not present within the study population, so this intercept has less meaning, but it is still good to check whether this value looks reasonable.

The b is also expressed in the logit context. A negative b means that the probability of the examination decreases with increasing age. The b is directly linked to the OR.

$OR = e^{b} = e^{-0.0556} = 0.95$

So keep in mind that the regression coefficients (a and b) cannot be interpreted in the same way as in linear regression.

Can I obtain adjusted proportions from my logistic model?

For the follow-up evaluation of the outcomes of an RCT (intervention versus usual care), we also want to perform an analysis in which we correct for baseline differences. For a dichotomous variable (neurologically abnormal or not), we made a univariable crosstab with a chi-square test. We then did this with a logistic regression, and after that a multivariable logistic regression including the covariates. The question now is how we should present the data. Normally we would give the frequencies (24% versus 14% abnormal), but the logistic regression gives an odds ratio. Would it then make more sense to present two odds ratios — one from the univariable and one from the multivariable log model? Or can we, similarly to adjusted means from an ANCOVA, also obtain an adjusted frequency? And if so, how?

Indeed, univariable and multivariable odds ratios are usually presented. However, you can also calculate adjusted proportions. To do this, it is customary to calculate the predicted frequencies for a patient who has the mean value for all the adjustment variables. Using the logistic regression formula, you fill in the mean value for the adjustment variables and, for the variable of interest (intervention versus usual care), fill in a 0 once and a 1 once. For example, if the model is:

ln(odds_neurological abnormality) = intercept + B1 * intervention + B2 * age_at_baseline,

and the mean baseline age of all patients was 63, then for the 'usual care group':

ln(odds_neurological abnormality}) = intercept + B2 * 63,

and for the intervention group:

ln(odds_neurological abnormality) = intercept + B1 * 1 + B2 * 63.

You read off the intercept, B1 and B2 from the coefficient table of the logistic regression.

The 'log odds' values found this way can be converted back to proportions, because ln(odds) = ln(P/1-P).

So if you want to recover P from the 'log odds', you can use the inverse transformation (worked out in detail here):

P = exp(ln(odds))/(1+exp(ln(odds))), where exp stands for 'e to the power of'.

P is the 'probability of the event', but can also be interpreted as the (predicted) proportion of events. The predicted proportion of events in the two groups, for patients with all other covariates at their mean, is a good representation of the 'adjusted' effect.

My logistic regression does not work because of a 0 in my 2x2 table, what now?

I have a binary outcome measure (detection or no detection) for which I have been using logistic regression. However, I now run into a predictive factor (also binary) that confuses me. I made a 2x2 table to calculate an odds ratio (as a check, because the logistic regression does not work), but I can't get any further.

Detection No detection Total
Factor present 17 3 20
Factor absent 7 0 7
Total 24 3 27

So the problem is that 0. How can I still perform a logistic regression despite this?

Calculating an odds ratio does not work because of the 0. The odds ratio for no detection here is (3/17)/(0/7) = infinity (dividing by zero is nonsensical). If you are only doing a univariable analysis (which seems to be the most that can be achieved with these numbers), a simple trick to still calculate an odds ratio is to add 0.5 to each of the 4 cells. You then get (3.5/17.5)/(0.5/7.5) = 3. To calculate a corresponding p-value, you could, for example, use a Fisher's exact test.

There are also more statistically advanced solutions for this. These are particularly useful if you want to perform a multivariable analysis that also needs to include this factor. You can then make use of so-called 'penalized-likelihood logistic regression'. During estimation, this ensures that the OR remains 'finite' by applying a 'penalty' to large ORs. Such methods have not yet been implemented in SPSS. They are available, for example, in the logistf package for R. In your case, a univariable penalized regression would give an OR of 3.000008.

I get a very large odds ratio from my logistic regression, what could be wrong?

I performed a simple (univariable) logistic regression analysis with prognosis as the outcome measure (not severe n=26 or severe n=14), and the variable is a continuous measure. Result: B 9.970, SE 5.722, Wald 3.035, df1, Sign 0.081, Exp(B) 21371.871, 95%CI 0.288 to 1587561235. However, I don't understand how the OR can be so extremely high, and online I read that this is often related to bias. Could you explain to me what kind of bias this concerns and how I can interpret these results? Am I allowed to use this analysis?

The first thing I would check in such a case is the order of magnitude of the continuous predictor. What values can this predictor take? If it is something expressed in small units (for example, 0.0012 or similar), this can lead to such large odds ratios. For interpreting the OR, it is helpful to express the continuous predictor in numbers within a normal range, for example between 0 and 100. To achieve this, you can express the measurements in, for example, 'per mille' (= thousandths). You can do this by multiplying all the values of the predictor by 1000. The example value of 0.0012 then becomes 1.2. The interpretation of the OR then becomes 'how much do the odds of the event increase when the predictor increases by 1 thousandth'. Something similar can of course also be done with other scaling factors, such as 10^4 (0.0012 then becomes 12) or 10^5 (0.0012 then becomes 120), etc.

I would also advise you to look at the distribution of the predictor. Strictly speaking, the predictor does not need to be normally distributed. However, the logistic model does make the assumption that a 1-unit increase in the predictor corresponds to a fixed increase in the (log) odds of the event (a linear relationship between the predictor and the log odds). If the predictor is very skewed, this linearity assumption is probably not plausible. A transformation of the predictor can produce a more symmetrically distributed predictor that may better fit the assumption.

I want to perform a logistic regression on a dataset with matched patients, how do I do that in SPSS?

When the data consist of matched pairs, for example cases and controls matched on age and sex, an ordinary logistic regression is not sufficient. To properly account for the paired nature of the data, a conditional logistic regression is needed. SPSS does not have a direct function for this, but the analysis can be done using a trick via the Cox procedure. This Youtube video by Robin Beaumont explains it very clearly.

The effect of a skewed predictor is different in a logistic model than in a Mann-Whitney test, what now?

I would like to perform a multivariable logistic regression for the outcome postoperative complications yes/no. One of the variables is 'number of days admitted to hospital before surgery'. It is known from the literature that patients who are admitted for longer often do worse postoperatively. This variable is continuous but skewed. First I looked at it univariably — I had first done a Mann-Whitney U test to see whether there was a difference between the group with and the group without complications; this gave P=0.019. Then, in SPSS, I did a univariable analysis via the logistic regression menus, and suddenly I got P=0.275. The P often does differ somewhat, but this time I find it very different. Is it because the variable is skewed, and the logistic regression does not take this into account?

The differences between the two analyses could indeed be caused by the skewness of the variable. In the logistic regression it is assumed that every additional day of admission gives a fixed increase in risk (on the log-odds scale) of postoperative complications. In the Mann-Whitney analysis this is not done, and the only assumption is that those with complications have, on average, a higher rank number when all admission durations are ranked from low to high.

To check whether the 'every day gives an equal amount of extra risk' assumption in the logistic model indeed explains the difference between the two analyses, you could do the following: create a new variable containing the rank numbers of the admission duration (see, for example, this instruction). Enter this rank variable into a logistic regression. If this 'rank regression' does agree in conclusion with the Mann-Whitney test, that is reason to think that the 'every day gives an equal amount of extra risk' assumption is not correct. Note that this analysis is only for checking purposes — it is not a cure-all, because the resulting OR gives an 'extra risk per rank number', and such information cannot be generalized to other populations/patients. If you would still like to do a multivariable logistic regression, you could consider a transformation of the length-of-stay variable, for example a log transformation. Check whether that log transformation gives you better-fitting information.

Does univariable logistic regression correspond to a Chi-square test?

Searching for risk factors for the development of PSC, I want to compare the frequency of occurrence of a risk factor (smoking) in the PSC group with the control group, and then correct for possible interacting variables by means of a multivariable logistic regression analysis. Initially I had performed a Chi-square test for the analysis of risk factors. But since a univariable logistic regression essentially tests the same thing, and I am going on to a multivariable model, I decided to do a univariable logistic regression instead of chi2. However, when I compare the outcomes of both tests, I notice that the OR and 95% CI are the same, but the p-value differs between the two tests. How can this be explained?

The 'exact' agreement between the two procedures lies in the test reported in the SPSS Crosstabs output under 'likelihood ratio' and the test reported in the logistic regression output in the 'omnibus tests of model coefficients' table. The p-values we usually use from these two procedures are, for the crosstabs, the 'Pearson chi-square', and for the logistic regression, the Wald test from the 'variables in the equation' table. These two can differ somewhat from each other because they are based on different formulas. The differences should, however, be very small. If there are large differences, that is in any case reason to look more closely at whether the model/test assumptions actually hold and what the best analysis would be. If the differences are small enough (say, if the p-values are equal when rounded to 2 decimal places), you can choose for yourself which test to use. If a multivariable analysis is planned, it increases consistency to also read off the univariable analysis within the regression framework. This obviously applies to categorical (nominal) predictors; for numerical predictors, the Chi-square test does not apply.

Where do I find logistic regression in SPSS?

You will find this technique in SPSS under Analyze -> Regression -> Binary Logistic.

How many variables am I allowed to include in my model?

I would like to perform a multivariable logistic analysis for the risk factors for obesity within a patient group. How many variables (risk factors) am I allowed to test on 893 patients?

A rule of thumb is that, in logistic regression, the number of parameters in a multivariable model should be at most 5% to 10% of the minimum of the number of people with and without obesity (the event) in your sample. So if you have 300 patients with and 593 without obesity, this rule of thumb allows you to include a maximum of 15 to 30 parameters in your model. I am talking about parameters here, not variables, because, for example, a categorical variable with 3 categories adds 2 extra parameters to your model. In logistic regression you therefore need at least 10, but preferably at least 20 or 30, 'events' per parameter. With fewer 'events', the parameter estimates can be misleading. Here you count the outcome that occurs least often as the event. Suppose you have a study in which 285 of the 300 patients die and 15 do not. Then there are 15 'events'. [1] See also the page on power on this wiki: power analysis for a prediction model

References

  1. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996 Dec;49(12):1373-9.

Additional resources

Back to OVERVIEW for an overview of all statistical topics on this wiki.

Back to SELECTION GUIDE for help in finding a suitable test or analysis.

⚠️ **GitHub.com Fallback** ⚠️