Deze pagina lezen in het Nederlands.

Author: prof dr R.J. de Haan
Authorship

This page was translated from the Dutch-language original using AI and has not been fully reviewed. Where the two differ, the Dutch version takes precedence.

• When should I use a linear regression model?
• How does (simple) linear regression work?
• How do I interpret my SPSS output when using a simple linear model?
• How does multiple linear regression work?
• How do I interpret my SPSS output when using a multiple linear model?
• How do I interpret the results of my linear model when the outcome measure has been log-transformed?
• Where do I find linear regression in SPSS?
• How do I include categorical variables in linear regression in SPSS?
• My predictor is not normally distributed, can I still use linear regression?
• How many predictors can I include in my model?
• How do I report my ANCOVA analysis?
• Additional resources

When you apply a correlation technique, you are interested in the degree of association between two variables $X$ and $Y$, for example in a study of the association between age and scores on a cognitive test. Linear regression takes this a step further. With this technique you try to predict the values of the outcome $Y$ from those of $X$ via a linear relationship. The outcome variable $Y$ is called the dependent variable, and the predictor $X$ the independent variable. We speak of simple (or univariable) regression when we want to predict the outcome with a single predictor. In practice this will often not be sufficient, and it is desirable to analyze the effects of two or more predictors. We then speak of multiple (or multivariable) linear regression.

In order to use linear regression, the research data must meet a number of statistical assumptions:

1. Across the entire range of observed values there must be a linear relationship between the independent variable $X$ and the dependent variable $Y$ (tip: first make a scatterplot to explore the nature of the relationship between $X$ and $Y$);
2. As for the type of data, the $X$ variables may be either discrete or continuous. The dependent variable $Y$ may only be continuous;
3. For each value of $X$, the observations of $Y$ must be independent of each other; that is, the $Y$ variable was measured only once per person. Linear regression is therefore not suitable for analyzing research data in which patients were measured repeatedly over time (other techniques exist for repeated measures).
4. The variable $Y$ has a normal distribution for each value of $X$.
5. The distributions of the $Y$ variable have equal variance (or standard deviation) for each value of $X$.

Assumptions 1, 4, and 5 can be checked using the residuals of the linear model.

In linear regression we determine the linear regression line that best describes the relationship between the $X$ and $Y$ variables. The linear regression line is estimated using the equation:

$Y = a+b \times X + \epsilon$

$Y$ is the dependent, outcome variable; $X$ the independent predictor; and 'a' and 'b' are the regression coefficients; $\epsilon$ represents the distance between the regression line and the data points (the residuals). Regression coefficient 'a' is also called the intercept, and indicates the value of $Y$ when $X=0$ (the intercept thus determines the height of the regression line). The regression coefficient 'b' reflects the slope of the regression line, and indicates the average increase in $Y$ when the $X$ variable increases by 1 unit.

Suppose that in a sample of n=25 the Pearson's correlation between the age of the subjects and their body fat percentage is 0.81. However, we do not want to merely describe this association, but also want to make a prediction of a person's body fat percentage based on their age. If, in SPSS, in the screens of the Linear Regression module we enter the variable 'fat percentage' as the dependent variable, and the variable age as the independent variable, the following tables will be generated: Model Summary, ANOVA, and Coefficients. NB: in the examples below, some SPSS tables are shown in abbreviated form.

Model Summary
Model	R	R square	Adjusted R square	SE of the estimate
1	.821	.659	.645	6.3898

The Model Summary table shows the Pearson's correlation between the relevant $X$ and $Y$ variables. The 'R square' column is a so-called goodness-of-fit measure. This measure expresses how well the observed data cluster around the estimated regression line. In simple linear regression this is the square of the correlation. The proportion is usually expressed as a percentage of 'explained variance'. In this example, R square therefore means that 66% of the total variation in fat percentages can be explained by the linear regression, i.e. by the differences in age.

ANOVA
Model	Sum of squares	df	Mean Square	F	Sig.
1
Regression	1818.362	1	1818.362	44.535	.000
Residuals	939.078	23	40.829
Total	2757.440	24

The F-test in the ANOVA table tests the null hypothesis: regression coefficient b = 0 ('there is no association between $X$ and $Y$, so the regression line runs horizontally - without slope - through the scatter of points'). From the table we can conclude that age is a significant predictor of the measured fat percentage (regression coefficient b ≠ 0). Or, put differently, we can conclude that the fat percentages of the subjects can be significantly explained by their differences in age.

Coefficients
		Unstandardized coefficients		t	Sig.
model		B	SE
1	(Constant)	2.507	3.932	.638	.530
	Age	.549	.082	6.673	.000

a. dependent variable: fat percentage

The t-test in the third table – Coefficients – does not provide more information than what we already knew from the F-test in the ANOVA table, namely that age is a significant predictor of the measured fat percentage. This table shows the linear equation $Y = a + b\times X$. We see the regression coefficient a (somewhat confusingly in SPSS) listed under the column B, to the right of the cell 'Constant', with a value of 2.507. Regression coefficient b is also listed under column B, to the right of the variable 'age', with a value of 0.55. We can now fill in the linear equation as follows: fat percentage = 2.51 + (0.55 $\times$ age). The expected fat percentage of a 60-year-old subject is therefore: 2.507+ (0.549 $\times$ 60) = 35.45%.

In multiple (or multivariable) linear regression we are interested in modeling two or more independent variables simultaneously. You can imagine that not only age, but also the sex of the subject contributes to the prediction of their fat percentage. Another example: because there is an association between a child's height and weight, in a study of the relationship between weight and systolic blood pressure we also want to include the child's height in the statistical analysis. In short, we use multiple linear regression to:

1. predict the value of a dependent variable using a set of independent variables, and/or
2. determine to what extent an independent $X$ variable is linearly related to the dependent $Y$ variable, taking into account (adjusting for) the influence of other $X$ variables.

In the multiple linear regression equation

$Y = a + b_1\times X_1+b_2\times X_2+\ldots+b_k\times X_k + \epsilon$

$X_i, (i = 1.2,\ldots,k),$ are the predictor variables or covariates, $Y$ is the dependent variable, and $b_1, b_2, \ldots, b_k$ form the (partial) regression coefficients; $\epsilon$ again represents the residual error (the distance of the data points to the regression line). The coefficient $b_1$ indicates how much $Y$ increases when $X_1$ increases by one unit, taking into account (adjusting for) the influence of the other $X$ variables by holding them constant. In this way we can analyze the impact of $X_1$ on $Y$ independently of the influence of the other $X$ variables.

Suppose we conduct a study in 100 children on the relationship between weight and systolic blood pressure. We know that there is a relationship between a child's weight and height, and sex may also play a role. In a multiple linear regression we now analyze the effect of weight (kg), height (cm), and sex (0 = boy; 1 = girl).

Model Summary
Model	R	R square	Adjusted R square
1	.564	.318	.297

a. Adjusted R square means that the number of explanatory variables in the model was taken into account when calculating the explained variance.

The Model Summary shows that the adjusted R square = 0.297. In other words, approximately 30% of the variation in systolic blood pressure can be explained by differences in weight, height, and sex among the children.

ANOVA
Model	Sum of squares	df	Mean Square	F	Sig.
1
Regression	2804.045	3	934.682	14.952	.0001
Residuals	6001.261	96	62.513
Total	8805.306	99

The F-test in the ANOVA table tests the null hypothesis: regression coefficients $b_i$ are 0. The analysis shows that there is a significant linear relationship between systolic blood pressure and at least one of the independent $X$ variables.

Coefficients
		Unstandardized coefficients		t	Sig.
model		B	SE
1	(Constant)	75.211	17.118	4.641	.0001
	Weight	1.180	.261	4.512	.0001
	Height	-.031	.172	-.181	.857
	Sex	4.230	1.611	2.626	.010

The t-test shows that the relationship between weight and systolic blood pressure is significant (p < 0.001). Adjusted for the effects of height and sex, a 1 kilogram increase in weight predicts an average systolic blood pressure increase of 1.18 mmHg. Adjusted for the influences of weight and sex, however, the relationship between height and systolic blood pressure is not significant (p = 0.86). There is also a significant relationship (p = 0.01) between sex and blood pressure: girls have - on average – a 4.23 mmHg higher systolic blood pressure than boys, taking into account differences in weight and height between children. In short, weight and sex appear to be independent predictors of systolic blood pressure. Using the regression coefficients given in the table, we can further fill in the linear equation. For example, the expected systolic blood pressure of a 37 kg girl with a height of 115 cm is: 75.21 + (1.18 $\times$ 37) – (0.03 $\times$ 115) + (4.23 $\times$ 1) = 119.65 mmHg.

I have a question about the interpretation of the B-values in my linear regression analysis. The dependent variable is a log-transformed cortisol value, which makes the B-value (and possibly also the standardized Beta) more difficult to interpret. I have therefore back-transformed the B-values (Transformed B (EXP(B-value)) and get values around 1. My question is now: how do I explain the B-values during the presentation at a conference next week? Normally you would say that a 1 unit increase in the independent variable corresponds to a ...(B-value) decrease in cortisol value. I myself thought that, for example, Transformed [EXP(beta)] = 0.97, CI (0.95-0.99), p = .027 could now be explained as: every unit increase of the independent variable gives a decrease of 3% in cortisol. Is this correct?

If you fit a linear model to assess the effect of X on the log-transformed cortisol value, it looks as follows:

log(cortisol) = intercept + B*X

Indeed, one unit increase in X gives a B increase in log(cortisol). If we want to translate this back to "ordinary" cortisol values, we use the inverse of the (natural) log:

exp(log(cortisol)) = exp(intercept + B*X), where exp stands for 'e to the power of'.

This is equal to:

cortisol = exp(intercept + B*X)

Let's now look at the difference in cortisol values with an increase of 1 unit X:

cortisol = exp(intercept + B*(X+1)) = exp(intercept + B*X + B)

Using the rules of arithmetic, this is equal to:

cortisol = exp(intercept + B*X)*exp(B)

Now the shaded part of the equation above is exactly equal to the cortisol value at the "starting point", so indeed: one extra unit of X gives multiplication of the cortisol value by the factor exp(B). And also indeed: if exp(B) = 0.97, then multiplying by this gives a decrease of 3%.

Your interpretation was therefore correct. (In the above, I have assumed that the cortisol values were transformed using the natural logarithm and not, for example, a base-10 log).

For a more detailed explanation of interpreting regression coefficients when using a log transformation, see this page from UCLA: Academic Technology Services, Statistical Consulting Group.

You can find linear regression in SPSS 28 under Analyze -> Regression -> Linear.

In order to use categorical variables as a predictor in a linear model, dummy variables must first be created in SPSS. For example, to investigate the effect of smoking (with categories 'no', 'previous', or 'current') on systolic blood pressure, two dummy variables must be created. If we consider the group that never smoked as the reference, we create a 0/1 variable for 'previous smoker' and a 0/1 variable for 'current smoker'. In general: with n categories, n-1 dummy variables must be created. In the output we now find, for the difference previous vs no smoking and current vs no smoking, the regression coefficient with the associated significance test. If desired, the "overall test", which shows whether the smoking variable contributes to the model at all, can be obtained by entering the predictors into the regression model in two blocks: one block with all variables except smoking, and a second block with the two dummy variables for smoking added. If 'R squared change' is checked under 'Statistics', SPSS also provides a p-value for adding all three smoking categories together.

Alternatively, instead of the standard Analyze -> Regression -> Linear procedure, you can use the Analyze -> General Linear Model -> Univariate module. Here you can indicate which variables are categorical by designating them as 'Fixed factor(s)', and you enter the numerical predictors under 'Covariate(s)'. Under 'Model...' you then indicate which terms you want in the model, plus any interactions. Under 'Options', request the usual regression coefficients in the output. This module does the same thing as an ordinary linear regression, but has more options/possibilities that are often not applicable.

I conducted a study in patients with a neurological disorder; these patients may have abnormalities in the brain. I therefore performed a brain MRI and a number of neuropsychological tests. I want to investigate whether children with many abnormalities also score worse on the neuropsychological tests. The variable indicating the MRI abnormalities is not normally distributed: half of the patients have no abnormalities (abnormality score is 0), the other half of the patients have an abnormality score ranging from 5 to >10,000. Ideally I would like to run a number of linear regressions with the abnormality score as predictor and, each time, one of the neuropsychological tests as outcome. Some of the neuropsychological tests are age-dependent, so age must also be included in the model as a predictor. I am unsure whether I am allowed to perform a (multiple) linear regression given the skewed distribution of the abnormality variable. Log-transforming helps somewhat, but even then it is not normally distributed. Is there a solution so that I can still perform the (multiple) linear regression? An alternative is to build a simple correlation model, which I can do non-parametrically using a Spearman's correlation. I believe I could then simply use the untransformed abnormality score as the variable and look at the Spearman's correlation with the neuropsychological tests. Is that correct? But then I have a problem in cases where I also need to include age as a variable — then I would need to do partial correlations and include age as a 'controlling for' variable. But that is again not non-parametric; is there a way to work around that?

Strictly speaking, the predictor in a linear regression does not need to be normally distributed. That requirement applies to the outcome measure (or actually to the residuals of the outcome measure minus the model prediction). However, if you were to use the abnormality score as a numerical covariate in the model, you would be making another strong assumption, namely that every 1-point increase in abnormality score corresponds to a fixed increase or decrease in the outcomes of the neuropsychological tests. Given the large spread in abnormality values, this is probably not realistic. The estimates in such a regression would be disproportionately influenced by the patients with a score >10000.

Calculating a non-parametric Spearman correlation is indeed possible. As you note yourself, however, this offers no way to correct for other factors (partial correlations are actually just multiple linear regressions).

An alternative you could try is dividing the abnormality variable into categories. A rough division into presence or absence of abnormality would work, or a more fine-grained one into, for example, 0, >0 to 50, >500. Ideally you would use 'logical cut-off points'. Do make sure that a reasonable number of patients still falls into each category so that it remains estimable. A regression with this categorical variable as predictor can then examine whether belonging to a different category is predictive of the test outcomes. Here too it is possible to correct for other factors. You could also try to make the variable less skewed by means of a transformation, for example a log transformation.

See the page about sample size on this wiki:

• power analysis for a prediction model.

We are conducting a study looking at the change in a biomarker after 26 weeks relative to baseline. In an ANCOVA model we examine the difference in change between two groups, corrected for the baseline value of the biomarker. The model in SPSS now produces output in which an "estimated marginal mean" is given for the "pairwise comparison", together with an associated 95% confidence interval. I thought this was a kind of corrected mean difference adjusted for the covariates in the model. My question is now: what should we report? The estimated marginal mean with 95% CI and p-value from the ANCOVA model, or is it better to report the difference in change over the original data (i.e. simply a t-test) with a 95% CI, indicating whether it is significant according to ANCOVA?

Take a look at this article by Vickers and Altman (Statistical Notes BMJ 2001). Click on the table, and you'll see that several scores are reported there — perhaps something for you as well? Also this paper by Gerard van Breukelen (J Clin Epidemiol 2006) might be interesting for you.

Course material on interaction and effect modification from the web based distance learning course - Regression Models as a Tool in Medical Research - International School of Quantitative Research, University of Freiburg.

---

Finished reading? You can go to the Overview of all topics, or to Choosing a test for help picking a suitable test or analysis. Want to learn more about biostatistics? Then follow the AMC e-learning Practical Biostatistics. Did you find something on these pages that isn't correct? Does a link not work? Or would you like to contribute to the wiki? Then get in touch with us.

This biostatistics wiki is an initiative of the former statistics helpdesk of Amsterdam UMC, location AMC. Amsterdam UMC staff can request statistical support via intranet.