Deze pagina lezen in het Nederlands.

Author: dr. ir. N. van Geloven
Authorship

This page was translated from the Dutch-language original using AI and has not been fully reviewed. Where the two differ, the Dutch version takes precedence.

• When do I use a confidence interval?
• Which confidence intervals belong with the tests I have performed?
• How do I calculate the confidence interval around a relative risk belonging to a paired (McNemar) comparison?
• How do I calculate a confidence interval for a difference between two medians?
• Possibly also of interest on this wiki
• References

The confidence interval (CI) is used to indicate how certain you are of an estimated value. It is an interval within which you expect the true value to lie. Formally, the interval says something about which values would be expected if an experiment were repeated multiple times. The 95% confidence interval is often used. With this you are essentially saying: if I were to repeat my study 100 times, I expect the true value (parameter) to lie within the (separately constructed, for each study) 95% confidence interval at least 95 times.

You can use a confidence interval for any estimate you make. For example: if I have measured the average weight of a group of patients, I can estimate the average weight of the entire patient population by taking the mean of my sample, and indicate the certainty of this estimate with a confidence interval.

In medical research, a confidence interval is often given around the estimate of a particular effect (or a particular difference between groups). For example: "women receiving treatment were less likely to report severe pain (risk ratio 0.41, 95% confidence interval 0.32 to 0.55)".

In my study I perform a Chi-square test, Mann-Whitney test and a Kaplan-Meier analysis to compare two groups with each other. In the first version of my article I had reported the p-values. Because we have only a limited number of participants, one of my co-authors suggested reporting the 95% confidence interval instead of the p-value. As I understand it, I cannot get this interval for the above tests from the analyses in SPSS. Is that correct? Do you have any ideas on how I can obtain the intervals?

You can calculate a 95% confidence interval around an 'effect measure'. For each of the comparisons mentioned, you would then need to choose which effect measure you want to report and calculate the confidence interval for that.

• For a Chi-square test (comparing proportions) you could think of a risk difference, risk ratio or odds ratio. The last 2 can be requested including confidence interval in the crosstabs procedure of SPSS (tick 'risk', paying close attention to which group is chosen as the reference). A risk difference is, as far as I know, not offered in the crosstabs procedure. You could obtain that, for example, via the 'two by two table' calculator on the OpenEpi website.

• for a Mann-Whitney U test you could think of reporting the confidence intervals around the two medians of both groups and an estimated difference between the medians + confidence interval. As far as I know, SPSS does not offer an option for the latter.

• Kaplan Meier analysis: if you only plot one (or several) survival curve(s) with it, it makes sense to report a 95% confidence interval around the estimated survival (or event) percentage. SPSS provides this for, for example, mean and median survival (if both can be calculated). For the curve itself, SPSS does not provide a confidence interval. You could possibly calculate this with R, function survfit. Perhaps you are more interested in an estimate of the difference in terms of a ratio between 2 curves at a certain point. You could possibly calculate that yourself using the standard errors that SPSS provides plus a delta method adjustment.

We compared our matched dichotomous outcome measure with paired tests (McNemar test), which yield some very nice p-values. However, to lend more weight to these findings, we would also like to present a relative risk with confidence interval alongside this. However, if you calculate these relative risks in the 'usual' way, the conclusion about significance based on the confidence interval does not always match the p-value from the McNemar test. For instance, the confidence interval may contain 1, yet the P may be <0.05, or vice versa. This difference can probably be explained by the fact that this relative risk with confidence interval does not take into account the fact that this is paired data. My question is therefore whether there is a way to calculate the relative risk with confidence interval for paired data, in such a way that we can present both this confidence interval and the p-value from the McNemar test together in a table.

After a brief search I came across the following method by Bonett and Price. On this website an implementation of this is offered in the form of an Excel calculator. Note that the calculator assumes the following data structure:

		Cases (or paired group 1)
		positive	negative
Controls (or paired group 2)	positive	a	b
	negative	c	d

Background: we are conducting a randomized cross-over study in which two interventions are compared. For normally distributed data, I report per intervention:

Mean + SD, plus a paired difference, an SD of the paired difference, and a 95%CI of the paired difference

A number of (secondary) outcome measures are not normally distributed. For these I now report per intervention:

Median + IQR, plus paired difference and an IQR of the paired difference.

I would also like to give a confidence interval for this non-normally distributed data. Is that possible?

Yes, that is possible. I understand that you want to calculate a confidence interval for the paired difference. You could use the Hodges-Lehmann procedure for this. In SPSS 28 you can request such a confidence interval under Analyze -> Nonparametric Tests -> Related Samples.

Do pay attention to exactly which difference you are estimating. This Hodges-Lehmann procedure estimates the median difference (= the median of all paired differences). This is not the same as the difference between the two medians per intervention group.

How do I determine a confidence interval for a proportion?

How do I calculate the confidence interval of an odds ratio?

• Douglas G. Altman, David Machin, Trevor N. Bryant, Martin J. Gardner, Statistics with confidence, BMJ books.
• Douglas G. Altman, J Martin Bland, Statistical Notes, Standard deviations and standard errors, BMJ 2005;331:903
• OpenEpi Dean AG, Sullivan KM, Soe MM. OpenEpi: Open Source Epidemiologic Statistics for Public Health, Version 2.3.1. www.OpenEpi.com, updated 2011/23/06, accessed 2011/11/28.
• Bonett DG, Price RM. Confidence intervals for a ratio of binomial proportions based on paired data. Stat Med. 2006 Sep 15;25(17):3039-47.

---

Finished reading? You can go to the Overview of all topics, or to Choosing a test for help picking a suitable test or analysis. Want to learn more about biostatistics? Then follow the AMC e-learning Practical Biostatistics. Did you find something on these pages that isn't correct? Does a link not work? Or would you like to contribute to the wiki? Then get in touch with us.

This biostatistics wiki is an initiative of the former statistics helpdesk of Amsterdam UMC, location AMC. Amsterdam UMC staff can request statistical support via intranet.