Deze pagina lezen in het Nederlands.

Author: dr. ir. N. van Geloven
Co-author: dr. R. Holman
Authorship

This page was translated from the Dutch-language original using AI and has not been fully reviewed. Where the two differ, the Dutch version takes precedence.

• When do I use Fisher's exact test?
• Should I use chi-square or Fisher's exact test?
• I have 0 events in my control arm; can I still test the difference between the two arms?
• I have 0 events in my control arm; how do I calculate the odds ratio?
• Where do I find Fisher's exact test in SPSS?
• I have more than two categories. Can I still use Fisher's exact test?
• I get a different p-value from my Fisher's exact test every time; how can that be?
• My calculator gives a different two-sided p-value for the Fisher exact test than SPSS; how can that be?
• Should I report a two-sided p-value or twice the one-sided p-value?
• Can I also use Fisher's exact test with larger samples?
• References

Fisher's exact test assesses whether the difference between proportions truly exists or is merely due to "chance".

Fisher's exact test can be used to test whether the difference between two proportions in a classic 2x2 table is significant. Such tables are usually analyzed with a chi-square test. Traditionally, Fisher's exact test is chosen over a chi-square test when there are low counts (observed count ~<10 or expected count <5) in the 2x2 table. In that case, the chi-square test is no longer accurate. Because Fisher's exact test can nowadays be calculated just as quickly as a chi-square test in current statistical packages, there is no longer any objection to always using this exact test (chi-square uses an approximation) when analyzing a 2x2 table. There is also a generalization of Fisher's exact test (the Fisher-Freeman-Halton test) to a (larger) m x n table. In general, the chi-square test will be less conservative (more likely to reach significance) than Fisher's exact test.

Example of a classic 2x2 table:

	Cases	Controls	Total
Men	0	10	10
Women	12	2	14
Total	12	12	24

For my research I analyzed a number of outcomes with chi-square (X²) tests. It is not a very large study (75 patients), so the numbers of patients with the symptoms in question are sometimes small. In principle I used X² tests, only in the case of an "expected count" of less than 5 in 20% or more of the cells in the table did I do a Fisher's exact test. Now we are doing this study together with a sponsor and they have also done statistical analyses. I just received the results, and it turns out they did Fisher's exact tests for all outcomes (instead of X²). This is rather unfortunate, because a number of statistically significant outcomes that I found with the X² tests (and which all pointed in the same direction, so seemed to be correct) are now just not significant anymore (at least not if you take the results of the two-tailed Fisher, as the sponsor did). What is your opinion on this?

The sponsor has a point: Fisher's exact test is more accurate than the X² test and can nowadays be calculated just as easily. In general: the outcome/interpretation of your study should not hinge on a p-value of 0.04 or 0.06; there are much more important points to make than whether or not a p-value comes out below 0.05.

I looked at the occurrence of hyperthyroidism in patients with venous thrombosis vs controls. Of the 173 cases, 3 patients had hyperthyroidism vs 0 of the 344 controls. Statistical analysis using Fisher's exact test shows that hyperthyroidism and venous thrombosis co-occur more often than would be expected based on chance (p=0.037). However, according to one of my professors, the numbers above cannot yield a significant difference. Can I still use Fisher's exact test in this case?

Your calculations are correct: 3 out of 173 (1.7%) is significantly different from 0 out of 344 (0%) and the p-value is indeed 0.037. I can understand the professor's skepticism, because 3 events is of course not very many, but perhaps it helps if you emphasize that 0 events out of 344 observations already provides fairly strong evidence that the event rate in the controls is very low; in fact, the 95% confidence interval runs from zero to 1.07%, so the probability that it is anywhere near 1.7% is very small. You will likely encounter the same skepticism if you want to publish this result, so you may need to gather some more evidence.

I have a 0 in a number of 2x2 tables, so I cannot calculate an OR. What is the usual solution for this?

The most commonly used method to still be able to calculate an OR when one of the cells in the table is zero, is to add 0.5 to ALL cells. This probably results in a wider confidence interval. A reference for this correction and the SE is: Agresti A (1990) Categorical Data Analysis. John Wiley & sons, New York. p. 54.

You will find the test in SPSS 28 under Analyze->Descriptive Statistics->Crosstabs. Under the "Statistics..." button, check Chi-square. When both variables specified have only 2 categories (i.e., it concerns a 2x2 table), you automatically get Fisher's exact test in the output alongside the chi-square test. For a (larger) m x n table, an additional Monte Carlo or Exact calculation must be requested under the heading Exact...

Strictly speaking, no. But the Fisher-Freeman-Halton exact test is an extension of Fisher's exact test to the situation in which one or both variables have more than two categories [1]. You can read here more about performing the Fisher-Freeman-Halton exact test.

For one of my sub-questions I want to determine whether there is a difference in the number of prescribed medications between different exposure groups. I wanted to present this in an 8 x 13 table, showing the percentage for each occurring number of medications. I believe the counts are unpaired and discrete, because only whole numbers can occur as an outcome. Looking at wikistatistiek under 'test selection', I could do a chi-square test. Because some cells contain small counts (sometimes even 0), I thought it would be better to do a Fisher's exact test. But when I repeat the test, the outcome of the Fisher's exact test keeps changing. The outcome of the chi-square test stays the same, and I haven't changed anything in the data in between. Am I using the wrong test, and would I be better off using the chi-square test after all?

For calculating the Fisher exact test for a table larger than 2x2, you can choose between 2 calculation methods in SPSS. One is the 'exact' method; because this can take a long time, you can specify the maximum time you want the calculation to run. The second is a 'Monte Carlo' simulation. This makes use of simulation. With simulation, a different set can be simulated at each run purely by chance, so you can get a different answer with each run. That probably explains your observation. You could try running the analysis with the 'exact' setting. Note, by the way, that with large tables with low frequencies there may be little power to detect differences. Combining categories is sometimes needed to make more powerful comparisons.

In some calculators (including www.openepi.com), the two-sided p-value of the Fisher exact test is approximated by calculating twice the one-sided p-value. This is too crude an approximation; the Fisher exact test statistic generally does not have a symmetric sampling distribution. This means that the probability of the observed 2x2 table, or of tables with an even more extreme difference in proportions, need not be equal to that of comparable tables where the difference goes the other way. This asymmetry occurs especially with low counts. SPSS does compute the correct (asymmetric) two-sided p-value.

A reviewer of my manuscript writes that I would be better off not reporting the two-sided p-value. He writes that I should instead report twice the one-sided p-value. What is best for me to use?

It seems the reviewer is referring to the "central" Fisher's exact test [2]. Its p-value is defined as twice the smaller of the one-sided p-values, with a maximum of 1. This way of calculating the p-value provides a way to resolve inconsistencies between mathematical definitions of exact confidence intervals and p-values. You could consider calculating the p-values this way and including a reference to the article in your manuscript.

I recently read that there is no objection to always using Fisher's exact test when analyzing a 2 by 2 table. This is new to me, since in earlier statistics courses I learned that Fisher's exact test is only used with low numbers in the 2 by 2 table, and that for larger samples the chi-square test is preferred.

Fisher's exact test is – as the name says – an exact test. The p-value you get from it is calculated exactly. At the time this test was developed (1922 was the first publication on it), computers did not exist, and these calculations took a great deal of time because they had to be done by hand (literally with pencil and paper). That is why results based on approximations of it (such as the chi-square test) were important. The chi-square test is easier to calculate. However, the chi-squared test is only a good approximation of the exact test under certain conditions. And even when the approximation is good, it remains an approximation. Until about 10 to 15 years ago, calculating Fisher's exact test still demanded a great deal from the computers of the time. But with today's computers, it is no problem to calculate exact p-values, even for fairly large samples. There is therefore now less objection to simply calculating these values, unless you have more than a few thousand rows in your dataset.

1. Freeman GH, Halton TR. Note on exact treatment of contingency, goodness-of-fit and other problems of significance. Biometrika. 1951; 38: 141-149. Pubmed
2. Fay MP. Confidence intervals that match Fisher's exact or Blaker's exact tests. Biostatistics. 2010 Apr;11(2):373-4. DOI 10.1093/biostatistics/kxp050

• Agresti A (1990) Categorical Data Analysis. John Wiley & sons, New York.
• Mehta, C. R.& Patel, N. R. 1997. Exact inference in categorical data. Biometrics, 53(1), 112-117.

---

Finished reading? You can go to the Overview of all topics, or to Choosing a test for help picking a suitable test or analysis. Want to learn more about biostatistics? Then follow the AMC e-learning Practical Biostatistics. Did you find something on these pages that isn't correct? Does a link not work? Or would you like to contribute to the wiki? Then get in touch with us.

This biostatistics wiki is an initiative of the former statistics helpdesk of Amsterdam UMC, location AMC. Amsterdam UMC staff can request statistical support via intranet.