Deze pagina lezen in het Nederlands.

Author: dr. J.M. Binnekade
Co-author: dr. ir. N. van Geloven
Authorship

This page was translated from the Dutch-language original using AI and has not been fully reviewed. Where the two differ, the Dutch version takes precedence.

• Why do I need a power analysis or sample size calculation?
• When do I need a power analysis?
• Which software is available for a power analysis?
• What is the power of a study?
• What information do I need for a power analysis?
• Where do I find the information needed for a power analysis?
• No one has ever looked at our outcome before, how do I get an estimate of the effect size?
• Can I combine multiple outcome measures in a power analysis?
• Does a correct sample size calculation correct for measurement errors?
• When is a post hoc power analysis useful?
• How do I perform a post hoc power analysis?
• Power analysis in specific study designs
  • How do I perform a power analysis for an equivalence study?
  • How do I calculate a sample size for a kappa coefficient?
  • How do I calculate a sample size for a Fleiss kappa coefficient for multiple categories and raters?
  • How do I calculate a sample size for an intraclass correlation coefficient?
  • How is a group size calculation adjusted for a 'clustered' trial?
  • How can I perform a sample size calculation for a hierarchical design?
  • Can I do a power analysis for a diagnostic study?
  • What information do I need for a power analysis if I use a non-parametric test?
  • Is a power analysis useful for a satisfaction survey?
  • I am calculating the sample size for comparing two proportions, should I use the continuity corrected chi-square version?
  • I am calculating the sample size for estimating a single proportion and notice that I need more people as the proportion gets closer to .5, is that correct?
  • How many patients do I need for a pilot study?
  • How many fewer patients do I need if I do a cross-over study instead of a study with parallel groups?
  • nQuery and PASS each calculate a different group size for my planned logistic regression, which one should I use?
  • How many patients do I need for a genome-wide association study?
  • How many patients do I need to build a prediction model?
  • How do I calculate the sample size for a study with one group, where the outcome is a proportion?
• References

A power analysis calculates the number of participants (sample size) needed for a study to detect a predefined minimal clinically relevant difference with a certain probability (power).

One of the most frequently asked questions before a study starts is: how many patients, participants or laboratory animals do I need for this study? This is an important question; an incorrect sample size can raise ethical objections. Underestimating the sample size can mean that a true effect is not detected by the study, so that participants were tested needlessly. The study will then lead to a false-negative conclusion. A sample size that is too large can also raise objections. If the intervention turns out to be effective, unnecessarily many participants are treated in the control group. If the intervention turns out not to work, too many participants are exposed to an ineffective intervention [1,2].

There are rules governing the conduct and reporting of (comparative) research. The CONSORT statement, where you can find these rules, states that a researcher must calculate a sample size before a study is carried out, and must report this in the methods section of the article. A Randomized Controlled Trial (RCT), including the sample size calculation, must be registered before it is carried out. Moreover, without a good sample size calculation you will not receive a positive assessment from the Medical Ethics Committee. It is also relevant for the economic justification of a grant application and for the logistical planning of a study.

nQuery is a validated, user-friendly program with which you can calculate group size and statistical power for various study designs and data types. nQuery Advisor is available at Amsterdam UMC via the OneView software store. For more advanced study designs, a limited number of staff members have a license for the PASS 15 program.

Other programs that offer power calculations include R, SAS or Stata. SPSS nowadays also offers a number of modules for this. Several free programs are also available on the internet, though these are not always reliable.

In the study we try to determine whether the groups are the same (null hypothesis) or different (alternative hypothesis) [3]. We can make two types of errors here: a type I error (α) and a type II error (β). We make a type I error if we incorrectly reject the null hypothesis (false positive). We make a type II error if we incorrectly accept the null hypothesis (false negative). The power of a study is 1 - β. It is therefore 1 minus the probability of incorrectly accepting the null hypothesis. Roughly speaking, it is the probability of detecting a true effect in the population within the study.

		Total population
		true effect H1 true	no true effect H0 true
Study result	effect detected H0 rejected	true positive power (1-β) 80%	false positive type I error (α) 5%
	no effect detected H0 not rejected	false negative type II error (β) 20%	true negative (1-α) 95%

A sample size calculation is based on (reasoned) assumptions. You need information on the following topics (more explanation per topic):

• The desired power of the study (1-β). The choice of power level determines how confident you can be that a type II error is avoided.
  
• The desired significance level (α). This α level shows which probability you accept of making a type I error.
  
• One-sided or two-sided test. Do you test the difference in only one direction, or do you take into account that a study arm can perform either worse or better than the other?
• Expected or clinically relevant difference. What difference or effect are you looking for?
• Expected spread / standard deviation. How much variation do you expect among participants within the same study group?
  
• Which statistical test. Which test will be used during the analysis of the study?
  
• Lost to follow-up. Estimate how many participants will be lost to the analysis so that you can correct for this in the sample size calculation.

• The desired power of the study (1-β) The power gives the probability of finding a difference that truly exists (in the population). In other words: the probability that the null hypothesis is correctly rejected. A common choice for the power is 1-β = 0.80. This means you accept a probability of 0.20 of missing a truly present effect in the study. If you want to be more certain of detecting a difference in the study, you should use a higher power (for example 0.90 or 0.95). Increasing the power will lead to a larger sample size. For example: increasing the power to 0.90 at an α of 0.05 will increase your sample size by about 30%.

• The desired significance level (α) The usual choice for α is 0.05, meaning you accept a 5% probability that your conclusion about a difference between the two groups is a false positive. Table 1 shows the relationship between the hypotheses and conclusions from the study, expressed as a 2 x 2 table. One reason to lower α could be a correction for multiple testing. A lower significance level will lead to a larger sample size. For example: if you lower α to 0.01 at a power of 0.80, your required sample size will increase by about 50%. The probability of a false positive conclusion is then only 1 percent. If an effective drug is compared with a cheaper alternative, you might also choose to reduce the chance of a false positive conclusion, i.e. use a lower α level. You do not want to incorrectly trade an effective drug for an ineffective one. In this same example, you might also want to lower the power, since you are less concerned about missing an effective treatment - you already have one.

• One-sided or two-sided test You need to decide whether the analysis will use a one-sided or two-sided test. Two-sided is standard. A one-sided test is only defensible if the effect of an intervention cannot possibly be anything other than expected. This then requires an extensive justification in a Medical Ethics Committee application or a publication. In some studies there are nonetheless valid reasons to test one-sided [4,5]. Two-sided testing gives a larger sample size than one-sided testing.

• Expected or clinically relevant effect that you want to be able to demonstrate For a study comparing two groups, for example a control and an experimental group, this could be the difference between the groups in the fraction of successes (dichotomous outcome measure: success/failure) or the difference between two means for a continuous outcome measure (difference in mean blood pressure). But it can be any type of effect, depending on what your study primarily aims to estimate. The size of the effect you use in your sample size calculation determines which effect you will later be able to estimate/demonstrate with sufficient statistical certainty. There are two ways to arrive at your choice.

First, you can think about what you expect the effect to be. You can partly base an estimate on earlier findings (literature or the results of pilot studies). Note: it is not necessarily true that if 1 or 2 small studies have described a (positive) effect, these studies also directly give the most realistic estimate of the size of the true effect. Usually there is some form of publication bias (positive results stand out the most and are published the fastest). It can therefore be wise to set a somewhat lower value for a realistic estimate of the effect. See also this article, which explains how point estimates from earlier small pilot studies are not always the best guide for planning a large new study [6].

Second, you can base the power on the smallest difference that you would still consider relevant. The (treatment) effect you use in the sample size calculation then represents the smallest clinically relevant difference. The idea is that if the true effect were smaller, it would not be worth conducting the study, because it would not lead to a change in practice. If you have strong indications that the true effect can be expected to be larger than the minimally relevant effect, you can also choose that larger expected effect [7].

It is wise to keep both aspects in mind: the chosen effect must be both relevant (so not too small - it must actually matter) and realistic (so not estimated too optimistically large).

• Expected spread / standard deviation The estimate of the spread is based on earlier findings (literature or a pilot study). When no direct estimate of the standard deviation is available, an estimate can be made with nQuery based on other known data. If an observation is accompanied by a lot of variation, it becomes harder to demonstrate a difference between groups. You therefore need more participants to find a difference. This is sometimes compared to the noise you hear on the radio - the more noise (variation), the harder it becomes to hear the real signal (effect).

• Which statistical test A sample size calculation is always based on a type of statistical analysis. You therefore need to already have an idea of how your data will ultimately be analyzed. Help with choosing a suitable test can be found on the page SELECTION GUIDE of this wiki.

• What should I do if I have insufficient information? If no information at all can be found in the literature for the required estimates, a pilot study can be considered. In this, a small number of participants are tested. The information from the pilot study can then be used to make a better estimate of the total number of people needed to answer the research question. The size of the pilot study is often influenced by factors such as invasiveness, but cost also plays a role.

For a number of outcomes in our study, nothing is yet known about quantities (means, standard deviations) in our setting (we are testing this for the first time and there is therefore no literature available on it), and so I think I cannot calculate the effect size. How should I deal with this?

For a sample size calculation, it is by no means always necessary to have an effect already described in the literature. You can plan based on effects that you consider relevant. You want to have enough patients to be able to find a relevant difference. You can reason, based on external knowledge, what a relevant difference is.

A power analysis can only be based on one outcome measure of the study. Usually this is the primary endpoint of the study. However, the calculated sample size may be insufficient for the secondary endpoints. You can perform a separate power analysis for the secondary endpoints and base the sample size on the highest number of people. Do keep in mind that the final criterion on which the sample size is based must still be sufficiently relevant to justify testing the (additional) people for it. Roughly speaking, there are therefore two possible approaches:

• you select 1 of the outcomes (and one of the comparisons) as the most important one, the primary outcome parameter. The sample size is then chosen such that there are sufficient numbers for this primary outcome. The other comparisons made in the study are 'secondary'. This means you were not necessarily powered for them. If you do not find significance, this could be due to too-small group sizes. Even if you do find significance, you cannot draw overly strong conclusions, because your study was not planned for this comparison - it is a kind of 'by-product'.

• you calculate the required group size for all desired comparisons and choose the largest required group size. This way you are powered for everything, and could therefore also label everything as a 'primary outcome measure'. If there are multiple comparisons, it is probably necessary to correct for multiple testing to keep the false-positive probability across your entire study/experiment under control.

A sample size calculation takes into account the influence of non-systematic errors, i.e. random variation in the sample. A sample size calculation does not provide a solution for other types of errors. Examples are systematic errors (for example, poor calibration of measuring equipment). This also applies to the (unwanted) influence of other variables on the study outcomes (for example, confounders). Systematic bias must be corrected for in the study design and/or in the analysis phase.

A power analysis is performed before a study is conducted. At that point it is useful for planning a study. However, when the analysis of a study finds no significant difference and no power analysis was done in advance, a post hoc power analysis can sometimes help interpret the study outcome. The idea is that a power calculation provides some insight into whether the non-significant result (p-value greater than 0.05) has to do with too little power, or whether you did have sufficient power to find a relevant effect but the effect simply was not found. In the latter case, there is a "true negative" result, whereas with too low a power you cannot rule out that you are dealing with a "false negative" result. For a significant study result, it is no longer relevant to look at the power of a study. An effect has been detected in the study, so the probability of missing a present effect is no longer at issue [8,9,10]. Note: when retrospectively assessing the power of a study, do not use the effect observed in the study. The effect found is (especially in small studies) a far too unreliable measure. It is better to look at what power there was to find an effect considered relevant (based on literature or experience) [11,12].

In a post-hoc power calculation, you try to say something afterwards (after the statistical analysis) about the power you had before the study. Here you do not want to know what the power of your study is at the observed difference, but rather how much power you had to be able to demonstrate a relevant difference in your study population.

For example, you calculate which effect you could have tested with 80% power given the current number of participants. A remark could look something like this: "Considering the number of included patients, this study would have 80% power to pick up a mean difference of >= ... between pre and post medication."

It is then interesting to compare the difference for which you had 80% power with what you consider a (clinically) relevant difference. This says something about whether you had sufficient power to pick up relevant differences. If you could only have detected extremely large differences, there was too little power.

A more direct way is to look at how much power you had to pick up a (minimally) relevant difference. If this is much lower than 80%, there was too little power.

In an equivalence study you try to demonstrate that groups are the same. Variation in the observations will still be present in these groups too. Therefore it must be specified which (small) difference between the groups is negligible, i.e. at which difference you still consider the groups equal (the equivalence margin). It must also be specified which difference is expected (expected difference). The expected difference will usually be smaller than the equivalence margin used. A power of 90% is often chosen, so that you have a lower chance of incorrectly rejecting the null hypothesis, which in an equivalence study means incorrectly rejecting the existence of a difference between the groups.

How often do two raters need to assess an object in order to make a valid statement about kappa? In the case of a dichotomous assessment (e.g. disease present or absent), you can calculate a sample size with nQuery as follows: In the Study Goal and Design screen (File -> New), select under Goal: Make Conclusion Using "Agreement", under Number of Groups "one" and under Analysis Method "test". Then select Kappa (binary outcome).

After a new table has been created, the following information must be entered:

• Test significance level, α (for example 0.05)
• 1 or 2 sided test (usually 2 sided)
• Proportion successes (expected percentage of positive tests)
• Null hypothesis agreement, enter a value here higher than 0 (e.g. 0.40), since you expect more agreement than would occur by chance
• Alternative agreement, enter the Kappa you want to detect (e.g. 0.70)
• Power (usually 80% or 90%)

nQuery calculates the sample size based on this information.

I want to conduct a study in which a still-unknown number of raters assess patients based on video footage. The raters will classify the patients into three or four categories. How do I calculate how many patients I need?

You can perform this calculation with the R package kappaSize.

The methods for this are described in this article [13].

In a 'clustered' trial you randomize groups (clusters) instead of individuals. Groups are, for example, wards in a hospital. Patients in the study are randomized per ward, and therefore receive the same intervention per ward. The advantage of this study design is that patients allocated to different interventions cannot influence (contaminate) each other. The disadvantage is that a clustered trial requires more patients than a 'regular' trial. There is a relationship between the number of participants in a cluster and the number of clusters you need. The more participants there are in a cluster, the fewer clusters you need. One example concerns general practices [14]:

After intervention (placebo/experimental), the goal is to demonstrate an average reduction of 0.1 mmol/L cholesterol in patients with a power of 90% and a significance of 5%. The intervention is randomized across general practices. With 10 patients per practice, 558 practices are needed (4% more patients than in a trial randomized at the individual patient level). With 50 patients per practice, 126 practices are needed (17% more patients than a regular trial). With 500 patients per practice, only 32 clusters are needed, but almost 3 times as many patients are required as in a regular randomized trial. The extra patients resulting from the cluster effect are called the design effect or the inflation factor.

For the sample size calculation of a cluster-randomized trial you need the following information: significance level; power; standard deviation; clinically relevant difference; the number of patients per cluster (k); the correlation between patients in the same cluster $\rho$ (usually $\rho > 0.05$). When calculating the sample size for cluster-randomized trials, it is often recommended to use a so-called inflation factor of $1+(n-1)\rho$, where n = the average cluster size and $\rho$ is the intraclass correlation coefficient. A step-by-step explanation of this has been published [15].

For the sample size calculation in a hierarchical design, the simple adjustment methods used for a clustered design can often be used. There are also more advanced approaches in which you can incorporate more expected relationships into the calculation. Please contact a statistician for this.

In a diagnostic study, the sensitivity and specificity of a diagnostic instrument is often analyzed. A power calculation tells you how many positive and negative people you need to be able to test sensitivity and specificity with a certain power. Depending on how often both types occur in the population, you will need a certain sample size. As you can imagine, the number of the rarest type is decisive for the sample size. Sample size for sensitivity and specificity is calculated separately. In nQuery you can, for example, do the following for a power analysis on sensitivity. Calculate the sample size for 1 proportion (choose: Proportions – one Group – Confidence Interval). The 'expected proportion' can be interpreted as the intended sensitivity, while the 'distance from proportion to limit' indicates the desired width (in one direction) of the confidence interval around the sensitivity. The sample size calculated this way relates to the number of patients who score positive on the gold standard. Suppose you expect a 'prevalence' of 0.33 in your study population, then you must multiply your calculated sample size by (1/0.33=3) to get the total number of people needed. Examples of these kinds of calculations have been published [16], and nomograms have also been published for this [17].

For a sample size calculation based on a non-parametric test, for example the Mann-Whitney U test, you can, via a workaround, still make use of the mean and standard deviation of both groups to determine the power or sample size. During a calculation in nQuery, you are asked for the probability that an observation in group 1 is smaller than in group 2 (P1 = P(X<Y)) if the alternative hypothesis is true. You can estimate this using the 'assistants' tab and then choosing the option 'calculate effect size'. This asks for the mean and standard deviation of both groups. You then enter the effect size calculated by the assistant into the previous screen. The rest of the calculation follows automatically.

If pilot data is available, an even more precise estimate can be made of the requested probability (P1 = P(X<Y)), since you can then simply count, among the observation pairs (measurement 1 group 1 vs measurement 1 group 2, measurement 1 group 1 vs measurement 2 group 2, etc.), in how many cases the measurement in group 1 was smaller than that in group 2. This number divided by the total number of pairs then gives the requested probability.

I am planning a study with questionnaires about the use and satisfaction of physiotherapy among patients with back complaints. Because this is not necessarily comparative research, I wondered whether it is necessary to perform a power calculation. If a power calculation is not needed, is there another way to determine the required number of participants? Or in this case should as many patients as possible simply participate in order to get a good picture of the current situation?

Even for non-comparative research, it is advisable to perform a sample size calculation. Because even then you want to ensure that you can support the results of the study with sufficient certainty. For example, you would have a very different level of evidence with 5 out of 10 satisfied patients than with 50 out of 100 satisfied patients collected. Also from another perspective, it is good to look at a maximum required number, because this can save patients (and yourself) unnecessary effort. So here too, it must be determined what the most important outcome parameters of the study are, what the expectation of these is, and with what degree of precision (e.g. width of the confidence interval) you wish to report on this.

We want to do the sample size calculation for a comparison of 2 proportions. We expect 2% events in one group and 30% events in the other group. In nQuery I find an option for a chi-square test and an option for a "continuity corrected" chi-square test. When should I use that correction?

The calculation in nQuery uses (in the background, visible in the help file) a z-test to calculate the numbers for comparing 2 proportions. You can imagine that when converting a proportion to such a (continuous) z-distribution, an approximation is made. For relatively large numbers this works well. For somewhat smaller numbers the approximation is less precise. This is what the 'continuity corrected' version is for. This makes an adjustment to account for the fact that proportions with small numbers cannot be 'smoothly' converted into a z-distribution, and it results in a somewhat more conservative (larger) sample size. Unfortunately, there are no absolute thresholds from which you 'must' use the continuity correction. A threshold of (total) sample size up to 100 is sometimes used.

In your case, where besides relatively small numbers (total n remains well under 100) a very small event rate is also expected in 1 group, I would always try to ensure a safe sample size by using the continuity corrected version.

I am doing a sample size calculation for estimating a single percentage, for which I want a fixed width of the confidence interval around that percentage. What surprises me is that the number of patients needed increases toward an expected percentage of 50% and then decreases again. I don't quite understand this. Is there a logical explanation for this?

That is correct. The explanation is that it is easier to estimate a low percentage than a percentage that approaches 50%. Intuitively, you can think about it as follows:

Suppose you have a (regular 1-6) die and don't know what its faces show.

• You first look at how often a 1 occurs. You roll the die very many times and count how often a 1 comes up. You will quickly notice that the 1 does not occur often. It quickly becomes apparent that this must be a low percentage. An estimate of between 0 and 25% can be made quickly.
• Suppose you want to see how often an even number (2, 4 or 6) comes up. Again you roll many times. But now it is harder to determine what the percentage of even numbers is. Sometimes it occurs often (in a row), then not. Your estimate fluctuates, for example between 25% and 75%.

You can see that in the second case, with a true percentage of 50%, it is harder to report this percentage with a narrow confidence band than in the first case.

There are no clear rules about how many patients to include when conducting a pilot study. If the goal of the pilot study is to obtain an estimate of the mean and standard deviation, some statisticians recommend 12 patients per group [18]. If you want to estimate a proportion, Pearson's correlation coefficient, Cronbach's alpha coefficient, or the effect size, different numbers are needed [19].

I am conducting a study in healthy volunteers; because I expose them to radiation, we want to limit the number of participants. We are therefore considering a cross-over design. To what extent can we limit the group size with this design?

You can apply the following calculation rule to determine the contrast in sample size between the parallel and cross-over design [20]

$n_{cross-over} = \frac{((1-r) * n_{parallel})}{2}$.

Here, r is the correlation between measurements within the same individuals. A drawback of a cross-over study can be that the total radiation dose per patient will be higher. You need to be able to justify well why giving fewer patients more radiation is better than giving everyone the same radiation. Furthermore, there must be no carry-over or cumulative effects.

In my study I want to look at the relationship between a continuous result of a diagnostic test and a dichotomous outcome (gold standard). I want to use logistic regression for this and am now thinking about the required sample size. I entered various assumptions into both nQuery and PASS, but the two programs return different required group sizes. Which one should I go with?

I looked at the references cited by nQuery [21] and PASS [22]. This shows that the nQuery calculation uses an 'upper bound' approximation, so it will generally calculate higher group sizes than needed. I would therefore advise you not to use nQuery for your calculation, but PASS, or to apply the correct formulas [22] yourself.

I want to do a whole genome SNP array in a large cohort. I am mainly interested in the most relevant nonsynSNPs with MAF >5%. This means that I will measure hundreds of thousands of SNPs in the DNA of patients using genome-wide association, such as the 'genome wide human SNP array'. I am taking a pre-selected group of patients, for example 300, of whom 50% have good kidney function and 50% have poor kidney function. I wondered whether it is possible to know in advance how many patients you need to use for sufficient power?

The standard power analysis for GWAS is similar to that for a study looking at 1 SNP, except that the significance level is lowered. Usually we use a Bonferroni correction, so the significance level then becomes 0.05 divided by the number of SNPs. So if you use a 250K chip and 200K SNPs meet all quality criteria, then the significance level becomes 0.05/200,000 = 0.00000025. If you use a larger platform with, say, 1,000,000 SNPs, or if you impute SNPs to GONL or 1000genomes, then we use the so-called "genomewide significance" of 8.5x10^9. If you limit the analysis to nonsynonymous SNPs with MAF>5%, this greatly limits the number. I think most general platforms contain very few such SNPs, and even dedicated platforms (such as the cardiochip, metabochip or cancer chips) have very few such SNPs with MAF>5%. Here too, the significance level equals 0.05/number of remaining SNPs.

The comparison you are going to make concerns the genotyping of a SNP in the groups of patients with good or poorer kidney function. Suppose you are going to look at the percentage of patients with genotypes AA or Aa versus genotype aa, then you can call these percentages $p_1$ and $p_2$. You need to make statements about those $p_1$ and $p_2$, and then you can use the standard sample size formula:

$N_{\rm{per, group}} = \frac{(z_{\alpha} + z_{\beta})^2 * (p_1*(1-p_1)+p_2*(1-p_2))}{(p_1-p_2)^2}$.

$\beta$ = 1-power and if power is 80%, then $z_{\beta}$=0.84 (if power is 90%, then $z_{\beta}$=1.28). $\alpha$ = significance level and if $\alpha$=0.05, then $z_{\alpha}$=1.96. If $\alpha=\frac{0.05}{200000}$, then $z_{\alpha}$=5.16, and if $\alpha=0.000000085$ then $z_{\alpha}$=5.36.

Suppose you assume that in the "healthy" population the AA+Aa genotype occurs in 10% of people, then $p_1$=0.10. Suppose further that the odds ratio of this SNP for having poorer kidney function equals 2.5. Then you expect that $p_2$ is approximately equal to 0.10*2.5 = 0.25 (you can calculate this a bit more precisely). To have 80% power to find such a SNP significant when testing 200,000 SNPs, you need

$N_{\rm{per\, group}} = \frac{(5.16+0.84)^2 * (0.10*(1-0.10)+0.25*(1-0.25))} {(0.10-0.25)^2} = 444$

patients with poor and 444 patients with good kidney function.

Incidentally, we rarely see such strong associations with SNPs; usually the odds ratios are 1.1 or 1.2 or thereabouts, and then you test $p_1$=0.10 versus $p_2$=0.11 or 0.12, and you will find that you need a very large number of patients.

I want to build a prediction model for a binary (0/1) outcome. How much patient data do I need?

In this situation, it is not about the number of patients, but about the number of events ("1") and non-events ("0"). For each parameter you estimate in your prediction model, you need at least 20 events and at least 20 non-events [23]. So you need enough patients that it is plausible that you have at least 20 events (for example, in-hospital mortality) and 20 non-events (for example, discharged alive from the hospital) per parameter you want to estimate. But to get a good estimate of the required number, it may be necessary to take other factors into account as well [24].

I want to build a prediction model for a continuous outcome. How much patient data do I need?

The answer to this is quite complex; there are a number of articles that provide guidance on the number of patients needed [25,26].

When planning a study to build a prediction model, also think about the data you may need for (internally) validating the model!

I want to calculate the sample size for a study with one group of patients. The patients are operated on using a new method. I want to compare the proportion of patients with a certain complication after the new operation with the proportion of patients with this complication after the current standard operation. The proportion of complications with the current standard operation is known from the literature and is 0.52. I expect that the proportion of complications with the new operating method will be 0.42. How can I calculate a sample size for this study?

In this type of study, you estimate a proportion based on the data of patients from one group. You then compare this proportion with a predetermined value of the proportion. This comparison can be performed with a chi-square test for one proportion. In nQuery you can select the option New under the File menu. Then click "proportions" under "Goals: make conclusions using", "One" under "Number of Groups", and "test" under "Analysis Method". Then click the "+" sign next to the text "Single proportion" on the screen below. Choose "One sample Chi-squared" and the calculation screen appears. Enter the desired values for alpha (usually 0.05), one- or two-sided test (usually two-sided), the "null hypothesis proportion" (here 0.52), the "Alternative proportion" (here 0.42) and the desired power (usually 80%). Then press "Enter" on your keyboard, and the required number of patients in the study group will appear in the row "n". In this example that is 195.

If you are working with other proportions, it may be necessary to use the option "Exact test for single proportion".

1. Jones SR, Carley S, Harrison M. An introduction to power and sample size estimation, Emergency Medicine Journal, 2003 20(5):453-458 DOI
2. Florey CD. Sample size for beginners. BMJ 1993 306 1181-4 DOI
3. Park HM. Hypothesis Testing and Statistical Power of a Test. 2010. Link
4. Knottnerus JA, Bouter LM. The ethics of sample size: Two-sided testing and one-sided thinking. Journal of Clinical Epidemiology. 2001 Feb; 54(2): 109-110. DOI
5. Peace KE. The alternative hypothesis: One-sided or two-sided? Journal of Clinical Epidemiology. 1989 42(5): 473-476. DOI
6. Westlund E, Stuart EA. The Nonuse, Misuse, and Proper Use of Pilot Studies in Experimental Evaluation Research. American Journal of Evaluation. 2017 38(2), 246–261. DOI
7. European Medicines Agency. Note for guidance on statistical principles for clinical trials (CPMP/ICH/363/96). download from ema website
8. Levine M, Ensom MHH. Post Hoc Power Analysis: An Idea Whose Time Has Passed? Pharmacotherapy. 2001 21(4):405-409. DOI
9. Goodman SN, Berlin JA. The Use of Predicted Confidence Intervals When Planning Experiments and the Misuse of Power When Interpreting the Results. Annals Internal Medicine. 1994 121(3): 200-206. DOI
10. Lenth RV. Some Practical Guidelines for Effective Sample Size Determination. The American Statistician. 2001 55(3): 187-193. DOI
11. Gelman A. Don't Calculate Post-hoc Power Using Observed Estimate of Effect Size. Annals of Surgery. 2019 269(1):e9–e10. DOI
12. Gelman A. Post-hoc Power Using Observed Estimate of Effect Size is too Noisy to be Useful. 2019 270(2):e64. DOI
13. Zou GY. Sample size formulas for estimating intraclass correlation coefficients with precision and assurance. Stat Med. 2012 31(29):3972-81. DOI
14. Kerry SM, Bland JM, Statistics notes: sample size in cluster randomization. BMJ 1998 volume 316(5):549. DOI
15. McCarthy WF. Assessment of Sample Size and Power for the Analysis of Clustered Matched-Pair Data. COBRA Preprint Series. Working Paper 28. July 2007. DOI.
16. Lameris W, van Randen A, Dijkgraaf MGW, Bossuyt PMM, Stoker J, Boermeester MA. Optimization of diagnostic imaging use in patients with acute abdominal pain (OPTIMA): Design and rationale. BMC Emerg Med. 2007 7: 9. DOI.
17. Carley S, Dosman S, Jones SR, Harrison M. Simple nomograms to calculate sample size in diagnostic studies. Emergency Medicine Journal 2005 22(5) 180-181. DOI
18. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharmaceutical Statistics. 2005 4(4) 287-291. DOI.
19. Hertzog MA. Considerations in determining sample size for pilot studies. Research in Nursing and Health. 2008 31(2) 180-191. DOI.
20. Bonten TN, Siegerink B, van der Bom JG. Cross-overstudies. Ned Tijdschr Geneeskd. 2012 156:A5542. Link
21. Hsieh FY. Sample size tables for logistic regression. Statistics in Medicine. 1989 8(7) 795-802. DOI
22. Hsieh FY, Bloch DA, Larsen MD. A simple method of sample size calculation for linear and logistic regression. Statistics in Medicine. 1998 17(14) 1623-1634. DOI
23. Ogundimu EO, Altman DG, Collins GS. Adequate sample size for developing prediction models is not simply related to events per variable. Journal of Clinical Epidemiology. 2016 76 175-182. DOI
24. van Smeden M, Moons KGM, de Groot JAH, Collins GS, Altman DG, Eijkemans MJC, Reitsma JB. Sample size for binary logistic prediction models: Beyond events per variable criteria. Statistical Methods in Medical Research 2018. DOI
25. Knofczynski GT, Mundfrom D. Sample Sizes When Using Multiple Linear Regression for Prediction. Educational and Psychological Measurement. 2007 68(3). DOI
26. Brooks GP, Barcikowski RS. The PEAR Method for Sample Sizes in Multiple Linear Regression. Multiple Linear Regression Viewpoints. 2012 38(2) 1-16. Link

See also Power analysis (English), a pre-existing, human-written English page on this topic.

---

Finished reading? You can go to the Overview of all topics, or to Choosing a test for help picking a suitable test or analysis. Want to learn more about biostatistics? Then follow the AMC e-learning Practical Biostatistics. Did you find something on these pages that isn't correct? Does a link not work? Or would you like to contribute to the wiki? Then get in touch with us.

This biostatistics wiki is an initiative of the former statistics helpdesk of Amsterdam UMC, location AMC. Amsterdam UMC staff can request statistical support via intranet.