Overview

We will be summarizing information from the textbook 'Doing Meta-Analysis in R: A Hands-on Guide'. Specifically, we are interested in noting information that will be beneficial to repeated measures design experiments.

Chapter 3: Effect Sizes

General Concepts

• Effect sizes capture the relationship between two entities. They give information about direction and magnitude in this relationship.

• Effect size formula: $$\hat\theta_{k} = \theta_{k} + \epsilon_{k}$$

• Where:

  • $$\hat\theta_{k}$$: Is the estimated effect size (effect sizes we will calculate)
  • $$\theta_{k}$$: The true effect size in the population
  • $$\epsilon_{k}$$: Error attributed to sampling error

• Studies with a larger sample size will have a smaller $$\epsilon$$, thus there will be more precision in the estimate of the true effect size.

• Both $$\theta_{k}$$ and $$\epsilon_{k}$$ are unknowns (we will never know). However, we can get an estimate of the influence from $$\epsilon_{k}$$ by calculating the standard error (SE).

• Standard error formula: $$SE = \frac{s}{\sqrt{n}}$$

• Where:

  • $$SE$$: Is the standard error (when multiplied by 2 can give a range of values where true population parameter lies)
  • $$s$$: The standard deviation, which is the average distance from the mean. A higher $$s$$ indicates more variability.
  • $$n$$: The sample size of the study

• When discussing descriptives of a particular study, we specifically want to know what the mean (x bar) of a sample is

• Mean formula: $$\bar{x} = \frac {\Sigma_{i=1}^{n} x_{i}}{n}$$

• By knowing a sample's mean, standard deviation, and sample size, effect sizes and standard errors can be calculated!

Concepts for Repeated Measures Design

• The information above is to give a general gist of effect sizes and standard errors. These calculations will need modifications for our paper of interest that incorporate repeated measures.

• Because GitHub Wiki sometimes sucks at producing the equations I want to share- I will be using images.

Standardized Mean Difference (Within)	Pooled Standard Deviation	Standard Errors of SMD (Within)

• We will be using the 'spooled' approach rather than 's1' as a denominator to standardized mean differences since pooling adjust for variability differences between time points. Thus, this should give us more accuracy if we expect variability to differ by conditions, which is likely to occur.

Calculating Repeated Measures Effect Sizes and Standard Errors

The example code below was made mostly from code in 'Doing Meta-Analysis in R: A Hands-on Guide' and from code that calculates s_pool available in statology. Note: Statology made a minor mistake that was correct for below.

• In an ideal study, the sample size of time point 2 would be the same as that of time point 1 (initial). This would indicate that everyone who provided for data on Day 1 also provided data on Day 2. However, this is rarely the case, and we would expect only some of the participants from Day 1 to also provide data for Day 2 (some dropped the study). Thus, only subjects with both days completed can be used to calculate 'effect sizes' and 'standard errors'. Therefore, the equation from below calculates the standard error using n2, since we will assume all subjects from n2 are present in n1 but not the reverse.
• Both ChatGPT and the textbook recommend I use n instead of n1 and n2 but I don't want to!

# Define example data needed for effect size calculation
x1 <- 20    # mean at t1
x2 <- 30    # mean at t2
sd1 <- 13   # sd at t1
sd2 <- 14    # sd at t2
n1 <- 45     # sample size at t1
n2 <- 45    # sample size at t2
r <- 0.5    # correlation between t1 and t2

# Calculate the raw mean difference
md_within <- x2 - x1

#calculate pooled standard deviation
s_pooled <- sqrt(((n1-1)*sd1^2 + (n2-1)*sd2^2) / (n1+n2-2)) # Can also use sd_pooled() from effectsize package

# Calculate the smd:
# Here, we use the pooled standard deviation
# to standardize the mean difference
smd_within <- md_within/s_pooled
smd_within

# Calculate standard error
se_smd_within <- sqrt(((2*(1-r))/(n2)) + 
                    (smd_within^2/(2*(n2))))
se_smd_within

• smd_within: 0.740
• se_smd_within: 0.168

Calculating Repeated Measures Effect Sizes and Standard Errors (from functions)

# Define example data needed for effect size calculation
x1 <- 20    # mean at t1
x2 <- 30    # mean at t2
sd1 <- 13   # sd at t1
sd2 <- 14    # sd at t2
n2 <- 45     # sample size (technically at t2 in repeated measures!)
r <- 0.5    # correlation between t1 and t2