Lecture 19
Comparing means of more than two groups

ABD 3e Chapter 15

Chris Merkord

Learning Objectives

• Explain why multiple pairwise tests inflate Type I error and why ANOVA is needed
• Describe how total variation is partitioned into between-group and within-group components
• Interpret sum of squares and mean squares as measures of variation
• Explain the \(F\)-statistic as a ratio of between-group to within-group variation
• State and interpret the hypotheses tested in a one-way ANOVA
• Interpret ANOVA output, including the meaning of a significant \(F\)-test
• Identify assumptions of ANOVA and assess when they may be violated
• Describe alternatives to ANOVA, including data transformation and the Kruskal–Wallis test

Many biological questions involve more than two groups

• Experiments often include multiple treatments, not just two
• Example: two medications and a placebo control
• This allows us to ask richer questions:
  • Are both treatments better than the control?
  • Is one treatment better than the other?
  • How large are these differences?

Figure 1: Image: La Trobe University (CC BY-NC-SA 4.0).

Comparing groups two at a time inflates false positives

• One approach is to run multiple two-sample tests:
  • Group 1 vs 2
  • Group 2 vs 3
  • Group 1 vs 3
• This seems reasonable, but it does not scale with more groups

Figure 2: Pairwise comparisons among three treatment groups. Each bracket represents one of the three two-group comparisons that would be made if the groups were analyzed two at a time.

Multiple tests inflate Type I error

• Problem:
  • Each test has a chance of a Type I error (false positive)
  • Multiple tests increase the chance of at least one false positive
• Example:
  • 5 groups → 10 pairwise tests
  • Up to ~40% chance of at least one false positive if all nulls are true

Figure 3: Probability of at least one Type I error as the number of pairwise comparisons increases (\(\alpha\) = 0.05). Assuming independent tests, the probability rises rapidly with more comparisons.

We need a single test for all groups

• Goal:
  • Test for differences across all groups at once
• Avoid:
  • Repeated testing
  • Inflated Type I error

Analysis of Variance (ANOVA)

ANOVA compares all group means simultaneously

• Analysis of variance (ANOVA) tests for differences among multiple means
• Uses a single overall test
• Based on:
  • Comparing variation among groups to variation within groups
• Tests:
  • Are individuals from different groups, on average, more different than individuals from the same group?

ANOVA tests variation to detect differences in means

• The name (analysis of variance) can be misleading:
  • We are interested in means, not variances
• Key idea:
  • If group means differ → there will be variation among groups
• Therefore:
  • Testing for variation among groups tells us whether means differ

One-way ANOVA analyzes one explanatory variable

• One-way ANOVA:
  • One explanatory variable (factor)
  • Multiple groups defined by that factor
• Examples:
  • Treatment type
  • Habitat type
  • Species

Case Study: Does light exposure affect circadian phase shift?

• Study of how light exposure shifts the body’s internal clock (circadian rhythm)

• 22 participants randomly assigned to one of three treatments:

  • No light (control)
  • Light to knees
  • Light to eyes

• Each person received a single 3-hour light exposure

• Researchers measured how much each person’s internal clock shifted

Figure 4: Phase shift in circadian rhythm (melatonin production) for participants exposed to different light treatments (control, knees, eyes). Open circles show individual participants; filled points with error bars show group means ± standard error. Image: Whitlock & Schluter, The Analysis of Biological Data, 3e © 2020 W. H. Freeman and Company. Data from Wright and Czeisler (2002).

ANOVA asks: where does the variation come from?

• Data vary across individuals, even within the same group
• Some variation is due to:
  • real differences among groups
  • random variation within groups
• Goal: Compare between-group variation to within-group variation

Total variation can be partitioned

• Total variation: how much all observations vary around the overall mean

• This can be split into two parts:

  • Between-group variation: differences among group means
  • Within-group variation: variation among individuals within groups

• ANOVA works by comparing these two sources of variation

Partitioning variation in a real dataset

• Same data shown three ways
  • Total: differences between each observation and the overall mean
  • Groups: differences between each group mean and the overall mean
  • Error: differences between observations and their group mean
• Total variation = Groups + Error
• ANOVA asks whether:
  • variation among group means is large relative to variation within groups

Partitioning total variation into between-group (Groups) and within-group (Error) components using circadian phase shift data for three light treatments. Image: Whitlock & Schluter, The Analysis of Biological Data, 3e © 2020 W. H. Freeman and Company. Data from Wright and Czeisler (2002).

Mean squares summarize variation

• ANOVA uses mean squares (MS) to measure variation
• Two key quantities:
  • \(MS_{groups}\): variation among group means
  • \(MS_{error}\): variation within groups
• Each is:
  • a measure of variability
  • converted to an average amount of variation per sample unit
• Larger values = more variation

• Start with sum of squares (SS):
  • sum of squared deviations from a mean
• Convert to mean squares: \(MS = \frac{SS}{df}\)
• So:
  • \(MS_{groups} = \frac{SS_{groups}}{df_{groups}}\)
  • \(MS_{error} = \frac{SS_{error}}{df_{error}}\)
• Dividing by \(df\) puts both on the same scale so they can be compared

The \(F\)-statistic compares two sources of variation

• ANOVA test statistic:

\[ F = \frac{MS_{groups}}{MS_{error}} \]

• Interpretation:
  • \(F \approx 1\) : groups are similar
  • \(F > 1\) : group means differ more than expected by chance
• The larger the ratio:
  • the stronger the evidence for differences among groups

ANOVA hypotheses

• Null hypothesis:

\[ H_0: \mu_1 = \mu_2 = \cdots = \mu_k \]

• Alternative hypothesis:
  • Not all group means are equal
• Important:
  • ANOVA tests for any difference, not which groups differ

Interpreting the ANOVA result

• If \(F\) is large → small p-value:
  • Reject \(H_0\)
  • Evidence that at least one group mean differs
• If \(F\) is near 1:
  • Fail to reject \(H_0\)
  • Differences are consistent with random variation
• Next step (if significant):
  • Determine which groups differ

ANOVA results are typically presented in a table

• Key components:
  • Source (Groups, Error)
  • Degrees of Freedom ( \(df\) )
  • Sum of squares ( \(\operatorname{SS}\) )
  • Mean Square ( \(\operatorname{MS}\) )
  • \(F\) and \(p\)-value
• Goal: compare variation among groups vs within groups

library(readr)
light_data <- read_csv("data/chap15e1KneesWhoSayNight.csv")
light_lm <- lm(shift ~ treatment, data = light_data)
anova(light_lm)

1

Read the data from the circadian rhythm shift experiment from Schluter and Whitlock 3e

2

Fit a linear model to the data using treatment as the (categorical) explanatory variable and shift as the (numerical) response variable.

3

Use the anova() function to show the ANOVA table

Analysis of Variance Table

Response: shift
          Df Sum Sq Mean Sq F value   Pr(>F)   
treatment  2 7.2245  3.6122  7.2894 0.004472 **
Residuals 19 9.4153  0.4955                    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Table 1: ANOVA table for circadian phase shift by light treatment. Data from Wright and Czeisler (2002), adapted by Whitlock & Schluter, The Analysis of Biological Data, 3e © 2020 W. H. Freeman and Company.

Interpreting an ANOVA table

Source	df	SS	MS	F	P
Groups (treatment)	2	7.22	3.61	7.29	0.004
Error (residuals)	19	9.42	0.50

Table 2: Formatted ANOVA table for circadian phase shift by light treatment, showing variation among groups (treatment) and within groups (residuals). Data from Wright and Czeisler (2002), adapted by Whitlock & Schluter, The Analysis of Biological Data, 3e © 2020 W. H. Freeman and Company.

• Rows represent different sources of variation
• Columns describe how variation is measured:
  • SS: total variation
  • df: degrees of freedom
  • MS: average variation
  • F and p-value: test results
• Goal: Compare variation among groups to variation within groups

Interpreting an ANOVA table: Groups

Source	df	SS	MS	F	p-value
Groups (treatment)	2	7.22	3.61	7.29	0.004
Error (residuals)	19	9.42	0.50	NA	NA

• Groups (treatment):
  • variation among group means
• “Treatment” = the explanatory variable defining groups
• This row measures:
  • how different the group means are from the overall mean
• Larger values → greater differences among groups

Interpreting an ANOVA table

Source	df	SS	MS	F	P
Groups (treatment)	2	7.22	3.61	7.29	0.004
Error (residuals)	19	9.42	0.50	NA	NA

• Error (residuals):
  • variation within groups
• Represents:
  • natural variability among individuals
  • measurement noise
• This is the baseline variation:
  • what we expect even if groups are the same
• Larger values → more within-group variability

Connecting the table to the F-statistic

Source	df	SS	MS	F	P
Groups (treatment)	2	7.22	3.61	7.29	0.004
Error (residuals)	19	9.42	0.50

\[ F = \frac{MS_{groups}}{MS_{error}} \]

• Mean squares (MS):
  • average variation for each source
• If groups differ → \(MS_{groups}\) is large relative to \(MS_{error}\)
• Larger \(F\) → stronger evidence of differences among groups
• P-value: probability of observing this \(F\) if all group means are equal

Proportion of variation explained

• ANOVA can quantify how much variation is explained by groups
• \(R^2\) (R-squared):

\[ R^2 = \frac{SS_{groups}}{SS_{total}} \]

• Interpretation:
  • Values closer to 1 → groups explain more variation
    
  • Values near 0 → little variation explained

ANOVA assumptions

• Observations are independent
  

• Data are approximately normally distributed within each group
  

• Groups have similar variance

• These assumptions affect validity of results

Robustness of ANOVA

• ANOVA is fairly robust to:
  • moderate departures from normality
    
  • moderate differences in variance
• Especially when:
  • sample sizes are large
    
  • sample sizes are similar across groups

What if assumptions are violated?

• If violations are mild:
  • results may still be valid
• If violations are strong:
  • consider alternative approaches

Data transformation as an option

• Transform the response variable to better meet assumptions
• Common transformations:
  • log
  • square root
• Goal:
  • stabilize variance
  • improve normality

Nonparametric alternative: Kruskal–Wallis test

• Used when ANOVA assumptions are not met
• Tests whether distributions differ among groups
• Does not assume normality
• Interpretation is similar:
  • significant result → differences among groups

A significant ANOVA does not tell you which groups differ

• ANOVA tests
  • whether any group means differ
• It does not identify:
  • which groups are different
• After a significant result:
  • we compare specific pairs or sets of groups
• But:
  • multiple comparisons increase Type I error
• This requires methods that control the overall error rate

Figure 5: Circadian phase shift under three light treatments, illustrating differences among group means and motivating follow-up comparisons after a significant ANOVA result. Image: Whitlock & Schluter, The Analysis of Biological Data, 3e © 2020 W. H. Freeman and Company. Data from Wright and Czeisler (2002).

Planned comparisons

• Comparisons specified before analyzing data

• Focus on specific, meaningful hypotheses:

  • e.g., control vs treatment
    
  • not all possible pairs

• Advantages:

  • fewer tests → lower risk of Type I error
    
  • more statistical power for those comparisons

• Interpreted similarly to:

  • t-tests, but within the ANOVA framework

Unplanned comparisons and Type I error

• Comparisons chosen after looking at the data
  
• Often involve:
  • examining many or all pairwise differences
    
• Problem:
  • each additional test increases the chance of a false positive
    
• Same issue as:
  • running many t-tests
    
• Solution:
  • use methods that control the overall (family-wise) Type I error rate

Tukey–Kramer method for multiple comparisons

• Compares all pairs of group means
  
• Adjusts for multiple comparisons:
  • controls the overall Type I error rate
    
• Used after a significant ANOVA:
  • to identify which pairs of groups differ
    
• Provides:
  • confidence intervals for differences between means
    
  • adjusted p-values for each comparison
    
• Note: other methods (e.g., Holm) are often used in practice and can be less conservative

If there are only two groups

• Use a two-sample t-test instead of ANOVA
• Prefer:
  • Welch’s t-test
• Reason:
  • does not assume equal variances