7.4 Analysis of variance (ANOVA)

Analysis of Variance (ANOVA) is a powerful statistical technique for comparing means across multiple groups. It's essential in reproducible research, allowing scientists to analyze complex experimental designs and draw meaningful conclusions from data.

ANOVA extends beyond simple comparisons, encompassing various types like one-way, two-way, and repeated measures. It requires specific assumptions and can be implemented in R, offering researchers a versatile tool for exploring group differences and interactions in their data.

Fundamentals of ANOVA

• Analysis of Variance (ANOVA) serves as a crucial statistical technique in Reproducible and Collaborative Statistical Data Science for comparing means across multiple groups
• ANOVA allows researchers to analyze complex experimental designs and draw meaningful conclusions from data, supporting reproducible research practices

Purpose and applications

Top images from around the web for Purpose and applications

• 

  Key design considerations for adaptive clinical trials: a primer for clinicians | The BMJ View original

  Is this image relevant?

• 

  Question about Tukey post-hoc ANOVA test results - Cross Validated View original

  Is this image relevant?

• 

  Meta-Research: Why we need to report more than 'Data were Analyzed by t-tests or ANOVA' | eLife View original

  Is this image relevant?

• 

  Key design considerations for adaptive clinical trials: a primer for clinicians | The BMJ View original

  Is this image relevant?

• 

  Question about Tukey post-hoc ANOVA test results - Cross Validated View original

  Is this image relevant?

1 of 3

Top images from around the web for Purpose and applications

• 

  Key design considerations for adaptive clinical trials: a primer for clinicians | The BMJ View original

  Is this image relevant?

• 

  Question about Tukey post-hoc ANOVA test results - Cross Validated View original

  Is this image relevant?

• 

  Meta-Research: Why we need to report more than 'Data were Analyzed by t-tests or ANOVA' | eLife View original

  Is this image relevant?

• 

  Key design considerations for adaptive clinical trials: a primer for clinicians | The BMJ View original

  Is this image relevant?

• 

  Question about Tukey post-hoc ANOVA test results - Cross Validated View original

  Is this image relevant?

1 of 3

• Compares means of three or more groups simultaneously to determine if significant differences exist
• Widely used in experimental research, clinical trials, and social sciences to assess treatment effects
• Helps control Type I error rate when making multiple comparisons between group means
• Applicable in various fields (psychology, biology, marketing) for analyzing group differences

Types of ANOVA

• One-way ANOVA examines the effect of a single independent variable on a continuous dependent variable
• Two-way ANOVA investigates the effects of two independent variables and their interaction
• Repeated measures ANOVA analyzes data from within-subjects designs where participants are measured multiple times
• MANOVA (Multivariate Analysis of Variance) extends ANOVA to multiple dependent variables
• Factorial ANOVA allows for the examination of multiple independent variables and their interactions

Assumptions and requirements

• Normality assumes the dependent variable is normally distributed within each group
• Homogeneity of variance requires equal variances across groups (tested using Levene's test)
• Independence of observations mandates that data points are not related or dependent on each other
• Continuous dependent variable measured on an interval or ratio scale
• Categorical independent variable(s) with two or more levels
• Random sampling from the population of interest enhances generalizability of results

One-way ANOVA

• One-way ANOVA forms the foundation for more complex ANOVA designs in statistical data science
• Understanding one-way ANOVA is crucial for reproducible research as it allows for consistent analysis of group differences across studies

Between-groups vs within-groups

• Between-groups design compares different groups of participants exposed to different conditions
  • Participants are only in one group (independent samples)
  • Reduces carry-over effects but requires larger sample sizes
• Within-groups design compares the same participants across different conditions
  • Each participant experiences all conditions (repeated measures)
  • More efficient use of participants but may introduce order effects
• Calculation of sum of squares differs between these designs
  • Between-groups SS = $\sum_{i=1}^{k} n_i(\bar{X}_i - \bar{X})^2$
  • Within-groups SS = $\sum_{i=1}^{k}\sum_{j=1}^{n_i} (X_{ij} - \bar{X}_i)^2$

Null and alternative hypotheses

• Null hypothesis (H0) states there are no significant differences between group means
  • H0: μ1 = μ2 = μ3 = ... = μk
• Alternative hypothesis (H1) states at least one group mean differs significantly from the others
  • H1: At least one μi ≠ μj (for i ≠ j)
• ANOVA tests whether between-group variance exceeds within-group variance beyond chance

F-statistic and p-value

• F-statistic represents the ratio of between-group variance to within-group variance
  • F = (Between-group MS) / (Within-group MS)
  • MS (Mean Square) = SS / df (degrees of freedom)
• Large F-values indicate greater between-group differences relative to within-group variability
• p-value derived from F-distribution determines statistical significance
  • p < α (typically 0.05) leads to rejection of the null hypothesis
  • Indicates probability of obtaining observed F-value by chance if null hypothesis is true

Effect size measures

• Eta-squared (η²) measures proportion of total variance explained by the independent variable
  • η² = SSbetween / SStotal
  • Ranges from 0 to 1, with larger values indicating stronger effects
• Partial eta-squared (ηp²) accounts for other variables in more complex designs
  • ηp² = SSeffect / (SSeffect + SSerror)
• Cohen's f provides standardized measure of effect size
  • f = √(η² / (1 - η²))
  • Small effect: f = 0.10, Medium effect: f = 0.25, Large effect: f = 0.40

Two-way ANOVA

• Two-way ANOVA extends one-way ANOVA by incorporating two independent variables, allowing for more complex analyses in reproducible data science
• This technique enables researchers to examine interactions between variables, providing deeper insights into data relationships

Main effects and interactions

• Main effects represent the independent influence of each factor on the dependent variable
  • Calculated by averaging across levels of the other factor
  • Significant main effect indicates one factor affects the outcome regardless of the other factor's level
• Interactions occur when the effect of one factor depends on the level of another factor
  • Visualized as non-parallel lines in interaction plots
  • Significant interaction suggests combined effects of factors differ from their individual effects
• F-tests conducted for each main effect and the interaction effect
  • Main effect A: F = MSA / MSwithin
  • Main effect B: F = MSB / MSwithin
  • Interaction effect: F = MSAB / MSwithin

Factorial designs

• Complete factorial design includes all possible combinations of factor levels
  • 2x2 design has two factors with two levels each, resulting in four groups
  • 3x3 design has two factors with three levels each, resulting in nine groups
• Balanced designs have equal sample sizes across all factor level combinations
  • Simplifies calculations and interpretation of results
  • Increases statistical power and robustness of the analysis
• Unbalanced designs have unequal sample sizes across groups
  • Requires careful consideration of Type I and Type II errors
  • May use different sums of squares methods (Type I, II, or III)

Interpretation of results

• Examine main effects first if interaction is non-significant
  • Interpret each factor's effect independently
  • Report mean differences and effect sizes for significant main effects
• Focus on interaction effect if significant
  • Describe how the effect of one factor changes across levels of the other factor
  • Conduct simple effects analyses to break down the interaction
• Use post-hoc tests for pairwise comparisons within significant effects
  • Tukey's HSD or Bonferroni correction to control for multiple comparisons
• Report F-values, degrees of freedom, p-values, and effect sizes for all effects
  • Include descriptive statistics (means, standard deviations) for each group

Repeated measures ANOVA

• Repeated measures ANOVA is essential in longitudinal studies and within-subjects designs, crucial for tracking changes over time in reproducible research
• This technique increases statistical power by reducing error variance associated with individual differences

Within-subjects designs

• Participants serve as their own controls, reducing between-subjects variability
  • Increases statistical power, requiring fewer participants
  • Allows for detection of smaller effect sizes
• Time-related effects can be examined (learning, fatigue, practice effects)
  • Useful for studying developmental processes or treatment efficacy over time
• Counterbalancing of conditions helps control for order effects
  • Latin square designs or randomized order of treatments
• Calculation of sum of squares accounts for individual differences
  • SSsubjects removes variability due to individual differences from error term

Sphericity assumption

• Sphericity assumes equal variances of differences between all pairs of related groups
  • Similar to homogeneity of variance assumption in between-subjects ANOVA
  • Tested using Mauchly's test of sphericity
• Violation of sphericity leads to increased Type I error rate
  • Epsilon (ε) correction factors adjust degrees of freedom
  • Greenhouse-Geisser correction (conservative) or Huynh-Feldt correction (less conservative)
• Multivariate approach (MANOVA) can be used as an alternative
  • Does not require sphericity assumption
  • May have lower power for small sample sizes

Post-hoc tests

• Pairwise comparisons between time points or conditions
  • Bonferroni correction adjusts p-values for multiple comparisons
  • Sidak correction provides slightly more power than Bonferroni
• Trend analysis examines patterns across time points
  • Linear trends indicate steady increase or decrease
  • Quadratic trends suggest curvilinear relationships
• Contrasts can be used to test specific hypotheses about differences between conditions
  • Planned comparisons have greater power than post-hoc tests
  • Must be specified before data analysis to maintain Type I error control

ANOVA in R

• R provides powerful tools for conducting ANOVA, supporting reproducible and collaborative statistical data science
• Implementing ANOVA in R allows for easy sharing and replication of analyses across research teams

Data preparation

• Import data using appropriate functions (

  read.csv()

  ,

  read_excel()

  )
  • Ensure correct data types for variables (factors for categorical, numeric for continuous)
• Check for missing values and outliers
  • Use

    is.na()

    to identify missing data
  • Create boxplots or use

    IQR()

    to detect outliers
• Verify ANOVA assumptions
  • Shapiro-Wilk test for normality:

    shapiro.test()

  • Levene's test for homogeneity of variance:

    leveneTest()

    from

    car

    package
• Organize data in long format for repeated measures designs
  • Use

    pivot_longer()

    from

    tidyr

    package to reshape data if necessary

Conducting ANOVA tests

• One-way ANOVA using

  aov()

  function

  • model <- aov(dependent_var ~ independent_var, data = dataset)

  • summary(model)

    to view results
• Two-way ANOVA with interaction term

  • model <- aov(dependent_var ~ factor1 * factor2, data = dataset)

• Repeated measures ANOVA using

  ezANOVA()

  from

  ez

  package

  • ezANOVA(data = dataset, dv = .(dependent_var), wid = .(subject_id), within = .(time))

• Post-hoc tests using

  TukeyHSD()

  for pairwise comparisons

  • TukeyHSD(model)

    for multiple comparisons
• Effect size calculation using

  effectsize

  package

  • eta_squared(model)

    or

    cohens_f(model)

    for effect size measures

Visualization of results

• Create interaction plots for two-way ANOVA

  • interaction.plot()

    function in base R

  • ggplot2

    package for more customizable plots
• Box plots to display group differences

  • boxplot()

    in base R or

    geom_boxplot()

    in

    ggplot2

• Mean plots with error bars
  • Use

    ggplot2

    with

    stat_summary()

    to add mean points and error bars
• Residual plots for checking ANOVA assumptions

  • plot(model)

    in base R for diagnostic plots

  • ggResidpanel

    package for ggplot-style residual diagnostics

ANOVA vs other methods

• Understanding the relationship between ANOVA and other statistical methods enhances the ability to choose appropriate analyses in reproducible data science
• Comparing ANOVA to other techniques helps researchers select the most suitable approach for their specific research questions

ANOVA vs t-tests

• ANOVA extends t-test concepts to compare multiple groups simultaneously
  • t-tests limited to comparing two groups at a time
  • ANOVA reduces Type I error rate when making multiple comparisons
• One-way ANOVA with two groups is mathematically equivalent to an independent samples t-test
  • F-statistic in ANOVA equals squared t-statistic from t-test
  • $F = t^2$
• ANOVA provides a more efficient alternative to multiple t-tests
  • Controls overall error rate across all comparisons
  • Allows for examination of interaction effects in factorial designs
• Power analysis considerations differ between ANOVA and t-tests
  • ANOVA typically requires larger sample sizes to detect effects
  • Power in ANOVA depends on number of groups and effect size measure (e.g., Cohen's f)

ANOVA vs regression

• ANOVA can be viewed as a special case of linear regression
  • Both techniques are part of the General Linear Model
  • ANOVA uses categorical predictors, while regression typically uses continuous predictors
• Regression can incorporate both categorical and continuous predictors
  • Allows for more flexible modeling of relationships between variables
  • Can include interaction terms similar to factorial ANOVA designs
• ANOVA results can be obtained through regression analysis
  • Dummy coding of categorical variables in regression yields equivalent results to ANOVA
  • R-squared in regression is equivalent to eta-squared in ANOVA
• ANCOVA (Analysis of Covariance) bridges ANOVA and regression
  • Combines ANOVA with regression by including continuous covariates
  • Allows for adjustment of group means based on covariate values
• Choice between ANOVA and regression depends on research questions and data structure
  • ANOVA focuses on mean differences between groups
  • Regression emphasizes relationships between variables and prediction of outcomes

Assumptions and diagnostics

• Verifying ANOVA assumptions is crucial for ensuring the validity and reproducibility of statistical analyses in data science
• Proper diagnostics help researchers identify potential violations and take appropriate corrective actions

Normality of residuals

• Assumes residuals (differences between observed and predicted values) are normally distributed
  • Check using Q-Q plots of residuals
  • Conduct Shapiro-Wilk test for formal assessment of normality
• Moderate violations generally do not severely impact ANOVA results
  • ANOVA robust to slight deviations from normality, especially with larger sample sizes
• Transformations can be applied to correct non-normality
  • Log transformation for right-skewed data
  • Square root transformation for count data
  • Box-Cox transformation for finding optimal power transformation

Homogeneity of variance

• Assumes equal variances across all groups (homoscedasticity)
  • Tested using Levene's test or Bartlett's test
  • Visual inspection using residual plots against fitted values
• Violation can lead to biased F-tests and increased Type I error
  • More problematic when group sizes are unequal
• Welch's ANOVA provides an alternative for heteroscedastic data
  • Does not assume equal variances
  • Uses weighted sum of squares and adjusted degrees of freedom
• Variance-stabilizing transformations can be applied
  • Log transformation for proportional relationship between mean and variance
  • Arcsine transformation for proportions or percentages

Independence of observations

• Assumes each observation is independent of others
  • No systematic relationship between residuals
  • Crucial for validity of F-tests
• Violated in repeated measures designs or clustered data
  • Use repeated measures ANOVA or mixed-effects models for dependent observations
• Check using Durbin-Watson test for time series data
  • Values close to 2 indicate no autocorrelation
• Plot residuals against time or order of data collection
  • Look for patterns indicating dependence
• Randomization in experimental design helps ensure independence
  • Random assignment to groups
  • Random order of treatments in repeated measures designs

Post-hoc analyses

• Post-hoc analyses are essential in reproducible data science for exploring significant ANOVA results in greater detail
• These techniques help researchers identify specific group differences while controlling for multiple comparisons

Tukey's HSD test

• Honest Significant Difference test compares all possible pairs of means
  • Controls familywise error rate at α level
  • Provides confidence intervals for mean differences
• Based on studentized range distribution
  • Uses critical values from this distribution instead of t-distribution
• Assumes equal sample sizes and homogeneity of variance
  • Relatively robust to moderate violations of these assumptions
• Calculates HSD (Honestly Significant Difference) value
  • HSD = $q_{\alpha,k,df} \sqrt{\frac{MS_{within}}{n}}$
  • q is the studentized range statistic, k is number of groups, df is degrees of freedom for error term
• Pairwise comparisons significant if mean difference exceeds HSD value

Bonferroni correction

• Controls Type I error rate by adjusting p-values for multiple comparisons
  • Divides α level by number of comparisons (α / m)
  • Very conservative, especially with large number of comparisons
• Simple to calculate and widely applicable
  • Can be used with any test statistic (t-tests, correlations)
• May lead to increased Type II error rate (decreased power)
  • More likely to miss true differences, especially with many comparisons
• Modified versions available (Holm's sequential Bonferroni)
  • Offer more power while still controlling Type I error
• Calculation of adjusted p-values
  • p_adjusted = min(1, m * p_original)
  • Where m is the number of comparisons

Planned comparisons

• A priori hypotheses tested using specific contrasts
  • Defined before data collection based on research questions
  • More powerful than post-hoc tests due to focused hypotheses
• Types of contrasts
  • Simple contrasts compare one group to another
  • Complex contrasts compare combinations of groups
• Orthogonal contrasts provide independent tests
  • Sum of products of contrast coefficients equals zero for all pairs
  • Number of orthogonal contrasts equals degrees of freedom between groups
• Non-orthogonal contrasts may require adjustment for multiple comparisons
  • Use Bonferroni or other correction methods
• Calculation of contrast value (L)
  • L = $\sum_{i=1}^{k} c_i \bar{X}_i$
  • Where ci are contrast coefficients and Xi are group means

Reporting ANOVA results

• Proper reporting of ANOVA results is crucial for reproducibility and transparency in statistical data science
• Clear and comprehensive reporting allows other researchers to understand and potentially replicate the analysis

Tables and figures

• ANOVA summary table
  • Include source of variation, degrees of freedom, sum of squares, mean squares, F-values, and p-values
  • Present in APA format or journal-specific style
• Descriptive statistics table
  • Report means, standard deviations, and sample sizes for each group
  • Include confidence intervals for means when relevant
• Main effects plot
  • Display group means with error bars (standard error or confidence intervals)
  • Use different colors or shapes to distinguish between groups
• Interaction plot for factorial designs
  • Show how the effect of one factor changes across levels of another factor
  • Use line graphs with different lines for each level of one factor
• Residual plots
  • Include Q-Q plot for normality check
  • Residuals vs. fitted values plot for homoscedasticity assessment

Interpretation guidelines

• State the research question and hypotheses clearly
  • Relate ANOVA results back to original research objectives
• Report overall ANOVA results
  • Include F-statistic, degrees of freedom, p-value, and effect size
  • Interpret significance in relation to chosen alpha level
• Describe main effects for each factor in factorial designs
  • Explain direction and magnitude of effects
  • Use mean differences to quantify effects
• Interpret interaction effects if present
  • Explain how the effect of one factor depends on levels of another
  • Use simple effects analysis to break down complex interactions
• Discuss post-hoc test results
  • Report specific group differences found to be significant
  • Include adjusted p-values and confidence intervals for pairwise comparisons

Effect size reporting

• Include appropriate effect size measures
  • Eta-squared (η²) or partial eta-squared (ηp²) for proportion of variance explained
  • Cohen's f for standardized measure of effect size
• Interpret effect sizes using established guidelines
  • Small effect: η² ≈ 0.01, f ≈ 0.10
  • Medium effect: η² ≈ 0.06, f ≈ 0.25
  • Large effect: η² ≈ 0.14, f ≈ 0.40
• Report confidence intervals for effect sizes when possible
  • Provides information about precision of effect size estimates
• Discuss practical significance alongside statistical significance
  • Consider real-world implications of observed effect sizes
  • Relate effect sizes to previous findings in the field

Advanced ANOVA techniques

• Advanced ANOVA techniques expand the capabilities of basic ANOVA, allowing for more complex and nuanced analyses in reproducible data science
• These methods provide researchers with tools to address specific research designs and questions that go beyond standard ANOVA applications

ANCOVA

• Analysis of Covariance combines ANOVA with regression analysis
  • Includes continuous covariates to adjust for their effects on the dependent variable
  • Increases statistical power by reducing error variance
• Assumptions include those of ANOVA plus:
  • Linear relationship between covariate and dependent variable
  • Homogeneity of regression slopes across groups
• Applications
  • Controlling for pre-existing differences in experimental designs
  • Adjusting for confounding variables in observational studies
• Interpretation focuses on adjusted means
  • Group means after accounting for covariate effects
  • Allows for more precise comparisons between groups

MANOVA

• Multivariate Analysis of Variance extends ANOVA to multiple dependent variables
  • Analyzes group differences across a combination of dependent variables
  • Controls overall Type I error rate for multiple outcomes
• Uses matrix algebra for calculations
  • Wilks' Lambda, Pillai's Trace, Hotelling's Trace, Roy's Largest Root as test statistics
• Assumptions include ANOVA assumptions plus:
  • Multivariate normality
  • Homogeneity of covariance matrices
• Post-hoc analyses often involve discriminant function analysis
  • Identifies which combination of dependent variables best distinguishes between groups
• Useful in studies with multiple related outcome measures
  • Psychological assessments with multiple subscales
  • Physiological studies measuring various biological markers

Mixed-effects models

• Combine fixed effects (systematic influences) and random effects (random variation)
  • Allow for modeling of hierarchical or nested data structures
  • Account for both within-subject and between-subject variability
• Advantages over traditional repeated measures ANOVA
  • Handle missing data more effectively
  • Allow for unequal time intervals in longitudinal designs
  • Incorporate time-varying covariates
• Specification includes:
  • Fixed effects (similar to standard ANOVA factors)
  • Random effects (e.g., subject-specific intercepts or slopes)
  • Covariance structure for random effects
• Interpretation focuses on:
  • Fixed effects estimates (similar to ANOVA main effects and interactions)
  • Variance components for random effects
  • Model comparisons using likelihood ratio tests or information criteria (AIC, BIC)
• Applications in longitudinal studies, multi-level designs, and clustered data analysis