🎲Data, Inference, and Decisions Unit 11 – Nonparametric & Robust Methods

What's the deal with nonparametric methods?

• Nonparametric methods make no assumptions about the underlying distribution of the data
• Useful when the data does not follow a normal distribution or when the sample size is small
• Rely on the rank order of the data rather than the actual values
• Can be more robust to outliers and extreme values compared to parametric methods
• Applicable to a wide range of data types, including ordinal and nominal data
• Provide a flexible alternative to parametric methods when assumptions are not met
• May have lower statistical power compared to parametric methods when assumptions are satisfied

Key concepts you need to know

• Rank-based tests assign ranks to the data points and analyze the ranks instead of the actual values
• Median is often used as a measure of central tendency in nonparametric methods
• Wilcoxon rank-sum test (Mann-Whitney U test) compares two independent samples
  • Null hypothesis: The two samples come from the same population
  • Alternative hypothesis: The two samples come from different populations
• Wilcoxon signed-rank test is used for paired or matched samples
• Kruskal-Wallis test is an extension of the Wilcoxon rank-sum test for comparing three or more groups
• Spearman's rank correlation coefficient measures the monotonic relationship between two variables
• Kendall's tau is another measure of rank correlation, more robust to ties in the data

Common nonparametric tests

• Sign test compares the median of a sample to a hypothesized value
• Runs test checks for randomness in a sequence of binary data
• Kolmogorov-Smirnov test compares the cumulative distribution functions of two samples
  • Used to test if two samples come from the same distribution
  • Can also be used to test if a sample comes from a specified distribution
• Friedman test is a nonparametric alternative to the repeated measures ANOVA
• Cochran's Q test is used for testing the equality of proportions in matched samples
• McNemar's test is used to compare paired proportions, often in before-after studies
• Chi-square test is used for testing the association between categorical variables

Robust statistics: When data gets messy

• Robust statistics aim to provide reliable results in the presence of outliers or deviations from assumptions
• Trimmed mean is a robust measure of central tendency that removes a specified percentage of the highest and lowest values
• Winsorized mean replaces the extreme values with the nearest non-extreme values instead of removing them
• Median absolute deviation (MAD) is a robust measure of dispersion, less sensitive to outliers than the standard deviation
• Huber's M-estimator is a robust alternative to the sample mean, minimizing the impact of outliers
  • Assigns weights to observations based on their distance from the center of the data
  • Observations far from the center receive lower weights
• Robust regression methods (Theil-Sen estimator) are less affected by outliers in the response variable
• Robust PCA (principal component analysis) can handle data with outliers or heavy-tailed distributions

Real-world applications

• Analyzing customer satisfaction surveys with Likert scale responses (ordinal data)
• Comparing the effectiveness of different treatments in a clinical trial with a small sample size
• Detecting anomalies or fraud in financial transactions using robust statistics
• Analyzing the impact of a new educational program on student performance, accounting for outliers
• Investigating the relationship between air pollution levels and respiratory illnesses in a city
  • Nonparametric methods can handle the non-normal distribution of pollutant concentrations
  • Robust statistics can account for extreme pollution events or measurement errors
• Comparing the preferences of different consumer groups for a new product using rank-based tests
• Evaluating the association between socioeconomic factors and health outcomes in a population

Pros and cons of nonparametric methods

Pros:

• Require fewer assumptions about the underlying distribution of the data
• Can handle a wide range of data types, including ordinal and nominal data
• More robust to outliers and extreme values compared to parametric methods
• Provide valid results even when the sample size is small or the data is not normally distributed
• Easy to understand and interpret, as they often rely on intuitive concepts like ranks

Cons:

• May have lower statistical power compared to parametric methods when assumptions are satisfied
• Some nonparametric tests may be less efficient than their parametric counterparts
• Results may be more difficult to generalize to the population, as they are based on the sample at hand
• May not provide quantitative estimates of effect sizes or confidence intervals
• Some nonparametric tests may be computationally intensive, especially for large datasets

Tools and software for analysis

• R programming language offers a wide range of nonparametric and robust methods through various packages

  • stats

    package includes basic nonparametric tests like Wilcoxon rank-sum and Kruskal-Wallis

  • robustbase

    package provides robust statistical methods, such as Huber's M-estimator and robust PCA

  • WRS2

    package offers robust statistical methods for comparing groups and measuring effect sizes
• Python's

  scipy.stats

  module includes several nonparametric tests, such as the Mann-Whitney U test and the Friedman test
• SPSS and SAS provide a range of nonparametric tests through their graphical user interfaces and programming languages
• Minitab offers a user-friendly interface for conducting nonparametric tests and robust statistical analyses
• Stata includes a variety of nonparametric and robust methods, accessible through its command-line interface

Tricky bits and how to tackle them

• Choosing the appropriate nonparametric test can be challenging, especially when dealing with complex study designs
  • Consider the type of data, the number of groups, and the research question to guide your choice
  • Consult with a statistician or refer to reliable sources when in doubt
• Interpreting the results of nonparametric tests may require a different approach compared to parametric methods
  • Focus on the median and interquartile range instead of the mean and standard deviation
  • Use rank-based effect sizes (Cliff's delta) to quantify the magnitude of the difference between groups
• Dealing with ties in rank-based tests can be problematic, as it may affect the test's validity and power
  • Use tie-corrected versions of the tests when available (Wilcoxon rank-sum test with continuity correction)
  • Consider alternative tests that are less sensitive to ties, such as the Brunner-Munzel test
• Robust methods may not always be the best choice, especially when the data is well-behaved and the assumptions are met
  • Compare the results of robust methods with their parametric counterparts to assess the impact of outliers or deviations from assumptions
  • Use diagnostic plots (QQ-plots) and tests (Shapiro-Wilk) to check the assumptions of parametric methods before deciding on a robust alternative