👩‍💻Foundations of Data Science Unit 7 – Statistical Inference & Hypothesis Testing

Key Concepts

• Statistical inference draws conclusions about a population based on a sample of data
• Hypothesis testing evaluates claims or conjectures about a population using sample data
• Null hypothesis ($H_0$) represents the default or status quo, while the alternative hypothesis ($H_a$ or $H_1$) represents the claim being tested
• P-value measures the probability of observing the sample data or more extreme results, assuming the null hypothesis is true
• Significance level ($\alpha$) is the threshold for rejecting the null hypothesis, commonly set at 0.05
• Type I error (false positive) occurs when rejecting a true null hypothesis, while Type II error (false negative) occurs when failing to reject a false null hypothesis
• Statistical power is the probability of correctly rejecting a false null hypothesis
• Effect size measures the magnitude of the difference or relationship between variables

Probability Basics

• Probability quantifies the likelihood of an event occurring, ranging from 0 (impossible) to 1 (certain)
• Sample space is the set of all possible outcomes of an experiment or random process
• Events are subsets of the sample space, representing specific outcomes or combinations of outcomes
• Probability distributions describe the likelihood of different outcomes for a random variable
  • Discrete probability distributions (binomial, Poisson) are used for countable outcomes
  • Continuous probability distributions (normal, exponential) are used for measurable outcomes
• Expected value is the average outcome of a random variable over many trials, calculated as the sum of each outcome multiplied by its probability
• Variance and standard deviation measure the spread or dispersion of a probability distribution around its expected value

Types of Statistical Inference

• Parameter estimation involves using sample data to estimate unknown population parameters (mean, proportion, standard deviation)
  • Point estimation provides a single value estimate of a parameter (sample mean, sample proportion)
  • Interval estimation provides a range of plausible values for a parameter, with a specified level of confidence (confidence intervals)
• Hypothesis testing involves using sample data to test claims or conjectures about population parameters
  • One-sample tests compare a sample statistic to a hypothesized population parameter (one-sample t-test, one-proportion z-test)
  • Two-sample tests compare statistics from two independent samples (two-sample t-test, two-proportion z-test)
  • Paired tests compare two measurements on the same individuals or matched pairs (paired t-test, McNemar's test)
• Regression analysis explores the relationship between a dependent variable and one or more independent variables
  • Simple linear regression models the relationship between two continuous variables using a straight line
  • Multiple linear regression extends simple linear regression to include multiple independent variables

Hypothesis Testing Fundamentals

• State the null and alternative hypotheses in terms of population parameters
  • One-tailed alternative hypotheses specify a direction (greater than or less than)
  • Two-tailed alternative hypotheses do not specify a direction (not equal to)
• Choose an appropriate test statistic based on the type of data and the hypothesis being tested
• Calculate the test statistic using sample data and compare it to the critical value or p-value
• Make a decision to reject or fail to reject the null hypothesis based on the comparison
• Interpret the results in the context of the original research question or problem
• Consider the practical significance of the findings, not just the statistical significance
• Report the results, including the test statistic, p-value, and confidence interval (if applicable)
• Discuss the limitations and potential sources of bias in the study design or data collection

Common Statistical Tests

• One-sample t-test compares a sample mean to a hypothesized population mean
• Two-sample t-test compares the means of two independent samples
• Paired t-test compares two measurements on the same individuals or matched pairs
• One-proportion z-test compares a sample proportion to a hypothesized population proportion
• Two-proportion z-test compares the proportions of two independent samples
• Chi-square goodness-of-fit test compares observed frequencies to expected frequencies for categorical data
• Chi-square test of independence assesses the relationship between two categorical variables
• Analysis of variance (ANOVA) compares the means of three or more groups
  • One-way ANOVA tests the effect of one categorical independent variable on a continuous dependent variable
  • Two-way ANOVA tests the effects of two categorical independent variables and their interaction on a continuous dependent variable

Interpreting Results

• Determine if the null hypothesis is rejected or not rejected based on the p-value and significance level
• Interpret the direction and magnitude of the effect, if applicable (positive or negative, strong or weak)
• Consider the confidence interval for the parameter estimate, which provides a range of plausible values
• Assess the practical significance of the findings, not just the statistical significance
  • Statistical significance indicates that the results are unlikely to be due to chance alone
  • Practical significance considers the magnitude and relevance of the effect in the real-world context
• Discuss the limitations and potential sources of bias in the study design or data collection
• Consider alternative explanations for the findings and suggest future research directions
• Communicate the results clearly and accurately, avoiding overgeneralization or misinterpretation

Real-World Applications

• A/B testing in marketing compares the effectiveness of two versions of a website or advertisement
• Clinical trials in medicine evaluate the safety and efficacy of new drugs or treatments
• Quality control in manufacturing tests whether products meet specified standards or requirements
• Polling and surveys in social sciences and market research estimate population opinions or preferences
• Environmental monitoring tests for differences in pollution levels or species abundance across locations or time periods
• Psychological research tests hypotheses about human behavior, cognition, or emotion
• Educational research evaluates the effectiveness of teaching methods or interventions on student learning outcomes
• Economic analysis tests hypotheses about market trends, consumer behavior, or policy impacts

Common Pitfalls and Misconceptions

• Confusing statistical significance with practical significance or importance
• Interpreting a failure to reject the null hypothesis as proof that the null hypothesis is true
• Overgeneralizing the results beyond the specific population or context studied
• Assuming that correlation implies causation without considering potential confounding variables
• Relying on small sample sizes that may not be representative of the population
• Conducting multiple hypothesis tests without adjusting the significance level for the increased risk of Type I errors
• Selectively reporting or interpreting results that support a desired conclusion (confirmation bias)
• Failing to consider the assumptions underlying the statistical tests and whether they are met by the data
• Misinterpreting the meaning of confidence intervals or p-values
• Overemphasizing the importance of a single study without considering the broader context of related research