📊Probabilistic Decision-Making Unit 6 – Hypothesis Testing: Single & Dual Populations

Key Concepts and Definitions

• Hypothesis testing a statistical method used to make decisions or draw conclusions about a population based on sample data
• Null hypothesis ($H_0$) represents the default or status quo assumption, typically stating no significant difference or effect
• Alternative hypothesis ($H_A$ or $H_1$) represents the claim or research question, suggesting a significant difference or effect
• Type I error (false positive) occurs when rejecting a true null hypothesis, denoted by $\alpha$ (significance level)
• Type II error (false negative) occurs when failing to reject a false null hypothesis, denoted by $\beta$
  • Power of a test ($1 - \beta$) represents the probability of correctly rejecting a false null hypothesis
• Test statistic a calculated value used to compare with a critical value or p-value to make a decision about the null hypothesis
• Critical value a threshold value determined by the significance level and the distribution of the test statistic under the null hypothesis
• P-value the probability of observing a test statistic as extreme as or more extreme than the one calculated, assuming the null hypothesis is true

Foundations of Hypothesis Testing

• Hypothesis testing relies on the principles of probability and sampling distributions
• The central limit theorem states that the sampling distribution of the mean approaches a normal distribution as the sample size increases, regardless of the population distribution
• Hypothesis tests assume random sampling, independence of observations, and a large enough sample size (typically $n \geq 30$)
• The significance level ($\alpha$) is determined before conducting the test and represents the maximum acceptable probability of making a Type I error
• The choice of $\alpha$ depends on the consequences of making a Type I error and the desired power of the test
• Hypothesis tests can be one-tailed (directional) or two-tailed (non-directional), depending on the alternative hypothesis
• The null and alternative hypotheses are mutually exclusive and exhaustive, covering all possible outcomes

Types of Hypotheses

• One-sample hypotheses test a claim about a single population parameter (mean, proportion, or variance)
  • Example: Testing if the average weight of a product differs from a specified value
• Two-sample hypotheses compare parameters between two independent populations
  • Example: Comparing the mean scores of two different teaching methods
• Paired-sample hypotheses test the difference between paired observations or repeated measures
  • Example: Comparing the effectiveness of a drug before and after treatment on the same individuals
• Analysis of Variance (ANOVA) tests the equality of means among three or more populations
  • Example: Comparing the average yield of multiple fertilizer treatments
• Chi-square tests assess the association between two categorical variables
  • Example: Testing the relationship between gender and preference for a product

Single Population Tests

• Z-test for a population mean ($\mu$) when the population standard deviation ($\sigma$) is known
  • Test statistic: $Z = \frac{\bar{X} - \mu_0}{\sigma / \sqrt{n}}$
• T-test for a population mean ($\mu$) when the population standard deviation ($\sigma$) is unknown
  • Test statistic: $t = \frac{\bar{X} - \mu_0}{s / \sqrt{n}}$, where $s$ is the sample standard deviation
• Z-test for a population proportion ($p$)
  • Test statistic: $Z = \frac{\hat{p} - p_0}{\sqrt{p_0(1 - p_0) / n}}$, where $\hat{p}$ is the sample proportion
• Chi-square test for a population variance ($\sigma^2$)
  • Test statistic: $\chi^2 = \frac{(n - 1)s^2}{\sigma_0^2}$

Dual Population Tests

• Two-sample Z-test for comparing means ($\mu_1$ and $\mu_2$) when population variances are known
  • Test statistic: $Z = \frac{(\bar{X}_1 - \bar{X}_2) - (\mu_1 - \mu_2)_0}{\sqrt{\sigma_1^2 / n_1 + \sigma_2^2 / n_2}}$
• Two-sample T-test for comparing means ($\mu_1$ and $\mu_2$) when population variances are unknown but assumed equal
  • Test statistic: $t = \frac{(\bar{X}_1 - \bar{X}_2) - (\mu_1 - \mu_2)_0}{s_p \sqrt{1/n_1 + 1/n_2}}$, where $s_p$ is the pooled standard deviation
• Welch's T-test for comparing means ($\mu_1$ and $\mu_2$) when population variances are unknown and unequal
  • Test statistic: $t = \frac{(\bar{X}_1 - \bar{X}_2) - (\mu_1 - \mu_2)_0}{\sqrt{s_1^2 / n_1 + s_2^2 / n_2}}$
• Two-proportion Z-test for comparing proportions ($p_1$ and $p_2$)
  • Test statistic: $Z = \frac{(\hat{p}_1 - \hat{p}_2) - (p_1 - p_2)_0}{\sqrt{\hat{p}(1 - \hat{p})(1/n_1 + 1/n_2)}}$, where $\hat{p}$ is the pooled sample proportion
• Paired T-test for comparing means of paired observations or repeated measures
  • Test statistic: $t = \frac{\bar{D} - \mu_{D_0}}{s_D / \sqrt{n}}$, where $\bar{D}$ is the mean difference and $s_D$ is the standard deviation of the differences

Test Statistics and Critical Values

• Test statistics are calculated from sample data and used to make decisions about the null hypothesis
• The distribution of the test statistic under the null hypothesis determines the critical value(s) or p-value
• For Z-tests, the test statistic follows a standard normal distribution (Z-distribution)
• For T-tests, the test statistic follows a T-distribution with degrees of freedom ($df$) based on the sample size(s)
• For Chi-square tests, the test statistic follows a Chi-square distribution with degrees of freedom ($df$) based on the sample size and number of parameters estimated
• Critical values are determined by the significance level ($\alpha$) and the type of test (one-tailed or two-tailed)
  • For a two-tailed test, the critical values are located at the $\alpha/2$ and $1 - \alpha/2$ percentiles of the distribution
  • For a one-tailed test, the critical value is located at the $\alpha$ or $1 - \alpha$ percentile, depending on the direction of the alternative hypothesis

Interpreting Results and Decision Making

• Compare the calculated test statistic with the critical value(s) or p-value to make a decision about the null hypothesis
• If the test statistic falls in the rejection region (beyond the critical value) or the p-value is less than the significance level ($\alpha$), reject the null hypothesis in favor of the alternative hypothesis
• If the test statistic falls in the non-rejection region (within the critical values) or the p-value is greater than the significance level ($\alpha$), fail to reject the null hypothesis
• Rejecting the null hypothesis suggests that there is sufficient evidence to support the alternative hypothesis
• Failing to reject the null hypothesis does not prove that the null hypothesis is true, but rather that there is insufficient evidence to support the alternative hypothesis
• The decision to reject or fail to reject the null hypothesis should be interpreted in the context of the research question and the practical significance of the results
• Confidence intervals can be constructed to estimate the range of plausible values for the population parameter(s) based on the sample data and the significance level

Real-World Applications

• Quality control testing the proportion of defective items in a production process to ensure it meets the desired specifications
• Clinical trials comparing the effectiveness of a new drug to a placebo or an existing treatment
• Market research testing the preference for a new product feature among different consumer segments
• Educational research comparing the performance of students under different teaching methods or curricula
• Environmental studies testing the difference in pollutant levels between two locations or time periods
• Psychological research comparing the mean scores of participants in different experimental conditions
• Financial analysis testing the difference in returns between two investment strategies
• Social science research testing the association between demographic variables and attitudes or behaviors