🔢Lower Division Math Foundations Unit 8 – Probability Theory Basics

Key Concepts and Definitions

• Probability measures the likelihood of an event occurring ranges from 0 (impossible) to 1 (certain)
• Sample space ($S$) set of all possible outcomes of an experiment or random process
• Event ($E$) subset of the sample space represents a specific outcome or set of outcomes
• Mutually exclusive events cannot occur simultaneously in a single trial (rolling a 1 and a 2 on a fair die)
• Exhaustive events collectively cover all possible outcomes in the sample space
  • Example: Rolling a fair die, the events "rolling an even number" and "rolling an odd number" are exhaustive
• Complementary events are mutually exclusive and exhaustive (event $A$ and its complement $A'$)
• Union of events ($A \cup B$) contains all outcomes that belong to either event $A$, event $B$, or both
• Intersection of events ($A \cap B$) contains outcomes common to both event $A$ and event $B$

Sample Spaces and Events

• Defining the sample space is crucial for calculating probabilities and analyzing outcomes
• Sample spaces can be discrete (finite or countably infinite) or continuous (uncountably infinite)
  • Example of a discrete sample space: Tossing a coin (Heads, Tails)
  • Example of a continuous sample space: Measuring the height of a randomly selected person
• Events are often represented using set notation and can be combined using set operations
• The empty set ($\emptyset$) represents an impossible event with a probability of 0
• The power set of a sample space contains all possible subsets (events) of the sample space
• Venn diagrams visually represent relationships between events and their probabilities
• Tree diagrams illustrate sequential events and their associated probabilities
• Counting techniques (permutations, combinations) help determine the size of sample spaces and events

Probability Axioms and Rules

• Axiom 1: Non-negativity - The probability of any event $E$ is non-negative, $P(E) \geq 0$
• Axiom 2: Normalization - The probability of the entire sample space $S$ is 1, $P(S) = 1$
• Axiom 3: Additivity - For any sequence of mutually exclusive events $E_1, E_2, \ldots$, the probability of their union is the sum of their individual probabilities, $P(\bigcup_{i=1}^{\infty} E_i) = \sum_{i=1}^{\infty} P(E_i)$
• Complement Rule: The probability of an event's complement is 1 minus the probability of the event, $P(A') = 1 - P(A)$
• Addition Rule: For any two events $A$ and $B$, $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
  • If $A$ and $B$ are mutually exclusive, $P(A \cup B) = P(A) + P(B)$
• Multiplication Rule: For any two events $A$ and $B$, $P(A \cap B) = P(A) \times P(B|A)$, where $P(B|A)$ is the conditional probability of $B$ given $A$
  • If $A$ and $B$ are independent, $P(A \cap B) = P(A) \times P(B)$

Conditional Probability

• Conditional probability measures the probability of an event $A$ occurring given that another event $B$ has already occurred, denoted as $P(A|B)$
• Formula for conditional probability: $P(A|B) = \frac{P(A \cap B)}{P(B)}$, where $P(B) \neq 0$
• Conditional probability is not commutative, meaning $P(A|B)$ is not necessarily equal to $P(B|A)$
• Bayes' Theorem relates conditional probabilities and helps update probabilities based on new information: $P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$
  • Example: Medical testing and disease diagnosis
• Law of Total Probability states that for a partition of the sample space $\{B_1, B_2, \ldots, B_n\}$, $P(A) = \sum_{i=1}^{n} P(A|B_i) \times P(B_i)$
• Conditional probability is essential for decision-making, risk assessment, and inference in various fields (finance, medicine, machine learning)

Independence and Dependence

• Two events $A$ and $B$ are independent if the occurrence of one does not affect the probability of the other
• Mathematically, $A$ and $B$ are independent if $P(A \cap B) = P(A) \times P(B)$
  • Equivalently, $P(A|B) = P(A)$ and $P(B|A) = P(B)$
• Independent events can occur in any order without changing their joint probability
• Dependent events have probabilities that are influenced by the occurrence of other events
• Conditional probability is used to calculate probabilities for dependent events
• Pairwise independence does not imply mutual independence for three or more events
  • Example: Pairwise independent but mutually dependent fair coin tosses
• Independence is a strong assumption and should be verified before applying in problem-solving

Random Variables

• A random variable ($X$) is a function that assigns a numerical value to each outcome in a sample space
• Random variables can be discrete (countable values) or continuous (uncountable values)
  • Example of a discrete random variable: Number of heads in three coin tosses
  • Example of a continuous random variable: Time taken for a chemical reaction to occur
• The probability distribution of a random variable describes the likelihood of each possible value
• Expected value (mean) of a random variable $X$, denoted $E(X)$, is the average value over many trials
  • For a discrete random variable, $E(X) = \sum_{x} x \times P(X = x)$
  • For a continuous random variable, $E(X) = \int_{-\infty}^{\infty} x \times f(x) dx$, where $f(x)$ is the probability density function
• Variance ($Var(X)$) measures the spread of a random variable around its expected value
  • $Var(X) = E[(X - E(X))^2]$
• Standard deviation ($\sigma$) is the square root of the variance and has the same units as the random variable

Probability Distributions

• A probability distribution assigns probabilities to the possible values of a random variable
• Discrete probability distributions:
  • Bernoulli distribution models a single binary outcome (success or failure)
  • Binomial distribution models the number of successes in a fixed number of independent Bernoulli trials
  • Poisson distribution models the number of rare events occurring in a fixed interval of time or space
• Continuous probability distributions:
  • Uniform distribution assigns equal probabilities to all values within a specified range
  • Normal (Gaussian) distribution is symmetric and bell-shaped, characterized by its mean and standard deviation
  • Exponential distribution models the time between independent events in a Poisson process
• Joint probability distributions describe the probabilities of multiple random variables simultaneously
• Marginal distributions are obtained by summing (discrete) or integrating (continuous) the joint distribution over the other variables
• Conditional distributions describe the probabilities of one random variable given the values of others

Applications and Problem Solving

• Probability theory has wide-ranging applications in science, engineering, finance, and everyday life
• Hypothesis testing uses probability to make decisions based on statistical evidence
  • Example: Determining if a new drug is more effective than a placebo
• Bayesian inference updates prior probabilities based on new data to obtain posterior probabilities
  • Example: Spam email filtering based on word frequencies
• Markov chains model systems that transition between states based on conditional probabilities
  • Example: Predicting weather patterns or stock prices
• Queuing theory applies probability to analyze waiting lines and optimize service systems
  • Example: Determining the optimal number of servers in a call center
• Monte Carlo simulations use random sampling to estimate probabilities and solve complex problems
  • Example: Estimating the value of pi by randomly throwing darts at a square
• Risk assessment quantifies the likelihood and impact of adverse events
  • Example: Calculating the probability of a nuclear power plant accident
• Probabilistic graphical models (Bayesian networks, Markov random fields) represent dependencies among random variables for reasoning and inference