📈Theoretical Statistics Unit 2 – Random Variables and Probability Distributions

Key Concepts and Definitions

• Random variable assigns a numerical value to each outcome in a sample space
• Probability distribution describes the likelihood of different values occurring for a random variable
• Cumulative distribution function (CDF) gives the probability that a random variable is less than or equal to a specific value
• Probability mass function (PMF) defines the probability of a discrete random variable taking on a specific value
• Probability density function (PDF) describes the relative likelihood of a continuous random variable falling within a particular range of values
  • PDF is used to calculate probabilities for continuous random variables
  • Area under the PDF curve between two points represents the probability of the variable falling within that range
• Expected value (mean) of a random variable is the average value obtained if an experiment is repeated many times
• Variance measures how far a random variable deviates from its expected value

Types of Random Variables

• Discrete random variables can only take on a countable number of distinct values (integers, whole numbers)
  • Examples include the number of heads in a coin toss or the number of defective items in a batch
• Continuous random variables can take on any value within a specified range or interval
  • Commonly represented by real numbers
  • Examples include height, weight, or time taken to complete a task
• Mixed random variables have both discrete and continuous components
• Bernoulli random variable is a special case of a discrete random variable with only two possible outcomes (success or failure)
• Random vectors are ordered collections of random variables
  • Used in multivariate analysis and modeling joint distributions

Probability Distributions

• Probability distributions assign probabilities to the possible values of a random variable
• Discrete probability distributions are used for discrete random variables
  • Probabilities are assigned to each possible value
  • Examples include the binomial distribution and Poisson distribution
• Continuous probability distributions are used for continuous random variables
  • Probabilities are assigned to ranges of values
  • Examples include the normal distribution and exponential distribution
• Joint probability distributions describe the probabilities of multiple random variables occurring together
• Marginal probability distributions are obtained by summing or integrating joint distributions over the values of one or more variables
• Conditional probability distributions describe the probabilities of one random variable given the values of another

Properties of Distributions

• Symmetry indicates that the probability distribution is the same when reflected about a central point
  • Normal distribution is an example of a symmetric distribution
• Skewness measures the asymmetry of a probability distribution
  • Positive skewness has a longer right tail, negative skewness has a longer left tail
• Kurtosis measures the heaviness of the tails of a distribution compared to a normal distribution
  • Higher kurtosis indicates heavier tails and more extreme values
• Moments are quantitative measures that describe the shape and properties of a probability distribution
  • First moment is the mean, second moment is the variance, third moment is skewness, fourth moment is kurtosis
• Moment-generating functions are used to calculate moments and characterize probability distributions

Expectation and Variance

• Expectation (expected value) is the average value of a random variable over many trials
  • For discrete random variables: $E(X) = \sum_{x} x \cdot P(X=x)$
  • For continuous random variables: $E(X) = \int_{-\infty}^{\infty} x \cdot f(x) dx$
• Linearity of expectation states that the expected value of the sum of random variables is the sum of their individual expected values
• Variance measures the average squared deviation from the mean
  • For discrete random variables: $Var(X) = E[(X-E(X))^2] = \sum_{x} (x-E(X))^2 \cdot P(X=x)$
  • For continuous random variables: $Var(X) = E[(X-E(X))^2] = \int_{-\infty}^{\infty} (x-E(X))^2 \cdot f(x) dx$
• Standard deviation is the square root of the variance and measures the average deviation from the mean
• Covariance measures the linear relationship between two random variables
  • Positive covariance indicates variables tend to increase or decrease together, negative covariance indicates an inverse relationship

Common Probability Distributions

• Bernoulli distribution models a single trial with two possible outcomes (success with probability $p$, failure with probability $1-p$)
• Binomial distribution models the number of successes in a fixed number of independent Bernoulli trials
  • Parameters: number of trials $n$, success probability $p$
  • PMF: $P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}$
• Poisson distribution models the number of events occurring in a fixed interval of time or space
  • Parameter: average rate of events $\lambda$
  • PMF: $P(X=k) = \frac{e^{-\lambda}\lambda^k}{k!}$
• Normal (Gaussian) distribution is a continuous probability distribution with a bell-shaped curve
  • Parameters: mean $\mu$, standard deviation $\sigma$
  • PDF: $f(x) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$
• Exponential distribution models the time between events in a Poisson process
  • Parameter: rate parameter $\lambda$
  • PDF: $f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$
• Uniform distribution has equal probability over a specified range
  • Parameters: minimum value $a$, maximum value $b$
  • PDF: $f(x) = \frac{1}{b-a}$ for $a \leq x \leq b$

Transformations of Random Variables

• Linear transformations involve multiplying a random variable by a constant and/or adding a constant
  • If $Y = aX + b$, then $E(Y) = aE(X) + b$ and $Var(Y) = a^2Var(X)$
• Nonlinear transformations change the shape of the probability distribution
  • Examples include exponential, logarithmic, and power transformations
• Convolution is used to find the distribution of the sum of independent random variables
  • For discrete random variables, convolution involves summing the product of the PMFs
  • For continuous random variables, convolution involves integrating the product of the PDFs
• Moment-generating functions can be used to derive the distribution of transformed random variables
• Central Limit Theorem states that the sum of a large number of independent random variables approaches a normal distribution
  • Applies regardless of the original distribution, under certain conditions

Applications and Examples

• Quality control uses the binomial and Poisson distributions to model defects in manufacturing processes
• Insurance companies use the exponential distribution to model the time between claims
• Normal distribution is used in hypothesis testing and confidence interval estimation
• Gaussian mixture models are used in machine learning for clustering and density estimation
• Markov chains model systems that transition between discrete states over time
  • Examples include weather patterns, stock prices, and customer behavior
• Queuing theory uses probability distributions to analyze waiting lines and service systems
  • Applications in call centers, traffic management, and resource allocation
• Monte Carlo simulations use random variables to model complex systems and estimate probabilities
  • Examples include financial risk assessment, particle physics, and engineering design optimization