3.3 Expectation, variance, and moments of discrete random variables
5 min read•august 14, 2024
Discrete random variables are the building blocks of probability theory. They help us understand and predict outcomes in situations with a finite number of possibilities, like coin flips or dice rolls.
Expectation, , and moments are key tools for analyzing discrete random variables. They give us insights into the average outcome, spread of values, and overall shape of the probability distribution, helping us make informed decisions in uncertain situations.
Expected Value of Discrete Variables
Calculating the Expected Value (Mean)
The (mean) of a discrete random variable X is denoted by E(X) and is calculated as the sum of the product of each possible value of X and its corresponding probability
The formula for the expected value of a discrete random variable X is E(X)=Σx⋅P(X=x), where x represents each possible value of X, and P(X=x) is the probability of X taking the value x
For example, if X represents the number of heads in two coin flips, with possible values of 0, 1, and 2, and corresponding probabilities of 0.25, 0.5, and 0.25, then E(X)=0⋅0.25+1⋅0.5+2⋅0.25=1
The expected value represents the average value of the random variable over a large number of trials or observations
In the coin flip example, if we repeat the two coin flips many times, the average number of heads observed will approach 1
The expected value is a weighted average, where each value is weighted by its probability of occurrence
Properties of Expected Value
The expected value is a linear operator, meaning that for any constants a and b, and random variables X and Y, E(aX+bY)=aE(X)+bE(Y)
For instance, if X and Y are independent random variables representing the numbers rolled on two fair dice, then E(2X+3Y)=2E(X)+3E(Y)=2⋅3.5+3⋅3.5=17.5
The holds regardless of whether the random variables are independent or not
The expected value of a constant c is the constant itself, i.e., E(c)=c
Variance and Standard Deviation of Discrete Variables
Calculating Variance and Standard Deviation
The variance of a discrete random variable X, denoted by Var(X) or σ2, measures the average squared deviation of the random variable from its expected value
The formula for the variance of a discrete random variable X is Var(X)=E[(X−E(X))2]=Σ(x−E(X))2⋅P(X=x), where x represents each possible value of X, and P(X=x) is the probability of X taking the value x
For example, if X represents the number of heads in two coin flips, with E(X)=1, then Var(X)=(0−1)2⋅0.25+(1−1)2⋅0.5+(2−1)2⋅0.25=0.5
The variance can also be calculated using the alternative formula Var(X)=E(X2)−[E(X)]2, where E(X2) is the expected value of the squared random variable X
In the coin flip example, E(X2)=02⋅0.25+12⋅0.5+22⋅0.25=1.5, so Var(X)=1.5−12=0.5
The standard deviation of a discrete random variable X, denoted by σ or Var(X), is the square root of the variance and measures the average deviation of the random variable from its expected value
In the coin flip example, σ=0.5≈0.707
Interpreting Variance and Standard Deviation
The variance and standard deviation provide a measure of the dispersion or spread of the random variable around its expected value
A higher variance or standard deviation indicates that the values of the random variable are more spread out from the mean, while a lower variance or standard deviation suggests that the values are more concentrated around the mean
The standard deviation has the same units as the random variable, making it easier to interpret than the variance, which has squared units
Moments of Distributions
Defining Moments
Moments are mathematical quantities that describe the shape and characteristics of a probability distribution
The n-th moment of a discrete random variable X is defined as E(Xn), where n is a non-negative integer
For example, the first moment (n = 1) of a fair six-sided die is E(X)=1⋅61+2⋅61+…+6⋅61=3.5
The first moment (n=1) is the expected value or mean of the random variable, E(X)
The second moment (n=2) is related to the variance of the random variable, as Var(X)=E(X2)−[E(X)]2
Higher-Order Moments and Their Significance
Higher-order moments (n>2) provide additional information about the shape of the distribution
The third moment is related to the skewness of the distribution, which measures the asymmetry of the distribution around its mean
A positive skewness indicates that the tail on the right side of the distribution is longer or fatter than the left side, while a negative skewness indicates the opposite
The fourth moment is related to the kurtosis of the distribution, which measures the heaviness of the tails and the peakedness of the distribution relative to a normal distribution
A higher kurtosis indicates heavier tails and a more peaked distribution, while a lower kurtosis indicates lighter tails and a flatter distribution
Moments can be used to compare and contrast different probability distributions and to estimate the parameters of a distribution from sample data
Linearity of Expectation
The Linearity Property
The linearity of expectation states that the expected value of the sum of random variables is equal to the sum of their individual expected values, regardless of whether the random variables are independent or not
Mathematically, for any random variables X and Y, E(X+Y)=E(X)+E(Y), and more generally, for any constants a and b, E(aX+bY)=aE(X)+bE(Y)
For example, if X and Y are the numbers rolled on two fair dice, then E(X+Y)=E(X)+E(Y)=3.5+3.5=7
The linearity of expectation allows for the calculation of the expected value of a sum of random variables without the need to determine their joint distribution or dependence structure
Applying Linearity of Expectation
The linearity of expectation is particularly useful when solving problems involving multiple random variables, as it simplifies the calculation of the expected value of their sum
For instance, in a game where a player rolls two fair dice and receives a payoff equal to the sum of the numbers rolled, the expected payoff can be easily calculated as E(X+Y)=E(X)+E(Y)=3.5+3.5=7
The linearity of expectation can be extended to any finite number of random variables, i.e., for random variables X1,X2,…,Xn, E(X1+X2+…+Xn)=E(X1)+E(X2)+…+E(Xn)
This property is useful in various applications, such as calculating the expected total waiting time in a queue with multiple independent service times or the expected total score in a game with multiple independent rounds
Key Terms to Review (13)
Bernoulli Distribution: The Bernoulli distribution is a discrete probability distribution that describes a random variable which has two possible outcomes: success (often represented as 1) and failure (often represented as 0). This distribution is foundational in probability and statistics, particularly in understanding events that can be modeled as yes/no or true/false scenarios, which connects to various concepts like independence, data analysis, and other common discrete distributions.
Binomial Random Variable: A binomial random variable is a type of discrete random variable that counts the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success. This concept is crucial for understanding how outcomes from these trials can be mathematically modeled and analyzed, revealing properties like distribution shape and expected values.
Central Moment: A central moment is a statistical measure that captures the average of the deviations of a random variable from its mean, raised to a certain power. Central moments provide valuable insights into the shape and characteristics of a probability distribution, particularly focusing on variance and higher moments such as skewness and kurtosis. The second central moment is especially important as it quantifies the variance, while higher-order central moments reveal additional information about the distribution's behavior.
E(x) = σ [x * p(x)]: The expression e(x) = σ [x * p(x)] represents the expected value of a discrete random variable, which is a fundamental concept in probability theory. This formula shows that the expected value is calculated by summing the product of each possible outcome of the random variable (x) and its corresponding probability (p(x)). Understanding this concept is crucial for evaluating the average or typical outcome of random processes, as well as for calculating variance and moments of the random variable.
Expected Value: Expected value is a fundamental concept in probability and statistics that provides a measure of the center of a random variable's distribution, representing the average outcome one would anticipate from an experiment if it were repeated many times. It connects to various aspects of probability theory, including the behaviors of discrete random variables, how probabilities are assigned through probability mass functions, and how to derive characteristics through moment-generating functions.
Geometric Distribution: The geometric distribution models the number of trials required to achieve the first success in a series of independent Bernoulli trials. It is a key concept in understanding the behavior of discrete random variables, particularly in contexts where events are repeated until a desired outcome occurs.
Linearity of Expectation: Linearity of expectation is a fundamental property in probability theory stating that the expected value of the sum of random variables is equal to the sum of their expected values, regardless of whether the variables are independent or dependent. This property simplifies calculations involving expectations, especially when dealing with complex random processes, by allowing you to add expectations directly.
N-th moment: The n-th moment of a random variable is a measure that provides information about its distribution, specifically how the values of the variable differ from the mean. It is defined mathematically as the expected value of the n-th power of the deviation of the random variable from its mean, which can be expressed as $$E[(X - ext{E}[X])^n]$$ for a random variable X. Moments are crucial because they help in understanding various properties like expectation, variance, and higher-order characteristics of distributions.
Poisson Random Variable: A Poisson random variable is a type of discrete random variable that models the number of events occurring within a fixed interval of time or space, where these events happen with a known constant mean rate and are independent of the time since the last event. This variable is often used in scenarios such as counting occurrences, like phone calls received at a call center or the number of decay events from a radioactive source, making it essential for understanding probabilities and expectations related to rare events.
Properties of Variance: Properties of variance refer to the key characteristics and rules that describe how variance behaves in statistical contexts, particularly when dealing with random variables. Understanding these properties is crucial for manipulating and interpreting variance calculations, especially in the context of discrete random variables where outcomes and their probabilities are defined. Variance provides insight into the spread or dispersion of data points around their mean, helping to assess the reliability and variability of predictions based on those random variables.
Risk Assessment: Risk assessment is the systematic process of evaluating the potential risks that may be involved in a projected activity or undertaking. This involves identifying hazards, analyzing potential consequences, and determining the likelihood of those consequences occurring, which connects deeply to understanding probabilities and making informed decisions based on various outcomes.
Statistical Inference: Statistical inference is the process of using data from a sample to make generalizations or predictions about a larger population. This concept relies on probability theory and provides tools for estimating population parameters, testing hypotheses, and making decisions based on data. It connects closely with concepts such as expectation, variance, and moments to quantify uncertainty, while also linking marginal and conditional distributions to analyze the relationships between different random variables.
Variance: Variance is a statistical measurement that describes the spread of a set of values in a dataset. It indicates how much individual data points differ from the mean (average) of the dataset, providing insight into the level of variability or consistency within that set. Understanding variance is crucial for analyzing both discrete and continuous random variables and their distributions.