5.4 Conditional distributions

Conditional distributions are a powerful tool in probability theory, allowing us to update our understanding of random events based on new information. They help us make more precise predictions and analyses by focusing on specific subsets of outcomes.

By using conditional distributions, we can calculate probabilities, expectations, and variances that take into account known conditions. This approach is crucial in various fields, from weather forecasting to medical diagnosis, enabling more accurate decision-making and risk assessment.

Definition of conditional distributions

• Conditional distributions describe the probability distribution of a random variable given specific information or conditions about another random variable
• Provides a way to update probabilities based on new information or evidence
• Allows for more precise and targeted analysis by focusing on a subset of outcomes that meet certain criteria

Notation for conditional distributions

• Conditional probability is denoted as $[P(A|B)](https://www.fiveableKeyTerm:p(a|b))$, which represents the probability of event A occurring given that event B has occurred
• For random variables, the conditional probability distribution of X given Y is denoted as $f_{X|Y}(x|y)$ or $P(X=x|Y=y)$
• The vertical bar "|" is used to separate the event or variable of interest from the conditioning event or variable

Conditional probability mass functions

Discrete random variables

Top images from around the web for Discrete random variables

• 

  Discrete Random Variables (3 of 5) | Concepts in Statistics View original

  Is this image relevant?

• 

  Discrete Random Variables | Boundless Statistics View original

  Is this image relevant?

• 

  Discrete Random Variables (5 of 5) | Concepts in Statistics View original

  Is this image relevant?

• 

  Discrete Random Variables (3 of 5) | Concepts in Statistics View original

  Is this image relevant?

• 

  Discrete Random Variables | Boundless Statistics View original

  Is this image relevant?

1 of 3

Top images from around the web for Discrete random variables

• 

  Discrete Random Variables (3 of 5) | Concepts in Statistics View original

  Is this image relevant?

• 

  Discrete Random Variables | Boundless Statistics View original

  Is this image relevant?

• 

  Discrete Random Variables (5 of 5) | Concepts in Statistics View original

  Is this image relevant?

• 

  Discrete Random Variables (3 of 5) | Concepts in Statistics View original

  Is this image relevant?

• 

  Discrete Random Variables | Boundless Statistics View original

  Is this image relevant?

1 of 3

• For discrete random variables, the conditional probability mass function (PMF) is used to describe the probability distribution given a specific condition
• The conditional PMF is defined as $P(X=x|Y=y) = \frac{P(X=x, Y=y)}{P(Y=y)}$, where $P(X=x, Y=y)$ is the joint probability of X and Y, and $P(Y=y)$ is the marginal probability of Y
• The conditional PMF satisfies the properties of a valid probability distribution, such as non-negativity and summing up to 1 over all possible values of X

Conditional probability density functions

Continuous random variables

• For continuous random variables, the conditional probability density function (PDF) is used to describe the probability distribution given a specific condition
• The conditional PDF is defined as $f_{X|Y}(x|y) = \frac{f_{X,Y}(x,y)}{f_Y(y)}$, where $f_{X,Y}(x,y)$ is the joint PDF of X and Y, and $f_Y(y)$ is the marginal PDF of Y
• The conditional PDF can be used to calculate probabilities and expectations for the random variable X given a specific value of Y

Independence and conditional distributions

• Two random variables X and Y are considered independent if their conditional distribution is equal to their marginal distribution
• Mathematically, X and Y are independent if $f_{X|Y}(x|y) = f_X(x)$ for all values of x and y, where $f_X(x)$ is the marginal PDF of X
• If X and Y are independent, then knowing the value of one variable does not provide any information about the other variable

Conditional expectation

Definition and properties

• The conditional expectation of a random variable X given another random variable Y is the expected value of X calculated using the conditional distribution of X given Y
• Denoted as $E[X|Y]$, the conditional expectation is a function of Y and can be calculated using the formula $E[X|Y=y] = \sum_{x} x \cdot P(X=x|Y=y)$ for discrete random variables or $E[X|Y=y] = \int_{-\infty}^{\infty} x \cdot f_{X|Y}(x|y) dx$ for continuous random variables
• Conditional expectation satisfies linearity properties, such as $E[aX+b|Y] = aE[X|Y] + b$ for constants a and b

Conditional expectation vs unconditional expectation

• The unconditional expectation (or simply expectation) of a random variable X is its average value over its entire range, denoted as $E[X]$
• The conditional expectation $E[X|Y]$ is a function of Y and represents the average value of X for each specific value of Y
• The law of total expectation states that $E[X] = E[E[X|Y]]$, meaning that the unconditional expectation can be obtained by taking the expectation of the conditional expectation over the distribution of Y

Conditional variance

Definition and properties

• The conditional variance of a random variable X given another random variable Y is the variance of X calculated using the conditional distribution of X given Y
• Denoted as $Var(X|Y)$, the conditional variance is a function of Y and can be calculated using the formula $Var(X|Y=y) = E[X^2|Y=y] - (E[X|Y=y])^2$
• Conditional variance satisfies properties similar to unconditional variance, such as $Var(aX+b|Y) = a^2Var(X|Y)$ for constants a and b

Conditional variance vs unconditional variance

• The unconditional variance (or simply variance) of a random variable X is a measure of its variability over its entire range, denoted as $Var(X)$
• The conditional variance $Var(X|Y)$ is a function of Y and represents the variability of X for each specific value of Y
• The law of total variance states that $Var(X) = E[Var(X|Y)] + Var(E[X|Y])$, which decomposes the unconditional variance into two components: the average conditional variance and the variance of the conditional expectation

Bayes' theorem

Statement of Bayes' theorem

• Bayes' theorem is a fundamental result in probability theory that describes how to update probabilities based on new evidence or information
• The theorem states that $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$, where A and B are events, $P(A)$ is the prior probability of A, $P(B|A)$ is the likelihood of B given A, and $P(B)$ is the marginal probability of B
• Bayes' theorem allows for the calculation of posterior probabilities, which are the updated probabilities of an event after considering new evidence

Applications of Bayes' theorem

• Bayes' theorem has numerous applications in various fields, such as machine learning, medical diagnosis, and spam email filtering
• In machine learning, Bayes' theorem is used in naive Bayes classifiers to predict the class of an instance based on its features
• Medical diagnosis uses Bayes' theorem to update the probability of a disease given the presence of certain symptoms or test results
• Spam email filters employ Bayes' theorem to classify emails as spam or not spam based on the occurrence of certain words or phrases

Conditional distributions and prediction

Predicting future events

• Conditional distributions can be used to make predictions about future events based on available information or past observations
• By conditioning on relevant variables or information, one can obtain more accurate and targeted predictions
• Example: In weather forecasting, the probability of rain tomorrow can be predicted based on today's weather conditions and historical data

Updating beliefs based on new information

• Conditional distributions allow for the updating of beliefs or probabilities as new information becomes available
• Bayes' theorem is a key tool for updating beliefs, as it combines prior probabilities with new evidence to obtain posterior probabilities
• Example: In a criminal investigation, the probability of a suspect's guilt can be updated based on new evidence, such as DNA matches or witness testimonies

Conditional distributions in multivariate settings

Joint conditional distributions

• In multivariate settings, where there are multiple random variables, joint conditional distributions describe the probability distribution of two or more variables given specific conditions
• The joint conditional distribution of random variables X and Y given a third variable Z is denoted as $f_{X,Y|Z}(x,y|z)$ or $P(X=x, Y=y|Z=z)$
• Joint conditional distributions can be used to model and analyze the relationships between multiple variables in the presence of conditioning information

Marginal conditional distributions

• Marginal conditional distributions are obtained by integrating or summing out one or more variables from a joint conditional distribution
• The marginal conditional distribution of a random variable X given a conditioning variable Z is denoted as $f_{X|Z}(x|z)$ or $P(X=x|Z=z)$
• Marginal conditional distributions allow for the analysis of a single variable's behavior while accounting for the influence of the conditioning variable

Sampling distributions and conditional distributions

• Sampling distributions describe the probability distribution of a sample statistic, such as the sample mean or sample variance
• Conditional distributions can be used to derive the sampling distribution of a statistic under certain conditions or assumptions
• Example: The sampling distribution of the sample mean, given that the population follows a normal distribution, is a normal distribution with mean equal to the population mean and variance equal to the population variance divided by the sample size

Conditional distributions in regression analysis

• In regression analysis, conditional distributions are used to model the relationship between a dependent variable and one or more independent variables
• The conditional distribution of the dependent variable given the independent variables is often assumed to follow a specific form, such as a normal distribution
• Regression coefficients and predictions are based on the estimated conditional distribution of the dependent variable given the observed values of the independent variables

Conditional distributions in Bayesian inference

Prior and posterior distributions

• In Bayesian inference, prior distributions represent the initial beliefs or knowledge about the parameters of interest before observing any data
• Posterior distributions are the updated distributions of the parameters after considering the observed data and combining it with the prior distribution using Bayes' theorem
• The posterior distribution is proportional to the product of the prior distribution and the likelihood function, which represents the probability of the observed data given the parameter values

Bayesian updating

• Bayesian updating is the process of updating the prior distribution to obtain the posterior distribution as new data becomes available
• The updating process involves multiplying the prior distribution by the likelihood function and normalizing the result to obtain a valid probability distribution
• Bayesian updating allows for the incorporation of new evidence or information into the existing knowledge or beliefs about the parameters, leading to more informed and accurate inferences