🎲Mathematical Probability Theory Unit 5 – Joint Distributions

Key Concepts

• Joint distributions describe the probability of two or more random variables occurring simultaneously
• Marginal distributions derived from joint distributions by summing over the values of one variable
• Conditional distributions calculate the probability of one variable given the value of another
• Independence occurs when the probability of one variable does not depend on the value of another
  • If two variables are independent, their joint probability is the product of their marginal probabilities
• Correlation measures the linear relationship between two variables
  • Positive correlation indicates that as one variable increases, the other tends to increase as well
  • Negative correlation indicates that as one variable increases, the other tends to decrease
• Joint probability mass functions (PMFs) used for discrete random variables
• Joint probability density functions (PDFs) used for continuous random variables

Types of Joint Distributions

• Bivariate distributions involve two random variables (X and Y)
• Multivariate distributions involve more than two random variables (X, Y, Z, etc.)
• Discrete joint distributions occur when both random variables can only take on a countable number of values
  • Example: the number of heads and tails in a series of coin flips
• Continuous joint distributions occur when both random variables can take on any value within a range
  • Example: the height and weight of individuals in a population
• Mixed joint distributions occur when one variable is discrete and the other is continuous
• Bernoulli joint distributions occur when both variables are binary (0 or 1)
• Poisson joint distributions occur when both variables are counts of rare events over a fixed interval

Properties and Characteristics

• The range of a joint distribution is the set of all possible values that the random variables can take on
• The support of a joint distribution is the set of all points where the joint PMF or PDF is non-zero
• Joint distributions must satisfy certain properties to be valid:
  • The joint PMF or PDF must be non-negative for all possible values of the random variables
  • The sum (for discrete) or integral (for continuous) of the joint PMF or PDF over all possible values must equal 1
• The expected value (mean) of a joint distribution is a vector containing the expected values of each random variable
• The variance of a joint distribution measures the spread of the distribution around its mean
• The covariance of a joint distribution measures the linear relationship between two random variables
  • Positive covariance indicates that the variables tend to increase or decrease together
  • Negative covariance indicates that the variables tend to move in opposite directions

Calculating Joint Probabilities

• For discrete joint distributions, the joint probability is the sum of the joint PMF over the desired values
  • Example: P(X=1, Y=2) = f(1, 2), where f is the joint PMF
• For continuous joint distributions, the joint probability is the double integral of the joint PDF over the desired region
  • Example: P(a ≤ X ≤ b, c ≤ Y ≤ d) = $\int_c^d \int_a^b f(x, y) dx dy$, where f is the joint PDF
• The law of total probability states that the marginal probability of one variable is the sum (discrete) or integral (continuous) of the joint probability over all values of the other variable
• Bayes' theorem allows for updating probabilities based on new information
  • P(A|B) = P(B|A) * P(A) / P(B), where A and B are events and P(B) > 0

Marginal Distributions

• Marginal distributions are obtained by summing (discrete) or integrating (continuous) the joint distribution over the values of one variable
• For discrete joint PMFs, the marginal PMF of X is given by: f_X(x) = $\sum_y f(x, y)$
• For continuous joint PDFs, the marginal PDF of X is given by: f_X(x) = $\int_{-\infty}^{\infty} f(x, y) dy$
• Marginal distributions represent the probability distribution of a single variable, ignoring the values of the other variable(s)
• The mean and variance of a marginal distribution can be calculated using the same formulas as for univariate distributions
• Marginal distributions are useful for understanding the behavior of individual variables in a joint distribution

Conditional Distributions

• Conditional distributions calculate the probability of one variable given the value of another
• For discrete joint PMFs, the conditional PMF of Y given X=x is: f_{Y|X}(y|x) = f(x, y) / f_X(x)
• For continuous joint PDFs, the conditional PDF of Y given X=x is: f_{Y|X}(y|x) = f(x, y) / f_X(x)
• Conditional distributions allow for updating probabilities based on new information
• The mean and variance of a conditional distribution can be calculated using modified formulas that account for the given value of the other variable
• Conditional distributions are useful for understanding the relationship between variables in a joint distribution

Independence and Correlation

• Two random variables are independent if their joint probability is the product of their marginal probabilities
  • For discrete joint PMFs: f(x, y) = f_X(x) * f_Y(y)
  • For continuous joint PDFs: f(x, y) = f_X(x) * f_Y(y)
• If two variables are independent, knowing the value of one variable does not provide any information about the other
• Correlation measures the linear relationship between two variables
  • The correlation coefficient (ρ) ranges from -1 to 1
  • ρ = 0 indicates no linear relationship, ρ = 1 indicates a perfect positive linear relationship, and ρ = -1 indicates a perfect negative linear relationship
• Independence implies zero correlation, but zero correlation does not imply independence
  • Example: X and Y are independent if and only if Cov(X, Y) = 0, but Cov(X, Y) = 0 does not necessarily mean X and Y are independent

Applications and Examples

• Joint distributions are used in various fields, such as finance (stock prices), engineering (component failures), and social sciences (income and education levels)
• Example: In a factory, the joint distribution of the number of defective items (X) and the production time (Y) can be used to optimize quality control and efficiency
• Example: In a medical study, the joint distribution of a patient's age (X) and blood pressure (Y) can be used to identify risk factors for heart disease
• Example: In a marketing survey, the joint distribution of a customer's income (X) and their likelihood to purchase a product (Y) can be used to target advertising campaigns
• Joint distributions can be used to calculate probabilities of complex events involving multiple variables
  • Example: The probability of a student scoring above 90% on both a math test (X) and a science test (Y)
• Marginal and conditional distributions derived from joint distributions provide insights into the individual behavior and relationships between variables
  • Example: The marginal distribution of a student's math test scores can be used to compare their performance to the class average
  • Example: The conditional distribution of a patient's blood pressure given their age can be used to determine if they are at a higher risk for heart disease compared to their age group