5.2 Joint probability density functions

Joint probability density functions describe the likelihood of multiple continuous random variables occurring together. They're essential for analyzing relationships between variables in fields like finance, physics, and engineering.

These functions allow us to calculate probabilities, find marginal and conditional distributions, and determine independence between variables. Understanding joint PDFs is crucial for modeling complex systems and making predictions based on multiple interrelated factors.

Definition of joint probability density functions

• Joint probability density functions (PDFs) are used to describe the probability distribution of two or more continuous random variables
• Denoted as $f(x,y)$ for two random variables $X$ and $Y$, the joint PDF gives the probability density at any point $(x,y)$ in the two-dimensional space
• The probability of the random variables falling within a specific region is given by the double integral of the joint PDF over that region

Properties of joint probability density functions

Non-negative values

• Joint PDFs are always non-negative for all values of the random variables
  • $f(x,y) \geq 0$ for all $x$ and $y$
• This property ensures that probabilities calculated using joint PDFs are always non-negative and meaningful

Integration to one

• The double integral of a joint PDF over the entire domain of the random variables must equal one
  • $\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(x,y) \, dy \, dx = 1$
• This property ensures that the total probability of all possible outcomes is one, as required by the axioms of probability

Marginal probability density functions

Definition of marginal probability density functions

• Marginal PDFs describe the probability distribution of a single random variable, ignoring the others
• For a joint PDF $f(x,y)$, the marginal PDFs are denoted as $f_X(x)$ and $f_Y(y)$
• Marginal PDFs are obtained by integrating the joint PDF over the domain of the other variable(s)

Relationship to joint probability density functions

• The marginal PDF of $X$ is given by $f_X(x) = \int_{-\infty}^{\infty} f(x,y) \, dy$
• The marginal PDF of $Y$ is given by $f_Y(y) = \int_{-\infty}^{\infty} f(x,y) \, dx$
• Marginal PDFs can be used to calculate probabilities and moments (mean, variance) for individual random variables

Conditional probability density functions

Definition of conditional probability density functions

• Conditional PDFs describe the probability distribution of one random variable given the value of another
• For a joint PDF $f(x,y)$, the conditional PDF of $Y$ given $X=x$ is denoted as $f_{Y|X}(y|x)$
• Conditional PDFs are obtained by dividing the joint PDF by the marginal PDF of the conditioning variable

Relationship to joint probability density functions

• The conditional PDF of $Y$ given $X=x$ is given by $f_{Y|X}(y|x) = \frac{f(x,y)}{f_X(x)}$, provided $f_X(x) > 0$
• Similarly, the conditional PDF of $X$ given $Y=y$ is given by $f_{X|Y}(x|y) = \frac{f(x,y)}{f_Y(y)}$, provided $f_Y(y) > 0$
• The joint PDF can be expressed as the product of a marginal PDF and a conditional PDF: $f(x,y) = f_X(x) \cdot f_{Y|X}(y|x) = f_Y(y) \cdot f_{X|Y}(x|y)$

Independence of random variables

Definition of independence

• Two random variables $X$ and $Y$ are independent if their joint PDF can be factored into the product of their marginal PDFs
  • $f(x,y) = f_X(x) \cdot f_Y(y)$ for all $x$ and $y$
• Independence implies that the value of one random variable does not affect the probability distribution of the other

Joint probability density functions for independent variables

• If $X$ and $Y$ are independent, their joint PDF is simply the product of their marginal PDFs
• This property simplifies the calculation of probabilities and moments for independent random variables
• For example, if $X \sim N(\mu_X, \sigma_X^2)$ and $Y \sim N(\mu_Y, \sigma_Y^2)$ are independent, their joint PDF is the bivariate normal distribution with zero correlation

Transformations of random variables

Jacobian matrix

• The Jacobian matrix is used to transform joint PDFs when changing variables
• For a transformation from $(X,Y)$ to $(U,V)$ given by $U=g(X,Y)$ and $V=h(X,Y)$, the Jacobian matrix is:
  • $J = \begin{bmatrix} \frac{\partial g}{\partial x} & \frac{\partial g}{\partial y} \\ \frac{\partial h}{\partial x} & \frac{\partial h}{\partial y} \end{bmatrix}$
• The determinant of the Jacobian matrix, $|J|$, is used to adjust the joint PDF under the transformation

Transforming joint probability density functions

• If $(X,Y)$ has a joint PDF $f(x,y)$ and the transformation $(U,V) = (g(X,Y), h(X,Y))$ is one-to-one, the joint PDF of $(U,V)$ is given by:
  • $f_{U,V}(u,v) = f_{X,Y}(x(u,v), y(u,v)) \cdot |J|$
  • where $x(u,v)$ and $y(u,v)$ are the inverse transformations and $|J|$ is the absolute value of the determinant of the Jacobian matrix
• Transformations are useful for simplifying calculations or generating new distributions from known ones

Applications of joint probability density functions

Calculating probabilities

• Joint PDFs are used to calculate the probability of events involving multiple random variables
• The probability of an event $A$ is given by the double integral of the joint PDF over the region defined by $A$
  • $P(A) = \int\int_A f(x,y) \, dy \, dx$
• Examples include calculating the probability that $X$ and $Y$ fall within a specific rectangle or that $X+Y$ exceeds a certain value

Expectation and variance

• Joint PDFs are used to calculate the expected values and variances of functions of multiple random variables
• The expected value of a function $g(X,Y)$ is given by:
  • $E[g(X,Y)] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(x,y) \cdot f(x,y) \, dy \, dx$
• The variance of $g(X,Y)$ is given by:
  • $Var[g(X,Y)] = E[g(X,Y)^2] - (E[g(X,Y)])^2$
• These calculations are useful for understanding the behavior and dispersion of functions of random variables

Examples of joint probability density functions

Bivariate normal distribution

• The bivariate normal distribution is a joint PDF for two normally distributed random variables $X$ and $Y$ with means $\mu_X$ and $\mu_Y$, variances $\sigma_X^2$ and $\sigma_Y^2$, and correlation coefficient $\rho$
• Its joint PDF is given by:
  • $f(x,y) = \frac{1}{2\pi\sigma_X\sigma_Y\sqrt{1-\rho^2}} \exp\left(-\frac{1}{2(1-\rho^2)}\left[\frac{(x-\mu_X)^2}{\sigma_X^2} - 2\rho\frac{(x-\mu_X)(y-\mu_Y)}{\sigma_X\sigma_Y} + \frac{(y-\mu_Y)^2}{\sigma_Y^2}\right]\right)$
• The bivariate normal distribution is widely used in various fields, such as finance (stock returns) and physics (particle positions)

Uniform distribution over a region

• A joint PDF can be uniform over a specific region $R$ in the $xy$-plane, meaning that the probability density is constant within $R$ and zero outside
• The joint PDF for a uniform distribution over a region $R$ with area $A$ is given by:
  • $f(x,y) = \begin{cases} \frac{1}{A}, & (x,y) \in R \\ 0, & (x,y) \notin R \end{cases}$
• Examples include the uniform distribution over a rectangle (product of two independent uniform distributions) or a circle (used in dartboard problems)

Visualization of joint probability density functions

Contour plots

• Contour plots are 2D graphs that show the level curves of a joint PDF in the $xy$-plane
• Each level curve represents points $(x,y)$ with the same probability density $f(x,y)$
• Contour plots provide a clear view of the shape and concentration of the probability distribution
• They are particularly useful for visualizing bivariate normal distributions and their correlations

3D surface plots

• 3D surface plots display the joint PDF as a surface in three-dimensional space, with the $x$ and $y$ axes representing the random variables and the $z$-axis representing the probability density $f(x,y)$
• Surface plots give a more intuitive understanding of the overall shape and peaks of the joint PDF
• They can be rotated and viewed from different angles to better appreciate the distribution's features
• Surface plots are helpful for comparing different joint PDFs and understanding their relative concentrations of probability mass