6.3 Transformations of random variables

Transformations of random variables are crucial in probability theory. They allow us to manipulate and analyze random variables in different ways, opening up new possibilities for modeling and problem-solving.

Linear transformations scale and shift random variables, affecting their mean and variance. Non-linear transformations can change the shape of probability distributions entirely, requiring special techniques to derive new distributions from existing ones.

Linear Transformations of Random Variables

Scaling and Shifting Random Variables

• Linear transformations involve scaling and shifting the original random variable
• If $X$ is a random variable and $a$ and $b$ are constants, the linear transformation is $Y = aX + b$
• Scaling a random variable by a constant factor $a$ changes the spread of the distribution
  • If $|a| > 1$, the distribution is stretched (wider spread)
  • If $|a| < 1$, the distribution is compressed (narrower spread)
• Shifting a random variable by a constant value $b$ changes the location of the distribution
  • Positive $b$ shifts the distribution to the right
  • Negative $b$ shifts the distribution to the left

Effects on Mean and Variance

• The mean of the transformed random variable $Y$ is related to the mean of $X$ by $E[Y] = aE[X] + b$, where $E[X]$ is the mean of $X$
  • The constant $a$ scales the mean of $X$
  • The constant $b$ shifts the mean of $X$
• The variance of the transformed random variable $Y$ is related to the variance of $X$ by $Var(Y) = a^2 Var(X)$, where $Var(X)$ is the variance of $X$
  • The constant $a$ scales the variance of $X$ by a factor of $a^2$
  • The constant $b$ does not affect the variance
• Linear transformations preserve the shape of the probability distribution
  • The location (mean) and scale (variance) of the distribution may change
  • Example: If $X$ follows a normal distribution, $Y = aX + b$ will also follow a normal distribution with a different mean and variance

Non-linear Transformations of Random Variables

Applying Non-linear Functions to Random Variables

• Non-linear transformations involve applying a non-linear function to the original random variable
• If $X$ is a random variable and $g(x)$ is a non-linear function, the transformed random variable is $Y = g(X)$
• Examples of non-linear transformations:
  • Exponential function: $Y = e^X$
  • Logarithmic function: $Y = \log(X)$
  • Power function: $Y = X^n$, where $n$ is a constant
• Non-linear transformations can change the shape of the probability distribution

Deriving the Probability Distribution of Transformed Variables

• To find the probability distribution of $Y$, determine the cumulative distribution function (CDF) of $Y$, denoted as $F_Y(y)$, using the CDF of $X$, denoted as $F_X(x)$
• The relationship between the CDFs is $F_Y(y) = P(Y \leq y) = P(g(X) \leq y)$
• For monotonically increasing functions $g(x)$, the CDF of $Y$ is $F_Y(y) = F_X(g^{-1}(y))$, where $g^{-1}(y)$ is the inverse function of $g(x)$
• For monotonically decreasing functions $g(x)$, the CDF of $Y$ is $F_Y(y) = 1 - F_X(g^{-1}(y))$
• The probability density function (PDF) of $Y$, denoted as $f_Y(y)$, is obtained by differentiating the CDF of $Y$ with respect to $y$
  • $f_Y(y) = \frac{d}{dy}F_Y(y)$
  • The PDF of $Y$ can also be expressed in terms of the PDF of $X$, denoted as $f_X(x)$, using the change of variables technique: $f_Y(y) = f_X(g^{-1}(y)) \cdot |\frac{d}{dy}g^{-1}(y)|$

Jacobian Determinant in Multivariate Transformations

Definition and Role of the Jacobian Determinant

• The Jacobian determinant is a matrix of partial derivatives used when transforming multivariate random variables
• Given a vector of random variables $X = (X_1, X_2, ..., X_n)$ and a vector of transformed variables $Y = (Y_1, Y_2, ..., Y_n)$, where each $Y_i$ is a function of $X_1, X_2, ..., X_n$, the Jacobian matrix $J$ is defined as $J_{ij} = \frac{\partial Y_i}{\partial X_j}$
• The Jacobian determinant, denoted as $|J|$, is the absolute value of the determinant of the Jacobian matrix $J$
• The Jacobian determinant represents the volume change factor when transforming from the $X$-space to the $Y$-space
  • If $|J| > 1$, the volume expands during the transformation
  • If $|J| < 1$, the volume contracts during the transformation

Calculating the Jacobian Determinant

• To calculate the Jacobian determinant, first determine the Jacobian matrix $J$ by finding the partial derivatives of each transformed variable $Y_i$ with respect to each original variable $X_j$
• Arrange the partial derivatives in a square matrix, with each row corresponding to a transformed variable and each column corresponding to an original variable
• Calculate the determinant of the Jacobian matrix using standard matrix determinant techniques (e.g., cofactor expansion, Laplace expansion, or Gaussian elimination)
• Take the absolute value of the determinant to obtain the Jacobian determinant $|J|$
• Example: For a transformation from polar coordinates $(R, \Theta)$ to Cartesian coordinates $(X, Y)$, where $X = R \cos(\Theta)$ and $Y = R \sin(\Theta)$, the Jacobian matrix is $J = \begin{bmatrix} \cos(\Theta) & -R \sin(\Theta) \\ \sin(\Theta) & R \cos(\Theta) \end{bmatrix}$, and the Jacobian determinant is $|J| = R$

Joint Distribution of Transformed Variables

Multivariate Change of Variables Technique

• The multivariate change of variables technique is used to find the joint probability density function (PDF) of transformed random variables
• Given a vector of random variables $X = (X_1, X_2, ..., X_n)$ with joint PDF $f_X(x_1, x_2, ..., x_n)$ and a vector of transformed variables $Y = (Y_1, Y_2, ..., Y_n)$, the joint PDF of $Y$, denoted as $f_Y(y_1, y_2, ..., y_n)$, is given by $f_Y(y_1, y_2, ..., y_n) = f_X(x_1, x_2, ..., x_n) \cdot |J|$, where $|J|$ is the absolute value of the Jacobian determinant
• The multivariate change of variables technique requires the transformation to be one-to-one and the inverse transformation to be differentiable

Steps to Apply the Multivariate Change of Variables Technique

1. Express the original variables $(X_1, X_2, ..., X_n)$ in terms of the transformed variables $(Y_1, Y_2, ..., Y_n)$
2. Calculate the Jacobian matrix $J$ by finding the partial derivatives of each original variable with respect to each transformed variable
3. Calculate the Jacobian determinant $|J|$ by taking the absolute value of the determinant of the Jacobian matrix
4. Substitute the expressions for the original variables and the Jacobian determinant into the joint PDF of $X$
5. Simplify the resulting expression to obtain the joint PDF of $Y$

• Example: For a transformation from Cartesian coordinates $(X, Y)$ to polar coordinates $(R, \Theta)$, where $R = \sqrt{X^2 + Y^2}$ and $\Theta = \arctan(Y/X)$, the joint PDF of $(R, \Theta)$ is $f_{R,\Theta}(r, \theta) = f_{X,Y}(r \cos(\theta), r \sin(\theta)) \cdot r$, where $f_{X,Y}(x, y)$ is the joint PDF of $(X, Y)$