2.5 Transformations of random variables

Transformations of random variables

Transforming a random variable means applying a function to it, producing a new random variable with a different distribution. This is one of the core skills in stochastic processes: if you know the distribution of $X$, and you define $Y = g(X)$, you need to figure out the distribution of $Y$. The main techniques for doing this are the CDF method, the MGF method, and for sums, convolution.

Functions of Random Variables

Discrete vs. continuous functions

The approach you take depends on whether you're working with discrete or continuous random variables.

• Discrete case: If $X$ takes values in a countable set, then $Y = g(X)$ is also discrete. You find its PMF by collecting all input values that map to the same output and summing their probabilities.
• Continuous case: If $X$ has a PDF, then $Y = g(X)$ is typically continuous (though not always). You'll use the CDF technique or the change-of-variables formula to find the PDF of $Y$.

Probability distribution of functions

For discrete random variables, the PMF of $Y = g(X)$ is:

$P(Y = y) = \sum_{x:\, g(x) = y} P(X = x)$

You're grouping together every $x$ value that lands on the same $y$, then adding up their probabilities.

For continuous random variables, you generally can't just "plug in" to the PDF. Instead, you work through the CDF first (described next) or use the MGF approach.

Cumulative Distribution Function Technique

Deriving CDFs from transformations

The CDF method is the most general approach and works for virtually any transformation. Here's the procedure:

1. Start with $Y = g(X)$ and write the CDF definition: $F_Y(y) = P(Y \leq y) = P(g(X) \leq y)$.
2. Manipulate the inequality $g(X) \leq y$ to isolate $X$. For example, if $g$ is strictly increasing, this becomes $P(X \leq g^{-1}(y))$.
3. Express the result in terms of $F_X$, the CDF of $X$.

If $g$ is strictly decreasing, the inequality flips: $P(g(X) \leq y) = P(X \geq g^{-1}(y)) = 1 - F_X(g^{-1}(y))$.

When $g$ is not monotone (e.g., $Y = X^2$), you need to split into cases and account for all regions of $X$ that satisfy $g(X) \leq y$.

Inverting CDFs to find distributions

Once you have $F_Y(y)$, differentiate with respect to $y$ to get the PDF:

$f_Y(y) = \frac{d}{dy} F_Y(y)$

For a monotone, differentiable transformation $Y = g(X)$ with inverse $X = g^{-1}(Y)$, this yields the change-of-variables formula:

$f_Y(y) = f_X(g^{-1}(y)) \cdot \left| \frac{d}{dy} g^{-1}(y) \right|$

The absolute value accounts for both increasing and decreasing transformations. This single formula handles most one-variable continuous problems you'll encounter.

Moment-Generating Function Technique

Uniqueness of moment-generating functions

The MGF of a random variable $X$ is $M_X(t) = E[e^{tX}]$, defined for $t$ in some neighborhood of zero. The key property: if two random variables have the same MGF in a neighborhood of zero, they have the same distribution. This uniqueness theorem is what makes the MGF method work.

Finding distributions using MGFs

The strategy is to compute the MGF of the transformed variable and then recognize it as belonging to a known distribution family.

1. Define $Y = g(X)$ and write $M_Y(t) = E[e^{tY}] = E[e^{t\,g(X)}]$.
2. Evaluate this expectation using the distribution of $X$.
3. If the resulting expression matches the MGF of a known distribution (normal, gamma, Poisson, etc.), you've identified the distribution of $Y$.

Example: If $X \sim N(\mu, \sigma^2)$ and $Y = aX + b$, then $M_Y(t) = e^{bt} M_X(at) = e^{bt} e^{a\mu t + a^2\sigma^2 t^2/2} = e^{(a\mu+b)t + a^2\sigma^2 t^2/2}$. This is the MGF of $N(a\mu + b,\, a^2\sigma^2)$, confirming that a linear transformation of a normal is still normal.

The MGF method is especially powerful for sums of independent random variables, since $M_{X+Y}(t) = M_X(t) \cdot M_Y(t)$ when $X$ and $Y$ are independent.

Convolutions of Independent Random Variables

Sums of independent random variables

When $X$ and $Y$ are independent and you want the distribution of $Z = X + Y$, the result is a convolution.

• Continuous case: $f_Z(z) = \int_{-\infty}^{\infty} f_X(x)\, f_Y(z - x)\, dx$
• Discrete case: $P(Z = z) = \sum_{x} P(X = x)\, P(Y = z - x)$

You're summing (or integrating) over all ways the two variables can combine to give the total $z$. In practice, the MGF method is often faster for sums: compute $M_Z(t) = M_X(t) \cdot M_Y(t)$ and recognize the result.

Products of independent random variables

For the product $W = XY$ of two independent continuous random variables, the PDF can be derived using a change of variables. One standard approach:

1. Define $W = XY$ and $V = X$ (an auxiliary variable).
2. Compute the joint PDF of $(W, V)$ using the Jacobian.
3. Integrate out $V$ to get the marginal PDF of $W$.

Note: unlike sums, the MGF of a product is not simply the product of the individual MGFs. The factoring property $M_{X+Y}(t) = M_X(t)M_Y(t)$ applies only to sums.

Transformations of Multiple Random Variables

Joint cumulative distribution functions

For a vector of random variables $(X_1, X_2, \ldots, X_n)$, the joint CDF is:

$F_{X_1, \ldots, X_n}(x_1, \ldots, x_n) = P(X_1 \leq x_1, X_2 \leq x_2, \ldots, X_n \leq x_n)$

When you apply a transformation $(Y_1, \ldots, Y_n) = \mathbf{g}(X_1, \ldots, X_n)$, you can use the multivariate CDF method: express events about the $Y_i$ in terms of the $X_i$ and use the joint distribution of $\mathbf{X}$.

Jacobian matrix for transformations

For an invertible transformation of continuous random variables, the multivariate change-of-variables formula is:

$f_{Y_1, \ldots, Y_n}(y_1, \ldots, y_n) = f_{X_1, \ldots, X_n}(x_1, \ldots, x_n) \cdot \left| \det(J) \right|^{-1}$

where $J$ is the Jacobian matrix with entries $J_{ij} = \frac{\partial y_i}{\partial x_j}$, and $(x_1, \ldots, x_n)$ is expressed in terms of $(y_1, \ldots, y_n)$ via the inverse transformation.

Equivalently, if you write the inverse transformation and define the Jacobian of the inverse, you get $|\det(J^{-1})|$ directly. Either way, the determinant corrects for how the transformation stretches or compresses volume in probability space.

Common Transformations and Distributions

Linear transformations

For $Y = aX + b$ (with $a \neq 0$):

• $E[Y] = aE[X] + b$
• $\text{Var}(Y) = a^2 \text{Var}(X)$
• The PDF transforms as: $f_Y(y) = \frac{1}{|a|} f_X\!\left(\frac{y - b}{a}\right)$

Linear transformations preserve distribution families in many cases. Normals stay normal, and Cauchy random variables stay Cauchy, for instance.

Exponential and logarithmic transformations

Exponential: If $Y = e^X$, apply the CDF method. Since $e^X$ is strictly increasing:

$f_Y(y) = f_X(\ln y) \cdot \frac{1}{y}, \quad y > 0$

A classic application: if $X \sim N(\mu, \sigma^2)$, then $Y = e^X$ follows a lognormal distribution.

Logarithmic: If $Y = \ln X$ for $X > 0$, then:

$f_Y(y) = f_X(e^y) \cdot e^y$

These transformations are useful for converting multiplicative relationships into additive ones.

Normal to standard normal transformation

Any normal random variable $X \sim N(\mu, \sigma^2)$ can be standardized:

$Z = \frac{X - \mu}{\sigma}$

This gives $Z \sim N(0, 1)$. The transformation lets you use standard normal tables or software to compute probabilities for any normal distribution. It's a special case of the linear transformation with $a = 1/\sigma$ and $b = -\mu/\sigma$.

Chi-square and gamma distributions

If $Z_1, Z_2, \ldots, Z_n$ are independent $N(0,1)$ variables, then:

$Y = \sum_{i=1}^{n} Z_i^2 \sim \chi^2(n)$

The chi-square distribution with $n$ degrees of freedom is actually a special case of the gamma distribution: $\chi^2(n) = \text{Gamma}(n/2,\, 1/2)$ (using the rate parameterization).

More generally, the gamma family is closed under summation of independent variables: if $X_i \sim \text{Gamma}(\alpha_i, \beta)$ are independent with the same rate $\beta$, then $\sum X_i \sim \text{Gamma}(\sum \alpha_i, \beta)$. This is easy to verify using MGFs.

Applications of Transformations

Signal processing and filtering

In signal processing, random signals pass through systems (filters) that transform their distributions. If the input to a linear time-invariant system is a random process, the output distribution depends on the system's transfer function. Fourier and Laplace transforms are used to move between time and frequency domains, simplifying the analysis of how noise and signals interact.

Reliability analysis and failure rates

Reliability engineering models component lifetimes as random variables. The exponential distribution models constant failure rates (memoryless property), while the Weibull distribution handles increasing or decreasing failure rates. A logarithmic transformation of Weibull data linearizes the survival function, making it easier to estimate parameters from observed failure data.

Stochastic modeling in physics and engineering

Transformations underpin many physical models. Brownian motion (particle diffusion) involves Gaussian random variables whose distributions evolve over time. Birth-death processes use transformations to derive steady-state distributions. In each case, knowing how to transform distributions lets you move from a simple model to the quantities you actually care about, like hitting times, equilibrium concentrations, or system reliability.