1.8 Moment generating functions and transformations

Definition of moment generating functions

The moment generating function (MGF) of a random variable uniquely characterizes its probability distribution. If two random variables share the same MGF, they follow the same distribution. This makes MGFs one of the most useful identification tools in actuarial probability.

The MGF of a random variable $X$ is defined as:

$M_X(t) = E[e^{tX}]$

where $t$ is a real number. For a continuous random variable, this expands to $M_X(t) = \int_{-\infty}^{\infty} e^{tx} f_X(x) \, dx$, and for a discrete random variable, $M_X(t) = \sum_x e^{tx} p_X(x)$.

The MGF exists if $E[e^{tX}]$ is finite for all $t$ in some open interval around zero (i.e., for $t \in (-h, h)$ where $h > 0$).

Laplace transforms

Laplace transforms are closely related to MGFs. The Laplace transform of a function $f(t)$ is defined as:

$F(s) = \int_0^{\infty} e^{-st} f(t) \, dt$

where $s$ is a complex number. Notice the structural similarity: for a non-negative random variable $X$ with density $f_X(x)$, the MGF evaluated at $-s$ equals the Laplace transform of the density. Laplace transforms appear in actuarial work when solving differential equations in ruin theory and analyzing aggregate claim distributions.

Existence of MGFs

Not every distribution has an MGF. For the MGF to exist, $E[e^{tX}]$ must be finite in a neighborhood of $t = 0$. Distributions with heavy tails can violate this condition.

• The Cauchy distribution has no MGF because $E[e^{tX}]$ is infinite for every $t \neq 0$.
• The lognormal distribution also lacks an MGF, despite having finite moments of all orders.
• When an MGF doesn't exist, you can use the characteristic function instead (covered at the end of this guide).

Properties of moment generating functions

MGFs have several properties that make distribution analysis much more tractable, especially when working with sums of independent random variables.

Uniqueness property

If two random variables $X$ and $Y$ have MGFs that are equal in a neighborhood of zero, i.e., $M_X(t) = M_Y(t)$ for all $t \in (-h, h)$, then $X$ and $Y$ have the same distribution.

This is the property you'll use most often on exam problems: compute an MGF, recognize it as belonging to a known distribution, and conclude the random variable follows that distribution.

MGF of linear transformations

For a linear transformation $Y = aX + b$, the MGF is:

$M_Y(t) = E[e^{t(aX+b)}] = e^{bt} \cdot M_X(at)$

This follows directly from the definition. The constant $b$ contributes a factor of $e^{bt}$, while the scaling constant $a$ replaces $t$ with $at$ in the original MGF.

MGF of linear combination of independent random variables

If $X$ and $Y$ are independent random variables, then for $Z = aX + bY$:

$M_Z(t) = M_X(at) \cdot M_Y(bt)$

Independence is critical here. Without it, you can't factor the joint expectation $E[e^{t(aX+bY)}]$ into the product of individual expectations. This extends naturally to $n$ independent random variables:

$M_{a_1X_1 + \cdots + a_nX_n}(t) = \prod_{i=1}^{n} M_{X_i}(a_i t)$

Moments and moment generating functions

The name "moment generating function" comes from the fact that you can extract every moment of a distribution by differentiating the MGF.

Relationship between moments and MGFs

The $n$-th moment of $X$ is obtained by taking the $n$-th derivative of the MGF and evaluating at $t = 0$:

$E[X^n] = M_X^{(n)}(0)$

Why does this work? Expand $e^{tX}$ as a Taylor series:

$M_X(t) = E[e^{tX}] = E\left[\sum_{n=0}^{\infty} \frac{(tX)^n}{n!}\right] = \sum_{n=0}^{\infty} \frac{E[X^n]}{n!} t^n$

Each coefficient of $t^n$ contains $E[X^n]/n!$, so differentiating $n$ times and setting $t = 0$ isolates $E[X^n]$.

Deriving moments from MGFs

Here's the step-by-step process:

1. Write down the MGF $M_X(t)$.

2. Differentiate with respect to $t$ once. Evaluate at $t = 0$ to get the mean: $E[X] = M_X'(0)$.

3. Differentiate again. Evaluate at $t = 0$ to get the second raw moment: $E[X^2] = M_X''(0)$.

4. Compute the variance using $\text{Var}(X) = E[X^2] - (E[X])^2 = M_X''(0) - [M_X'(0)]^2$.

5. Continue differentiating for higher moments as needed.

Central moments vs raw moments

• Raw moments are computed about the origin: $E[X^n]$
• Central moments are computed about the mean: $E[(X - \mu)^n]$

The first central moment is always zero. The second central moment is the variance. The third and fourth central moments (often standardized) give skewness and kurtosis, which describe the asymmetry and tail weight of the distribution.

You can convert between raw and central moments. For example, the variance equals the second raw moment minus the square of the first: $\text{Var}(X) = E[X^2] - \mu^2$.

Transformations using moment generating functions

MGFs are especially handy for finding the distribution of sums and differences of independent random variables, since multiplication of MGFs is much simpler than convolving densities.

MGF of sum of independent random variables

If $X$ and $Y$ are independent:

$M_{X+Y}(t) = M_X(t) \cdot M_Y(t)$

This is the special case of the linear combination formula with $a = b = 1$. It extends to any finite number of independent summands:

$M_{X_1 + X_2 + \cdots + X_n}(t) = \prod_{i=1}^{n} M_{X_i}(t)$

This property is what makes MGFs so powerful for proving closure properties (e.g., the sum of independent normals is normal, the sum of independent Poissons is Poisson).

MGF of difference of random variables

For independent $X$ and $Y$, write $X - Y = X + (-Y)$. The MGF of $-Y$ is $M_Y(-t)$, so:

$M_{X-Y}(t) = M_X(t) \cdot M_Y(-t)$

MGF of product of independent random variables

There is no general product rule analogous to the sum rule. The MGF of $XY$ cannot typically be expressed in terms of $M_X(t)$ and $M_Y(t)$ alone. For products, you usually need to work directly from the definition or use other techniques (such as the distribution of the product derived from the joint density).

Applications of moment generating functions

Determining distributions from MGFs

The standard exam technique:

1. Compute the MGF of the random variable in question (often a sum of independent random variables).
2. Simplify the expression.
3. Compare the result to the table of known MGFs.
4. By the uniqueness property, identify the distribution.

Example: Suppose $X_1, X_2, \ldots, X_n$ are i.i.d. Exponential with rate $\lambda$. The MGF of each is $M_{X_i}(t) = \frac{\lambda}{\lambda - t}$. The MGF of the sum $S = X_1 + \cdots + X_n$ is:

$M_S(t) = \left(\frac{\lambda}{\lambda - t}\right)^n$

This is the MGF of a Gamma distribution with shape $n$ and rate $\lambda$. So $S \sim \text{Gamma}(n, \lambda)$.

Deriving probability distributions

You can recover a PDF or PMF from an MGF by expanding it as a Taylor series around $t = 0$ and reading off the moments, then matching to a known distribution family. In practice, direct recognition (as above) is far more common on exams than full inversion.

Calculating probabilities using MGFs

MGFs are not typically used to compute individual probabilities like $P(X \leq x)$ directly. For that, you'd use the CDF. However, once you've identified the distribution via its MGF, you can use the known CDF or probability tables for that distribution to find probabilities.

Common moment generating functions

The following table summarizes MGFs you should have memorized:

MGF of normal distribution

$M_X(t) = e^{\mu t + \frac{1}{2}\sigma^2 t^2}$

This exists for all real $t$. For the standard normal ($\mu = 0, \sigma^2 = 1$), it simplifies to $M_X(t) = e^{t^2/2}$. You can verify: $M_X'(0) = \mu$ and $M_X''(0) - [M_X'(0)]^2 = \sigma^2$.

MGF of exponential distribution

$M_X(t) = \frac{\lambda}{\lambda - t}, \quad t < \lambda$

The restriction $t < \lambda$ is important. Differentiating: $M_X'(t) = \frac{\lambda}{(\lambda - t)^2}$, so $E[X] = M_X'(0) = \frac{1}{\lambda}$. Differentiating again gives $E[X^2] = \frac{2}{\lambda^2}$, so $\text{Var}(X) = \frac{1}{\lambda^2}$.

MGF of gamma distribution

$M_X(t) = \left(\frac{\beta}{\beta - t}\right)^\alpha, \quad t < \beta$

Note that the exponential distribution is the special case $\alpha = 1$. The mean is $E[X] = \frac{\alpha}{\beta}$ and the variance is $\text{Var}(X) = \frac{\alpha}{\beta^2}$.

MGF of binomial distribution

$M_X(t) = (pe^t + 1 - p)^n$

This exists for all $t$. Differentiating and evaluating at $t = 0$: $E[X] = np$ and $\text{Var}(X) = np(1-p)$. You can also use this MGF to prove that the sum of independent Bernoulli random variables is binomial, since the MGF of a single Bernoulli trial is $(pe^t + 1 - p)$, and multiplying $n$ of these gives the binomial MGF.

Limitations of moment generating functions

Non-existence of MGFs for certain distributions

The main limitation: some distributions simply don't have an MGF. Heavy-tailed distributions like the Cauchy and lognormal are the classic examples. If you encounter a distribution where $E[e^{tX}] = \infty$ for all $t \neq 0$, the MGF approach won't work, and you'll need to use characteristic functions instead.

Convergence issues with MGFs

Even when an MGF exists, it may only converge on a restricted interval around zero (e.g., $t < \lambda$ for the exponential). This can limit its usefulness in certain theoretical arguments. The characteristic function avoids this problem entirely because $|e^{itX}| = 1$ for all $t$, guaranteeing convergence.

Characteristic functions vs moment generating functions

Definition of characteristic functions

The characteristic function of a random variable $X$ replaces the real exponential in the MGF with a complex exponential:

$\phi_X(t) = E[e^{itX}]$

where $i = \sqrt{-1}$ and $t$ is real. For continuous random variables: $\phi_X(t) = \int_{-\infty}^{\infty} e^{itx} f_X(x) \, dx$. This is the Fourier transform of the density.

Properties of characteristic functions

Characteristic functions share the key properties of MGFs:

• Uniqueness: $\phi_X(t) = \phi_Y(t)$ for all $t$ implies $X$ and $Y$ have the same distribution.
• Independence and sums: If $X$ and $Y$ are independent, $\phi_{X+Y}(t) = \phi_X(t) \cdot \phi_Y(t)$.
• Moment extraction: $E[X^n] = \frac{\phi_X^{(n)}(0)}{i^n}$, provided the $n$-th moment exists.

Advantages of characteristic functions over MGFs

The characteristic function always exists for any random variable, because $|e^{itX}| = 1$ ensures the expectation is bounded. This is the decisive advantage over MGFs.

Characteristic functions are the standard tool for proving the Central Limit Theorem: you show that the characteristic function of the standardized sum converges to $e^{-t^2/2}$ (the characteristic function of the standard normal), then invoke the continuity theorem.

The tradeoff is that characteristic functions involve complex arithmetic, which makes direct computation less convenient than MGFs when the MGF exists. For exam purposes, use MGFs whenever they exist. Reserve characteristic functions for distributions where MGFs fail or for theoretical proofs requiring universal applicability.