Cumulative Distribution Functions

Why This Matters

Cumulative Distribution Functions (CDFs) are one of the most versatile tools in probability. They show up everywhere from calculating probabilities to generating random samples in simulations. When you're tested on probability distributions, you're really being tested on your ability to move between CDFs, PDFs, and probability calculations.

The core idea: a CDF tracks how probability accumulates as you move along the number line. Don't just memorize that $F(x)$ goes from 0 to 1. Understand why it must be non-decreasing (probability can't "un-accumulate") and how the CDF-PDF relationship lets you convert between cumulative and instantaneous probability descriptions. Those conceptual links are what application problems actually test.

---

Foundational Definitions and Properties

You need a rock-solid understanding of what a CDF is and what mathematical properties it must satisfy. These properties aren't arbitrary; they follow directly from the axioms of probability.

Definition of Cumulative Distribution Function

$F(x) = P(X \leq x)$

This is the probability that random variable $X$ takes a value less than or equal to $x$. Two things to note:

• Complete distribution description. Knowing the CDF tells you everything about the random variable's probabilistic behavior. You can recover the PDF, compute any probability, find percentiles, etc.
• Range is always $[0, 1]$. Since $F(x)$ is a probability, it can never go below 0 or above 1.

Properties of CDFs

Every valid CDF must satisfy three properties:

1. Non-decreasing. If $x_1 < x_2$, then $F(x_1) \leq F(x_2)$. This is because the event $\{X \leq x_1\}$ is a subset of $\{X \leq x_2\}$, so its probability can't be larger.
2. Boundary limits. $\lim_{x \to -\infty} F(x) = 0$ and $\lim_{x \to \infty} F(x) = 1$. Far enough to the left, no probability has accumulated yet; far enough to the right, all of it has.
3. Right-continuous. $F(x) = \lim_{h \to 0^+} F(x + h)$. This convention ensures that $F(x)$ includes the probability at exactly $x$. It matters most for discrete distributions, where the CDF has jump discontinuities.

---

The CDF-PDF Relationship

The PDF tells you probability density at a point; the CDF tells you accumulated probability up to that point. Knowing how to go back and forth between them is essential.

Relationship for Continuous Variables

• PDF is the derivative of CDF: $f(x) = \frac{dF(x)}{dx}$. The PDF measures the rate at which probability accumulates.
• CDF is the integral of PDF: $F(x) = \int_{-\infty}^{x} f(t) \, dt$. This gives the area under the PDF curve from $-\infty$ to $x$.
• Probability over an interval is the area under $f(x)$ between the endpoints: $P(a < X \leq b) = \int_a^b f(t) \, dt = F(b) - F(a)$.

Continuous vs. Discrete CDFs

• Continuous CDFs are smooth curves with no jumps. You can differentiate them to get the PDF.
• Discrete CDFs are step functions. Each jump occurs at a possible outcome $x_i$, and the jump height equals the PMF value $P(X = x_i)$. You use differences (not derivatives) to recover probabilities.
• Both types completely describe their distribution. The difference is how probability accumulates: smoothly vs. in discrete chunks.

---

Probability Calculations with CDFs

This is where CDFs prove their practical value. They make probability calculations straightforward, especially for ranges and tail probabilities.

Calculating Probabilities Using CDFs

The key formulas:

• Range probability: $P(a < X \leq b) = F(b) - F(a)$. This works for both continuous and discrete variables.
• Tail probability: $P(X > x) = 1 - F(x)$. You'll use this constantly in reliability and survival analysis contexts.
• For continuous variables specifically, $P(X = x) = 0$ for any single point, so $P(a < X \leq b) = P(a \leq X \leq b)$. The inequality direction doesn't matter. For discrete variables, it does matter whether the endpoints are included.

Inverse CDF and Its Applications

The quantile function $F^{-1}(p)$ goes the other direction: given a probability $p$, it returns the value $x$ such that $P(X \leq x) = p$.

• Finding percentiles. The median is $F^{-1}(0.5)$. The 95th percentile is $F^{-1}(0.95)$.
• Random sample generation. The inverse transform method works like this: generate a uniform random number $U$ on $[0, 1]$, then compute $X = F^{-1}(U)$. The result $X$ follows the target distribution. This works for any distribution with an invertible CDF.
• Confidence intervals. Critical values for hypothesis testing come from the inverse CDF of the test statistic's distribution.

---

CDFs for Standard Distributions

Knowing the CDF shapes and formulas for common distributions lets you quickly identify distribution types and apply the right methods.

CDFs for Common Probability Distributions

Normal distribution. The CDF is the S-shaped curve written as $\Phi(z)$ for the standard normal. There's no closed-form expression, so you look up values in a table or use software. The symmetry property $\Phi(-z) = 1 - \Phi(z)$ is useful for quick calculations.

Exponential distribution. $F(x) = 1 - e^{-\lambda x}$ for $x \geq 0$. Probability accumulates quickly at first, then slows. This distribution has the memoryless property: $P(X > s + t \mid X > s) = P(X > t)$. The parameter $\lambda$ is the rate; the mean is $1/\lambda$.

Uniform distribution. $F(x) = \frac{x - a}{b - a}$ for $x \in [a, b]$. This is just a straight line from 0 to 1, reflecting the constant density across the interval.

---

Extensions and Advanced Applications

These topics extend CDF concepts to multiple variables and real-world data analysis.

Joint CDFs for Multiple Random Variables

The joint CDF for two random variables is defined as:

$F(x, y) = P(X \leq x, Y \leq y)$

This describes the simultaneous behavior of $X$ and $Y$. It extends naturally to $n$ variables: $F(x_1, x_2, \ldots, x_n)$.

A key use: checking independence. If $X$ and $Y$ are independent, then $F(x, y) = F_X(x) \cdot F_Y(y)$ for all $x, y$. If this product relationship fails anywhere, the variables are dependent.

Empirical CDF and Data Analysis

When you have actual data (say $n$ observations), the empirical CDF is:

$\hat{F}(x) = \frac{1}{n} \sum_{i=1}^{n} \mathbf{1}(X_i \leq x)$

In plain terms, $\hat{F}(x)$ is the fraction of your data points that fall at or below $x$. This is a non-parametric estimate, meaning you don't need to assume any particular distribution.

The empirical CDF is especially useful for goodness-of-fit testing. The Kolmogorov-Smirnov test compares the empirical CDF to a theoretical CDF and measures the maximum vertical distance between them. A large gap suggests the data doesn't follow the hypothesized distribution.

CDF Transformations and Applications

• Probability integral transform. If $X$ has a continuous CDF $F$, then $F(X)$ follows a $\text{Uniform}(0, 1)$ distribution. This is the theoretical foundation behind the inverse transform method for simulation.
• Standardization. Transforming data using CDFs can map any continuous distribution to a uniform one, which is useful for comparing distributions or meeting statistical assumptions.

---

Quick Reference Table

---

Self-Check Questions

1. A function $G(x)$ satisfies $G(-\infty) = 0$ and $G(\infty) = 1$, but decreases on some interval. Can $G(x)$ be a valid CDF? Why or why not?

2. Compare how you would find $P(2 < X \leq 5)$ using a CDF versus using a PDF. When might one approach be preferable?

3. You need to generate random samples from an Exponential distribution with rate $\lambda$. Describe how the inverse CDF method works and write the transformation formula.

4. Given a dataset of 100 observations, explain how you would construct the empirical CDF and what information it gives you that a histogram doesn't.

5. If two random variables $X$ and $Y$ are independent, what relationship must hold between their joint CDF $F(x, y)$ and their marginal CDFs $F_X(x)$ and $F_Y(y)$?