Cumulative Distribution Functions

Why This Matters

Cumulative Distribution Functions (CDFs) are one of the most versatile tools in your probability toolkit—they show up everywhere from calculating probabilities to generating random samples in simulations. When you're tested on probability distributions, you're really being tested on your ability to move fluidly between CDFs, PDFs, and probability calculations. Mastering CDFs means understanding how distributions accumulate probability, why certain properties must hold, and when to use CDFs versus other representations.

The concepts here connect directly to hypothesis testing, confidence intervals, and reliability engineering—all core topics in engineering probability. Don't just memorize that $F(x)$ goes from 0 to 1; understand why it must be non-decreasing (probability can't "un-accumulate") and how the CDF-PDF relationship lets you convert between cumulative and instantaneous probability descriptions. These conceptual links are what FRQs and application problems actually test.

---

Foundational Definitions and Properties

Before diving into applications, you need rock-solid understanding of what a CDF is and what mathematical properties it must satisfy. These properties aren't arbitrary—they follow directly from the axioms of probability.

Definition of Cumulative Distribution Function

• $F(x) = P(X \leq x)$—the probability that random variable $X$ takes a value less than or equal to $x$
• Complete distribution description—knowing the CDF tells you everything about the random variable's probabilistic behavior
• Range is always $[0, 1]$—since $F(x)$ represents a probability, it cannot exceed these bounds

Properties of CDFs

• Non-decreasing function—if $x_1 < x_2$, then $F(x_1) \leq F(x_2)$, because probability accumulates as you move right
• Boundary limits—$F(-\infty) = 0$ and $F(\infty) = 1$, capturing impossible and certain events respectively
• Right-continuous—$F(x)$ approaches its value from the right, which matters for discrete distributions with jump discontinuities

---

The CDF-PDF Relationship

Understanding how CDFs and PDFs connect is essential for switching between representations. The PDF tells you probability density at a point; the CDF tells you accumulated probability up to that point.

Relationship for Continuous Variables

• PDF is the derivative of CDF—$f(x) = \frac{dF(x)}{dx}$, meaning the PDF measures the rate at which probability accumulates
• CDF is the integral of PDF—$F(x) = \int_{-\infty}^{x} f(t) \, dt$, giving the area under the PDF curve from $-\infty$ to $x$
• Area interpretation—the probability $P(a < X \leq b)$ equals the area under $f(x)$ between $a$ and $b$

Continuous vs. Discrete CDFs

• Continuous CDFs are smooth—derived from continuous random variables, they have no jumps and are differentiable almost everywhere
• Discrete CDFs are step functions—jumps occur at each possible outcome, with jump height equal to the PMF value $P(X = x_i)$
• Same fundamental role—both types completely describe their distribution; the difference is how probability accumulates (continuously vs. in discrete chunks)

---

Probability Calculations with CDFs

This is where CDFs prove their practical value—they make probability calculations straightforward, especially for ranges and tail probabilities.

Calculating Probabilities Using CDFs

• Range probability formula—$P(a < X \leq b) = F(b) - F(a)$, which works for both continuous and discrete variables
• Tail probabilities—$P(X > x) = 1 - F(x)$, essential for reliability and survival analysis applications
• Simplifies complex distributions—even when the PDF is complicated, the CDF often has a closed form that makes calculations tractable

Inverse CDF and Its Applications

• Quantile function $F^{-1}(p)$—returns the value $x$ such that $P(X \leq x) = p$, answering "what value corresponds to this probability?"
• Random sample generation—the inverse transform method generates samples from any distribution by applying $F^{-1}$ to uniform random numbers
• Statistical inference—critical for finding confidence interval bounds and critical values in hypothesis testing

---

CDFs for Standard Distributions

Knowing the CDF shapes and formulas for common distributions lets you quickly identify distribution types and apply appropriate methods.

CDFs for Common Probability Distributions

• Normal distribution—CDF is the S-shaped error function with no closed form; values come from tables or $\Phi(z)$ notation for standard normal
• Exponential distribution—$F(x) = 1 - e^{-\lambda x}$ for $x \geq 0$, showing rapid initial accumulation that slows over time (memoryless property)
• Uniform distribution—$F(x) = \frac{x - a}{b - a}$ for $x \in [a, b]$, a simple linear ramp reflecting constant density

---

Extensions and Advanced Applications

These topics extend CDF concepts to multiple variables and real-world data analysis—common in engineering applications and upper-level exam questions.

Joint CDFs for Multiple Random Variables

• Definition—$F(x, y) = P(X \leq x, Y \leq y)$ describes simultaneous behavior of two random variables
• Extends to $n$ variables—$F(x_1, x_2, \ldots, x_n)$ captures joint probability structure for any number of variables
• Reveals dependencies—comparing joint CDF to product of marginal CDFs shows whether variables are independent

Empirical CDF and Data Analysis

• Constructed from data—$\hat{F}(x) = \frac{1}{n} \sum_{i=1}^{n} \mathbf{1}(X_i \leq x)$, the fraction of observations at or below $x$
• Non-parametric estimation—requires no assumptions about the underlying distribution, making it robust for exploratory analysis
• Goodness-of-fit testing—comparing empirical CDF to theoretical CDF (via Kolmogorov-Smirnov test) assesses whether data follows a hypothesized distribution

CDF Transformations and Applications

• Probability integral transform—if $X$ has CDF $F$, then $F(X)$ follows a Uniform(0,1) distribution, a powerful result for simulation
• Standardization—transforming data using CDFs can normalize distributions or meet statistical assumptions
• Model improvement—transformations like Box-Cox use CDF-based reasoning to improve model fit and interpretability

---

Quick Reference Table

---

Self-Check Questions

1. A function $G(x)$ satisfies $G(-\infty) = 0$ and $G(\infty) = 1$, but decreases on some interval. Can $G(x)$ be a valid CDF? Why or why not?

2. Compare and contrast how you would find $P(2 < X \leq 5)$ using a CDF versus using a PDF. When might one approach be preferable?

3. You need to generate random samples from an Exponential distribution with rate $\lambda$. Describe how the inverse CDF method works and write the transformation formula.

4. Given a dataset of 100 observations, explain how you would construct the empirical CDF and what it tells you that a histogram doesn't.

5. If two random variables $X$ and $Y$ are independent, what relationship must hold between their joint CDF $F(x, y)$ and their marginal CDFs $F_X(x)$ and $F_Y(y)$?