Key Concepts of Probability Density Functions

Why This Matters

Probability density functions (PDFs) are how you model uncertainty and predict outcomes when results aren't deterministic. Mastering PDFs means being able to recognize which distribution fits which scenario, understand how parameters shape behavior, and apply the right PDF to problems involving reliability, quality control, hypothesis testing, and more.

Don't just memorize the formulas. Know why each distribution exists, what real-world processes it models, and how changing parameters affects the shape. When you see a problem describing waiting times, failure rates, or sample statistics, you should immediately recognize which distribution family applies.

---

Foundational Continuous Distributions

These distributions form the building blocks of probability theory. They model idealized scenarios and serve as the basis for more complex distributions.

Uniform Distribution

• Equal probability across a bounded interval: every value between $a$ and $b$ is equally likely, with PDF $f(x) = \frac{1}{b-a}$ for $a \leq x \leq b$
• Two parameters define the support: minimum $a$ and maximum $b$, giving mean $\frac{a+b}{2}$ and variance $\frac{(b-a)^2}{12}$
• Foundation for random number generation: transforming uniform samples into other distributions is a core simulation technique (this is called the inverse transform method)

Normal (Gaussian) Distribution

• Bell-shaped and symmetric around the mean $\mu$: the PDF is $f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$
• Defined by mean $\mu$ and standard deviation $\sigma$, where approximately 68% of values fall within $\pm 1\sigma$, 95% within $\pm 2\sigma$, and 99.7% within $\pm 3\sigma$ of the mean
• The Central Limit Theorem makes this universal: sums (and averages) of independent random variables converge to normal as the sample size grows, which explains why measurement errors and natural phenomena so often follow this distribution

---

Time-to-Event and Reliability Distributions

These distributions model when something happens: failure times, arrival processes, and system lifetimes. A key concept here is the hazard rate, which represents the instantaneous probability of an event occurring given that it hasn't occurred yet.

Exponential Distribution

• Models time until a single event with constant hazard rate $\lambda$. The "memoryless" property means the probability of the event occurring in the next interval doesn't depend on how much time has already passed. Mathematically: $P(X > s + t \mid X > s) = P(X > t)$.
• Single parameter $\lambda$ (rate) gives mean $\frac{1}{\lambda}$ and PDF $f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$
• Fundamental to queuing theory and reliability: use this when the failure rate doesn't change with age (think electronic components, not mechanical wear)

Weibull Distribution

• Generalizes exponential to handle varying failure rates: the shape parameter $k$ determines whether hazard increases ($k > 1$), decreases ($k < 1$), or stays constant ($k = 1$, which reduces to exponential)
• Two parameters: shape $k$ and scale $\lambda$, with PDF $f(x) = \frac{k}{\lambda}\left(\frac{x}{\lambda}\right)^{k-1} e^{-(x/\lambda)^k}$ for $x \geq 0$
• Industry standard for reliability engineering: models infant mortality ($k < 1$), random failures ($k = 1$), and wear-out ($k > 1$) in a single framework

Gamma Distribution

• Models waiting time for multiple events: if exponential gives the time to one event, gamma gives the time to the $k$-th event
• Two parameters: shape $k$ and scale $\theta$ (or equivalently rate $\beta = 1/\theta$), with mean $k\theta$ and variance $k\theta^2$
• Connects to other distributions: it reduces to exponential when $k = 1$, and to chi-square when $\theta = 2$ and $k = \nu/2$. These connections come up often in problems.

---

Bounded and Proportion Distributions

When your random variable is constrained to a specific interval, these distributions apply. They're essential for modeling probabilities, percentages, and ratios.

Beta Distribution

• Defined only on $[0, 1]$: perfect for modeling probabilities, proportions, and Bayesian prior distributions
• Two shape parameters $\alpha$ and $\beta$ control asymmetry: $\alpha > \beta$ skews right (density concentrated toward 1), $\alpha < \beta$ skews left (density concentrated toward 0), and $\alpha = \beta$ is symmetric
• Extremely flexible: it becomes uniform when $\alpha = \beta = 1$, U-shaped when $\alpha, \beta < 1$, or unimodal when $\alpha, \beta > 1$. It's also the conjugate prior for binomial likelihood in Bayesian inference, which makes updating beliefs mathematically clean.

Lognormal Distribution

• Models positive-only variables where multiplicative effects dominate: if $\ln(X)$ is normally distributed, then $X$ is lognormal
• Parameters $\mu$ and $\sigma$ are the mean and standard deviation of $\ln(X)$, not of $X$ itself. This is a common source of mistakes on problems. The actual mean of $X$ is $e^{\mu + \sigma^2/2}$.
• Right-skewed with a heavy tail: models income distributions, stock prices, particle sizes, and any quantity that grows by percentages rather than by fixed amounts

---

Sampling and Inference Distributions

These distributions arise from sampling processes and are essential for hypothesis testing, confidence intervals, and ANOVA. They're all derived from normal distributions.

Chi-Square Distribution

• Sum of squared standard normal variables: if $Z_i \sim N(0,1)$, then $\sum_{i=1}^{k} Z_i^2 \sim \chi^2_k$
• Single parameter: degrees of freedom $k$, with mean $k$ and variance $2k$. The distribution is right-skewed for small $k$ and approaches normal as $k \to \infty$.
• Primary use is variance testing: the sample variance $s^2$ follows a scaled chi-square distribution, specifically $\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}$. This makes it essential for confidence intervals on $\sigma^2$ and goodness-of-fit tests.

Student's t-Distribution

• Ratio of a standard normal to the square root of a chi-square divided by its df: this arises naturally when you estimate a population mean but have to use the sample standard deviation instead of the true $\sigma$
• Degrees of freedom $\nu$ control tail heaviness: smaller $\nu$ means heavier tails (more probability in the extremes), and as $\nu \to \infty$ the t-distribution approaches the standard normal
• Critical for small-sample inference: use the t-distribution instead of the normal when $\sigma$ is unknown and you're relying on $s$. The common rule of thumb is that this matters most when $n < 30$, though technically you should use t whenever $\sigma$ is unknown.

F-Distribution

• Ratio of two independent chi-square variables, each divided by its df: used to compare two variances or mean squares in ANOVA
• Two parameters: $d_1$ (numerator df) and $d_2$ (denominator df): order matters, so $F_{d_1, d_2} \neq F_{d_2, d_1}$
• Right-skewed and positive-only: the test statistic answers "is the variance ratio significantly different from 1?"

---

Quick Reference Table

---

Self-Check Questions

1. Which two distributions are memoryless, and what does this property mean mathematically? (Hint: one is continuous, one is discrete.)

2. You're modeling the proportion of defective items in a batch (values between 0 and 1). Which distribution is most appropriate, and what parameters would you adjust to reflect a prior belief that defect rates are typically low?

3. Compare and contrast the chi-square and F-distributions: how are they mathematically related, and when would you use each in hypothesis testing?

4. A reliability engineer observes that component failure rates increase with age due to wear. Which distribution should they use, and what constraint on the shape parameter reflects this behavior?

5. You have sample data and need to construct a confidence interval for the population mean with unknown variance. Which distribution do you use for the critical value, and how does your answer change as sample size grows large?