Key Concepts of Probability Density Functions

Why This Matters

Probability density functions are the backbone of engineering analysis—they're how you model uncertainty, predict system behavior, and make decisions when outcomes aren't deterministic. You're being tested on your ability to recognize which distribution fits which scenario, understand how parameters shape behavior, and apply the right PDF to problems involving reliability, quality control, hypothesis testing, and signal processing.

Don't just memorize the formulas. Know why each distribution exists, what real-world processes it models, and how changing parameters affects the shape. When you see an exam problem describing waiting times, failure rates, or sample statistics, you should immediately recognize which distribution family applies—and understand the mathematical reasoning behind that choice.

---

Foundational Continuous Distributions

These distributions form the building blocks of probability theory. They model idealized scenarios and serve as the basis for more complex distributions.

Uniform Distribution

• Equal probability across a bounded interval—every value between $a$ and $b$ is equally likely, with PDF $f(x) = \frac{1}{b-a}$
• Two parameters define the support: minimum $a$ and maximum $b$, giving mean $\frac{a+b}{2}$ and variance $\frac{(b-a)^2}{12}$
• Foundation for random number generation—transforming uniform samples into other distributions is a core simulation technique

Normal (Gaussian) Distribution

• Bell-shaped and symmetric around the mean $\mu$—the PDF is $f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$
• Defined by mean $\mu$ and standard deviation $\sigma$, where approximately 68% of values fall within $\pm 1\sigma$ of the mean
• Central Limit Theorem makes this universal—sums of independent random variables converge to normal, explaining why measurement errors and natural phenomena follow this distribution

---

Time-to-Event and Reliability Distributions

These distributions model when something happens—failure times, arrival processes, and system lifetimes. The key concept is the hazard rate (instantaneous failure probability).

Exponential Distribution

• Models time until a single event with constant hazard rate $\lambda$—the "memoryless" property means past time doesn't affect future probability
• Single parameter $\lambda$ (rate) gives mean $\frac{1}{\lambda}$ and PDF $f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$
• Fundamental to queuing theory and reliability—use this when failure rate doesn't change with age (electronic components, not mechanical wear)

Weibull Distribution

• Generalizes exponential to handle varying failure rates—shape parameter $k$ determines whether hazard increases ($k > 1$), decreases ($k < 1$), or stays constant ($k = 1$)
• Two parameters: shape $k$ and scale $\lambda$, with PDF $f(x) = \frac{k}{\lambda}\left(\frac{x}{\lambda}\right)^{k-1} e^{-(x/\lambda)^k}$
• Industry standard for reliability engineering—models infant mortality ($k < 1$), random failures ($k = 1$), and wear-out ($k > 1$) in a single framework

Gamma Distribution

• Models waiting time for multiple events—if exponential is time to one event, gamma is time to k events
• Two parameters: shape $k$ (number of events) and scale $\theta$ (or rate $\beta = 1/\theta$), with mean $k\theta$
• Reduces to exponential when $k = 1$ and to chi-square when $\theta = 2$ and $k = \nu/2$—understanding these connections is frequently tested

---

Bounded and Proportion Distributions

When your random variable is constrained to a specific interval, these distributions apply. They're essential for modeling probabilities, percentages, and ratios.

Beta Distribution

• Defined only on $[0, 1]$—perfect for modeling probabilities, proportions, and Bayesian prior distributions
• Two shape parameters $\alpha$ and $\beta$ control asymmetry: $\alpha > \beta$ skews right, $\alpha < \beta$ skews left, $\alpha = \beta$ is symmetric
• Extremely flexible: uniform ($\alpha = \beta = 1$), U-shaped ($\alpha, \beta < 1$), or unimodal ($\alpha, \beta > 1$)—the conjugate prior for binomial likelihood in Bayesian inference

Lognormal Distribution

• Models positive-only variables where multiplicative effects dominate—if $\ln(X)$ is normal, then $X$ is lognormal
• Parameters $\mu$ and $\sigma$ are the mean and standard deviation of $\ln(X)$, not of $X$ itself—a common exam trap
• Right-skewed with heavy tail—models income distributions, stock prices, particle sizes, and any quantity that grows by percentages

---

Sampling and Inference Distributions

These distributions arise from sampling processes and are essential for hypothesis testing, confidence intervals, and ANOVA. They're derived from normal distributions.

Chi-Square Distribution

• Sum of squared standard normal variables—if $Z_i \sim N(0,1)$, then $\sum_{i=1}^{k} Z_i^2 \sim \chi^2_k$
• Single parameter: degrees of freedom $k$, with mean $k$ and variance $2k$; right-skewed for small $k$, approaches normal as $k \to \infty$
• Primary use: variance testing—the sample variance $s^2$ follows a scaled chi-square, making this essential for confidence intervals on $\sigma^2$

Student's t-Distribution

• Ratio of normal to chi-square—arises when estimating means with unknown population variance
• Degrees of freedom $\nu$ control tail heaviness: smaller $\nu$ means heavier tails, $\nu \to \infty$ approaches standard normal
• Critical for small-sample inference—use t-distribution instead of normal when $n < 30$ and $\sigma$ is unknown

F-Distribution

• Ratio of two chi-square variables—used to compare two variances or mean squares in ANOVA
• Two parameters: $d_1$ (numerator df) and $d_2$ (denominator df)—order matters, so $F_{d_1, d_2} \neq F_{d_2, d_1}$
• Right-skewed and positive-only—test statistic for "is the variance ratio significantly different from 1?"

---

Quick Reference Table

---

Self-Check Questions

1. Which two distributions are memoryless, and what does this property mean mathematically?

2. You're modeling the proportion of defective items in a batch (values between 0 and 1). Which distribution is most appropriate, and what parameters would you adjust to reflect prior belief that defect rates are typically low?

3. Compare and contrast the chi-square and F-distributions: how are they mathematically related, and when would you use each in hypothesis testing?

4. A reliability engineer observes that component failure rates increase with age due to wear. Which distribution should they use, and what constraint on the shape parameter reflects this behavior?

5. If an FRQ gives you sample data and asks you to construct a confidence interval for the population mean with unknown variance, which distribution do you use for the critical value—and how does your answer change as sample size grows large?