Fundamental Probability Axioms

Why This Matters

Probability axioms aren't just abstract rules—they're the logical foundation that makes all of statistics, data science, and scientific inference possible. When you're tested on probabilistic methods, you're being evaluated on whether you understand why these axioms work together as a coherent system. Every calculation you'll ever do in probability—from simple coin flips to complex Bayesian inference—traces back to these core principles.

Think of the axioms as the "rules of the game" that guarantee mathematical consistency. Without non-negativity, probabilities could be meaningless negative numbers. Without the unity axiom, we'd have no reference point. Without additivity, we couldn't combine events logically. Don't just memorize the formulas—understand what each axiom prevents from going wrong and how the derived rules (complement, inclusion-exclusion, Bayes) flow directly from these foundations.

---

The Core Axioms: Building the Foundation

The three Kolmogorov axioms define what a valid probability measure must satisfy. Every theorem and rule in probability theory derives from these three statements.

Sample Space and Events

• The sample space $S$ is the complete set of all possible outcomes—defining it correctly is your first step in any probability problem
• Events are subsets of $S$—a simple event contains one outcome, while a compound event groups multiple outcomes together
• The event structure forms a σ-algebra—this technical requirement ensures we can take complements, unions, and intersections without breaking the system

Probability Measure

• A probability measure $P$ is a function mapping events to real numbers in $[0, 1]$—it quantifies uncertainty numerically
• $P$ must satisfy all three axioms—any function that violates even one axiom isn't a valid probability measure
• Different probability measures on the same sample space represent different models of uncertainty—choosing the right one is the art of applied probability

Non-Negativity Axiom

• For any event $A$, we require $P(A) \geq 0$—probabilities cannot be negative, which matches our intuition about "likelihood"
• This axiom rules out nonsensical assignments—you can't have a $-30\%$ chance of rain
• Combined with the other axioms, non-negativity ensures probabilities stay bounded between 0 and 1

Unity Axiom

• $P(S) = 1$ establishes the reference point—the certain event (something happens) has probability one
• This normalizes the probability scale—all other probabilities are measured relative to this baseline
• Equivalently, $P(\emptyset) = 0$—the impossible event has zero probability, derivable from unity plus additivity

Additivity Axiom

• For mutually exclusive events $A$ and $B$: $P(A \cup B) = P(A) + P(B)$—probabilities add when events can't overlap
• Extends to countable additivity—for infinitely many disjoint events $A_1, A_2, \ldots$, we have $P(\bigcup_{i=1}^{\infty} A_i) = \sum_{i=1}^{\infty} P(A_i)$
• This axiom enables calculation—without it, we couldn't decompose complex events into simpler pieces

---

Derived Rules: Direct Consequences of the Axioms

These aren't separate axioms—they're theorems you can prove using only the three core axioms. Understanding the derivations helps you reconstruct formulas under exam pressure.

Complement Rule

• $P(A') = 1 - P(A)$ follows directly from additivity and unity—since $A$ and $A'$ are disjoint and $A \cup A' = S$
• Use this when "not happening" is easier to calculate—finding $P(\text{at least one})$ often means computing $1 - P(\text{none})$
• The complement rule converts difficult problems into simpler ones—a key problem-solving strategy throughout the course

Inclusion-Exclusion Principle

• For any two events: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$—we subtract the intersection to correct for double-counting
• Generalizes to $n$ events with alternating addition and subtraction—the pattern involves all possible intersections
• Reduces to additivity when $A \cap B = \emptyset$—inclusion-exclusion is the general case, additivity the special case

---

Conditional Probability and Dependence

Conditioning is how we update probabilities when we gain partial information. This concept bridges the axioms to real-world inference.

Conditional Probability

• $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$ for $P(B) > 0$—we rescale to the new sample space where $B$ has occurred
• Conditional probability is itself a valid probability measure—it satisfies all three axioms on the reduced sample space
• Independence means $P(A \mid B) = P(A)$—knowing $B$ occurred doesn't change our assessment of $A$

Law of Total Probability

• If $B_1, B_2, \ldots, B_n$ partition $S$, then $P(A) = \sum_{i=1}^{n} P(A \mid B_i)P(B_i)$—we decompose by cases
• "Partition" means mutually exclusive and exhaustive—every outcome belongs to exactly one $B_i$
• This law structures complex calculations—break the problem into scenarios, solve each, then combine

Bayes' Theorem

• $P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}$—this "flips" the conditioning direction
• The denominator often uses total probability—$P(B) = P(B \mid A)P(A) + P(B \mid A')P(A')$ for binary partitions
• Bayes' theorem is the engine of statistical inference—it tells us how to update beliefs (prior → posterior) given evidence

---

Quick Reference Table

---

Self-Check Questions

1. Which two axioms together imply that $P(A) \leq 1$ for any event $A$? Explain the logical chain.

2. You're told $P(A) = 0.4$, $P(B) = 0.5$, and $P(A \cup B) = 0.7$. Are $A$ and $B$ mutually exclusive? Are they independent? Show your reasoning.

3. Compare and contrast the additivity axiom with the inclusion-exclusion principle—when does each apply, and how is inclusion-exclusion derived from additivity?

4. A diagnostic test has $P(\text{positive} \mid \text{disease}) = 0.95$ and $P(\text{positive} \mid \text{no disease}) = 0.08$. If 2% of the population has the disease, find $P(\text{disease} \mid \text{positive})$. Which rules did you use?

5. Why must we require $P(B) > 0$ in the definition of conditional probability? What would go wrong mathematically if we allowed $P(B) = 0$?