1.2 Basic Probability Concepts and Axioms

Probability is the backbone of statistical inference. It gives us tools to measure uncertainty and make predictions. From coin flips to complex data analysis, probability concepts help us understand and quantify the likelihood of events occurring.

Random variables bridge the gap between abstract probability theory and real-world applications. Whether discrete like dice rolls or continuous like height measurements, random variables allow us to model and analyze various phenomena in fields ranging from medicine to finance.

Foundations of Probability

Definition of probability axioms

• Probability measures likelihood of event occurring ranges from 0 (impossible) to 1 (certain)
• Axioms of probability form mathematical foundation:
  • Non-negativity: $P(A) \geq 0$ for any event A ensures probabilities are always positive
  • Normalization: $P(S) = 1$, where S is sample space guarantees total probability is 1
  • Additivity: For mutually exclusive events A and B, $P(A \cup B) = P(A) + P(B)$ allows combining probabilities
• Sample space encompasses all possible outcomes in experiment (dice roll: 1-6)
• Event represents subset of sample space (rolling an even number: 2, 4, 6)

Rules for probability calculations

• Addition rule combines probabilities of events: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ (probability of rain or snow)
• Multiplication rule for independent events: $P(A \cap B) = P(A) \times P(B)$ (flipping two coins)
• Dependent events: $P(A \cap B) = P(A) \times P(B|A)$ (drawing cards without replacement)
• Conditional probability $P(A|B) = \frac{P(A \cap B)}{P(B)}$ calculates probability of A given B occurred
• Complement rule $P(A^c) = 1 - P(A)$ finds probability of event not occurring
• Law of total probability $P(A) = P(A|B_1)P(B_1) + P(A|B_2)P(B_2) + ... + P(A|B_n)P(B_n)$ considers all possible scenarios

Random Variables and Applications

Discrete vs continuous random variables

• Discrete random variables take countable distinct values:
  • Number of customers in store
  • Dice rolls (1-6)
  • Coin flips (heads or tails)
  • Described by probability mass function (PMF) $P(X = x)$ for each value x
• Continuous random variables take any value within range:
  • Height of person
  • Weight of object
  • Time to complete task
  • Described by probability density function (PDF), area under curve represents probability
• Cumulative distribution function (CDF) $F(x) = P(X \leq x)$ applies to both types, gives probability of variable being less than or equal to x

Applications of probability concepts

• Bayes' theorem $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$ used in:
  1. Medical diagnoses (probability of disease given test result)
  2. Spam filtering (probability email is spam given certain words)
  3. Machine learning (updating beliefs based on new data)
• Expected value calculates average outcome:
  • Discrete: $E(X) = \sum_{x} xP(X = x)$ (average dice roll)
  • Continuous: $E(X) = \int_{-\infty}^{\infty} xf(x)dx$ (average height in population)
• Variance $Var(X) = E[(X - \mu)^2]$ measures spread of distribution (consistency of dice rolls)
• Practical applications:
  • Finance: risk assessment, portfolio diversification
  • Insurance: policy pricing based on claim probability
  • Quality control: monitoring defect rates in manufacturing
  • Epidemiology: predicting disease spread in populations