📈Preparatory Statistics Unit 6 – Probability Rules & Conditional Probability

Key Concepts and Definitions

• Probability measures the likelihood of an event occurring ranges from 0 (impossible) to 1 (certain)
• Sample space ($S$) set of all possible outcomes of an experiment or random process
• Event ($E$) subset of the sample space represents one or more outcomes of interest
• Mutually exclusive events cannot occur at the same time (rolling a 1 and a 2 on a die)
• Collectively exhaustive events cover all possible outcomes in the sample space
• Independent events occurrence of one event does not affect the probability of another event (flipping a coin twice)
• Dependent events probability of one event is influenced by the occurrence of another event (drawing cards without replacement)
• Complement of an event ($E^c$ or $\overline{E}$) includes all outcomes in the sample space that are not in the event $E$

Basic Probability Rules

• Addition rule for mutually exclusive events: $P(A \cup B) = P(A) + P(B)$
• Addition rule for non-mutually exclusive events: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
• Multiplication rule for independent events: $P(A \cap B) = P(A) \times P(B)$
• Multiplication rule for dependent events: $P(A \cap B) = P(A) \times P(B|A)$
  • $P(B|A)$ conditional probability of event $B$ given that event $A$ has occurred
• Complement rule: $P(E^c) = 1 - P(E)$
• Law of total probability: $P(B) = P(A_1 \cap B) + P(A_2 \cap B) + \ldots + P(A_n \cap B)$
  • $A_1, A_2, \ldots, A_n$ partition the sample space into mutually exclusive and collectively exhaustive events

Types of Events

• Simple event consists of a single outcome (rolling a 3 on a die)
• Compound event combination of two or more simple events (drawing a king or a queen from a deck of cards)
• Mutually exclusive events cannot occur simultaneously (drawing a red card and a black card from a deck in a single draw)
• Independent events occurrence of one event does not affect the probability of another event (rolling a die and flipping a coin)
• Dependent events probability of one event is influenced by the occurrence of another event (selecting marbles from a bag without replacement)
• Complementary events an event and its complement make up the entire sample space (passing or failing an exam)

Probability Calculations

• Classical probability: $P(E) = \frac{\text{number of favorable outcomes}}{\text{total number of possible outcomes}}$
  • Assumes equally likely outcomes (rolling a fair die)
• Empirical probability: $P(E) = \frac{\text{frequency of event E}}{\text{total number of trials}}$
  • Based on observed data or experiments (calculating the probability of heads after 100 coin flips)
• Subjective probability assigns probabilities based on personal belief or judgment
  • Used when limited information is available (estimating the probability of a team winning a game)
• Expected value: $E(X) = \sum_{i=1}^{n} x_i \times P(X = x_i)$
  • $X$ random variable, $x_i$ possible values of $X$, $P(X = x_i)$ probability of $X$ taking the value $x_i$
• Variance: $Var(X) = E[(X - E(X))^2] = E(X^2) - [E(X)]^2$
  • Measures the spread of a random variable around its expected value

Conditional Probability

• Conditional probability: $P(A|B) = \frac{P(A \cap B)}{P(B)}$
  • Probability of event $A$ occurring given that event $B$ has occurred
• Independence: $P(A|B) = P(A)$ and $P(B|A) = P(B)$
  • Events $A$ and $B$ are independent if the occurrence of one does not affect the probability of the other
• Multiplication rule for conditional probability: $P(A \cap B) = P(A|B) \times P(B) = P(B|A) \times P(A)$
• Chain rule for conditional probability: $P(A \cap B \cap C) = P(A|B \cap C) \times P(B|C) \times P(C)$
  • Extends to more than three events
• Conditional probability tree diagrams visually represent the relationships between events and their probabilities
  • Useful for solving complex conditional probability problems

Bayes' Theorem

• Bayes' theorem: $P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$
  • Relates the conditional probabilities of events $A$ and $B$
• Prior probability: $P(A)$ initial probability of event $A$ before considering any additional information
• Posterior probability: $P(A|B)$ updated probability of event $A$ after considering the additional information (event $B$)
• Likelihood: $P(B|A)$ probability of observing event $B$ given that event $A$ has occurred
• Normalizing constant: $P(B) = P(B|A) \times P(A) + P(B|A^c) \times P(A^c)$
  • Ensures the posterior probabilities sum to 1
• Bayes' theorem is used in various fields (medical diagnosis, machine learning, spam filters) to update probabilities based on new evidence

Practical Applications

• Quality control testing products for defects and calculating the probability of accepting or rejecting a batch
• Insurance companies use probability to determine premiums based on the likelihood of claims
• Medical diagnosis calculating the probability of a patient having a disease given their test results (sensitivity and specificity)
• Machine learning algorithms use probability to classify data and make predictions (spam filters, recommendation systems)
• Genetics predicting the probability of inheriting certain traits based on parental genotypes (Punnett squares)
• Weather forecasting estimating the probability of rain, snow, or other weather events based on historical data and current conditions
• Financial risk management assessing the probability of investment losses or defaults to make informed decisions

Common Mistakes and Tips

• Confusing mutually exclusive and independent events
  • Mutually exclusive events cannot occur simultaneously, while independent events do not affect each other's probabilities
• Forgetting to account for the intersection when adding probabilities of non-mutually exclusive events
  • Use the addition rule: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
• Misinterpreting conditional probability as the probability of the condition
  • $P(A|B)$ is the probability of event $A$ given that event $B$ has occurred, not the probability of event $B$
• Incorrectly assuming events are independent without verifying the conditions
  • Check if $P(A|B) = P(A)$ and $P(B|A) = P(B)$ to confirm independence
• Misapplying Bayes' theorem by confusing the terms or forgetting the normalizing constant
  • Carefully identify the prior probability, likelihood, and normalizing constant in the given context
• Double-check calculations and ensure probabilities are between 0 and 1
• Practice solving a variety of problems to develop a strong understanding of the concepts and techniques
• Utilize visual aids (Venn diagrams, tree diagrams) to organize information and solve complex problems