Conditional Probability & Independence | Intro to Probabilistic Methods Class Notes

Conditional probability and independence are fundamental concepts in probability theory. They help us understand how events relate to each other and calculate probabilities in complex scenarios. These ideas are crucial for analyzing real-world situations and making informed decisions based on available information. Mastering conditional probability and independence enables us to solve problems in various fields, from medical diagnosis to risk assessment. By understanding these concepts, we can update our beliefs based on new evidence and make more accurate predictions in uncertain situations.

2.1

Conditional probability and the multiplication rule

2.2

Total probability theorem and Bayes' theorem

2.3

Independence of events

unit 2 review

Key Concepts

Conditional probability measures the probability of an event A occurring given that another event B has already occurred, denoted as P(A|B)
Independence in probability means that the occurrence of one event does not affect the probability of another event
- Two events A and B are independent if P(A|B) = P(A) and P(B|A) = P(B)
The multiplication rule for independent events states that the probability of the intersection of two independent events is the product of their individual probabilities: P(A ∩ B) = P(A) × P(B)
Bayes' theorem relates conditional probabilities and can be used to update probabilities based on new information
The law of total probability states that the probability of an event A can be calculated by summing the probabilities of A occurring given each possible outcome of another event B
The concept of mutual exclusivity is important in conditional probability, as it affects the calculation of probabilities for multiple events

Conditional Probability Basics

Conditional probability is calculated using the formula: P(A|B) = P(A ∩ B) / P(B), where P(B) > 0
The probability of the intersection of two events, P(A ∩ B), can be calculated using the multiplication rule: P(A ∩ B) = P(A|B) × P(B)
Conditional probability is not commutative, meaning that P(A|B) is not necessarily equal to P(B|A)
The complement rule for conditional probability states that P(A'|B) = 1 - P(A|B), where A' is the complement of event A
Conditional probability can be visualized using tree diagrams or Venn diagrams
- In a tree diagram, each branch represents a possible outcome, and the probabilities along the branches are conditional probabilities
- In a Venn diagram, the overlapping region represents the intersection of two events, and the conditional probability is the ratio of the area of the intersection to the area of the given event

Independence in Probability

If two events A and B are independent, then the occurrence of one event does not affect the probability of the other event occurring
The multiplication rule for independent events simplifies to P(A ∩ B) = P(A) × P(B) when A and B are independent
If A and B are independent, then their complements A' and B' are also independent
Pairwise independence does not imply mutual independence for three or more events
- Pairwise independence means that each pair of events is independent, but the events may not be independent when considered together
Conditional independence means that two events A and B are independent given the occurrence of a third event C
- Mathematically, P(A ∩ B|C) = P(A|C) × P(B|C)
Independence is a strong assumption and should be verified before applying it to solve problems

Formulas and Calculations

The multiplication rule for dependent events: P(A ∩ B) = P(A) × P(B|A)
Bayes' theorem: P(A|B) = (P(B|A) × P(A)) / P(B)
- This formula is useful for updating probabilities based on new information or evidence
The law of total probability: P(A) = P(A|B₁) × P(B₁) + P(A|B₂) × P(B₂) + ... + P(A|Bₙ) × P(Bₙ), where B₁, B₂, ..., Bₙ are mutually exclusive and exhaustive events
The addition rule for mutually exclusive events: P(A ∪ B) = P(A) + P(B)
The general addition rule for non-mutually exclusive events: P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
When solving conditional probability problems, it is essential to identify the given information and the desired probability, then select the appropriate formula to calculate the result

Real-World Applications

Conditional probability is used in medical diagnosis to determine the probability of a patient having a disease given their symptoms or test results
- For example, calculating the probability of a patient having COVID-19 given a positive test result
In machine learning, conditional probability is used in classification algorithms such as Naive Bayes classifiers
Conditional probability is applied in risk assessment and decision-making in various fields, such as finance, insurance, and engineering
- For instance, calculating the probability of a loan defaulting given the borrower's credit score and income
Independence assumptions are often used in modeling complex systems to simplify calculations and make problems more tractable
Bayes' theorem is used in spam email filters to update the probability of an email being spam based on the presence of certain words or phrases
In genetics, conditional probability is used to calculate the probability of an offspring inheriting specific traits given the genotypes of the parents

Common Mistakes to Avoid

Confusing conditional probability with joint probability
- Joint probability, P(A ∩ B), is the probability of both events A and B occurring, while conditional probability, P(A|B), is the probability of event A occurring given that event B has already occurred
Assuming events are independent without proper justification
- Independence should be verified using the definition or by checking if the multiplication rule for independent events holds
Incorrectly applying the multiplication rule for dependent events
- When events are dependent, the multiplication rule is P(A ∩ B) = P(A) × P(B|A), not P(A) × P(B)
Misinterpreting conditional probability statements
- For example, confusing P(A|B) with P(B|A) or misinterpreting the meaning of the given event and the event being conditioned on
Neglecting to check if events are mutually exclusive or exhaustive when applying the law of total probability or the addition rule
Attempting to calculate probabilities without clearly defining the sample space and the events involved

Practice Problems

A bag contains 4 red balls and 6 blue balls. If two balls are drawn at random without replacement, what is the probability that both balls are red?
In a certain population, 10% of people are left-handed, and 25% have blue eyes. If having blue eyes and being left-handed are independent traits, what is the probability that a randomly selected person has blue eyes and is left-handed?
A factory produces widgets in three sizes: small, medium, and large. The probability of a widget being small is 0.3, medium is 0.5, and large is 0.2. The probability of a defective widget given that it is small is 0.05, medium is 0.02, and large is 0.01. If a widget is selected at random and found to be defective, what is the probability that it is a medium-sized widget?
Two events A and B have the following probabilities: P(A) = 0.6, P(B) = 0.4, and P(A ∩ B) = 0.3. Are events A and B independent? Justify your answer.
A medical test for a rare disease has a 95% accuracy rate for detecting the disease when it is present and a 98% accuracy rate for correctly identifying the absence of the disease when it is not present. If 0.1% of the population has this rare disease, what is the probability that a person has the disease given that they test positive?

Intro to Probabilistic Methods Unit 2 ReviewConditional Probability & Independence