🃏Engineering Probability Unit 2 – Probability Axioms and Set Theory

Key Concepts and Definitions

• Probability a branch of mathematics dealing with the likelihood of an event occurring
• Sample space the set of all possible outcomes of an experiment or random process, denoted by $S$
• Event a subset of the sample space, representing a collection of outcomes
• Probability of an event a measure of the likelihood that the event will occur, denoted by $P(A)$ for event $A$
• Mutually exclusive events events that cannot occur simultaneously (rolling a 1 and a 2 on a single die roll)
• Independent events events where the occurrence of one does not affect the probability of the other (flipping a coin and rolling a die)
• Conditional probability the probability of an event occurring given that another event has already occurred, denoted by $P(A|B)$ for events $A$ and $B$
• Complement of an event the set of all outcomes in the sample space that are not in the given event, denoted by $A^c$ for event $A$

Set Theory Fundamentals

• Set a well-defined collection of distinct objects
• Element an object that belongs to a set
• Subset a set whose elements are all contained within another set, denoted by $A \subseteq B$ for sets $A$ and $B$
  • Proper subset a subset that is not equal to the original set, denoted by $A \subset B$
• Union of sets the set containing all elements that belong to at least one of the given sets, denoted by $A \cup B$ for sets $A$ and $B$
• Intersection of sets the set containing all elements that belong to all of the given sets, denoted by $A \cap B$ for sets $A$ and $B$
• Difference of sets the set containing all elements that belong to the first set but not the second, denoted by $A \setminus B$ or $A - B$ for sets $A$ and $B$
• Cartesian product of sets the set of all ordered pairs $(a, b)$ where $a$ belongs to the first set and $b$ belongs to the second set, denoted by $A \times B$ for sets $A$ and $B$

Probability Axioms

• Non-negativity axiom the probability of any event is greater than or equal to zero, $P(A) \geq 0$ for any event $A$
• Normalization axiom the probability of the entire sample space is equal to one, $P(S) = 1$
• Additivity axiom for any two mutually exclusive events $A$ and $B$, the probability of their union is the sum of their individual probabilities, $P(A \cup B) = P(A) + P(B)$
  • Generalized additivity axiom for any countable sequence of mutually exclusive events $A_1, A_2, \ldots$, the probability of their union is the sum of their individual probabilities, $P(\bigcup_{i=1}^{\infty} A_i) = \sum_{i=1}^{\infty} P(A_i)$
• Complementary event axiom the sum of the probabilities of an event and its complement is equal to one, $P(A) + P(A^c) = 1$ for any event $A$
• Monotonicity axiom if $A \subseteq B$, then $P(A) \leq P(B)$ for any events $A$ and $B$

Set Operations and Probability

• Union of events the probability of the union of two events $A$ and $B$ is given by $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
  • Generalized union formula for $n$ events $A_1, A_2, \ldots, A_n$, the probability of their union is given by the inclusion-exclusion principle
• Intersection of events the probability of the intersection of two events $A$ and $B$ is given by $P(A \cap B) = P(A) + P(B) - P(A \cup B)$
• Difference of events the probability of the difference of two events $A$ and $B$ is given by $P(A \setminus B) = P(A) - P(A \cap B)$
• Independent events for independent events $A$ and $B$, the probability of their intersection is the product of their individual probabilities, $P(A \cap B) = P(A) \cdot P(B)$
  • Generalized independence for $n$ events $A_1, A_2, \ldots, A_n$, they are independent if and only if for any subset of indices $\{i_1, i_2, \ldots, i_k\}$, $P(\bigcap_{j=1}^{k} A_{i_j}) = \prod_{j=1}^{k} P(A_{i_j})$

Venn Diagrams and Probability

• Venn diagram a graphical representation of sets and their relationships using overlapping circles or other shapes
• Visualizing unions the union of two sets $A$ and $B$ is represented by the entire shaded region in a Venn diagram
• Visualizing intersections the intersection of two sets $A$ and $B$ is represented by the overlapping region in a Venn diagram
• Visualizing complements the complement of a set $A$ is represented by the region outside the circle representing set $A$ in a Venn diagram
• Visualizing mutually exclusive events mutually exclusive events have no overlapping region in a Venn diagram
• Visualizing independent events independent events have an overlapping region that is proportional to the product of their individual areas in a Venn diagram

Sample Spaces and Events

• Discrete sample space a sample space that consists of a finite or countably infinite number of outcomes (rolling a die, flipping a coin)
• Continuous sample space a sample space that consists of an uncountably infinite number of outcomes (selecting a random point on a line segment)
• Simple event an event that consists of a single outcome from the sample space
• Compound event an event that consists of multiple outcomes from the sample space
• Equally likely outcomes outcomes that have the same probability of occurring (each side of a fair die)
  • For equally likely outcomes, the probability of an event $A$ is given by $P(A) = \frac{|A|}{|S|}$, where $|A|$ is the number of outcomes in event $A$ and $|S|$ is the total number of outcomes in the sample space

Conditional Probability

• Conditional probability the probability of an event $A$ occurring given that another event $B$ has already occurred, denoted by $P(A|B)$
  • Formula for conditional probability $P(A|B) = \frac{P(A \cap B)}{P(B)}$, where $P(B) > 0$
• Multiplication rule for any two events $A$ and $B$, $P(A \cap B) = P(A) \cdot P(B|A) = P(B) \cdot P(A|B)$
  • Generalized multiplication rule for $n$ events $A_1, A_2, \ldots, A_n$, $P(\bigcap_{i=1}^{n} A_i) = P(A_1) \cdot P(A_2|A_1) \cdot P(A_3|A_1 \cap A_2) \cdots P(A_n|\bigcap_{i=1}^{n-1} A_i)$
• Law of total probability for a partition of the sample space $\{B_1, B_2, \ldots, B_n\}$ and any event $A$, $P(A) = \sum_{i=1}^{n} P(A \cap B_i) = \sum_{i=1}^{n} P(A|B_i) \cdot P(B_i)$
• Bayes' theorem for events $A$ and $B$, $P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$, where $P(B) > 0$
  • Generalized Bayes' theorem for a partition of the sample space $\{A_1, A_2, \ldots, A_n\}$ and any event $B$, $P(A_i|B) = \frac{P(B|A_i) \cdot P(A_i)}{\sum_{j=1}^{n} P(B|A_j) \cdot P(A_j)}$, where $P(B) > 0$

Real-World Applications

• Quality control using probability to determine the likelihood of defective products in a manufacturing process
• Risk assessment evaluating the probability of adverse events in various industries (finance, healthcare, engineering)
• Genetics calculating the probability of inheriting specific traits based on parental genotypes and Mendelian inheritance
• Machine learning and artificial intelligence using probability theory to develop algorithms for classification, regression, and decision-making tasks
• Cryptography employing probability theory to analyze the security of cryptographic systems and develop secure communication protocols
• Weather forecasting using probabilistic models to predict the likelihood of various weather events based on historical data and current conditions
• Insurance industry setting premiums and assessing risk based on the probability of certain events occurring (accidents, natural disasters, health issues)
• Quantum mechanics applying probability theory to describe the behavior of subatomic particles and develop models for quantum systems