3.4 Independence and Conditional Independence

Independence in statistics is crucial for understanding how events and variables relate. It simplifies calculations and helps us make sense of complex data. When things are independent, we can treat them separately, making analysis easier.

Conditional independence adds another layer, showing how relationships change when we consider specific conditions. This concept is super useful in real-world applications, from medical diagnoses to machine learning models. It helps us uncover hidden patterns in data.

Fundamentals of Independence

Independence of random variables

• Events A and B independent when $P(A \cap B) = P(A) \times P(B)$, occurrence of one doesn't affect probability of other
• Random variables X and Y independent if $P(X \leq x, Y \leq y) = P(X \leq x) \times P(X \leq y)$ for all x and y, joint distribution factors into product of marginals
• Simplifies joint probability computations, eases calculation of expectations and variances of sums, enables multiplication rule (coin flips, dice rolls)

Determining variable independence

• Compare joint distribution to product of marginals, equal for all values indicates independence
• Discrete variables: construct contingency table, check if each cell equals product of row and column totals (survey responses, card draws)
• Continuous variables: examine joint PDF, verify if expressible as product of marginal PDFs (height and weight, temperature and humidity)
• Zero covariance and correlation suggest independence, but not definitive proof

Conditional Independence and Applications

Conditional independence concept

• Events A and B conditionally independent given C if $P(A \cap B | C) = P(A | C) \times P(B | C)$
• Random variables X and Y conditionally independent given Z if $P(X \leq x, Y \leq y | Z = z) = P(X \leq x | Z = z) \times P(X \leq y | Z = z)$ for all x, y, z
• Doesn't imply unconditional independence and vice versa
• Uses conditional probabilities, allows dependencies to be "explained away" by third variable (medical symptoms given disease, academic performance given study habits)

Applications of independence concepts

• Simplify joint probability calculations in Bayesian networks and graphical models
• Assume observation independence in hypothesis testing, residual independence in regression analysis
• Test residual independence using autocorrelation in time series analysis
• Naive Bayes classifier assumes conditional feature independence given class
• Hidden Markov Models utilize conditional independence assumptions
• Randomization ensures treatment group independence in experimental design
• Simple random sampling ensures observation independence, cluster sampling considers within-cluster dependencies