2.3 Expectation and Variance

Random variables are the building blocks of statistical inference. They help us model uncertainty and variability in real-world phenomena. Understanding their expected values and variances is crucial for making predictions and drawing conclusions from data.

Expectation gives us the average outcome, while variance measures spread. These concepts are fundamental in fields like finance, insurance, and scientific research. They allow us to quantify risk, estimate probabilities, and make informed decisions based on statistical analysis.

Expectation of Random Variables

Expected value calculation

• Expected value (E[X]) measures central tendency of random variable, represents average outcome over many trials
• Discrete random variables: $E[X] = \sum_{x} x \cdot P(X=x)$, sum each value multiplied by probability (coin flips, dice rolls)
• Continuous random variables: $E[X] = \int_{-\infty}^{\infty} x \cdot f(x) dx$, integral of value and probability density function (height, weight)

Long-run average interpretation

• Theoretical mean of infinite repetitions, converges to expected value as sample size increases
• Law of Large Numbers states sample mean approaches expected value with larger samples
• Applied in gambling (roulette), insurance premiums, and financial modeling (stock returns)

Variance and Properties

Variance and standard deviation

• Variance (Var(X)) measures spread around expected value: $Var(X) = E[(X - E[X])^2]$
• Standard deviation: $SD(X) = \sqrt{Var(X)}$, same units as original data
• Discrete: $Var(X) = \sum_{x} (x - E[X])^2 \cdot P(X=x)$ (exam scores, number of customers)
• Continuous: $Var(X) = \int_{-\infty}^{\infty} (x - E[X])^2 \cdot f(x) dx$ (temperature, rainfall)

Properties of expectation and variance

• Linearity of expectation: $E[aX + b] = aE[X] + b$, $E[X + Y] = E[X] + E[Y]$
• Variance properties: $Var(aX + b) = a^2Var(X)$, $Var(X + Y) = Var(X) + Var(Y)$ for independent variables
• Independence and covariance: Cov(X, Y) = 0 for independent variables
• Expectation of products: $E[XY] = E[X]E[Y]$ for independent variables (investment returns, biological measurements)

Law of unconscious statistician

• LOTUS calculates expectations of functions of random variables
• Discrete: $E[g(X)] = \sum_{x} g(x) \cdot P(X=x)$
• Continuous: $E[g(X)] = \int_{-\infty}^{\infty} g(x) \cdot f(x) dx$
• Used for moments of random variables, deriving probability distributions of transformed variables
• Relates to change of variables technique for complex expectation calculations (squared returns, logarithmic transformations)