7.2 Properties of variance

Variance and Standard Deviation

Definition and Basic Properties

Variance measures how spread out a random variable's values are from its mean. It's defined as the expected value of the squared deviation from the mean:

$Var(X) = E[(X - \mu)^2]$

You'll also see it written as $\sigma^2(X)$. Standard deviation is just the square root of variance, denoted $\sigma(X)$ or $SD(X)$. The reason standard deviation gets used so often is that it's in the same units as the original data, while variance is in squared units.

A few core properties to know:

• Variance is always non-negative. It equals zero only when the random variable is constant (no spread at all).
• Adding a constant shifts the distribution but doesn't change the spread: $Var(X + c) = Var(X)$
• Scaling by a constant $a$ squares the effect on variance: $Var(aX) = a^2 \cdot Var(X)$
• For standard deviation, the scaling rule uses absolute value: $SD(aX) = |a| \cdot SD(X)$
• For independent random variables $X$ and $Y$: $Var(X + Y) = Var(X) + Var(Y)$

That squaring in the scaling rule is what makes variance "non-linear." If you double $X$, the variance quadruples, not doubles. This trips people up on exams.

Interpretation and Significance

• Larger variance means data points are more spread out from the mean (think volatile stock prices).
• Smaller variance means values cluster tightly around the mean (think consistent manufacturing output).
• Variance and standard deviation show up everywhere: risk assessment in finance, quality control in engineering, confidence intervals and hypothesis testing in statistics.
• Standard deviation is often preferred for interpretation because its units match the data.

Calculating Variance and Standard Deviation

Discrete Random Variables

For a discrete random variable $X$ with probability mass function $p(x)$, here's the process:

1. Find the mean: $\mu = E(X) = \sum x \cdot p(x)$

2. Compute variance using the definition: $Var(X) = \sum (x - \mu)^2 \cdot p(x)$

3. Take the square root for standard deviation: $SD(X) = \sqrt{Var(X)}$

There's also a shortcut formula that's often easier to compute:

$Var(X) = E(X^2) - [E(X)]^2$

where $E(X^2) = \sum x^2 \cdot p(x)$. This avoids subtracting the mean from every value. Both formulas give the same result, so use whichever feels cleaner for the problem.

Calculation Examples

Fair six-sided die:

• Each outcome has probability $p(x) = 1/6$ for $x = 1, 2, 3, 4, 5, 6$
• Mean: $\mu = (1+2+3+4+5+6)/6 = 3.5$
• Variance: $Var(X) = \sum (x - 3.5)^2 \cdot (1/6) = \frac{(2.5)^2 + (1.5)^2 + (0.5)^2 + (0.5)^2 + (1.5)^2 + (2.5)^2}{6} = \frac{17.5}{6} \approx 2.917$

Biased coin with P(Heads) = 0.6:

• Let $X = 1$ for Heads, $X = 0$ for Tails
• Mean: $\mu = 1(0.6) + 0(0.4) = 0.6$
• Variance: $Var(X) = (1 - 0.6)^2(0.6) + (0 - 0.6)^2(0.4) = (0.16)(0.6) + (0.36)(0.4) = 0.096 + 0.144 = 0.24$

This is a Bernoulli random variable, and you can verify with the general Bernoulli variance formula: $Var(X) = p(1-p) = 0.6 \times 0.4 = 0.24$.

Variance and Standard Deviation for Continuous Variables

Computation Methods

For a continuous random variable $X$ with probability density function $f(x)$, the formulas mirror the discrete case but with integrals replacing sums:

1. Find the mean: $\mu = E(X) = \int x \cdot f(x) \, dx$

2. Compute variance: $Var(X) = \int (x - \mu)^2 \cdot f(x) \, dx$

3. Take the square root: $SD(X) = \sqrt{Var(X)}$

The shortcut formula works here too: $Var(X) = E(X^2) - [E(X)]^2$ where $E(X^2) = \int x^2 \cdot f(x) \, dx$. All integrals are taken over the entire support of $f(x)$.

Application to Specific Distributions

For common continuous distributions, the variance formulas are already worked out:

• Uniform distribution $U(a, b)$: $Var(X) = \frac{(b-a)^2}{12}$
  • Example: $U(0,1)$ has variance $\frac{(1-0)^2}{12} = \frac{1}{12} \approx 0.0833$
• Exponential distribution with rate $\lambda$: $Var(X) = \frac{1}{\lambda^2}$
  • Example: $\text{Exponential}(0.5)$ has variance $\frac{1}{(0.5)^2} = 4$
• Normal distribution $N(\mu, \sigma^2)$: Variance is $\sigma^2$ by definition.
  • Example: The standard normal $N(0,1)$ has variance 1.

Knowing these formulas saves you from doing the integral every time. On exams, you'll typically be expected to recognize which distribution you're working with and apply the right formula.

Variance Properties for Independent Variables

Additive Properties

These properties only hold when the random variables are independent:

• Sum: $Var(X + Y) = Var(X) + Var(Y)$
• Difference: $Var(X - Y) = Var(X) + Var(Y)$

That second one catches many students off guard. Variance measures spread, and subtracting one random variable from another still introduces variability from both. The variances add either way.

More generally, for a linear combination with constants $a$ and $b$:

$Var(aX + bY) = a^2 Var(X) + b^2 Var(Y)$

And for standard deviation of a sum or difference:

$SD(X \pm Y) = \sqrt{Var(X) + Var(Y)}$

These extend naturally to more variables. For independent $X, Y, Z$:

$Var(X + Y + Z) = Var(X) + Var(Y) + Var(Z)$

Applications and Limitations

• Portfolio analysis: If two investments have independent returns, the variance of the combined portfolio uses the additive property. This is the math behind diversification.
• Error propagation: When combining multiple independent measurement sources, total error variance is the sum of individual error variances.
• Experimental design: In multi-factor experiments, overall variability comes from adding the variance contributions of each independent factor.

The big limitation: if $X$ and $Y$ are not independent, you need a covariance term:

$Var(X + Y) = Var(X) + Var(Y) + 2\text{Cov}(X, Y)$

Forgetting this when variables are dependent is a common mistake. Always check whether independence is stated in the problem before using the simpler additive rule. Also note that these linear combination rules don't generalize to non-linear functions of random variables; for something like $Var(XY)$, you'd need a different approach entirely.