4.3 Expectation and variance

Expectation and variance are crucial concepts in continuous random variables. They help us understand the average value and spread of a distribution. These measures provide key insights into the behavior of random variables, forming the foundation for statistical analysis and probability theory.

Calculating expectation and variance often involves integration techniques. By mastering these calculations, we can better characterize continuous distributions and make informed predictions. This knowledge is essential for understanding more complex probabilistic concepts and their real-world applications.

Expectation of Continuous Random Variables

Definition and Interpretation

Top images from around the web for Definition and Interpretation

• 

  Introduction to Continuous Random Variables | Introduction to Statistics View original

  Is this image relevant?

• 

  Probability density function - Wikipedia View original

  Is this image relevant?

• 

  Continuous Probability Distribution (2 of 2) | Concepts in Statistics View original

  Is this image relevant?

• 

  Introduction to Continuous Random Variables | Introduction to Statistics View original

  Is this image relevant?

• 

  Probability density function - Wikipedia View original

  Is this image relevant?

1 of 3

Top images from around the web for Definition and Interpretation

• 

  Introduction to Continuous Random Variables | Introduction to Statistics View original

  Is this image relevant?

• 

  Probability density function - Wikipedia View original

  Is this image relevant?

• 

  Continuous Probability Distribution (2 of 2) | Concepts in Statistics View original

  Is this image relevant?

• 

  Introduction to Continuous Random Variables | Introduction to Statistics View original

  Is this image relevant?

• 

  Probability density function - Wikipedia View original

  Is this image relevant?

1 of 3

• Expectation for continuous random variables, denoted as E[X], represents the average or mean value of the random variable over its entire range
• Calculated using the probability density function (PDF) of the continuous random variable
• For a continuous random variable X with PDF f(x), the expectation defined as $E[X] = \int x f(x) dx$, where the integral taken over the entire support of X
• Provides a measure of central tendency for the distribution of the continuous random variable
• Interpreted geometrically as the center of mass of the probability distribution
• May not always exist for all continuous distributions, particularly those with heavy tails or infinite support (Cauchy distribution)
• Functions as a key parameter in characterizing the distribution of a continuous random variable when it exists

Mathematical Properties and Considerations

• Expectation may be finite or infinite, depending on the distribution
• For symmetric distributions, expectation coincides with the median and mode (normal distribution)
• Expectation sensitive to outliers in skewed distributions (log-normal distribution)
• Existence of expectation requires the integral $\int |x| f(x) dx$ to be finite
• Higher moments of a distribution defined using expectations (skewness, kurtosis)
• Expectation used in defining other important concepts (variance, covariance, moment-generating functions)

Calculating Expectation with Integration

Integration Techniques

• Compute expectation using the integral $E[X] = \int x f(x) dx$, where f(x) the probability density function of X
• Integration limits cover the entire support of the random variable X
• Calculate expectation for piecewise continuous probability density functions by summing integrals over each piece of the function
• Determine expectation of a function g(X) of a random variable X using $E[g(X)] = \int g(x) f(x) dx$
• Apply proper techniques for evaluating definite integrals (integration by parts, substitution)
• Use closed-form expressions for expectations of common continuous distributions (normal, exponential)
• Employ numerical integration techniques for complex probability density functions without analytical solutions (Monte Carlo methods)

Practical Considerations and Examples

• Consider the support of the random variable when setting up the integral (uniform distribution on [a,b])
• Handle improper integrals carefully for distributions with infinite support (exponential distribution)
• Simplify calculations by recognizing symmetry in probability density functions (normal distribution)
• Utilize the linearity of expectation to break down complex functions into simpler components
• Apply change of variables technique for transformed random variables (log-normal distribution)
• Leverage known results for standard distributions to compute expectations of related distributions (chi-square distribution)

Properties of Expectation

Linearity and Additivity

• Linearity of expectation states $E[aX + bY] = aE[X] + bE[Y]$ for constants a and b, and random variables X and Y
• Extends linearity property to any finite linear combination of random variables
• Apply linearity to simplify calculations involving multiple random variables (portfolio returns)
• Utilize linearity in proofs and derivations of more complex probabilistic results
• Expectation of a constant c equals the constant itself: $E[c] = c$
• Leverage additivity to compute expectations of sums of independent random variables (compound Poisson process)

Monotonicity and Inequalities

• Monotonicity of expectation implies $E[X] \leq E[Y]$ if X ≤ Y (almost surely)
• For a non-negative random variable X, $E[X] \geq 0$
• Apply Jensen's inequality for convex functions: $E[g(X)] \geq g(E[X])$ (financial option pricing)
• Use law of the unconscious statistician (LOTUS) to calculate $E[g(X)]$ using the distribution of X rather than deriving the distribution of g(X)
• Employ Markov's inequality to bound probabilities: $P(X \geq a) \leq \frac{E[X]}{a}$ for non-negative X and positive a
• Utilize Chebyshev's inequality to bound deviations from the mean: $P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}$

Variance of Continuous Random Variables

Definition and Interpretation

• Variance, denoted as Var(X) or σ², measures the spread or dispersion of a continuous random variable around its expectation
• Define variance for a continuous random variable X as $Var(X) = E[(X - \mu)^2]$, where $\mu = E[X]$
• Use alternative formula for variance: $Var(X) = E[X^2] - (E[X])^2$, often easier to compute
• Calculate standard deviation as the square root of variance, $\sigma = \sqrt{Var(X)}$, in the same units as the random variable
• Variance always non-negative, equals zero if and only if the random variable constant (with probability 1)
• May not exist for all continuous distributions, particularly those with very heavy tails (Cauchy distribution)
• Functions as an important parameter in statistical analyses and probability models, characterizing the variability of the distribution

Properties and Applications

• Variance invariant under location shifts: $Var(X + c) = Var(X)$ for any constant c
• Scale variance by a constant: $Var(aX) = a^2Var(X)$ for any constant a
• Utilize variance in risk assessment and portfolio theory (Sharpe ratio)
• Apply Chebyshev's inequality using variance to bound probabilities of deviations from the mean
• Compare variances of different distributions to assess relative spread (coefficient of variation)
• Use variance in statistical inference (confidence intervals, hypothesis testing)

Calculating Variance with Integration

Integration Techniques

• Calculate variance using integration by first computing $E[X]$ and $E[X^2]$ separately using $E[g(X)] = \int g(x) f(x) dx$
• Apply the formula $Var(X) = E[X^2] - (E[X])^2$ to find the variance
• Compute $E[X^2]$ using the integral $E[X^2] = \int x^2 f(x) dx$ over the support of X
• Employ more advanced integration techniques for $E[X^2]$ due to the squared term (integration by parts)
• Consider using the definition $Var(X) = E[(X - \mu)^2]$ directly for some distributions, especially if $E[X]$ known
• Simplify calculations using properties of variance, such as $Var(aX + b) = a^2Var(X)$ for constants a and b
• Utilize closed-form expressions for variance of common continuous distributions without explicit integration (normal, exponential)

Practical Considerations and Examples

• Handle improper integrals carefully when calculating variance for distributions with infinite support (Pareto distribution)
• Leverage symmetry of probability density functions to simplify variance calculations (normal distribution)
• Apply transformation techniques to compute variances of functions of random variables (log-normal distribution)
• Use moment-generating functions to derive variances for some distributions (gamma distribution)
• Consider numerical methods for complex probability density functions without analytical solutions (bootstrap methods)
• Employ variance decomposition techniques for compound distributions (negative binomial distribution)