All Study Guides Theoretical Statistics Unit 2
📈 Theoretical Statistics Unit 2 – Random Variables and Probability DistributionsRandom variables and probability distributions form the backbone of statistical analysis. They provide a framework for modeling uncertainty and variability in real-world phenomena, allowing us to quantify and predict outcomes in various fields.
This unit covers key concepts like types of random variables, probability mass and density functions, and common distributions. We explore properties such as expectation, variance, and transformations, laying the groundwork for advanced statistical techniques and practical applications.
Key Concepts and Definitions
Random variable assigns a numerical value to each outcome in a sample space
Probability distribution describes the likelihood of different values occurring for a random variable
Cumulative distribution function (CDF) gives the probability that a random variable is less than or equal to a specific value
Probability mass function (PMF) defines the probability of a discrete random variable taking on a specific value
Probability density function (PDF) describes the relative likelihood of a continuous random variable falling within a particular range of values
PDF is used to calculate probabilities for continuous random variables
Area under the PDF curve between two points represents the probability of the variable falling within that range
Expected value (mean) of a random variable is the average value obtained if an experiment is repeated many times
Variance measures how far a random variable deviates from its expected value
Types of Random Variables
Discrete random variables can only take on a countable number of distinct values (integers, whole numbers)
Examples include the number of heads in a coin toss or the number of defective items in a batch
Continuous random variables can take on any value within a specified range or interval
Commonly represented by real numbers
Examples include height, weight, or time taken to complete a task
Mixed random variables have both discrete and continuous components
Bernoulli random variable is a special case of a discrete random variable with only two possible outcomes (success or failure)
Random vectors are ordered collections of random variables
Used in multivariate analysis and modeling joint distributions
Probability Distributions
Probability distributions assign probabilities to the possible values of a random variable
Discrete probability distributions are used for discrete random variables
Probabilities are assigned to each possible value
Examples include the binomial distribution and Poisson distribution
Continuous probability distributions are used for continuous random variables
Probabilities are assigned to ranges of values
Examples include the normal distribution and exponential distribution
Joint probability distributions describe the probabilities of multiple random variables occurring together
Marginal probability distributions are obtained by summing or integrating joint distributions over the values of one or more variables
Conditional probability distributions describe the probabilities of one random variable given the values of another
Properties of Distributions
Symmetry indicates that the probability distribution is the same when reflected about a central point
Normal distribution is an example of a symmetric distribution
Skewness measures the asymmetry of a probability distribution
Positive skewness has a longer right tail, negative skewness has a longer left tail
Kurtosis measures the heaviness of the tails of a distribution compared to a normal distribution
Higher kurtosis indicates heavier tails and more extreme values
Moments are quantitative measures that describe the shape and properties of a probability distribution
First moment is the mean, second moment is the variance, third moment is skewness, fourth moment is kurtosis
Moment-generating functions are used to calculate moments and characterize probability distributions
Expectation and Variance
Expectation (expected value) is the average value of a random variable over many trials
For discrete random variables: E ( X ) = ∑ x x ⋅ P ( X = x ) E(X) = \sum_{x} x \cdot P(X=x) E ( X ) = ∑ x x ⋅ P ( X = x )
For continuous random variables: E ( X ) = ∫ − ∞ ∞ x ⋅ f ( x ) d x E(X) = \int_{-\infty}^{\infty} x \cdot f(x) dx E ( X ) = ∫ − ∞ ∞ x ⋅ f ( x ) d x
Linearity of expectation states that the expected value of the sum of random variables is the sum of their individual expected values
Variance measures the average squared deviation from the mean
For discrete random variables: V a r ( X ) = E [ ( X − E ( X ) ) 2 ] = ∑ x ( x − E ( X ) ) 2 ⋅ P ( X = x ) Var(X) = E[(X-E(X))^2] = \sum_{x} (x-E(X))^2 \cdot P(X=x) Va r ( X ) = E [( X − E ( X ) ) 2 ] = ∑ x ( x − E ( X ) ) 2 ⋅ P ( X = x )
For continuous random variables: V a r ( X ) = E [ ( X − E ( X ) ) 2 ] = ∫ − ∞ ∞ ( x − E ( X ) ) 2 ⋅ f ( x ) d x Var(X) = E[(X-E(X))^2] = \int_{-\infty}^{\infty} (x-E(X))^2 \cdot f(x) dx Va r ( X ) = E [( X − E ( X ) ) 2 ] = ∫ − ∞ ∞ ( x − E ( X ) ) 2 ⋅ f ( x ) d x
Standard deviation is the square root of the variance and measures the average deviation from the mean
Covariance measures the linear relationship between two random variables
Positive covariance indicates variables tend to increase or decrease together, negative covariance indicates an inverse relationship
Common Probability Distributions
Bernoulli distribution models a single trial with two possible outcomes (success with probability p p p , failure with probability 1 − p 1-p 1 − p )
Binomial distribution models the number of successes in a fixed number of independent Bernoulli trials
Parameters: number of trials n n n , success probability p p p
PMF: P ( X = k ) = ( n k ) p k ( 1 − p ) n − k P(X=k) = \binom{n}{k} p^k (1-p)^{n-k} P ( X = k ) = ( k n ) p k ( 1 − p ) n − k
Poisson distribution models the number of events occurring in a fixed interval of time or space
Parameter: average rate of events λ \lambda λ
PMF: P ( X = k ) = e − λ λ k k ! P(X=k) = \frac{e^{-\lambda}\lambda^k}{k!} P ( X = k ) = k ! e − λ λ k
Normal (Gaussian) distribution is a continuous probability distribution with a bell-shaped curve
Parameters: mean μ \mu μ , standard deviation σ \sigma σ
PDF: f ( x ) = 1 σ 2 π e − ( x − μ ) 2 2 σ 2 f(x) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} f ( x ) = σ 2 π 1 e − 2 σ 2 ( x − μ ) 2
Exponential distribution models the time between events in a Poisson process
Parameter: rate parameter λ \lambda λ
PDF: f ( x ) = λ e − λ x f(x) = \lambda e^{-\lambda x} f ( x ) = λ e − λ x for x ≥ 0 x \geq 0 x ≥ 0
Uniform distribution has equal probability over a specified range
Parameters: minimum value a a a , maximum value b b b
PDF: f ( x ) = 1 b − a f(x) = \frac{1}{b-a} f ( x ) = b − a 1 for a ≤ x ≤ b a \leq x \leq b a ≤ x ≤ b
Linear transformations involve multiplying a random variable by a constant and/or adding a constant
If Y = a X + b Y = aX + b Y = a X + b , then E ( Y ) = a E ( X ) + b E(Y) = aE(X) + b E ( Y ) = a E ( X ) + b and V a r ( Y ) = a 2 V a r ( X ) Var(Y) = a^2Var(X) Va r ( Y ) = a 2 Va r ( X )
Nonlinear transformations change the shape of the probability distribution
Examples include exponential, logarithmic, and power transformations
Convolution is used to find the distribution of the sum of independent random variables
For discrete random variables, convolution involves summing the product of the PMFs
For continuous random variables, convolution involves integrating the product of the PDFs
Moment-generating functions can be used to derive the distribution of transformed random variables
Central Limit Theorem states that the sum of a large number of independent random variables approaches a normal distribution
Applies regardless of the original distribution, under certain conditions
Applications and Examples
Quality control uses the binomial and Poisson distributions to model defects in manufacturing processes
Insurance companies use the exponential distribution to model the time between claims
Normal distribution is used in hypothesis testing and confidence interval estimation
Gaussian mixture models are used in machine learning for clustering and density estimation
Markov chains model systems that transition between discrete states over time
Examples include weather patterns, stock prices, and customer behavior
Queuing theory uses probability distributions to analyze waiting lines and service systems
Applications in call centers, traffic management, and resource allocation
Monte Carlo simulations use random variables to model complex systems and estimate probabilities
Examples include financial risk assessment, particle physics, and engineering design optimization