Bayesian Statistics Unit 7 ReviewMarkov Chain Monte Carlo (MCMC) Methods

Key Concepts and Foundations

• Bayesian inference updates prior beliefs about parameters using observed data to obtain posterior distributions
• Markov chains are stochastic processes where the next state depends only on the current state, not the entire history
• Monte Carlo methods use random sampling to approximate complex integrals and distributions
• MCMC combines Markov chains and Monte Carlo sampling to draw samples from high-dimensional posterior distributions
• Metropolis-Hastings algorithm is a general MCMC method that proposes new states and accepts or rejects them based on a probability
• Gibbs sampling is an MCMC algorithm that samples from the full conditional distributions of each parameter
• Convergence diagnostics assess whether the MCMC chain has reached its stationary distribution and is sampling effectively
• Effective sample size (ESS) measures the number of independent samples in an MCMC chain, accounting for autocorrelation

Markov Chain Basics

• A Markov chain is a sequence of random variables where the distribution of each variable depends only on the state of the previous variable
• The Markov property states that the future state of a system depends only on its current state, not on its past history
• Transition probabilities define the likelihood of moving from one state to another in a Markov chain
• A state space is the set of all possible states that a Markov chain can occupy
• Stationary distribution is the long-run equilibrium distribution of a Markov chain, where the probability of being in each state remains constant over time
• Irreducibility means that any state can be reached from any other state in a finite number of steps
• Aperiodicity ensures that the chain does not get stuck in cycles and can converge to its stationary distribution
• Ergodicity combines irreducibility and aperiodicity, guaranteeing that the chain will converge to a unique stationary distribution regardless of its initial state

Monte Carlo Methods Overview

• Monte Carlo methods rely on repeated random sampling to estimate numerical results and solve problems
• They are particularly useful for approximating complex integrals, expectations, and distributions that are difficult to compute analytically
• Simple Monte Carlo estimation involves drawing independent samples from a target distribution and averaging them to approximate expectations
• Importance sampling improves efficiency by sampling from a proposal distribution and reweighting the samples based on the ratio of the target and proposal densities
• Rejection sampling generates samples from a target distribution by accepting or rejecting samples from a proposal distribution based on a acceptance probability
• Variance reduction techniques, such as antithetic variates and control variates, can reduce the variance of Monte Carlo estimates and improve their accuracy
• Quasi-Monte Carlo methods use low-discrepancy sequences (Sobol, Halton) instead of random numbers to achieve faster convergence rates
• Monte Carlo integration approximates definite integrals by sampling points from the integration domain and averaging the function values

MCMC Algorithms and Techniques

• Metropolis-Hastings (MH) algorithm proposes new states from a proposal distribution and accepts or rejects them based on an acceptance probability that ensures the chain converges to the target distribution
• The acceptance probability in MH balances the proposal distribution and the target distribution to maintain detailed balance and ensure convergence
• Gibbs sampling updates each parameter by sampling from its full conditional distribution given the current values of all other parameters
• Gibbs sampling is a special case of MH where the proposal distribution is the full conditional and the acceptance probability is always 1
• Random-walk Metropolis uses a symmetric proposal distribution centered at the current state, such as a Gaussian with a tunable variance
• Independence sampler proposes new states independently of the current state, using a proposal distribution that approximates the target distribution
• Adaptive MCMC methods adjust the proposal distribution or other parameters during the simulation to improve efficiency and convergence
• Hamiltonian Monte Carlo (HMC) uses the gradient of the log-posterior to propose efficient moves and explore the parameter space more effectively

Implementing MCMC in Practice

• Choose an appropriate MCMC algorithm based on the structure of the model, the complexity of the posterior, and the available computational resources
• Specify the prior distributions for all parameters based on domain knowledge or previous studies
• Define the likelihood function that relates the parameters to the observed data and incorporates any assumptions or constraints
• Combine the prior and likelihood to obtain the unnormalized posterior distribution, which is the target distribution for MCMC sampling
• Initialize the MCMC chain by setting starting values for all parameters, either randomly or based on prior information
• Implement the chosen MCMC algorithm in a programming language (Python, R, Stan) or use existing software packages (PyMC3, JAGS, BUGS)
• Run the MCMC chain for a sufficient number of iterations to ensure convergence and obtain reliable posterior samples
  • Discard an initial portion of the chain as burn-in to allow the chain to reach its stationary distribution
  • Thin the chain by keeping only every k-th sample to reduce autocorrelation and storage requirements
• Monitor convergence and mixing using diagnostic tools and visual inspection of trace plots and posterior distributions
• Summarize the posterior samples by computing point estimates (mean, median, mode), intervals (credible intervals), and other relevant quantities
• Assess model fit and compare alternative models using posterior predictive checks, information criteria (DIC, WAIC), or Bayes factors

Convergence and Diagnostics

• Convergence refers to the MCMC chain reaching its stationary distribution and providing reliable samples from the posterior
• Visual inspection of trace plots helps assess mixing and identify any trends, patterns, or stuck regions in the chain
• Autocorrelation plots show the correlation between samples at different lags and indicate the effective sample size
• Gelman-Rubin diagnostic (Rhat) compares the between-chain and within-chain variances of multiple chains to check for convergence
  • Rhat values close to 1 suggest convergence, while values greater than 1.1 indicate lack of convergence
• Geweke diagnostic compares the means of the first and last parts of the chain to check for equality, indicating convergence
• Heidelberger-Welch diagnostic assesses the stationarity of the chain by testing for equality of means and variances in different segments
• Effective sample size (ESS) estimates the number of independent samples in the chain, accounting for autocorrelation
  • Higher ESS values indicate better mixing and more reliable posterior estimates
• Potential scale reduction factor (PSRF) compares the variance of the pooled chains to the average within-chain variance, with values close to 1 suggesting convergence
• Posterior predictive checks generate replicated data from the posterior predictive distribution and compare them to the observed data to assess model fit

Applications in Bayesian Statistics

• Bayesian inference is widely used in various fields, including machine learning, data science, economics, and social sciences
• Bayesian regression models (linear, logistic, Poisson) incorporate prior information and provide posterior distributions for the regression coefficients
• Hierarchical models allow for borrowing strength across groups or units by specifying priors on the parameters of the group-level distributions
• Bayesian model selection and averaging use MCMC to estimate posterior model probabilities and account for model uncertainty
• Gaussian processes are flexible non-parametric models that use MCMC to infer the posterior distribution of the underlying function
• Bayesian neural networks specify prior distributions on the weights and biases and use MCMC to obtain posterior samples and quantify uncertainty
• Bayesian time series models (ARIMA, state-space) incorporate prior knowledge and provide probabilistic forecasts and uncertainty quantification
• Spatial and spatio-temporal models use MCMC to estimate the posterior distribution of the spatial random effects and other parameters
• Bayesian nonparametrics (Dirichlet processes, Gaussian processes) allow for flexible modeling of complex data structures and use MCMC for posterior inference

Advanced Topics and Extensions

• Reversible jump MCMC (RJMCMC) enables transdimensional sampling, allowing the number of parameters to vary across models
• Pseudo-marginal MCMC uses unbiased estimators of the likelihood (particle filters, importance sampling) to perform exact inference when the likelihood is intractable
• Hamiltonian Monte Carlo (HMC) and its variants (NUTS, RHMC) use the gradient of the log-posterior to propose efficient moves and explore the parameter space more effectively
• Riemannian manifold HMC (RMHMC) adapts the proposal distribution to the local geometry of the posterior, improving efficiency in high-dimensional and highly correlated spaces
• Stein variational gradient descent (SVGD) is a deterministic alternative to MCMC that approximates the posterior using a set of particles that are updated based on a kernelized Stein discrepancy
• Variational inference (VI) approximates the posterior with a simpler distribution by minimizing the Kullback-Leibler divergence, providing a faster but less accurate alternative to MCMC
• Stochastic gradient MCMC (SGMCMC) combines stochastic optimization with MCMC, enabling posterior sampling for large-scale datasets and complex models
• Parallel tempering (PT) runs multiple MCMC chains at different temperatures and proposes swaps between them to improve mixing and exploration of multimodal posteriors
• Sequential Monte Carlo (SMC) methods, such as particle filters and SMC samplers, provide an alternative to MCMC for online inference and model comparison in sequential settings