📈Theoretical Statistics Unit 10 – Stochastic processes

Key Concepts and Definitions

• Stochastic process: A collection of random variables indexed by time or space that represents the evolution of a system subject to randomness
• State space: The set of all possible values that a stochastic process can take at any given time
• Transition probability: The probability of moving from one state to another in a stochastic process
  • Denoted as $P(X_{t+1} = j | X_t = i)$ for discrete-time processes
  • Represents the likelihood of transitioning from state $i$ at time $t$ to state $j$ at time $t+1$
• Stationarity: A property of a stochastic process where the joint probability distribution does not change over time or space
• Ergodicity: A property of a stochastic process where the time average of a single realization converges to the ensemble average as time tends to infinity
• Martingale: A stochastic process where the expected value of the next observation, given the current state, is equal to the current state
• Filtration: An increasing sequence of σ-algebras that represents the information available up to a certain time in a stochastic process

Probability Theory Foundations

• Probability space: A mathematical construct consisting of a sample space, a σ-algebra, and a probability measure
  • Sample space ($\Omega$): The set of all possible outcomes of a random experiment
  • σ-algebra ($\mathcal{F}$): A collection of subsets of the sample space that is closed under complement and countable unions
  • Probability measure ($P$): A function that assigns probabilities to events in the σ-algebra, satisfying certain axioms
• Random variable: A function that maps outcomes from the sample space to real numbers
• Expectation: The average value of a random variable, calculated as the sum or integral of the product of the random variable and its probability density or mass function
• Conditional probability: The probability of an event occurring given that another event has already occurred
  • Denoted as $P(A|B)$ and calculated as $\frac{P(A \cap B)}{P(B)}$ when $P(B) > 0$
• Independence: Two events are independent if the occurrence of one does not affect the probability of the other occurring
• Bayes' theorem: A formula for updating the probability of an event based on new information or evidence
  • Expressed as $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$ when $P(B) > 0$

Types of Stochastic Processes

• Discrete-time processes: Stochastic processes where the time index takes integer values (e.g., random walks, Markov chains)
• Continuous-time processes: Stochastic processes where the time index takes real values (e.g., Brownian motion, Poisson processes)
• Markov processes: Stochastic processes where the future state depends only on the current state and not on the past states
  • Memoryless property: The future evolution of the process depends only on the current state
• Gaussian processes: Stochastic processes where any finite collection of random variables has a multivariate normal distribution
  • Characterized by a mean function and a covariance function
• Renewal processes: Stochastic processes where events occur at random times, and the inter-arrival times are independent and identically distributed (i.i.d.)
• Point processes: Stochastic processes that model the occurrence of events in time or space (e.g., Poisson processes, Cox processes)
• Lévy processes: Stochastic processes with independent and stationary increments, continuous in probability, and starting at zero almost surely

Markov Chains

• Markov chain: A discrete-time stochastic process with the Markov property, where the future state depends only on the current state
• Transition matrix: A square matrix that contains the transition probabilities between states in a Markov chain
  • Denoted as $P = (p_{ij})$, where $p_{ij} = P(X_{t+1} = j | X_t = i)$
  • Each row of the transition matrix sums to 1
• Chapman-Kolmogorov equations: A set of equations that describe the probability of transitioning from one state to another in a Markov chain over multiple time steps
  • For an $n$-step transition probability, $p_{ij}^{(n)} = \sum_{k} p_{ik}^{(m)} p_{kj}^{(n-m)}$ for $0 < m < n$
• Stationary distribution: A probability distribution over the states of a Markov chain that remains unchanged under the transition matrix
  • Denoted as $\pi = (\pi_1, \pi_2, \ldots, \pi_n)$, where $\pi_j = \lim_{t \to \infty} P(X_t = j)$
  • Satisfies the equation $\pi P = \pi$
• Absorbing states: States in a Markov chain from which there is no possibility of transitioning to any other state
• Ergodic Markov chains: Markov chains that are irreducible, aperiodic, and positive recurrent, guaranteeing the existence of a unique stationary distribution
• Reversibility: A property of a Markov chain where the reversed process is also a Markov chain with the same stationary distribution

Poisson Processes

• Poisson process: A continuous-time stochastic process that models the occurrence of events in time or space
  • Events occur independently and with a constant average rate $\lambda$
  • Inter-arrival times are exponentially distributed with parameter $\lambda$
• Poisson distribution: A discrete probability distribution that describes the number of events occurring in a fixed interval of time or space
  • Probability mass function: $P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}$ for $k = 0, 1, 2, \ldots$
  • Mean and variance equal to $\lambda$
• Homogeneous Poisson process: A Poisson process with a constant rate $\lambda$ over time
• Non-homogeneous Poisson process: A Poisson process with a time-varying rate $\lambda(t)$
  • The expected number of events in an interval $[a, b]$ is given by $\int_a^b \lambda(t) dt$
• Thinning (splitting) of a Poisson process: The process of decomposing a Poisson process into multiple independent Poisson processes based on a set of probabilities
• Superposition (merging) of Poisson processes: The process of combining multiple independent Poisson processes into a single Poisson process with a rate equal to the sum of the individual rates

Brownian Motion

• Brownian motion (Wiener process): A continuous-time stochastic process with independent and normally distributed increments
  • Denoted as $\{B(t), t \geq 0\}$
  • $B(0) = 0$ almost surely
  • Increments $B(t) - B(s)$ are independent and normally distributed with mean 0 and variance $t - s$ for $0 \leq s < t$
• Standard Brownian motion: A Brownian motion with zero drift and unit variance per unit time
• Drift: The deterministic component of a stochastic process, representing the average change per unit time
• Diffusion: The random component of a stochastic process, representing the spread or variability of the process
• Itô calculus: A framework for defining and manipulating stochastic integrals with respect to Brownian motion
  • Itô integral: $\int_0^t f(s) dB(s)$, where $f$ is a suitable random function and $B$ is a Brownian motion
  • Itô's lemma: A stochastic version of the chain rule for computing the differential of a function of a stochastic process
• Stochastic differential equations (SDEs): Differential equations that incorporate random terms, often driven by Brownian motion
  • Example: $dX(t) = \mu(t, X(t)) dt + \sigma(t, X(t)) dB(t)$, where $\mu$ is the drift and $\sigma$ is the diffusion coefficient
• Geometric Brownian motion: A stochastic process used to model asset prices, where the logarithm of the process follows a Brownian motion with drift
  • Commonly used in financial mathematics and option pricing (e.g., Black-Scholes model)

Applications in Statistics

• Time series analysis: The study of data collected over time, often modeled using stochastic processes
  • Autoregressive (AR) models: A class of models where the current value of a process depends linearly on its own previous values and a stochastic term
  • Moving average (MA) models: A class of models where the current value of a process depends linearly on the current and previous values of a stochastic term
  • Autoregressive moving average (ARMA) models: A class of models that combine AR and MA components
  • Autoregressive integrated moving average (ARIMA) models: An extension of ARMA models that can handle non-stationary processes by differencing the data
• State space models: A framework for modeling time series data using a latent stochastic process (state) and an observation process
  • Kalman filter: An algorithm for estimating the state of a linear state space model based on noisy observations
• Hidden Markov models (HMMs): A class of models where the observed data is generated by a hidden Markov process
  • Commonly used in speech recognition, bioinformatics, and machine learning
• Bayesian inference: A framework for updating beliefs about model parameters or latent variables based on observed data and prior knowledge
  • Markov chain Monte Carlo (MCMC) methods: A class of algorithms for sampling from complex probability distributions, often used in Bayesian inference (e.g., Gibbs sampling, Metropolis-Hastings algorithm)
• Stochastic optimization: The study of optimization problems involving random variables or stochastic processes
  • Stochastic gradient descent (SGD): An optimization algorithm that uses noisy gradients estimated from random subsets of data to update model parameters

Problem-Solving Techniques

• Moment generating functions (MGFs): A tool for characterizing the probability distribution of a random variable or stochastic process
  • Defined as $M_X(t) = \mathbb{E}[e^{tX}]$ for a random variable $X$
  • Can be used to compute moments, cumulants, and other properties of the distribution
• Characteristic functions: A complex-valued function that uniquely determines the probability distribution of a random variable or stochastic process
  • Defined as $\varphi_X(t) = \mathbb{E}[e^{itX}]$ for a random variable $X$
  • Can be used to prove limit theorems and study the convergence of stochastic processes
• Martingale techniques: Exploiting the martingale property of a stochastic process to derive bounds, convergence results, or optimal stopping rules
  • Optional stopping theorem: States that the expected value of a martingale at a stopping time is equal to its initial value, under certain conditions
  • Martingale convergence theorems: A set of results that provide conditions under which a martingale converges to a limit (e.g., Doob's martingale convergence theorem)
• Coupling: A technique for comparing two stochastic processes by constructing them on the same probability space
  • Useful for proving convergence results, bounding distances between distributions, or studying the mixing time of Markov chains
• Simulation methods: Generating realizations of a stochastic process using random number generators and numerical algorithms
  • Monte Carlo methods: A class of algorithms that rely on repeated random sampling to estimate quantities or solve problems
  • Importance sampling: A variance reduction technique that involves sampling from a different distribution and reweighting the samples to estimate expectations under the original distribution
• Stochastic calculus techniques: Applying Itô calculus and related tools to analyze and solve problems involving stochastic processes
  • Girsanov's theorem: A result that allows for changing the drift of a stochastic process by an equivalent change of measure
  • Feynman-Kac formula: A theorem that relates the solution of a partial differential equation to the expectation of a functional of a stochastic process