Key Concepts in Stochastic Processes

Why This Matters

Stochastic processes form the mathematical backbone for modeling anything that evolves randomly over time—from stock prices and particle movement to customer arrivals and population dynamics. You're being tested on your ability to recognize which process fits which scenario, understand the assumptions underlying each model, and apply them to solve real-world problems in finance, engineering, and data science.

Don't just memorize definitions. For each concept, know what makes it unique, what assumptions it requires, and when to use it over alternatives. Exam questions often present a scenario and ask you to identify the appropriate model or compare two processes—so focus on the distinguishing characteristics and mathematical properties that set each apart.

---

Memoryless Processes and the Markov Property

The Markov property—where the future depends only on the present, not the past—simplifies analysis dramatically. This "memorylessness" allows us to use transition matrices and focus solely on current state information.

Markov Chains

• The Markov property states that $P(X_{n+1} | X_n, X_{n-1}, ..., X_0) = P(X_{n+1} | X_n)$—future states depend only on the current state
• Transition probabilities form a matrix $P$ where entry $p_{ij}$ represents the probability of moving from state $i$ to state $j$
• Classification of states into transient, recurrent, or absorbing determines long-run behavior and equilibrium distributions

Birth-Death Processes

• A special Markov process where transitions occur only between adjacent states—population increases by 1 (birth) or decreases by 1 (death)
• Transition rates $\lambda_n$ (birth) and $\mu_n$ (death) can depend on current population size $n$, enabling realistic population modeling
• Steady-state distributions exist when birth and death rates balance, crucial for queueing applications and biological population models

---

Counting and Arrival Processes

When modeling how many events occur over time—customer arrivals, radioactive decays, or system failures—these processes provide the mathematical framework. The key is understanding the assumptions about event independence and timing.

Poisson Processes

• Defined by a single rate parameter $\lambda$, representing the average number of events per unit time, with $P(N(t) = k) = \frac{(\lambda t)^k e^{-\lambda t}}{k!}$
• Three key assumptions: events occur independently, the rate is constant, and no two events occur simultaneously
• Interarrival times follow an exponential distribution with mean $1/\lambda$, connecting counting processes to waiting time analysis

Renewal Processes

• Generalizes Poisson processes by allowing interarrival times to follow any distribution, not just exponential
• The renewal function $M(t) = E[N(t)]$ gives expected number of renewals by time $t$, essential for replacement and maintenance scheduling
• Renewal theorem states that $\lim_{t \to \infty} \frac{N(t)}{t} = \frac{1}{\mu}$ where $\mu$ is mean interarrival time—long-run rate equals reciprocal of mean

Queueing Theory

• Analyzes waiting lines using notation $A/S/c$ where $A$ = arrival process, $S$ = service distribution, $c$ = number of servers
• Little's Law states $L = \lambda W$—average customers in system equals arrival rate times average wait time, remarkably general and model-independent
• Traffic intensity $\rho = \lambda/\mu$ must be less than 1 for stable queues; values approaching 1 cause wait times to explode

---

Continuous-Path Processes

These processes model phenomena that evolve continuously over time with random fluctuations. The mathematics here underpins modern financial theory and physics.

Brownian Motion

• Characterized by four properties: starts at zero, has independent increments, increments are normally distributed $B(t) - B(s) \sim N(0, t-s)$, and paths are continuous
• Quadratic variation equals $t$ over interval $[0,t]$, a key property distinguishing it from smooth functions and enabling stochastic calculus
• Foundation of the Black-Scholes model where stock prices follow geometric Brownian motion $dS = \mu S dt + \sigma S dW$

Wiener Processes

• Mathematically identical to standard Brownian motion—the terms are used interchangeably, with "Wiener process" common in probability theory
• Increments satisfy $W(t) - W(s) \sim N(0, t-s)$ and are independent for non-overlapping intervals
• Drives stochastic differential equations (SDEs) of the form $dX = a(X,t)dt + b(X,t)dW$, the foundation of continuous-time finance

Random Walks

• Discrete-time precursor to Brownian motion—sum of i.i.d. random steps $S_n = X_1 + X_2 + ... + X_n$
• Recurrence depends on dimension: 1D and 2D walks return to origin with probability 1; 3D+ walks are transient (escape to infinity)
• Central Limit Theorem connection: properly scaled random walks converge to Brownian motion as step size shrinks—this is Donsker's theorem

---

Fair Game and No-Arbitrage Foundations

Martingales capture the mathematical essence of "fair games" where no betting strategy can create systematic advantage. This property is essential for pricing derivatives and understanding market efficiency.

Martingales

• Defining property: $E[X_{n+1} | X_1, ..., X_n] = X_n$—the best prediction of tomorrow's value is today's value
• Optional stopping theorem governs when you can "stop" a martingale and still preserve the fairness property—critical for analyzing gambling strategies
• Risk-neutral pricing in finance relies on converting price processes to martingales under an equivalent measure, eliminating arbitrage

---

Flexible Modeling Frameworks

These processes provide powerful tools for modeling complex, uncertain systems where relationships aren't easily specified parametrically.

Gaussian Processes

• Any finite collection of variables $\{X_{t_1}, ..., X_{t_n}\}$ follows a multivariate normal distribution, fully specified by mean and covariance functions
• Covariance function $k(t, s) = Cov(X_t, X_s)$ encodes assumptions about smoothness, periodicity, and correlation structure
• Machine learning applications use GPs for regression with built-in uncertainty quantification—predictions come with confidence intervals automatically

---

Quick Reference Table

---

Self-Check Questions

1. A process has the property that $E[X_{t+1} | X_t, X_{t-1}, ...] = X_t$. Is this describing a Markov chain, a martingale, or both? What additional condition would make it both?

2. Compare and contrast Poisson processes and renewal processes. Under what assumption does a renewal process reduce to a Poisson process?

3. You're modeling customer arrivals where the time between arrivals follows a gamma distribution. Which process framework applies—Poisson, renewal, or birth-death? Justify your choice.

4. A random walk in 2D returns to the origin infinitely often, but a 3D random walk does not. What property distinguishes these behaviors, and how does this connect to Brownian motion?

5. An FRQ asks you to model stock prices that can't go negative and exhibit random percentage changes. Which process is most appropriate, and what is its stochastic differential equation form?