🔬Mathematical Biology Unit 10 – Stochastic Models: Markov Chains & Gillespie

Key Concepts

• Stochastic models describe systems with random variables and probabilistic transitions between states
• Markov chains model systems where the future state depends only on the current state, not the past (memoryless property)
• Transition matrices represent the probabilities of moving from one state to another in a Markov chain
• Steady state refers to the long-term behavior of a Markov chain where the state probabilities remain constant over time
• Equilibrium is reached when the system's state distribution no longer changes with time
• Gillespie algorithm simulates stochastic systems by generating random events based on reaction rates and probabilities
• Stochastic models have wide applications in biology, including population dynamics, gene expression, and epidemiology

Markov Chains Basics

• Markov chains consist of a set of states and transition probabilities between those states
• The Markov property states that the probability of moving to the next state depends only on the current state
• Discrete-time Markov chains (DTMCs) update state probabilities at fixed time intervals
• Continuous-time Markov chains (CTMCs) model systems where transitions occur at random times
• The state space of a Markov chain can be finite (a limited number of states) or infinite (an unlimited number of states)
• Absorbing states are states that, once entered, cannot be left (probability of leaving is 0)
• Ergodic Markov chains have a unique steady-state distribution independent of the initial state

Transition Matrices

• A transition matrix $P$ represents the probabilities of moving from one state to another in a single step
• The element $p_{ij}$ in a transition matrix represents the probability of moving from state $i$ to state $j$
• Transition matrices are square (number of rows equals number of columns) and stochastic (each row sums to 1)
  • Example: For a 3-state Markov chain, a transition matrix might look like: 0.6 & 0.3 & 0.1 \\ 0.2 & 0.7 & 0.1 \\ 0.1 & 0.2 & 0.7 \end{bmatrix}$$
• The $n$-step transition probability matrix $P^{(n)}$ gives the probabilities of moving between states after $n$ steps
• $P^{(n)}$ is calculated by multiplying the transition matrix $P$ by itself $n$ times

Steady State and Equilibrium

• The steady-state distribution $\pi$ is a vector of probabilities that remains unchanged under further transitions
• In steady state, the probability of being in each state remains constant over time
• The steady-state distribution satisfies the equation $\pi P = \pi$, where $P$ is the transition matrix
• Equilibrium is reached when the system's state distribution no longer changes with time
• For ergodic Markov chains, the steady-state distribution is unique and independent of the initial state
• The limiting distribution of a Markov chain is the steady-state distribution it converges to as the number of steps approaches infinity
• Mean first passage time is the expected number of steps to reach a specific state starting from another state

Gillespie Algorithm

• The Gillespie algorithm is a stochastic simulation method for modeling chemical reactions and other systems with discrete events
• It generates a sequence of random events (reactions) and their corresponding times based on reaction rates and probabilities
• The algorithm consists of two main steps: (1) determining the time until the next reaction and (2) selecting which reaction occurs
• The time until the next reaction is drawn from an exponential distribution with a rate equal to the sum of all reaction rates
• The probability of selecting a particular reaction is proportional to its reaction rate
• The system's state is updated after each reaction, and the process continues until a specified end time or condition is met
• The Gillespie algorithm is exact, meaning it generates samples from the same probability distribution as the underlying stochastic process

Applications in Biology

• Stochastic models are used to study population dynamics, such as birth-death processes and predator-prey interactions
• Gene expression can be modeled using Markov chains, considering the stochastic nature of transcription and translation
• Epidemiological models, such as the SIR (Susceptible-Infected-Recovered) model, use Markov chains to study disease spread
• Ion channel dynamics in neurons can be modeled as Markov processes, with states representing open, closed, and inactivated states
• Evolutionary processes, such as genetic drift and natural selection, can be studied using stochastic models
• Cell differentiation and lineage commitment can be modeled as stochastic transitions between cell states
• Stochastic models help understand the role of noise and variability in biological systems

Problem-Solving Techniques

• Identify the states and transition probabilities of the Markov chain based on the given problem
• Construct the transition matrix $P$ using the identified states and probabilities
• Determine the type of Markov chain (absorbing, ergodic, etc.) and its properties
• Calculate the steady-state distribution $\pi$ by solving the equation $\pi P = \pi$
• Use the Gillespie algorithm to simulate stochastic systems by generating random events and updating the system state
• Analyze the long-term behavior of the system using the steady-state distribution and other relevant metrics
• Interpret the results in the context of the biological problem and draw meaningful conclusions

Further Reading and Resources

• "Introduction to Stochastic Processes" by Gregory F. Lawler (book)
• "Stochastic Modelling for Systems Biology" by Darren J. Wilkinson (book)
• "Stochastic Processes in Physics and Chemistry" by N.G. Van Kampen (book)
• "Gillespie's Stochastic Simulation Algorithm" by Mario Pineda-Krch (article)
• "Markov Chains: From Theory to Implementation and Experimentation" by Paul A. Gagniuc (book)
• "Stochastic Processes in Cell Biology" by Paul C. Bressloff (book)
• Online courses on platforms like Coursera, edX, and MIT OpenCourseWare covering stochastic processes and their applications in biology