🔀Stochastic Processes Unit 4 Review

A Poisson process is a stochastic process that models events occurring randomly and independently over time (or space). It serves as the foundation for many applied probability models because of its clean mathematical properties: independent increments, stationary increments, and a single rate parameter that controls everything.

The process is characterized by a constant rate parameter $\lambda$ , representing the average number of events per unit time. The number of events in any interval of length $t$ follows a Poisson distribution with mean $\lambda t$ .

Counting process

A Poisson process is formally a counting process $\{N(t), t \geq 0\}$ , where $N(t)$ counts how many events have occurred up to time $t$ . This counting process must satisfy three conditions:

$N(0) = 0$ (no events have occurred at time zero)
$N(t)$ takes non-negative integer values for all $t \geq 0$
$N(t)$ is non-decreasing (once an event happens, it can't un-happen)

The increments $N(t+s) - N(t)$ represent the number of events in the interval $(t, t+s]$ . The two properties below pin down exactly which counting process qualifies as Poisson.

Independent increments

The number of events in non-overlapping time intervals must be independent. Knowing how many events fell in one window tells you nothing about how many fell in another.

Formally, for any $0 \leq t_1 < t_2 \leq t_3 < t_4$ , the random variables $N(t_2) - N(t_1)$ and $N(t_4) - N(t_3)$ are independent. This extends to any finite collection of disjoint intervals.

Stationary increments

The distribution of the number of events in an interval depends only on the interval's length, not on when it starts. For any $s, t \geq 0$ :

$N(t+s) - N(t) \;\overset{d}{=}\; N(s)$

This is what makes the rate "constant." A Poisson process with rate $\lambda = 3$ per hour behaves the same way at 2 PM as it does at 10 PM.

Poisson distribution for increments

Combining the above properties with a regularity condition (events don't arrive in simultaneous clumps) forces the increment distribution to be Poisson. The probability mass function for the count in an interval of length $t$ is:

$P(N(t) = k) = \frac{(\lambda t)^k \, e^{-\lambda t}}{k!}, \quad k = 0, 1, 2, \ldots$

Both the mean and variance of $N(t)$ equal $\lambda t$ . That equality is a quick diagnostic: if the sample mean and sample variance of your counts are close, a Poisson model may be reasonable.

Properties of Poisson processes

Memoryless property

Because the inter-arrival times are exponentially distributed, the process is memoryless. The time you've already waited since the last event gives you no information about when the next event will occur.

In terms of increments, this is expressed as:

$P(N(t+s) - N(t) = k \mid N(t) = n) = P(N(s) = k)$

This follows directly from independent and stationary increments. The conditioning on the past ( $N(t) = n$ ) drops out entirely.

Superposition of independent Poisson processes

If $N_1(t)$ and $N_2(t)$ are independent Poisson processes with rates $\lambda_1$ and $\lambda_2$ , their sum $N(t) = N_1(t) + N_2(t)$ is again a Poisson process, now with rate $\lambda_1 + \lambda_2$ .

This generalizes to any finite number of independent Poisson processes. For example, if walk-in customers arrive at rate 8/hour and online-order pickups arrive at rate 5/hour (independently), total arrivals form a Poisson process at rate 13/hour.

Splitting (thinning) of Poisson processes

Superposition runs in reverse. If each event in a rate- $\lambda$ Poisson process is independently classified as Type 1 with probability $p$ or Type 2 with probability $1-p$ , then:

The Type 1 events form a Poisson process with rate $\lambda p$
The Type 2 events form a Poisson process with rate $\lambda(1-p)$
These two processes are independent of each other

For instance, if calls arrive at a call center at rate 20/hour and 30% are technical support calls, then technical support calls alone form a Poisson process at rate 6/hour.

Conditional distribution of arrival times

Given that exactly $n$ events occurred in $[0, t]$ , the $n$ arrival times are distributed as the order statistics of $n$ i.i.d. Uniform $(0, t)$ random variables.

This is a powerful result. It means that, conditional on the total count, the events are "scattered uniformly" across the interval. Note that the inter-arrival times in this conditional picture are not independent uniform random variables. Rather, the $n$ unordered arrival times are i.i.d. Uniform $(0, t)$ , and you then sort them.

Poisson process as limit of binomial processes

Binomial process definition

A binomial process is a discrete-time counting process built from i.i.d. Bernoulli trials. Let $X_1, X_2, \ldots, X_n$ be i.i.d. Bernoulli $(p)$ random variables, and define $S_n = \sum_{i=1}^n X_i$ . Then $S_n \sim \text{Binomial}(n, p)$ .

You can think of this as dividing time into $n$ tiny slots, each of which independently produces an event with probability $p$ .

Counting process, Poisson Process and Its Application to the Storm Water Overflows

Limiting behavior as n → ∞

The Poisson process emerges when you take more and more slots while shrinking the per-slot probability, keeping the expected total count fixed. Specifically, let $n \to \infty$ and $p \to 0$ with $np \to \lambda$ . Then for any fixed $t > 0$ :

$\lim_{n \to \infty} P(S_{\lfloor nt \rfloor} = k) = \frac{(\lambda t)^k \, e^{-\lambda t}}{k!}, \quad k = 0, 1, 2, \ldots$

This is the classical Poisson limit theorem (sometimes called the "law of rare events"). It tells you why Poisson models work well for rare events: whenever you have many independent opportunities for something unlikely to happen, the total count is approximately Poisson.

A concrete example: a factory produces 10,000 items per day, each independently defective with probability 0.0003. The expected defect count is $\lambda = 3$ , and the actual count is well-approximated by a Poisson(3) distribution.

Relationship to other processes

Comparison to renewal processes

A renewal process generalizes the Poisson process by allowing the inter-arrival times to follow any distribution, not just the exponential. In a Poisson process, inter-arrival times are i.i.d. Exponential $({\lambda})$ . In a general renewal process, they are i.i.d. with some distribution $F$ .

So the Poisson process is the unique renewal process whose inter-arrival distribution is exponential. This is also the only renewal process that has both independent and stationary increments.

Poisson process as special case of birth-death process

A birth-death process is a continuous-time Markov chain on the non-negative integers where transitions go only to neighboring states (up by 1 or down by 1). Birth and death rates can depend on the current state.

A Poisson process is the special case where:

The birth rate is constant at $\lambda$ regardless of state
The death rate is zero

Because there are no deaths, the state only increases, which is exactly the counting behavior of a Poisson process.

Applications of Poisson processes

Modeling rare events

When events are rare and occur independently, the Poisson process is a natural model. Examples include earthquakes in a region, radioactive decays, or hardware failures in a data center. The Poisson limit theorem justifies this: many independent low-probability opportunities aggregate into a Poisson count.

Queueing theory

Customer or job arrivals to a service system are often modeled as a Poisson process. The classic M/M/1 queue, for instance, assumes Poisson arrivals at rate $\lambda$ and exponential service times. From this, you can derive key performance metrics like average waiting time, queue length, and server utilization.

Reliability engineering

The number of component failures over time is frequently modeled as a Poisson process, especially when failures are independent and occur at a roughly constant rate. The rate $\lambda$ directly gives you the mean time between failures as $\text{MTBF} = 1/\lambda$ . For example, if server hard drives fail at a rate of 0.02 per month, the MTBF is 50 months, and you can use the Poisson model to plan spare-parts inventory.

Simulating Poisson processes

Inverse transform method

This method exploits the fact that inter-arrival times are Exponential $(\lambda)$ . You generate each inter-arrival time by inverting the exponential CDF applied to a uniform random variable.

Generate i.i.d. uniform random variables $U_1, U_2, \ldots$ on $[0, 1]$ .
Set $T_0 = 0$ . Compute successive event times as:

$T_n = T_{n-1} - \frac{1}{\lambda} \ln(U_n), \quad n = 1, 2, \ldots$

Stop once $T_n$ exceeds your time horizon. The sequence $T_1, T_2, \ldots$ gives the simulated event times.

Each gap $T_n - T_{n-1} = -\frac{1}{\lambda}\ln(U_n)$ is an Exponential $(\lambda)$ random variable, which is exactly what a Poisson process requires.

Thinning method

The thinning method handles non-homogeneous Poisson processes where the rate $\lambda(t)$ varies with time.

Find an upper bound $\lambda^*$ such that $\lambda(t) \leq \lambda^*$ for all $t$ in your interval.
Simulate a homogeneous Poisson process at rate $\lambda^*$ using the inverse transform method, producing candidate event times $T_1^*, T_2^*, \ldots$
For each candidate $T_n^*$ , draw $U_n \sim \text{Uniform}(0,1)$ . Accept the event if:

$U_n \leq \frac{\lambda(T_n^*)}{\lambda^*}$

The accepted times form a non-homogeneous Poisson process with rate $\lambda(t)$ .

The tighter your bound $\lambda^*$ is to the actual peak rate, the fewer candidates you waste. This method is especially useful for modeling time-varying arrival patterns, such as customer traffic that peaks during certain hours.

🔀Stochastic Processes Unit 4 Review