Derivation of the Poisson Distribution as a Limiting Case of the Binomial PDF

The Poisson distribution is a discrete probability distribution with identical mean and variance that expresses the probability of a given number of events occurring in a fixed interval of time. In this post, we derive the Poisson probability mass function beginning with the Binomial PDF.

The binomial distribution represents the number of successes, $k$ in $n$ independent Bernoulli trials, where the probability of success $p$ for any given trial is $0 \leq p \leq 1$. Formally, the binomial PDF is given by

$$ f(k; p, n) = \binom{n}{k}p^{k}(1-p)^{n-k}, $$

where $\binom{n}{k}$ represents the binomial coefficient, which equates to

$$ \binom{n}{k} = \frac{n!}{n!(n-k)!}. $$

The expanded binomial PDF then appears as

$$ f(k; p, n) = \frac{n!}{n!(n-k)!}p^{k}(1-p)^{n-k}. $$

The mean of the binomial distribution is

$$ \mu = \lambda = np, $$

which can be rearranged in terms of $p$ to yield

$$ p = \frac{\lambda}{n}. $$

Substituting this expression for $p$ into the binomial PDF results in

$$ f(k; p, n) = \frac{n!}{n!(n-k)!}\Big(\frac{\lambda}{n}\Big)^{k}\Big(1-\frac{\lambda}{n}\Big)^{n-k} $$

If we let $n$ grow very large and $p$ grow very small, after a bit of re-expression we can reformulate the binomial distribution as

$$ \lim_{n\to\infty , p \to 0} f(k; p, n) = \frac{n(n-1) \cdots (n-k+1)}{k!} \frac{\lambda^{k}}{n^{k}} \Big(1-\frac{\lambda}{n}\Big)^{n} \Big(1-\frac{\lambda}{n}\Big)^{-k}. $$

Swapping denominator terms yields

$$ = \frac{n(n-1) \cdots (n-k+1)}{n^{k}} \frac{\lambda^{k}}{k!} \Big(1-\frac{\lambda}{n}\Big)^{n} \Big(1-\frac{\lambda}{n}\Big)^{-k} $$

Recall from calculus that $e = \lim_{k\to\infty} (1 + 1/k)^{k}$ and similarly $e^{x} = \lim_{k\to\infty} (1 + x/k)^{k}$. Leveraging these identities, $(1 - \frac{\lambda}{n})^{n}$ becomes $e^{-\lambda}$ in the limit. With very large $n$, the expression for the PDF reduces to

$$ \lim_{n\to\infty} f(k; p, n) = (1) \Big( \frac{\lambda^{k}}{k!}\Big)(e^{-\lambda})(1). $$

After a little cleanup, we obtain the familiar expression for the Poisson density with mean $\lambda$:

$$ f(k; \lambda) = \frac{\lambda^{k}}{k!}e^{-\lambda} \hspace{1cm} k \in 0, 1, 2, \cdots. $$

An interesting property of the Poisson distribution (as well as the Binomial and Negative Binomial distributions) is that successive probabilities can be found recursively. The recurrence relation is given by

$$ \frac{p_{k}}{p_{k-1}} = a +\frac{b}{k}, \hspace{2mm} k = 2,3,4, \cdots, $$

where in the case of the Poisson distribution, $a=0$ and $b=\lambda$, resulting in

$$ p_{k} = \frac{\lambda}{k}p_{k-1}, \hspace{2mm} k = 1,2,3, \cdots. $$

To demonstrate, for a system approximated by a Poisson distribution with $\lambda = 3$, the probabilities for $0 \leq k \leq 6$ are:

$$ \begin{align*} p_{0} &= .049787068 \\ p_{1} &= .149361205 \\ p_{2} &= .224041808 \\ p_{3} &= .224041808 \\ p_{4} &= .168031356 \\ p_{5} &= .100818813 \\ p_{6} &= .050409407. \end{align*} $$

To use the recurrence relation, we first calculate $p_0$ as above. Then:

$$ \begin{align*} p_{0} &=.049787068 \\ p_{1} &=p_{0}\frac{\lambda}{k} =.049787068*\frac{3}{1}=.149361205 \\ p_{2} &=p_{1}\frac{\lambda}{k} =.149361205*\frac{3}{2}=.224041808 \\ p_{3} &=p_{2}\frac{\lambda}{k} =.224041808*\frac{3}{3}=.224041808 \\ p_{4} &=p_{3}\frac{\lambda}{k} =.224041808*\frac{3}{4}=.168031356 \\ p_{5} &=p_{4}\frac{\lambda}{k} =.168031356*\frac{3}{5}=.100818813 \\ p_{6} &=p_{5}\frac{\lambda}{k} =.100818813*\frac{3}{6}=.050409407, \end{align*} $$

which are identical to the probabilities obtained from evaluating the Poisson PDF directly.