Stirling's approximation

In fields from statistical physics to pure mathematics, we often encounter factorials of enormous numbers, such as $10^{23}!$. These quantities are computationally impossible to handle directly, creating a fundamental barrier to understanding systems with vast numbers of components. How can we tame these mathematical beasts and extract meaningful physical insights? This article delves into Stirling's approximation, a remarkably powerful formula that bridges the discrete world of counting with the continuous realm of calculus. It provides the key to unlocking the secrets hidden within these colossal numbers. The following chapters will first explore the core principles and mechanisms of the approximation, revealing how it works and what makes it so effective. Subsequently, we will journey through its diverse applications, demonstrating how this single formula illuminates everything from random walks and probability distributions to the very definition of entropy in thermodynamics.

Imagine you are trying to count something simple, like the number of ways to arrange a deck of 52 cards. The answer, as you might know, is $52!$, which is 52 factorial. This number starts with an 8 and is followed by 67 other digits. It's a number so colossally large that there are not that many atoms in our entire galaxy. Now, imagine you're a physicist in the 19th century, like Ludwig Boltzmann, trying to understand heat and gases. You're not dealing with 52 cards, but with moles of gas, containing somewhere around $10^{23}$ particles. The number of ways to arrange these particles involves factorials of numbers that make $52!$ look like pocket change.

How on Earth can you work with such numbers? You can't write them down. You can't punch them into any calculator. You are stuck. This is not just a mathematical curiosity; it's a fundamental barrier to understanding the statistical nature of the world. What we need is not the exact number, but its essence, its approximate size, its character. We need a way to tame this factorial beast.

The escape route from this prison of gigantic numbers is one of the most beautiful and surprising formulas in all of science: ​​Stirling's approximation​​. In its most common form, it tells us that for a large number $n$, the factorial $n!$ is wonderfully close to a continuous and much more manageable function:

Take a moment to look at this. On the left, we have $n! = 1 \times 2 \times \dots \times n$, a purely discrete, arithmetic object. On the right, we have a formula built from the most fundamental constants of continuous mathematics: $\pi$, the soul of every circle, and $e$, the base of natural growth. How did they get in there? This connection is a profound hint at a deep unity between the discrete world of counting and the continuous world of calculus.

What does this "approximately equal to" sign, $\approx$, really mean? It's more precise than you might think. It signifies an ​​asymptotic relationship​​. It means that as $n$ gets larger and larger, the ratio between $n!$ and Stirling's formula gets closer and closer to exactly 1. The formula doesn't just give a ballpark number; it captures the dominant behavior, the very soul of the factorial function's growth. It tells us that, for large $n$, $n!$ grows a bit faster than a simple exponential function, guided by this peculiar combination of powers and constants.

In many areas of science, especially in statistical mechanics, we are often interested not in a quantity itself, but in its logarithm. This is because entropy, a measure of disorder or the number of ways a system can be arranged, is proportional to the logarithm of the number of available states, $\Omega$. If $\Omega$ involves gigantic factorials, the entropy $S = k_B \ln(\Omega)$ becomes manageable.

Taking the natural logarithm of Stirling's formula simplifies it beautifully. The multiplications and powers turn into additions and multiplications:

For very large $n$, the first two terms, $n\ln(n) - n$, dominate completely. So, a powerful and widely used version of the formula is simply:

Let's see how this incredible simplification works in practice. Imagine a hypothetical physical system where its statistical entropy is given by $S(z) = \ln(\Gamma(z+1))$, where $\Gamma$ is the Gamma function (the generalization of the factorial, so $\ln(\Gamma(z+1)) = \ln(z!)$ for integer $z$). If we want to find the change in entropy when a "complexity parameter" changes from $z_1 = 150$ to $z_2 = 151$, we don't need to compute insane factorials. We can just use our approximation: $\Delta S = S(z_2) - S(z_1) \approx (151\ln(151) - 151) - (150\ln(150) - 150)$. This calculation is straightforward and gives an answer of about $5.01$. The formula turns an impossible factorial calculation into a simple problem of functions.

This all sounds wonderful, but how good is the approximation really? Let's try it on a number we can actually check. We know that $10! = 3,628,800$. What does Stirling's formula give?

Comparing this to the exact value, we see the approximation is off by only about $0.8\%$. For a number as "small" as 10, this is already remarkably accurate!. Imagine how good it gets for $n=10^{23}$.

The true power of Stirling's formula, however, shines when we deal with expressions involving ratios of factorials. This is common in probability and combinatorics. Consider the ​​central binomial coefficient​​, $\binom{2n}{n} = \frac{(2n)!}{(n!)^2}$. This tells you the number of ways to choose $n$ items out of $2n$, and it appears in problems like a random walk. Calculating this for large $n$ is a nightmare. But watch what happens when we apply Stirling's approximation to the numerator and denominator:

A cascade of cancellations occurs! The $n^{2n}$ terms vanish, the $e^{-2n}$ terms vanish, and the prefactors simplify, leaving an astonishingly simple and elegant result:

This beautiful formula reveals the essential growth of the central binomial coefficient. It grows exponentially as $4^n$, but is tempered by a factor of $1/\sqrt{n}$. This kind of simplification is routine when using Stirling's approximation to evaluate complex limits or to determine the asymptotic behavior of series terms, allowing us to easily determine if an infinite series converges or diverges.

So far, we've talked about factorials of integers. But the true home of Stirling's formula is the ​​Gamma function​​, $\Gamma(z)$, a beautiful function that extends the factorial to all complex numbers (except non-positive integers). It is defined by the integral $\Gamma(z) = \int_0^\infty t^{z-1} e^{-t} dt$, and it satisfies $\Gamma(n+1) = n!$ for integers.

Stirling's formula is really an approximation for the Gamma function, $\Gamma(z+1) \approx \sqrt{2\pi z} (z/e)^z$. This means it must be consistent with all the deep and exact properties of $\Gamma(z)$. For example, the Gamma function satisfies a remarkable identity called the ​​Legendre duplication formula​​: $\Gamma(z) \Gamma(z+\frac{1}{2}) = 2^{1-2z} \sqrt{\pi} \Gamma(2z)$. If we apply Stirling's approximation to every Gamma function in this identity, a flurry of algebraic steps reveals that the ratio of the two sides indeed approaches 1 for large $z$. The approximation is so good that it "knows about" and respects the hidden symmetries of the Gamma function.

The formula can also reveal surprising behaviors. For instance, the ratio of two Gamma functions whose arguments are very close, like $\Gamma(n+1)/\Gamma(n+1/2)$, turns out to behave simply as $\sqrt{n}$ for large $n$. Even more striking is what happens when we venture off the real number line. What is the size of the Gamma function on the imaginary axis, say at $z = 1+iy$ for a large real number $y$? One's intuition might be that it grows, but Stirling's approximation reveals the startling truth:

It decays exponentially! The magnitude plummets to zero as you move up the imaginary axis. This is a critical piece of information in fields like high-energy physics, and it's a testament to the power of a single approximation to illuminate the rich and complex landscape of mathematical functions.

With such a powerful tool at our disposal, it is easy to become overzealous and apply it everywhere. But a good physicist, or any scientist, knows that understanding the limitations of a tool is just as important as knowing how to use it. Stirling's approximation is an asymptotic formula. It works beautifully when its argument is large. "Large" compared to what? Compared to 1.

The danger of forgetting this is famously illustrated in derivations of quantum statistics. When deriving the ​​Bose-Einstein distribution​​, which describes the behavior of particles like photons, one must count the ways of distributing $n_i$ particles among $g_i$ states at a certain energy level. This involves factorials like $n_i!$. It is tempting to immediately apply Stirling's approximation to $\ln(n_i!)$ for all energy levels $i$.

But this is a serious mistake. While the total number of particles in the system might be enormous, the occupation numbers $n_i$ for many individual energy levels, especially high-energy ones, can be very small. They might be 0, 1, or 2. Applying an approximation that requires $n_i \gg 1$ to these small numbers is mathematically invalid and leads to a faulty derivation. For $n=1$, $\ln(1!) = 0$, while the simple form of Stirling's approximation gives $1\ln(1) - 1 = -1$, a huge relative error.

This is a profound lesson. The universe is not always in the asymptotic limit. Nature has its small numbers as well as its large ones. And a truly deep understanding comes not just from wielding powerful formulas, but from knowing with precision and care where their power ends and where a different kind of thinking must begin. Stirling's approximation gives us a brilliant lens to view the world of large numbers, but wisdom lies in knowing when to put that lens down.

Now that we have acquainted ourselves with the machinery of Stirling’s approximation, let us step back and appreciate its phenomenal reach. To a beginner, the formula $n! \sim \sqrt{2\pi n} (n/e)^n$ might seem like little more than a clever trick for taming unwieldy calculations. But to a physicist or a mathematician, it is something far more profound. It is a bridge, a Rosetta Stone that translates the discrete, granular language of counting into the smooth, flowing language of continuous functions and analysis. It allows us to stand back from a system of immense combinatorial complexity and see the simple, elegant, and often surprising patterns that govern its collective behavior. In this journey, we will see how this single mathematical idea illuminates phenomena ranging from a gambler’s luck to the very nature of entropy.

At its core, Stirling’s formula is about counting. And nowhere is the challenge of counting more apparent than in the realm of probability. Imagine a "drunkard's walk," where a person at each step flips a coin and moves one step to the right for heads or one step to the left for tails. This simple model, known as a one-dimensional random walk, is a surprisingly powerful description for everything from stock market fluctuations to the diffusion of molecules.

A natural question to ask is: after an even number of steps, say $2n$, what is the probability that the walker is right back where they started? To be at the origin, the walker must have taken exactly $n$ steps to the right and $n$ steps to the left. The total number of ways to arrange these $n$ right steps and $n$ left steps among the $2n$ total steps is given by the binomial coefficient $\binom{2n}{n}$. Since every specific path of $2n$ steps is equally likely, with a probability of $(1/2)^{2n}$, the probability of returning to the origin is $p_{2n}(0) = \binom{2n}{n} / 4^n$.

For small $n$, this is easy to calculate. But what happens when $n$ is a million, or a billion? Here, Stirling’s formula comes to our rescue. By approximating the factorials in $\binom{2n}{n} = (2n)! / (n!)^2$, we discover a law of stunning simplicity. The intricate combinatorial expression melts away, revealing that for large $n$, the central binomial coefficient behaves as $\binom{2n}{n} \sim 4^n / \sqrt{\pi n}$. Plugging this into our probability formula, the $4^n$ terms cancel, leaving us with a beautiful result:

This isn't just a numerical approximation; it's a physical law governing the random walk. It tells us that while a return to the origin is possible, the probability of being there at any specific (large) time fades away with the square root of time. The same tool can be used to find the growth rate of more complex combinatorial objects, like the Catalan numbers, which appear in an astonishing variety of counting problems in computer science and mathematics, or to analyze other binomial coefficients. In each case, Stirling's formula acts as a mathematical microscope, revealing the simple asymptotic laws hidden within daunting combinatorial expressions.

The magic of Stirling’s approximation extends far beyond a single point in a distribution. It can reveal the shape of the entire landscape of chance. Consider the Poisson distribution, which governs the probability of a certain number of events occurring in a fixed interval, such as the number of radioactive nuclei that decay in a second, or the number of calls arriving at a call center in an hour. If the average number of events is $\lambda$, the probability of observing exactly $k$ events is $P(k; \lambda) = e^{-\lambda}\lambda^k/k!$.

What happens when the average, $\lambda$, is very large? Let's say $\lambda=1000$. We might expect the distribution to be peaked around $k=1000$. Using Stirling's formula for $k!$, we can approximate the probability right at the peak, $P(X=\lambda)$, and we find it is approximately $1/\sqrt{2\pi\lambda}$. But we can do much more.

If we look at the probabilities not just at the peak, but in its neighborhood—for values of $k$ that are some deviation away from the mean—a remarkable transformation occurs. By applying the logarithmic form of Stirling's approximation and making the substitution $k = \lambda + x\sqrt{\lambda}$ (where $x$ measures the deviation from the mean in units of standard deviation), the discrete, lopsided Poisson formula miraculously morphs into the familiar, symmetric form of the Gaussian (or Normal) distribution:

This is a deep and beautiful result, a concrete example of the Central Limit Theorem at work. The messy, discrete factorial has been smoothed out by the power of large numbers into the most elegant and ubiquitous curve in all of statistics. Stirling’s formula is the mathematical engine that drives this convergence, allowing us to see how the universal bell curve emerges from the chaos of countless individual random events.

The utility of Stirling’s approximation is not confined to probability and statistics; it is a fundamental tool throughout mathematical analysis. For example, consider a function defined by a power series, like $f(z) = \sum_{n=0}^\infty a_n z^n$. A key question is its radius of convergence—the boundary in the complex plane beyond which the series diverges and the function ceases to be well-behaved. This radius is determined by the rate of growth of the coefficients $a_n$. If these coefficients involve factorials, as in the series with $a_n=1/\binom{2n}{n}$, a direct calculation of the limit is difficult. However, by using Stirling’s formula to find the asymptotic behavior of $a_n$, we can readily compute the radius of convergence, revealing the analytic structure of the function from its combinatorial definition.

Furthermore, the factorial function is just the integer-valued version of the more general Gamma function, $\Gamma(z+1)=z!$. Stirling’s formula is, at heart, an asymptotic formula for the Gamma function, valid even for complex arguments. This allows us to explore the vast, interconnected landscape of special functions. For instance, we can determine the large-argument behavior of the Beta function, $B(z,z)$, by expressing it in terms of Gamma functions and applying the approximation. We can even uncover the surprisingly rapid growth of the Bernoulli numbers, $B_{2n}$, which are defined implicitly through their connection to the Riemann zeta function, by relating them back to factorials and unleashing Stirling's formula once more. It forges a link, showing a deep unity across different branches of mathematics.

Perhaps the most breathtaking applications of Stirling’s approximation are found when we use it to probe the structure of our physical world.

Let's start with a mind-bending question from geometry. We all have an intuition for the volume of a sphere. But what happens to the volume of a unit ball in a space with a very large number of dimensions, $n$? The formula for this volume, $V_n$, involves Gamma functions. When we apply Stirling’s approximation to this formula for large $n$, we find an astonishing result: the volume of the unit ball, regardless of the specific norm used to define it, plummets towards zero as the number of dimensions increases. This completely defies our three-dimensional intuition. It tells us that in high-dimensional spaces, "volume" is not concentrated near the center but is instead found in a thin shell near the surface; the vast majority of points in a high-dimensional hypercube lie far from its center. This strange geometry, made visible by Stirling's formula, is not just a curiosity; it is the everyday reality for data scientists, machine learning engineers, and theoretical physicists working with models in many thousands of dimensions.

Finally, we arrive at one of the cornerstones of 19th-century physics: thermodynamics and the concept of entropy. Why does a perfume, when opened, fill a room? The answer from statistical mechanics, pioneered by Ludwig Boltzmann, is that the system evolves toward the macroscopic state that has the largest number of possible microscopic arrangements of its particles. Entropy is simply the logarithm of this number.

To calculate this number for an ideal gas of $N$ particles, one must count the available states. Crucially, because the atoms are identical, we must divide by $N!$ to avoid overcounting states that are just permutations of one another. The entropy, $S$, is thus related to $\ln(1/N!)=-\ln(N!)$. For a mole of gas, $N$ is Avogadro's number, roughly $6 \times 10^{23}$. Calculating $\ln(N!)$ is an impossible task. But with Stirling’s approximation, $\ln(N!) \approx N \ln N - N$, the problem becomes tractable.

This one step is the key that unlocks thermodynamics. By incorporating this approximation, we resolve the famous Gibbs paradox (which incorrectly predicts an entropy increase when mixing identical gases) and derive the celebrated Sackur-Tetrode equation—an explicit formula for the entropy of a monatomic ideal gas. This equation connects macroscopic, measurable quantities like temperature and volume to microscopic constants like the mass of an atom and Planck's constant. It is the triumphant culmination of statistical mechanics, a solid bridge between the microscopic world of atoms and the macroscopic world of heat and energy that we experience. And this bridge is built, almost single-handedly, by Stirling’s approximation.

From the simple toss of a coin to the arrow of time itself, Stirling's formula is more than an approximation. It is a fundamental principle of scale, revealing how simplicity and predictable regularity emerge from staggering complexity, a testament to the profound and often hidden unity of the sciences.