While defining the Gibbs entropy, I quoted an equation called Stirling’s approximation, which says that
if is a large number. In this post we’ll discover where this equation comes from.
Unfortunately, the method I’m about to show you uses another equation which completely eclipses the result in beauty. It is
This expression warrants a discussion of its own, but for now we’ll take it as read. If this equation is unfamiliar to you, take a look at the Gamma function.
The Stirling approximation can be derived by approximating the integrand in the equation above. For shorthand, we’ll name it little gamma:
Before we decide how to approximate this function, let’s work out what it looks like. First of all, we see that
That is, the integrand vanishes at the limits of integration. Why is this? That goes to 0 at the origin is clear enough, since
What about as tends to infinity? Although blows up quickly, it doesn’t do so quick enough to counteract the crushing force of . The limit
can be formally calculated in a number of ways, the most convincing of which uses l’Hôpital’s rule.
So starts and ‘ends’ at zero. How does it behave in-between? Let’s look at its derivative:
Note that, like itself, the gradient of goes to zero at the limits of integration. More importantly, the first factor tells us that
That is, the function has an extremum (which must be a maximum since for ) at . Using what we’ve learnt, we can draw a fairly comprehensive sketch of . The diagram below shows sketches of for .
The diagram above shows that it would be a good idea to approximate about the point . We want our expansion of to be most accurate around its maximum, so that we don’t lose a significant amount of its area when we finally integrate it. With this in mind, let’s write in the form
The new independent variable measures how far to the left or right of ‘s maximum we are. So now we have
There are far too many powers going on here, so let’s temporarily work with the natural logarithm of :
We’re now going to Taylor expand the logarithm on the right-hand side using the equation
which is most accurate for small . To get the logarithm into this form, we’re going to take out a ‘factor’ of :
We can now use the Taylor expansion because in the area around ‘s maximum, is larger than , hence is ‘small’. So
Exponentiating both sides, we find
So approximately, the integrand is a Gaussian. The diagram below compares our approximation of to its exact shape for =15, the former show in brown and the latter in gold.
Not too bad!
Let’s reconsider the equation for given above:
Using our expression for the integrand ,
Note that the lower limit of integration has changed, since we’re using the new variable which is equal to when is zero. One last approximation: we’ve demanded that is ‘large’, so we’ll extend the lower limit of integration to negative infinity. This doesn’t hurt too much, since the integrand has all but vanished by the time reaches , so the additional area that will be included is only small. So,
The integral is a ‘standard’ one, and evaluation gives
Finally, we take the logarithm of both sides:
This method actually gives us an additional term on order of the logarithm of . This term is often omitted since it increases much more slowly than either or ln() as increases. Doing so yields the desired result
Test how good Stirling’s approximation is for a few values of for yourself. If you want to check really big values of you’ll have to use something powerful like Mathematica – most handheld calculators conk out at 69!, and both Google and Excel can’t handle anything bigger than 170!.