Before we explore the Gibbs entropy further, it is necessary to introduce a technique called the method of Lagrange multipliers. The following is a sketch proof, one I hope will be satisfactory for the average amateur physicist!
Consider some function of the variables . At a stationary point, the function’s differential is equal to 0:
Since the variables are independent, this equation is only generally true provided each partial derivative is equal to 0. But what if we want to extremise subject to some constraint of the form
where is some known function. This equation provides a relation between the variables, hence they are not all independent. The constraint above can at least in principal be rearranged to give one variable in terms of the others. Hence only of the variables are independent. So we cannot say for definite that for to be 0, each partial derivative of must be 0: we have to treat the problem more carefully.
We have the constraint . Then we can say for certain that its differential everywhere:
This equation can be rearranged to give one of the variable differentials, say the last one for sake of argument:
We substitute this into the equation for :
Here, we introduce the new variable
The quantity is called a Lagrange multiplier. The above becomes
These variables are independent, so we can say with certainty that
for . What about the last variable? From the definition of ,
So we can extend the identity above to include too:
Now consider the function
where . At stationary points,
For this to be true,
So we recover the equations relating the partial derivatives that we derived above, and the constraint. This is a set of simultaneous equations with unknowns (since has been added to the mix). The conclusion: to extremise a function subject to the constraint , we can instead extremise the functon subject to no constraints.
It can be shown (though I will not!) that multiple constraints can be accounted for in a similar way by adding additional terms to our function . For instance, say . Then we would extremise the function
where is our new Lagrange multiplier.
How does this pertain to the Gibbs entropy?
The Gibbs entropy is a function of the variables . We want to find the probabilities that maximise . So far, we’ve purported to know absolutely nothing about the system we’re describing. In reality, we are not completely ignorant of the system we’re describing – we are able to make measurements of certain macroscopic quantities, which manifest themselves as a mean average over all possible microstates. We can convey what we know about the system through constraints.
The implementation of a constraint will be used in the next post.