Explain the constraints we have to incorporate when we find the probability distribution that maximises the entropy
Write the extended function that needs to optimised and the derivative of this function with respect to $p_i$
Write an expression for the generalised partition function
What is the partial derivative $\left( \frac{\partial \psi}{\partial \lambda^{(j)}} \right)$ equal to
Write an expression for the entropy in terms of $\psi$, $\{\lambda^{(j)}\}$ and $\{\langle B^{(j)}\rangle \}$
Write an expression for the differential $\textrm{d}S$ as a linear combination of the differentials $\{\textrm{d}\alpha_k\}$ and $\{\textrm{d}\langle B^{(j)} \rangle\}$
Explain the meaning of the term generalised force
The axiom of equal a priori probabilities tells us that we can find the probability of being in a state by maximising the entropy:
$S=-k_B \sum_{i=1}^N p_i \log p_i$
However, when doing this we must recognise that the problem to be solved here is a constrained optimisation. In particular, normalisation requires that we have:
$\sum_i p_i = 1$
Furthermore, if each of the microstates in phase space has some set of properties $\{b^{(j)}\}_i$ (a volume, energy, number of atoms, etc) we also assert that the ensemble average for each of these properties $\langle B^{(j)} \rangle$ must be finite. In other words:
$\sum_i p_i b^{(j)}_i = \langle B^{(j)} \rangle$
The extended function we generate using Lagrange's method of undetermined multipliers is thus:
$I(\mathbf{p},\lambda_0,\{\lambda^{(j)}\} ) = -k_B \sum_{i} p_i \ln p_i - k_B \lambda_0 \left( \sum_i p_i - 1 \right) - k_B \sum_j \lambda_j\left( \sum_i p_i b^{(j)}_i - \langle B^{(j)} \rangle \right)$
Differentiating this with respect to $p_i$ we obtain:
$\begin{aligned} \left( \frac{\partial I}{\partial p_i} \right) = -\ln p_i -\psi - \sum_j \lambda^{(j)} b^{(j)}_i = 0\\ \rightarrow \quad p_i = e^{-\psi} e^{-\sum_j \lambda^{(j)} b^{(j)}_i} \end{aligned}$
where we define $\psi=\lambda_0 + 1.$
We can derive a value for the $e^{-\psi}$ that appeared in our final expression for the probability, $p_i$, of being in microstate $i$ by remembering that our probability mass vector must be normalised:
$\sum_i p_i = e^{-\psi} \sum_i e^{-\sum_j \lambda^{(j)} b^{(j)}_i} = 1 \quad \rightarrow \quad e^{\psi} = \sum_i e^{-\sum_j \lambda^{(j)} b^{(j)}_i}$
This quantity, $e^\psi$, is called the generalised partition function. It is generally given the symbol $Z$ and we can use it to rewrite the probability of being in a microstate as:
$p_i = \frac{e^{-\sum_j \lambda^{(j)} b^{(j)}_i} }{Z}$
We can also determine from the partition function by using:
$\begin{aligned} \frac{\partial \psi}{\partial \lambda^{(j)}} & = \frac{\partial }{\partial \lambda^{(j)}} \ln\left[ \sum_i e^{-\sum_j \lambda^{(j)} b^{(j)}_i} \right] \\ & = \frac{-\sum_i b^{(j)}_i e^{-\sum_j \lambda^{(j)} b^{(j)}_i}}{Z} \\ &= - \langle B^{(j)} \rangle \end{aligned}$
https://www.youtube.com/watch?v=66OywtwENus&ab_channel=GarethTribello
We next remember that the entropy can be calculated using:
$S = -k_B \sum_{i} p_i \log p_i$.
Inserting the formula for the probability that we derived above this is:
$\begin{aligned} \frac{S}{k_B} & = \sum_i p_i \left[ \psi + \sum_j \lambda^{(j)} b^{(j)}_i \right] \\ & = \psi \sum_i p_i + \sum_j \lambda^{(j)} \sum_i p_i b^{(j)}_i = \psi + \lambda^{(j)} \langle B^{(j)} \rangle \end{aligned}$
https://www.youtube.com/watch?v=mtQtcDA_7sI&t=316s&ab_channel=GarethTribello
We now suppose that the values of the set of properties $\{b^{(j)}\}_i$ for each of the microstates depend on some set of external parameters $\{\alpha_k\}$. We then ask if it is possible to write an expression for the differential, $\textrm{d}S$, of the entropy change. Using the expression that we arrived at for the entropy in the previous part we can write:
$\frac{\textrm{d}S}{k_B} = \textrm{d}\psi + \sum_j \lambda^{(j)}d\langle B^{(j)} \rangle + \sum_j \langle B^{(j)} \rangle \textrm{d}\lambda^{(j)}$
Furthermore, remembering that the value of the properties of each of the microstates depend on $\alpha$ and recalling our old friend from PDEs, we arrive at:
$\begin{aligned} \textrm{d}\psi &= \sum_k \left( \frac{\partial \psi}{\partial \alpha_k} \right) \textrm{d}\alpha_k + \sum_j \left( \frac{\partial \psi}{\partial \lambda^{(j)} } \right) \textrm{d}\lambda^{(j)} \\ &= \sum_k \left( \frac{\partial \psi}{\partial \alpha_k} \right) \textrm{d}\alpha - \sum_j \langle B^{(j)} \rangle \textrm{d}\lambda^{(j)} \end{aligned}$
Inserting this result into the previous expression and calculating $\left( \frac{\partial \psi}{\partial \alpha_k} \right)$ using the definition of the generalised partition function we arrive at:
$\begin{aligned} \frac{\textrm{d}S}{k_B} & = \sum_k \langle F_k \rangle \textrm{d}\alpha_k + \sum_j \lambda^{(j)} \textrm{d}\langle B^{(j)} \rangle \\ \textrm{where} \quad \frac{F_k}{k_B T} & = -\sum_j \lambda^{(j)} \left\langle \frac{\partial b^{(j)}}{\partial \alpha_k} \right\rangle \end{aligned}$
$F_k$ is a quantity known as a generalised force. Each of the terms inside the summation is an ensemble average of the derivative of the $b^{(j)}$ properties with respect to $\alpha_k$.
https://www.youtube.com/watch?v=-Tbv5CGd_H4&ab_channel=GarethTribello
<aside> 📌 SUMMARY: We can find the probability of being in any microstate by performing a constrained optimisation using the method of Lagrange's undetermined multipliers. This process leads us to a natural definition for the generalised partition function. We can then calculate thermodynamic variables for equilibrated macro states by taking ensemble averages over all the microstates in phase space
</aside>
(a volume, energy, number of atoms, etc) we also assert that the ensemble average for each of these properties
⟨B(j)⟩\langle B^{(j)} \rangle
⟨B(j)
(a volume, energy, number of atoms, etc) we also assert that the ensemble average for each of these properties
⟨B(j)⟩\langle B^{(j)} \rangle
⟨B(j)
(a volume, energy, number of atoms, etc) we also assert that the ensemble average for each of these properties
⟨B(j)⟩\langle B^{(j)} \rangle
⟨B(j)