Statistical Mechanics 1

Leo Lue

Department of Chemical & Process Engineering

University of Strathclyde

Statistical mechanics

Ludwig Boltzmann, who spent much of his life studying statistical mechanics, died in 1906, by his own hand. Paul Ehrenfest, carrying on the work, died similarly in 1933. Now it is our turn to study statistical mechanics. Perhaps it will be wise to approach the subject cautiously.

(Opening lines of "States of Matter", by D.L. Goodstein).

Statistical mechanics

And so this is the kind of system mechanics is going to be one of the cornerstones of what you're going to learn in this module. So this kind of interesting question is, you know, so that you know that there's like 10 to the 23 you know in in a gram of water, around 10 or 20, three kind of molecules of water. And if you think about all the different numbers, you have to specify to get the positions and velocities of every single particle. Every single water molecule in the system. You just think that you know there's too much information. How do you know that you have the same sort of bunch of water than someone else, right? But what's kind of interesting is that if you think of water from thermodynamics, how many different parameters you need to specify in order to kind of make sure that you have the same water in the same state as someone else. And the answer is like 2 right? You need to know like temperature and pressure or temperature and density.

So how did you actually reduce something that actually is is 10 to the 23 degrees of freedom. Down to something with just two degrees of freedom.

Alright, and what's sort of interesting is that that's kind of related to, well, this. This sort of how everything kind of reduces to Gaussian Statistics.

It's also kind of interesting in terms of like when you get huge numbers of multiple when you have huge numbers of events. Actually, things are much more predictable than if you just have single events, right? So that's why sort of when you're gambling in a casino. You know whether you win or lose in a particular day or whatever, is actually very difficult to kind of predict, right?

But the casino always wins 'cause they know very precisely. Given that you know 1000 people gamble everyday or 10,000 people gamble or there's 10,000 kind of games going on. They can tell you pretty precisely how many people are gonna win and how many people going to lose and what the distribution of that's going to be.

And so even though the individual events may be very difficult to. To determine The actual statistics of them are actually quite reproducible, and you can actually do a lot to kind of model that. Yep, next slide, thanks.

Overview

Thermodynamics
Microscopic dynamics
Statistical mechanics

Basic concepts in thermodynamics

Thermodynamics is a phenomenological theory that describes the properties of systems, and their relationships, when in equilibrium.

Properties:
- state vs path
- intensive vs extensive
Processes:
- reversible vs irreversible
- spontaneous vs non-spontaneous

First law of thermodynamics

First law of thermodynamics: Energy cannot be created or destroyed, but it can transform from one form to another.

\begin{align*} d(E + {\rm KE} + {\rm PE} + \cdots) &= \delta Q - \delta W \end{align*}

What do we mean by energy?
- kinetic, internal, thermal, electromagnetic, gravitational, etc.
Heat
Work
- pressure: \(-pdV\)
- interfacial tension: \(-\sigma dA\)
- chemical potential: \(\mu_\alpha dN_\alpha\)
- \(\delta W'\): other

\begin{align*} d(E + {\rm KE} + {\rm PE} + \cdots) &= \delta Q - pdV + \mu dN - \delta W' \end{align*}

Second law of thermodynamics

Clausius statement: No process is possible whose sole result is the transfer of heat from a body of lower temperature to a body of higher temperature.

Kelvin-Planck statement: No process is possible in which the sole result is the absorption of heat from a reservoir and its complete conversion into work.

\begin{align*} dS \ge \frac{\delta Q}{T} \end{align*}

The second law of thermodynamics is interesting in that it is one of the few laws of physics that is not reversible. Most physical dynamical equations, such as Newton's laws of motion or even the Schrodinger equation, are the same moving forward and backward in time. This is not the case for the second law of thermodynamics, which does show a difference.

If you accidentally view a film backward, you will immediately notice it because it will look "unusual". This

While different forms of energy can be converted to each other, some

In this sense, not all forms of energy have equal "value".

This provides an

While different forms of energy can be converted to each other, some forms of energy

In this sense, not all forms of energy have equal "value".

This provides an

Combined first and second laws

first law

\begin{align*} dE &= \delta Q - pdV + \sum_\alpha \mu_\alpha dN_\alpha - \delta W' \end{align*}
second law

\begin{align*} \delta Q \le T dS \end{align*}
combined first and second laws

\begin{align*} dE &\le TdS - pdV + \sum_\alpha \mu_\alpha dN_\alpha - \delta W' \\ \delta W' &\le - dE + TdS - pdV + \sum_\alpha \mu_\alpha dN_\alpha \end{align*}

Fundamental equation of thermodynamics

For general transformations:

\begin{align*} dE &\le TdS - pdV + \sum_\alpha \mu_\alpha dN_\alpha - \delta W' \end{align*}

For quasi-equilibrium transformation without extracting work:

\begin{align*} dE &= TdS - pdV + \sum_\alpha \mu_\alpha dN_\alpha \end{align*}

Calculus of thermodynamics

For quasi-equilibrium transformations:

\begin{equation*} dS = - \frac{\mu}{T}dN + \frac{p}{T}dV + \frac{dE}{T} \end{equation*}

For \(f(x,y,z)\):

\begin{equation*} df = \left(\frac{\partial f}{\partial x}\right)_{y,z} dx + \left(\frac{\partial f}{\partial y}\right)_{x,z} dy + \left(\frac{\partial f}{\partial z}\right)_{x,y} dz \end{equation*}

This implies \(S(N,V,E)\) and

\begin{align*} \left(\frac{\partial S}{\partial N}\right)_{V,E} &= - \frac{\mu}{T} \qquad \left(\frac{\partial S}{\partial V}\right)_{N,E} = \frac{p}{T}; \qquad \left(\frac{\partial S}{\partial E}\right)_{N,V} = \frac{1}{T}; \end{align*}

Calculus of thermodynamics

For quasi-equilibrium transformations:

\begin{align*} dE &= \mu dN - pdV + TdS \end{align*}

This implies \(E(N,V,S)\) and

\begin{align*} \left(\frac{\partial E}{\partial N}\right)_{V,S} &= \mu; \qquad \left(\frac{\partial E}{\partial V}\right)_{N,S} = - p; \qquad \left(\frac{\partial E}{\partial S}\right)_{N,V} = - T \end{align*}

Maxwell relations

At equilibrium, \(S(N,V,E)\) and

\begin{equation*} dS = - \frac{\mu}{T}dN + \frac{p}{T}dV + \frac{dE}{T} \end{equation*}
Mixed partial derivatives:

\begin{equation*} \frac{\partial}{\partial x} \left(\frac{\partial f}{\partial y}\right) = \frac{\partial}{\partial y} \left(\frac{\partial f}{\partial x}\right) \end{equation*}
Maxwell relations:

\begin{align*} \left[\frac{\partial}{\partial V}\left(\frac{1}{T}\right)\right]_{E,N} &= \left[\frac{\partial}{\partial E} \left(\frac{p}{T}\right)\right]_{V,N}; \\ \left[\frac{\partial}{\partial N}\left(\frac{1}{T}\right)\right]_{E,V} &= - \left[\frac{\partial}{\partial E} \left(\frac{\mu}{T}\right)\right]_{V,N}; \\ \left[\frac{\partial}{\partial N} \left(\frac{p}{T}\right)\right]_{E,V} &= - \left[\frac{\partial}{\partial V}\left(\frac{\mu}{T}\right)\right]_{E,N} \end{align*}

maximum work

For general transformations:

\begin{align*} dW' &\le -dE + TdS - pdV + \mu dN \end{align*}

maximum entropy

For general transformations:

\begin{equation*} dS \ge \frac{dE}{T} + \frac{p}{T}dV - \frac{\mu}{T}dN + \frac{1}{T}\delta W' \end{equation*}

If we do not extract or do work on the system (i.e. \(\delta W'=0\)) and we hold \(E\), \(V\), and \(N\) constant:

\begin{equation*} dS \ge 0 \end{equation*}

For a system at fixed \(N\), \(V\), and \(E\), the entropy is maximized at equilibrium.

Legendre transforms and free energies

\(E(N,V,S)\):

\begin{align*} dE &\le \mu dN - pdV + TdS - \delta W' \end{align*}
Helmholtz free energy: \(A=E - TS\)

\begin{align*} dA &= dE - TdS - SdT \\ dA &\le \mu dN - pdV - SdT - \delta W' \\ \end{align*}

So we find \(A(N,V,T)\).

Legendre transforms and free energies

internal energy: \(E(N,V,S)\)

\begin{align*} dE &\le \mu dN - pdV + TdS - \delta W' \end{align*}
Helmholtz free energy: \(A(N,V,T)=E-TS\)

\begin{align*} dA &\le \mu dN - pdV - SdT - \delta W' \end{align*}
Gibbs free energy: \(G(N,T,p)=E+pV-TS=A+pV\)

\begin{align*} dG &\le \mu dN + Vdp - SdT - \delta W' \end{align*}
enthalpy: \(H(N,p,S)=U+pV\)

\begin{align*} dH &\le \mu dN + Vdp + TdS - \delta W' \end{align*}

Importance of free energy

Knowledge of the free energy of the system in terms of its "natural" variables, all equilibrium thermodynamics properties can be determined.
The free energy represents the maximum work that can be extracted from a transformation if it is spontaneous or the minimum work required to perform a transformation if it is non-spontaneous.

\begin{align*} \delta W' &\le -dE + \mu dN - pdV + TdS \\ \delta W' &\le -dA + \mu dN - pdV - SdT \\ \delta W' &\le -dG +\mu dN + Vdp - SdT \\ \delta W' &\le -dH +\mu dN + Vdp + TdS \end{align*}

Microscopic dynamics

particle systems

classical mechanics
quantum mechanics
stochastic dynamics

Example: Simple fluid

\(N\) spherical atoms of mass \(m\)
position of atom \(\alpha\) is \({\bf r}_\alpha\)
momentum of atom \(\alpha\) is \({\bf p}_\alpha\)
point in phase space: \({\boldsymbol\Gamma}=({\bf r}_1,\dots{\bf r}_N,{\bf p}_1,\dots,{\bf p}_N)\)
pairwise additive potential \(u(|{\bf r}_\alpha-{\bf r}_{\alpha'}|)\)
Hamiltonian: \(H({\boldsymbol\Gamma})=U({\bf r}_1,\dots{\bf r}_N)+K({\bf p}_1,\dots,{\bf p}_N)\)

\begin{align*} U({\bf r}_1,\dots{\bf r}_N) &= \frac{1}{2} \sum_{\alpha\ne\alpha'} u(|{\bf r}_\alpha-{\bf r}_{\alpha'}|) \\ K({\bf p}_1,\dots,{\bf p}_N) &= \sum_\alpha \frac{p_\alpha^2}{2m} \end{align*}

Classical mechanics

Newton's equation of motion

\begin{align*} \frac{d{\bf r}_\alpha}{dt} &= \frac{{\bf p}_\alpha}{m_\alpha} \\ \frac{d{\bf p}_\alpha}{dt} &= {\bf F}_{\alpha} \end{align*}

where \(m_\alpha\) is the mass, \({\bf r}_\alpha\) is the position, and \({\bf p}_\alpha\) is the momentum of particle \(\alpha\).

Conservative force:

\begin{equation*} {\bf F}_\alpha = - \frac{\partial}{\partial {\bf r}_\alpha} V({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N) \end{equation*}

where \(V({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N)\) is the interaction potential.

Hamiltonian formulation

Hamiltonian:

\begin{align*} H({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N, {\bf p}_1, {\bf p}_2,\dots,{\bf p}_N) &= U({\bf r}_1,\dots{\bf r}_N) + K({\bf p}_1,\dots,{\bf p}_N) \end{align*}

Equations of motion:

\begin{align*} \frac{d{\bf r}_\alpha}{dt} &= \frac{\partial H}{\partial {\bf p}_\alpha} \\ \frac{d{\bf p}_\alpha}{dt} &= - \frac{\partial H}{\partial {\bf r}_\alpha} \end{align*}

The Hamiltonian dictates the dynamics (i.e. time evolution) of the system.

phase space

Liouville's Theorem

We can treat an ensemble of systems flowing around phase space as an incompressible fluid with density \(\rho(\boldsymbol{\Gamma})\), thus it behaves as follows:

\begin{align*} \frac{\partial}{\partial t} \rho(\boldsymbol{\Gamma},t) + \nabla_{\boldsymbol{\Gamma}} \cdot [\dot{\boldsymbol\Gamma}\,\rho({\boldsymbol\Gamma},t)] &= 0 \end{align*}

where

\begin{align*} \nabla_{\boldsymbol{\Gamma}} \cdot [\dot{\boldsymbol\Gamma}\, \rho({\boldsymbol\Gamma})] = \sum_\alpha \left( \dot{ {\bf r}}_\alpha\cdot \frac{\partial}{\partial {\bf r}_\alpha} + \dot{{\bf p}}_\alpha \cdot \frac{\partial}{\partial {\bf p}_\alpha} \right) \rho({\bf r}_1,{\bf r}_2,\dots,{\bf r}_N,{\bf p}_1,{\bf p}_2,\dots,{\bf p}_N) \end{align*}

which should be familiar to you as a simplification of the continuity equation from any fluid flow class you might have had:

\begin{align*} \frac{\partial \rho}{\partial t} + \nabla \cdot [\rho\,\boldsymbol{v}] = 0 \end{align*}

This is the foundation of kinetic theory, as from here comes the BBGKY heirarchy and ultimately Boltzmann's/Enskog's equation.

Microstates

The state of the system is specified by the positions and momenta of all its particles (\(2Nd\) quantities).
The structural and mechanical properties (e.g., energy, pressure, etc.) of the system are given by functions of the positions and momenta of its particles.
The dynamics is given by a set of \(2Nd\) coupled first order differential equations, which can be solved, in principle (e.g., using numerical methods).
For a system in a given state, its future evolution and its past history is known and governed by Newton's laws of motion.

Quantum mechanics

The state of a system is characterized by a function (abstract vector): the wave function. For a system of \(N\) particles:

\begin{align*} \psi({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N) & \end{align*}
The wave function is a continuous and smooth function of the particle positions.
Probability distribution that the particles are in configuration \({\bf r}_1\), \({\bf r}_2\),… \({\bf r}_N\) is

\begin{align*} P({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N) &\propto |\psi({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N)|^2 = \psi^*({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N) \psi({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N) \end{align*}
The properties of the system are determined by operators on the wave function.

Quantum mechanics

Hamiltonian:

\begin{equation*} \hat{H} = - \sum_\alpha \frac{\hbar^2}{2m_\alpha} \nabla_\alpha^2 + V({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N) \end{equation*}

Time dependent Schrodinger equation:

\begin{equation*} \frac{\partial}{\partial t} \psi({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N,t) = - \frac{i}{\hbar}\hat{H}\psi({\bf r}_1, {\bf r}_2,\dots,{\bf r}_N,t) \end{equation*}

The Hamiltonian dictates the dynamics (i.e. time evolution) of the system.

Stochastic dynamics

state: \({\boldsymbol\Gamma}\) (can be continuous or discrete)
transition probabilities: \(\Pi({\boldsymbol\Gamma}'\leftarrow{\boldsymbol\Gamma})\)
- Detailed balance: There exists a stationary distribution \(\rho^*({\boldsymbol\Gamma})\) such that
  
  \begin{align*} \Pi({\boldsymbol\Gamma}'\leftarrow{\boldsymbol\Gamma}) \rho^*({\boldsymbol\Gamma}) &= \Pi({\boldsymbol\Gamma}\leftarrow{\boldsymbol\Gamma}') \rho^*({\boldsymbol\Gamma'}) \end{align*}
dynamics (discrete states)

\begin{align*} \frac{\partial \rho({\boldsymbol\Gamma},t)}{\partial t} &= \sum_{{\boldsymbol\Gamma}'\ne{\boldsymbol\Gamma}} \Pi({\boldsymbol\Gamma}\leftarrow{\boldsymbol\Gamma}') \rho({\boldsymbol\Gamma'},t) + \Pi({\boldsymbol\Gamma}\leftarrow{\boldsymbol\Gamma}) \rho({\boldsymbol\Gamma},t) \end{align*}

Observable properties

Observable: \({\mathcal A}({\boldsymbol\Gamma})\)
trajectory average:

\begin{align*} \langle{\mathcal A}\rangle &= \lim_{t_{\rm obs}\to\infty} \frac{1}{t_{\rm obs}} \int_0^{t_{\rm obs}} {\rm d}t\,{\mathcal A}({\boldsymbol\Gamma}(t)) \end{align*}
ensemble average

\begin{align*} \langle{\mathcal A}(t)\rangle &= \int {\rm d}{\boldsymbol\Gamma}\, \rho({\boldsymbol\Gamma},t) {\mathcal A}({\boldsymbol\Gamma}) \\ \langle{\mathcal A}\rangle &= \lim_{t_{\rm obs}\to\infty} \frac{1}{t_{\rm obs}} \int_0^{t_{\rm obs}}{\rm d}t\, \langle{\mathcal A}(t)\rangle = \lim_{t_{\rm obs}\to\infty} \frac{1}{t_{\rm obs}} \int_0^{t_{\rm obs}}{\rm d}t,\ \int {\rm d}{\boldsymbol\Gamma}\, \rho({\boldsymbol\Gamma},t) {\mathcal A}({\boldsymbol\Gamma}) \end{align*}

Density of states

The density of states \(\Omega(N,V,E)\) is the number of ways the system can have an energy \(E\). In other words, it is the "volume" of phase space with an energy \(E\).
Mathematically, the density of states is given by

\begin{equation*} \Omega(N,V,E) = \int_{E \lt H({\boldsymbol\Gamma})\lt E+\delta E} d{\boldsymbol\Gamma} \end{equation*}

where \(\delta E\ll E\).
Volume element in phase space:

\begin{equation*} d{\boldsymbol\Gamma} = \frac{1}{N!} \frac{d{\bf r}_1d{\bf p}_1}{h^3}\cdots \frac{d{\bf r}_N d{\bf p}_N}{h^3} \end{equation*}

where \(h\) is Planck's constant.

Averaging

God does not play dice. A. Einstein

God does play dice with the universe. All the evidence points to him being an inveterate gambler, who throws the dice on every possible occasion. S. Hawking

You are the omnipotent being of your simulation, so you get to choose:

\begin{align*} \langle{\mathcal A}\rangle &= \int d{\boldsymbol\Gamma} d{\boldsymbol\Gamma}\,P({\boldsymbol\Gamma})\, {\mathcal A}({\boldsymbol\Gamma}) & \text{(Monte Carlo)} \\ &\text{or} \\ \langle{\mathcal A}\rangle &= \lim_{t_{\rm obs}\to\infty} \frac{1}{t_{\rm obs}}\int_0^{t_{\rm obs}}\mathcal A(t)\,{\rm d}t & \text{(Molecular Dynamics)} \end{align*}

But are these equivalent? If so, the system is ergodic

Ergodic hypothesis

A system with fixed \(N\), \(V\), and \(E\) is equally likely to be found in any of its \(\Omega(N,V,E)\) microscopic states. Microstates with the same energy has an equal probability of being occupied.

Consider two subsets of states, \(\Omega_A\) and \(\Omega_B\). A system is more likely to be found in set \(A\) if \(\Omega_A>\Omega_B\).

Therefore \(S(\Omega_A)>S(\Omega_B)\) as \(A\) is more likely, thus \(S\) must be a monotonically increasing function in \(\Omega\).

As states increase multiplicatively, yet entropy is additive, the relationship must be logarithmic.

Statistical mechanics

where \(W=\Omega\), and \(k.=k_B\).

Statistical mechanics

\begin{align*} S(N,V,E) &= k_B \ln \Omega(N,V,E) \end{align*}

where \(k_B=1.3806503\times10^{-23}\) J K\(^{-1}\) is the Boltzmann constant.

Fundamental equation of thermodynamics:

\begin{align*} {\rm d}S(N,V,E) &= \frac{{\rm d}E}{T} + \frac{p}{T}{\rm d}V - \frac{\mu}{T}{\rm d}N \\ {\rm d}\ln\Omega(N,V,E) &= \beta\,{\rm d}E + \beta\,p\,{\rm d} V - \beta\,\mu\,{\rm d}N \end{align*}

where \(\beta=1/(k_B\,T)\).

At fixed \(N\), \(V\), and \(E\), \(\Omega(N,V,E)\) is maximized at equilibrium.

Once the density of states \(\Omega(N,V,E)\) is known, all the thermodynamic properties of the system can be determined.

Ergodicity

Phase space is so unbelievably big even for small systems it actually "feels" impossible for any trajectory to meaningfully visit all microstates (or come close enough to do so) in finite time.

Practically, in simulation we typically just check averages are converging and hope we are "effectively" ergodic. Like a lot of things in simulation, you presume it might be true but verify as best you can.

There are many techniques for accelerating dynamics past energy barriers and "encourage" good sampling of phase space which you will learn about in your MC lectures. These are also applied more frequently to MD simulations too.

Entropy

The second law of thermodynamics, put simply, is that the most likely things will eventually happen and stay there.
Richard Feynman says in his Lectures on Physics "…entropy is just the logarithm of the number of ways of internally arranging a system while having it look the same from the outside".
Entropy is a measure of how much we do not know (i.e. it measures our lack of control-over/measurement-of the microstate).
Entropy is NOT disorder/chaos, it is just that there are so many ways to make a mess…
Sometimes order is more likely (i.e., look at the fluid to crystal transition of hard spheres).

Statistical Mechanics 1

Leo Lue Department of Chemical & Process Engineering University of Strathclyde

Statistical mechanics

Statistical mechanics

Overview

Basic concepts in thermodynamics

First law of thermodynamics

Second law of thermodynamics

Combined first and second laws

Fundamental equation of thermodynamics

Calculus of thermodynamics

Calculus of thermodynamics

Maxwell relations

maximum work

maximum entropy

Legendre transforms and free energies

Legendre transforms and free energies

Importance of free energy

Microscopic dynamics

Example: Simple fluid

Classical mechanics

Hamiltonian formulation

phase space

Liouville's Theorem

Microstates

Quantum mechanics

Quantum mechanics

Stochastic dynamics

Observable properties

Density of states

Averaging

Ergodic hypothesis

Statistical mechanics

Statistical mechanics

Ergodicity

Entropy

Questions?

Leo Lue

Department of Chemical & Process Engineering

University of Strathclyde