In
mathematical physicsMathematical physics is the scientific discipline concerned with the interface of mathematics and physics. The Journal of Mathematical Physics defines it as: "the application of mathematics to problems in physics and the development of mathematical methods suitable for such applications and for the...
, especially as introduced into
statistical mechanicsStatistical mechanics is the application of probability theory, which includes mathematical tools for dealing with large populations, to the field of mechanics, which is concerned with the motion of particles or objects when subjected to a force...
and
thermodynamicsIn physics, thermodynamics is the study of the conversion of energy into work and heat and its relation to macroscopic variables such as temperature, volume and pressure...
by J. Willard Gibbs in 1878, an
ensemble (also
statistical ensemble or
thermodynamic ensemble) is an idealization consisting of a large number of mental copies (sometimes infinitely many) of a
systemSystem is a set of interacting or interdependent entities forming an integrated whole....
, considered all at once, each of which represents a possible state that the real system might be in. This article treats the notion of ensembles in a mathematically rigorous fashion, although relevant physical aspects will be mentioned.
Physical considerations
The ensemble formalises the notion that a physicist repeating an experiment again and again under the same macroscopic conditions, but unable to control the microscopic details, may expect to observe a range of different outcomes.
The notional size of the mental ensembles in thermodynamics, statistical mechanics and
quantum statistical mechanicsQuantum statistical mechanics is the study of statistical ensembles of quantum mechanical systems. A statistical ensemble is described by a density operator S, which is a non-negative, self-adjoint, trace-class operator of trace 1 on the Hilbert space H describing the quantum system. This can be...
can be very large indeed, to include every possible
microscopic stateIn statistical mechanics, a microstate describes a specific detailed microscopic configuration of a thermodynamic system that the system visits in the course of its thermal fluctuations....
the system could be in, consistent with its observed
macroscopicMacroscopic is a word commonly used to describe physical objects that are measurable and observable by the naked eye.When applied to phenomena and abstract objects, it describes existence in the world as we perceive it, often in contrast to experiences or theories considering objects of geometric...
properties. But for important physical cases it can be possible to calculate averages directly over the whole of the thermodynamic ensemble, to obtain explicit formulas for many of the thermodynamic quantities of interest, often in terms of the appropriate partition function (see below). Some of these results are presented in the article
Statistical mechanicsStatistical mechanics is the application of probability theory, which includes mathematical tools for dealing with large populations, to the field of mechanics, which is concerned with the motion of particles or objects when subjected to a force...
.
Note on terminology
- The word ensemble is also sometimes used for smaller sets of possibilities, sampled
In statistics, a sample is a subset of a population. Typically, the population is very large, making a census or a complete enumeration of all the values in the population impractical or impossible. The sample represents a subset of manageable size...
from the full set of possible states. Thus for example, an ensemble of walkers in a Markov chain Monte CarloMarkov chain Monte Carlo methods , are a class of algorithms for sampling from probability distributions based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. The state of the chain after a large number of steps is then used as a sample from the...
iteration.
- The word ensemble is particularly used in thermodynamics; by some physicists working in Bayesian probability
Bayesian probability is one of the most popular interpretations of the concept of probability. The Bayesian interpretation of probability can be seen as an extension of logic that enables reasoning with uncertain statements. To evaluate the probability of a hypothesis, the Bayesian probabilist...
theory; and by mathematicians whose work in probability theory is heavily influenced by physicists, especially those working on random matrices. Most "pure" mathematicians working in probability theoryProbability theory is the branch of mathematics concerned with analysis of random phenomena. The central objects of probability theory are random variables, stochastic processes, and events: mathematical abstractions of non-deterministic events or measured quantities that may either be single...
do not use the term, preferring to use the terminology of probability spaceIn probability theory, the probability space, or probability triple, is a concept which serves as a rigorous mathematical ground for the conventional idea of randomness...
s.
Ensembles of classical mechanical systems
For an ensemble of a
classical mechanical systemHamiltonian mechanics is a reformulation of classical mechanics that was introduced in 1833 by Irish mathematician William Rowan Hamilton. It arose from Lagrangian mechanics, a previous reformulation of classical mechanics introduced by Joseph Louis Lagrange in 1788, but can be formulated without...
, one considers the phase space of the given system. A collection of elements from the ensemble can be viewed as a swarm of representative points in the phase space. The statistical properties of the ensemble then depend on a chosen probability measure on the phase space. If a region
A of the phase space has larger measure than region B, then a system chosen at random from the ensemble is more likely to be in a microstate belonging to
A than
B. The choice of this measure is dictated by the specific details of the system and the assumptions one makes about the ensemble in general. For example, the phase space measure of the
microcanonical ensembleIn statistical physics, the microcanonical ensemble is a theoretical tool used to describe the thermodynamic properties of an isolated system. In such a system, the possible states of the system all have the same energy and the probability for the system to be in any given state is the same.-...
(see below) is different from that of the
canonical ensembleA canonical ensemble in statistical mechanics is a statistical ensemble representing a probability distribution of microscopic states of the system...
. The normalizing factor of the probability measure is referred to as the
partition functionPartition function may refer to:*Partition function *Partition function , which generalizes its use in statistical mechanics and quantum field theory:**Partition function...
of the ensemble. Physically, the partition function encodes the underlying physical structure of the system.
When the measure is time-independent, the ensemble is said to be
stationary.
Principal ensembles of statistical thermodynamics
Different macroscopic environmental constraints lead to different types of ensembles, with particular statistical characteristics. The following are the most important:
- Microcanonical ensemble
In statistical physics, the microcanonical ensemble is a theoretical tool used to describe the thermodynamic properties of an isolated system. In such a system, the possible states of the system all have the same energy and the probability for the system to be in any given state is the same.-...
or NVE ensemble -- an ensemble of systems, each of which is required to have the same total energy (i.e. thermally isolated).
- Canonical ensemble
A canonical ensemble in statistical mechanics is a statistical ensemble representing a probability distribution of microscopic states of the system...
or NVT ensemble -- an ensemble of systems, each of which can share its energy with a large heat reservoir or heat bath. The system is allowed to exchange energy with the reservoir, and the heat capacity of the reservoir is assumed to be so large as to maintain a fixed temperature for the coupled system.
- Grand canonical ensemble
In statistical mechanics, a grand canonical ensemble is an imaginary collection of model systems put together to mirror the calculated probability distribution of microscopic states of a given physical system which is being maintained in a given macroscopic state...
-- an ensemble of systems, each of which is again in thermal contact with a reservoir. But now in addition to energy, there is also exchange of particles. The temperature is still assumed to be fixed.
The calculations that can be made over each of these ensembles are explored further in the article
Statistical mechanicsStatistical mechanics is the application of probability theory, which includes mathematical tools for dealing with large populations, to the field of mechanics, which is concerned with the motion of particles or objects when subjected to a force...
. The main result for each ensemble however, is its characteristic state function:
Microcanonical:
Canonical:
Grand canonical:
For these ensembles, the choice for the appropriate probability measure is dictated by the expressions above.
Other thermodynamic ensembles can be also defined, corresponding to different physical requirements, for which analogous formulae can often similarly be derived.
Properties of "good" ensembles
The following properties are considered desirable for a classical mechanical ensemble.
The chosen probability measure on the phase space should be a
Gibbs stateA Gibbs state in probability theory and statistical mechanics is an equilibrium probability distribution which remains invariant under future evolution of the system .In physics there may be several physically...
of the ensemble, i.e. it should be invariant under time evolution. A standard example of this is the natural measure (locally, it is just the Lebesgue measure) on a constant energy surface for a classical mechanical system.
Liouville's theoremLiouville's theorem has various meanings, all mathematical results named after Joseph Liouville:*In complex analysis, see Liouville's theorem .*In conformal mappings, see Liouville's theorem ....
states this measure is invariant under the Hamiltonian flow.
Once a probability measure μ on the phase space is specified, one can define the
ensemble average of an observable, i.e. real-valued function
f defined on via this measure by
,
where we have restricted to those observables which are μ-integrable.
On the other hand, let denote a representative point in the phase space, and be its image under the flow, specified by the system in question, at time
t. The
time average of
f is defined to be
provided that this limit exists μ-almost everywhere and is independent of .
The
ergodicityErgodic theory is a branch of mathematics that studies dynamical systemswith an invariant measure and related problems. Its initial development was motivated by problems of statistical physics....
requirement is that the ensemble average coincide with the time average. A sufficient condition for ergodicity is that the time evolution of the system is a
mixingIn physics, a dynamical system is said to be mixing if the phase space of the system becomes strongly intertwined, according to at least one of several mathematical definitions. For example, a measure-preserving transformation T is said to be strong mixing ifwhenever A and B are any measurable...
. (See also
ergodic hypothesisIn physics and thermodynamics, the ergodic hypothesis says that, over long periods of time, the time spent by a particle in some region of the phase space of microstates with the same energy is proportional to the volume of this region, i.e., that all accessible microstates are equiprobable over a...
.) Not all systems are ergodic. For instance, it is unknown at this time whether classical mechanical flows on a constant energy surface are ergodic in general. Physically, when a system fails to be ergodic, we may infer that there is more macroscopically discoverable information available about the microscopic state of the system than what we first thought. In turn this may be used to create a better-
conditionedConditional probability is the probability of some event A, given the occurrence of some other event B. Conditional probability is written P, and is read "the probability of A, given B"....
ensemble.
Ensembles in quantum statistical mechanics
Putting aside for the moment the question of how statistical ensembles are generated
operationallyAn operational definition is a demonstration of a process – such as a variable, term, or object – in terms of the specific process or set of validation tests used to determine its presence and quantity. The term was coined by Percy Williams Bridgman...
, we should be able to perform the following two operations on ensembles
A,
B of the same system:
- Test whether A, B are statistically equivalent.
- If p is a real number such that 0 < p < 1, then produce a new ensemble by probabilistic sampling from A with probability p and from B with probability 1 – p.
Under certain conditions therefore,
equivalence classIn mathematics, given a set X and an equivalence relation ~ on X, the equivalence class of an element a in X is the subset of all elements in X which are equivalent to a:...
es of statistical ensembles have the structure of a convex set. In quantum physics, a general model for this convex set is the set of
density operatorsIn quantum mechanics, a density matrix is a self-adjoint positive-semidefinite matrix, , of trace one, that describes the statistical state of a quantum system...
on a
Hilbert spaceThe mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...
. Accordingly, there are two types of ensembles:
- Pure ensembles cannot be decomposed as a convex combination of different ensembles. In quantum mechanics, a pure density matrix is one of the form . Accordingly, a ray in a Hilbert space
The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...
can be used to represent such an ensemble in quantum mechanics. A pure ensemble corresponds to having many copies of the same (up to a global phase) quantum state.
- Mixed ensembles are decomposable into a convex combination of different ensembles. In general, an infinite number of distinct decompositions will be possible.
Thus a quantum mechanical ensemble is specified by a mixed state in general. For example, one can specify the density operators describing microcanonical, canonical, and grand canonical ensembles of quantum mechanical systems, in a mathematically rigorous fashion.
The normalization factor required for the density operator to have trace 1 is the quantum mechanical version of the partition function.
We note here that ensembles of quantum mechanical system are sometimes treated by physicists in a
semi-classical fashion. Namely, one considers the phase space of the corresponding classical system (e.g. for an ensemble of quantum harmonic oscillators, the phase space of a classical harmonic oscillator is considered). Then, using physical arguments, one derives a suitable "fundamental volume" for the particular system to reflect the fact that quantum mechanical microstates are discretely distributed on the phase space. From the uncertainly principle, it is expected this fundamental volume to be related to the Planck constant, , in some way.
Ensembles in statistics
The formulation of statistical ensembles used in physics has now been widely adopted in other fields, in part because it has been recognized that the
Boltzmann distributionIn physics and mathematics, the Boltzmann distribution is a certain distribution function or probability measure for the distribution of the states of a system. It underpins the concept of the canonical ensemble, providing its underlying distribution...
or
Gibbs measureIn mathematics, the Gibbs measure, named after Josiah Willard Gibbs, is a probability measure frequently seen in many problems of probability theory and statistical mechanics. It is the measure associated with the Boltzmann distribution, and generalizes the notion of the canonical ensemble...
serves to maximize the entropy of a system, subject to a set of constraints: this is the
principle of maximum entropyIn subjectivist probability, the principle of maximum entropy is a postulate which states that, subject to known constraints , the probability distribution which best represents the current state of knowledge is the one with largest entropy.Let some testable information about a probability...
. This principle has now been widely applied to problems in
linguisticsLinguistics is the scientific study of natural language. Linguistics encompasses a number of sub-fields. An important topical division is between the study of language structure and the study of meaning...
,
roboticsRobotics is the engineering science and technology of robots, and their design, manufacture, and application. Robotics is related to electronics, mechanics, and software. The word robot was introduced to the public by Czech writer Karel Čapek in his play R.U.R. , published in 1920...
, and the like.
In addition, statistical ensembles in physics are often built on a
principle of localityIn physics, the principle of locality states that an object is influenced directly only by its immediate surroundings. Experiments have shown that quantum mechanically entangled particles must violate either the principle of locality or the form of philosophical realism known as counterfactual...
: that all interactions are only between neighboring atoms or nearby molecules. Thus, for example,
lattice modelsIn physics, a lattice model is a physical model that is defined on a lattice, as opposed to the continuum of space or spacetime. Lattice models originally occurred in the context of condensed matter physics, where the atoms of a crystal automatically form a lattice. Currently, lattice models are...
, such as the
Ising modelThe Ising model, named after the physicist Ernst Ising, is a mathematical model in statistical mechanics. It has since been used to model diverse phenomena in which bits of information, interacting in pairs, produce collectiveeffects.-Definition :...
, model ferromagnetic materials by means of nearest-neighbor interactions between spins. The statistical formulation of the principle of locality is now seen to be a form of the
Markov propertyIn mathematics, the term Markov property or Markov-type property can refer to either of two closely-related things.In the narrowest sense, a stochastic process has the Markov property if the conditional probability distribution of future states of the process, given the present state and a constant...
in the broad sense; nearest neighbors are now
Markov blanketIn machine learning, the Markov blanket for a node in a Bayesian network is the set of nodes composed of 's parents, its children, and its children's other parents. In a Markov network, the Markov blanket of a node is its set of neighbouring nodes...
s. Thus, the general notion of a statistical ensemble with nearest-neighbor interactions leads to Markov random fields, which again find broad applicability; for example in Hopfield networks.
Operational interpretation
In the discussion given so far, while rigorous, we have taken for granted that the notion of an ensemble is valid a priori, as is commonly done in physical context. What has not been shown is that the ensemble
itself (not the consequent results) is a precisely defined object mathematically. For instance,
- It is not clear where this very large set of systems exists (for example, is it a gas of particles inside a container
In quantum mechanics, the results of the quantum particle in a box can be used to look at the equilibrium situation for a quantum ideal gas in a box which is a box containing a large number of molecules which do not interact with each other except for instantaneous thermalizing collisions...
?)
- It is not clear how to physically generate an ensemble.
In this section we attempt to partially answer this question.
Suppose we have a
preparation procedure for a system in a physics
lab: For example, the procedure might involve a physical apparatus and
some protocols for manipulating the apparatus. As a result of this preparation procedure some system
is produced and maintained in isolation for some small period of time.
By repeating this laboratory preparation procedure we obtain a
sequence of systems
X1,
X2,
....,
Xk, which in our mathematical idealization, we assume is an infinite sequence of systems. The systems are similar in that they were all produced in the same way. This infinite sequence is an ensemble.
In a laboratory setting, each one of these prepped systems might be used as input
for
one subsequent
testing procedure. Again, the testing procedure
involves a physical apparatus and some protocols; as a result of the
testing procedure we obtain a
yes or
no answer.
Given a testing procedure
E applied to each prepared system, we obtain a sequence of values
Meas (
E,
X1), Meas (
E,
X2),
...., Meas (
E,
Xk). Each one of these values is a 0 (or no) or a 1 (yes).
Assume the following time average exists:
For quantum mechanical systems, an important assumption made in the
quantum logicIn quantum mechanics, quantum logic is a set of rules for reasoning about propositions which takes the principles of quantum theory into account...
approach to quantum mechanics is the identification of
yes-no questions to the
lattice of closed subspaces of a Hilbert space. With some additional
technical assumptions one can then infer that states are given by
density operators
S so that:
We see this reflects the definition of quantum states in general: A quantum state is a mapping from the observables to their expectation values.
See also
- density matrix
In quantum mechanics, a density matrix is a self-adjoint positive-semidefinite matrix, , of trace one, that describes the statistical state of a quantum system...
- Partition function (statistical mechanics)
In statistical mechanics, the partition function Z is an important quantity that encodes the statistical properties of a system in thermodynamic equilibrium. It is a function of temperature and other parameters, such as the volume enclosing a gas...
- Partition function (mathematics)
The partition function or configuration integral, as used in probability theory, information science and dynamical systems, is an abstraction of the definition of a partition function in statistical mechanics. It is a special case of a normalizing constant in probability theory, for the Boltzmann...
- isothermal-isobaric ensemble
The isothermal–isobaric ensemble is a statistical mechanical ensemble that maintains constant temperature and constant pressure applied. It is also called the -ensemble, where the number of particles is also kept as a constant...
- phase space
In mathematics and physics, a phase space, introduced by Willard Gibbs in 1901, is a space in which all possible states of a system are represented, with each possible state of the system corresponding to one unique point in the phase space. For mechanical systems, the phase space usually consists...
- Liouville's theorem (Hamiltonian)
In physics, Liouville's theorem, named after the French mathematician Joseph Liouville, is a key theorem in classical statistical and Hamiltonian mechanics...