Physical information
Encyclopedia
In physics
, physical information refers generally to the information
that is contained in a physical system
. Its usage in quantum mechanics
(i.e. quantum information
) is important, for example in the concept of quantum entanglement
to describe effectively direct or causal
relationships between apparently distinct or spatially separated particles.
Information itself may be loosely defined as "that which can distinguish one thing from another".
The information embodied by a thing can thus be said to be the identity of the particular thing itself, that is, all of its properties, all that makes it distinct from other (real or potential) things.
It is a complete description of the thing, but in a sense that is divorced from any particular language.
When clarifying the subject of information, care should be taken to distinguish between the following specific cases:
The above usages are clearly all conceptually distinct from each other. However, many people insist on overloading the word "information" (by itself) to denote (or connote) several of these concepts simultaneously.
(Since this may lead to confusion, this article uses more detailed phrases, such as those shown in bold above, whenever the intended meaning is not made clear by the context.)
that system's "true" state. (In many practical situations, a system's true state may be largely unknown, but a realist would insist that a physical system regardless always has, in principle, a true state of some sort—whether classical or quantum.)
When discussing the information that is contained in physical systems according to modern quantum physics, we must distinguish between classical information and quantum information
. Quantum information specifies the complete quantum state vector (or equivalently, wavefunction) of a system, whereas classical information, roughly speaking, only picks out a definite (pure) quantum state if we are already given a prespecified set of distinguishable (orthogonal) quantum states to choose from; such a set forms a basis for the vector space
of all the possible pure quantum states (see pure state). Quantum information could thus be expressed by providing (1) a choice of a basis such that the actual quantum state is equal to one of the basis vectors, together with (2) the classical information specifying which of these basis vectors is the actual one. (However, the quantum information by itself does not include a specification of the basis, indeed, an uncountable number of different bases will include any given state vector.)
Note that the amount of classical information in a quantum system gives the maximum amount of information that can actually be measured and extracted from that quantum system for use by external classical (decoherent) systems, since only basis states are operationally distinguishable from each other. The impossibility of differentiating between non-orthogonal states is a fundamental principle of quantum mechanics, equivalent to Heisenberg's uncertainty principle
. Because of its more general utility, the remainder of this article will deal primarily with classical information, although quantum information theory does also have some potential applications (quantum computing, quantum cryptography
, quantum teleportation
) that are currently being actively explored by both theoreticians and experimentalists.
, as follows. For a system S, defined abstractly in such a way that it has N distinguishable states (orthogonal quantum states) that are consistent with its description, the amount of information I(S) contained in the system's state can be said to be log(N). The logarithm is selected for this definition since it has the advantage that this measure of information content is additive when concatenating independent, unrelated subsystems; e.g., if subsystem A has N distinguishable states (I(A) = log(N) information content) and an independent subsystem B has M distinguishable states (I(B) = log(M) information content), then the concatenated system has NM distinguishable states and an information content I(AB) = log(NM) = log(N) + log(M) = I(A) + I(B). We expect information to be additive from our everyday associations with the meaning of the word, e.g., that two pages of a book can contain twice as much information as one page.
The base of the logarithm used in this definition is arbitrary, since it affects the result by only a multiplicative constant, which determines the unit of information that is implied. If the log is taken base 2, the unit of information is the binary digit or bit (so named by John Tukey
); if we use a natural logarithm instead, we might call the resulting unit the "nat." In magnitude, a nat is apparently identical to Boltzmann's constant k or the ideal gas constant R, although these particular quantities are usually reserved to measure physical information that happens to be entropy, and that are expressed in physical units such as joules per kelvin, or kilocalories per mole-kelvin.
and information-theoretic entropy is as follows: Entropy is simply that portion of the (classical) physical information contained in a system of interest (whether it is an entire physical system, or just a subsystem delineated by a set of possible messages) whose identity (as opposed to amount) is unknown (from the point of view of a particular knower). This informal characterization corresponds to both von Neumann's formal definition of the entropy of a mixed quantum state (which is just a statistical mixture of pure states; see von Neumann entropy), as well as Claude Shannon's definition of the entropy of a probability distribution
over classical signal states or messages (see information entropy
). Incidentally, the credit for Shannon's entropy formula (though not for its use in an information theory
context) really belongs to Boltzmann, who derived it much earlier for use in his H-theorem
of statistical mechanics. (Shannon himself references Boltzmann in his monograph.)
Furthermore, even when the state of a system is known, we can say that the information in the system is still effectively entropy if that information is effectively incompressible, that is, if there are no known or feasibly determinable correlations or redundancies between different pieces of information within the system. Note that this definition of entropy can even be viewed as equivalent to the previous one (unknown information) if we take a meta-perspective, and say that for observer A to "know" the state of system B means simply that there is a definite correlation between the state of observer A and the state of system B; this correlation could thus be used by a meta-observer (that is, whoever is discussing the overall situation regarding A's state of knowledge about B) to compress his own description of the joint system AB.
Due to this connection with algorithmic information theory
, entropy can be said to be that portion of a system's information capacity which is "used up," that is, unavailable for storing new information (even if the existing information content were to be compressed). The rest of a system's information capacity (aside from its entropy) might be called extropy, and it represents the part of the system's information capacity which is potentially still available for storing newly derived information. The fact that physical entropy is basically "used-up storage capacity" is a direct concern in the engineering of computing systems; e.g., a computer must first remove the entropy from a given physical subsystem (eventually expelling it to the environment, and emitting heat) in order for that subsystem to be used to store some newly computed information.
, "physical information" can be defined to be the loss of Fisher information
that is incurred during the observation of a "physical effect".
Frieden states, if the effect has an intrinsic information level J, and is observed with information level I, then the physical information is defined to be the difference I − J, which Frieden calls the information Lagrangian. Frieden's so-called principle of extreme physical information
or EPI states that extremalizing I − J with respect to variation of the system probability amplitudes can be used the correct Lagrangians for most or even all physical theories.
Physics
Physics is a natural science that involves the study of matter and its motion through spacetime, along with related concepts such as energy and force. More broadly, it is the general analysis of nature, conducted in order to understand how the universe behaves.Physics is one of the oldest academic...
, physical information refers generally to the information
Information
Information in its most restricted technical sense is a message or collection of messages that consists of an ordered sequence of symbols, or it is the meaning that can be interpreted from such a message or collection of messages. Information can be recorded or transmitted. It can be recorded as...
that is contained in a physical system
Physical system
In physics, the word system has a technical meaning, namely, it is the portion of the physical universe chosen for analysis. Everything outside the system is known as the environment, which in analysis is ignored except for its effects on the system. The cut between system and the world is a free...
. Its usage in quantum mechanics
Quantum mechanics
Quantum mechanics, also known as quantum physics or quantum theory, is a branch of physics providing a mathematical description of much of the dual particle-like and wave-like behavior and interactions of energy and matter. It departs from classical mechanics primarily at the atomic and subatomic...
(i.e. quantum information
Quantum information
In quantum mechanics, quantum information is physical information that is held in the "state" of a quantum system. The most popular unit of quantum information is the qubit, a two-level quantum system...
) is important, for example in the concept of quantum entanglement
Quantum entanglement
Quantum entanglement occurs when electrons, molecules even as large as "buckyballs", photons, etc., interact physically and then become separated; the type of interaction is such that each resulting member of a pair is properly described by the same quantum mechanical description , which is...
to describe effectively direct or causal
Causality
Causality is the relationship between an event and a second event , where the second event is understood as a consequence of the first....
relationships between apparently distinct or spatially separated particles.
Information itself may be loosely defined as "that which can distinguish one thing from another".
The information embodied by a thing can thus be said to be the identity of the particular thing itself, that is, all of its properties, all that makes it distinct from other (real or potential) things.
It is a complete description of the thing, but in a sense that is divorced from any particular language.
When clarifying the subject of information, care should be taken to distinguish between the following specific cases:
- The phrase instance of information refers to the specific instantiationInstantiationInstantiation or instance*Philosophy:*A modern concept similar to participation in classical Platonism, see the Theory of Forms.*The principle of instantiation, the idea that in order for a property to exist, it must be had by some object or substance.*Universal instantiation and existential...
of information (identity, form, essence) that is associated with the being of a particular example of a thing. (This allows for the reference to separate instances of information that happen to share identical patterns.) - A holder of information is a variable or mutable instance that can have different forms at different times (or in different situations).
- A piece of information is a particular fact about a thing's identity or properties, i.e., a portion of its instance.
- A pattern of information (or form) is the pattern or content of an instance or piece of information. Many separate pieces of information may share the same form. We can say that those pieces are perfectly correlated or say that they are copies of each other, as in copies of a book.
- An embodiment of information is the thing whose essence is a given instance of information.
- A representation of information is an encoding of some pattern of information within some other pattern or instance.
- An interpretation of information is a decoding of a pattern of information as being a representation of another specific pattern or fact.
- A subject of information is the thing that is identified or described by a given instance or piece of information. (Most generally, a thing that is a subject of information could be either abstract or concrete; either mathematical or physical.)
- An amount of information is a quantification of how large a given instance, piece, or pattern of information is, or how much of a given system's information content (its instance) has a given attribute, such as being known or unknown. Amounts of information are most naturally characterized in logarithmLogarithmThe logarithm of a number is the exponent by which another fixed value, the base, has to be raised to produce that number. For example, the logarithm of 1000 to base 10 is 3, because 1000 is 10 to the power 3: More generally, if x = by, then y is the logarithm of x to base b, and is written...
ic units.
The above usages are clearly all conceptually distinct from each other. However, many people insist on overloading the word "information" (by itself) to denote (or connote) several of these concepts simultaneously.
(Since this may lead to confusion, this article uses more detailed phrases, such as those shown in bold above, whenever the intended meaning is not made clear by the context.)
Classical versus quantum information
The instance of information that is contained in a physical system is generally considered to specifythat system's "true" state. (In many practical situations, a system's true state may be largely unknown, but a realist would insist that a physical system regardless always has, in principle, a true state of some sort—whether classical or quantum.)
When discussing the information that is contained in physical systems according to modern quantum physics, we must distinguish between classical information and quantum information
Quantum information
In quantum mechanics, quantum information is physical information that is held in the "state" of a quantum system. The most popular unit of quantum information is the qubit, a two-level quantum system...
. Quantum information specifies the complete quantum state vector (or equivalently, wavefunction) of a system, whereas classical information, roughly speaking, only picks out a definite (pure) quantum state if we are already given a prespecified set of distinguishable (orthogonal) quantum states to choose from; such a set forms a basis for the vector space
Vector space
A vector space is a mathematical structure formed by a collection of vectors: objects that may be added together and multiplied by numbers, called scalars in this context. Scalars are often taken to be real numbers, but one may also consider vector spaces with scalar multiplication by complex...
of all the possible pure quantum states (see pure state). Quantum information could thus be expressed by providing (1) a choice of a basis such that the actual quantum state is equal to one of the basis vectors, together with (2) the classical information specifying which of these basis vectors is the actual one. (However, the quantum information by itself does not include a specification of the basis, indeed, an uncountable number of different bases will include any given state vector.)
Note that the amount of classical information in a quantum system gives the maximum amount of information that can actually be measured and extracted from that quantum system for use by external classical (decoherent) systems, since only basis states are operationally distinguishable from each other. The impossibility of differentiating between non-orthogonal states is a fundamental principle of quantum mechanics, equivalent to Heisenberg's uncertainty principle
Uncertainty principle
In quantum mechanics, the Heisenberg uncertainty principle states a fundamental limit on the accuracy with which certain pairs of physical properties of a particle, such as position and momentum, can be simultaneously known...
. Because of its more general utility, the remainder of this article will deal primarily with classical information, although quantum information theory does also have some potential applications (quantum computing, quantum cryptography
Quantum cryptography
Quantum key distribution uses quantum mechanics to guarantee secure communication. It enables two parties to produce a shared random secret key known only to them, which can then be used to encrypt and decrypt messages...
, quantum teleportation
Quantum teleportation
Quantum teleportation, or entanglement-assisted teleportation, is a process by which a qubit can be transmitted exactly from one location to another, without the qubit being transmitted through the intervening space...
) that are currently being actively explored by both theoreticians and experimentalists.
Quantifying classical physical information
An amount of (classical) physical information may be quantified, as in information theoryInformation theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
, as follows. For a system S, defined abstractly in such a way that it has N distinguishable states (orthogonal quantum states) that are consistent with its description, the amount of information I(S) contained in the system's state can be said to be log(N). The logarithm is selected for this definition since it has the advantage that this measure of information content is additive when concatenating independent, unrelated subsystems; e.g., if subsystem A has N distinguishable states (I(A) = log(N) information content) and an independent subsystem B has M distinguishable states (I(B) = log(M) information content), then the concatenated system has NM distinguishable states and an information content I(AB) = log(NM) = log(N) + log(M) = I(A) + I(B). We expect information to be additive from our everyday associations with the meaning of the word, e.g., that two pages of a book can contain twice as much information as one page.
The base of the logarithm used in this definition is arbitrary, since it affects the result by only a multiplicative constant, which determines the unit of information that is implied. If the log is taken base 2, the unit of information is the binary digit or bit (so named by John Tukey
John Tukey
John Wilder Tukey ForMemRS was an American statistician.- Biography :Tukey was born in New Bedford, Massachusetts in 1915, and obtained a B.A. in 1936 and M.Sc. in 1937, in chemistry, from Brown University, before moving to Princeton University where he received a Ph.D...
); if we use a natural logarithm instead, we might call the resulting unit the "nat." In magnitude, a nat is apparently identical to Boltzmann's constant k or the ideal gas constant R, although these particular quantities are usually reserved to measure physical information that happens to be entropy, and that are expressed in physical units such as joules per kelvin, or kilocalories per mole-kelvin.
Physical information and entropy
An easy way to understand the underlying unity between physical (as in thermodynamic) entropyEntropy
Entropy is a thermodynamic property that can be used to determine the energy available for useful work in a thermodynamic process, such as in energy conversion devices, engines, or machines. Such devices can only be driven by convertible energy, and have a theoretical maximum efficiency when...
and information-theoretic entropy is as follows: Entropy is simply that portion of the (classical) physical information contained in a system of interest (whether it is an entire physical system, or just a subsystem delineated by a set of possible messages) whose identity (as opposed to amount) is unknown (from the point of view of a particular knower). This informal characterization corresponds to both von Neumann's formal definition of the entropy of a mixed quantum state (which is just a statistical mixture of pure states; see von Neumann entropy), as well as Claude Shannon's definition of the entropy of a probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....
over classical signal states or messages (see information entropy
Information entropy
In information theory, entropy is a measure of the uncertainty associated with a random variable. In this context, the term usually refers to the Shannon entropy, which quantifies the expected value of the information contained in a message, usually in units such as bits...
). Incidentally, the credit for Shannon's entropy formula (though not for its use in an information theory
Information theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
context) really belongs to Boltzmann, who derived it much earlier for use in his H-theorem
H-theorem
In Classical Statistical Mechanics, the H-theorem, introduced by Ludwig Boltzmann in 1872, describes the increase in the entropy of an ideal gas in an irreversible process. H-theorem follows from considerations of Boltzmann's equation...
of statistical mechanics. (Shannon himself references Boltzmann in his monograph.)
Furthermore, even when the state of a system is known, we can say that the information in the system is still effectively entropy if that information is effectively incompressible, that is, if there are no known or feasibly determinable correlations or redundancies between different pieces of information within the system. Note that this definition of entropy can even be viewed as equivalent to the previous one (unknown information) if we take a meta-perspective, and say that for observer A to "know" the state of system B means simply that there is a definite correlation between the state of observer A and the state of system B; this correlation could thus be used by a meta-observer (that is, whoever is discussing the overall situation regarding A's state of knowledge about B) to compress his own description of the joint system AB.
Due to this connection with algorithmic information theory
Algorithmic information theory
Algorithmic information theory is a subfield of information theory and computer science that concerns itself with the relationship between computation and information...
, entropy can be said to be that portion of a system's information capacity which is "used up," that is, unavailable for storing new information (even if the existing information content were to be compressed). The rest of a system's information capacity (aside from its entropy) might be called extropy, and it represents the part of the system's information capacity which is potentially still available for storing newly derived information. The fact that physical entropy is basically "used-up storage capacity" is a direct concern in the engineering of computing systems; e.g., a computer must first remove the entropy from a given physical subsystem (eventually expelling it to the environment, and emitting heat) in order for that subsystem to be used to store some newly computed information.
Extreme physical information
According to a theory developed by B. Roy FriedenB. Roy Frieden
B. Roy Frieden is an American mathematical physicist.Frieden obtained a Ph.D. in Optics from The Institute of Optics at the University of Rochester. His doctoral thesis advisor was Robert E. Hopkins...
, "physical information" can be defined to be the loss of Fisher information
Fisher information
In mathematical statistics and information theory, the Fisher information is the variance of the score. In Bayesian statistics, the asymptotic distribution of the posterior mode depends on the Fisher information and not on the prior...
that is incurred during the observation of a "physical effect".
Frieden states, if the effect has an intrinsic information level J, and is observed with information level I, then the physical information is defined to be the difference I − J, which Frieden calls the information Lagrangian. Frieden's so-called principle of extreme physical information
Extreme physical information
Extreme physical information is a principle, first described and formulated in 1998 by B. Roy Frieden, Emeritus Professor of Optical Sciences at the University of Arizona, that states, the precipitation of scientific laws can be derived through Fisher information, taking the form of differential...
or EPI states that extremalizing I − J with respect to variation of the system probability amplitudes can be used the correct Lagrangians for most or even all physical theories.
See also
- Digital physicsDigital physicsIn physics and cosmology, digital physics is a collection of theoretical perspectives based on the premise that the universe is, at heart, describable by information, and is therefore computable...
- Entropy in thermodynamics and information theoryEntropy in thermodynamics and information theoryThere are close parallels between the mathematical expressions for the thermodynamic entropy, usually denoted by S, of a physical system in the statistical thermodynamics established by Ludwig Boltzmann and J. Willard Gibbs in the 1870s; and the information-theoretic entropy, usually expressed as...
- History of information theoryHistory of information theoryThe decisive event which established the discipline of information theory, and brought it to immediate worldwide attention, was the publication of Claude E...
- Information entropyInformation entropyIn information theory, entropy is a measure of the uncertainty associated with a random variable. In this context, the term usually refers to the Shannon entropy, which quantifies the expected value of the information contained in a message, usually in units such as bits...
- Information theoryInformation theoryInformation theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
- Logarithmic scaleLogarithmic scaleA logarithmic scale is a scale of measurement using the logarithm of a physical quantity instead of the quantity itself.A simple example is a chart whose vertical axis increments are labeled 1, 10, 100, 1000, instead of 1, 2, 3, 4...
- Logarithmic units
- Reversible computingReversible computingReversible computing is a model of computing where the computational process to some extent is reversible, i.e., time-invertible. A necessary condition for reversibility of a computational model is that the transition function mapping states to their successors at a given later time should be...
(for relations between information and energy) - Philosophy of informationPhilosophy of informationThe philosophy of information is the area of research that studies conceptual issues arising at the intersection of computer science, information technology, and philosophy.It includes:...
- Thermodynamic entropyEntropyEntropy is a thermodynamic property that can be used to determine the energy available for useful work in a thermodynamic process, such as in energy conversion devices, engines, or machines. Such devices can only be driven by convertible energy, and have a theoretical maximum efficiency when...
Further reading
- J. G. Hey, ed., Feynman and Computation: Exploring the Limits of Computers, Perseus, 1999.
- Harvey S. Leff and Andrew F. Rex, Maxwell's Demon 2: Entropy, Classical and Quantum Information, Computing, Institute of Physics Publishing, 2003.