Information projection
Encyclopedia
In information theory
, the information projection or I-projection of a probability distribution
q onto a set of distributions P is
where
is the Kullback–Leibler divergence
from p to q. Viewing the Kullback–Leibler divergence as a measure of distance, the I-projection
is the "closest" distribution to q of all the distributions in P.
Information theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
, the information projection or I-projection of a probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....
q onto a set of distributions P is
where
is the Kullback–Leibler divergenceKullback–Leibler divergence
In probability theory and information theory, the Kullback–Leibler divergence is a non-symmetric measure of the difference between two probability distributions P and Q...
from p to q. Viewing the Kullback–Leibler divergence as a measure of distance, the I-projection
is the "closest" distribution to q of all the distributions in P.