Denormal number
Encyclopedia
In computer science
, denormal numbers or denormalized numbers (now often called subnormal numbers) fill the underflow
gap around zero
in floating point
arithmetic: any non-zero number which is smaller than the smallest normal number
is 'sub-normal'.
For example, if the smallest positive 'normal' number is 1×β−n (where β is the base of the floating-point system, usually 2 or 10), then any smaller positive numbers that can be represented are denormal.
The significand
(or mantissa) of an IEEE number is the part of a floating point
number that represents the significant digits. For a positive normalised number it can be represented as m0.m1m2m3...mp-2mp-1 (where m represents a significant digit and p is the precision, and m0 is non-zero). Notice that for a binary radix, the leading binary digit is one. In a denormal number, since the exponent is the least that it can be, zero is the lead significand digit (0.m1m2m3...mp-2mp-1) in order to represent numbers closer to zero than the smallest normal number.
By filling the underflow gap like this, significant digits are lost, but not to the extent as when doing flush to zero on underflow (losing all significant digits all through the underflow gap). Hence the production of a denormal number is sometimes called gradual underflow because it allows a calculation to lose precision slowly when the result is small.
In IEEE 754-2008, denormal numbers are renamed subnormal numbers, and are supported in both binary and decimal formats. In binary interchange formats, subnormal numbers are encoded with a biased exponent
of 0, but are interpreted with the value of the smallest allowed exponent, which is one greater (i.e., as if it were encoded as a 1). In decimal interchange formats they require no special encoding because the format supports unnormalized numbers directly.
errors that cannot occur when gradual underflow is used.
Denormal numbers were implemented in the Intel 8087
while the IEEE 754 standard was being written. They were by far the most controversial feature in the K-C-S format proposal that was eventually adopted, but this implementation demonstrated that denormals could be supported in a practical implementation. Some implementations of floating point unit
s do not directly support denormal numbers in hardware, but rather trap to some kind of software support. While this may be transparent to the user, it can result in calculations which produce or consume denormal numbers being much slower than similar calculations on normal numbers.
Some applications need to contain code to avoid denormal numbers, either to maintain accuracy, or in order to avoid the performance penalty in some processors. For instance, in audio processing applications, denormal values usually represent a signal
so quiet that it's out of the human hearing range. Because of this, a common measure to avoid denormals on processors where there would be a performance penalty is to cut the signal to zero once it reaches denormal levels or mix in an extremely quiet noise signal. Since the SSE2 processor extension, Intel has provided such a functionality in CPU hardware, which rounds denormalized numbers to zero.
's web site http://www.cs.berkeley.edu/~wkahan/ for examples of where denormal numbers help improve the results of calculations.
Computer science
Computer science or computing science is the study of the theoretical foundations of information and computation and of practical techniques for their implementation and application in computer systems...
, denormal numbers or denormalized numbers (now often called subnormal numbers) fill the underflow
Arithmetic underflow
The term arithmetic underflow is a condition in a computer program that can occur when the true result of afloating point operation is smaller in magnitude...
gap around zero
0 (number)
0 is both a numberand the numerical digit used to represent that number in numerals.It fulfills a central role in mathematics as the additive identity of the integers, real numbers, and many other algebraic structures. As a digit, 0 is used as a placeholder in place value systems...
in floating point
Floating point
In computing, floating point describes a method of representing real numbers in a way that can support a wide range of values. Numbers are, in general, represented approximately to a fixed number of significant digits and scaled using an exponent. The base for the scaling is normally 2, 10 or 16...
arithmetic: any non-zero number which is smaller than the smallest normal number
Normal number (computing)
In computing, a normal number is a non-zero number in a floating-point representation which is within the balanced range supported by a given floating-point format....
is 'sub-normal'.
For example, if the smallest positive 'normal' number is 1×β−n (where β is the base of the floating-point system, usually 2 or 10), then any smaller positive numbers that can be represented are denormal.
The significand
Significand
The significand is part of a floating-point number, consisting of its significant digits. Depending on the interpretation of the exponent, the significand may represent an integer or a fraction.-Examples:...
(or mantissa) of an IEEE number is the part of a floating point
Floating point
In computing, floating point describes a method of representing real numbers in a way that can support a wide range of values. Numbers are, in general, represented approximately to a fixed number of significant digits and scaled using an exponent. The base for the scaling is normally 2, 10 or 16...
number that represents the significant digits. For a positive normalised number it can be represented as m0.m1m2m3...mp-2mp-1 (where m represents a significant digit and p is the precision, and m0 is non-zero). Notice that for a binary radix, the leading binary digit is one. In a denormal number, since the exponent is the least that it can be, zero is the lead significand digit (0.m1m2m3...mp-2mp-1) in order to represent numbers closer to zero than the smallest normal number.
By filling the underflow gap like this, significant digits are lost, but not to the extent as when doing flush to zero on underflow (losing all significant digits all through the underflow gap). Hence the production of a denormal number is sometimes called gradual underflow because it allows a calculation to lose precision slowly when the result is small.
In IEEE 754-2008, denormal numbers are renamed subnormal numbers, and are supported in both binary and decimal formats. In binary interchange formats, subnormal numbers are encoded with a biased exponent
Exponent bias
In IEEE 754 floating point numbers, the exponent is biased in the engineering sense of the word – the value stored is offset from the actual value by the exponent bias....
of 0, but are interpreted with the value of the smallest allowed exponent, which is one greater (i.e., as if it were encoded as a 1). In decimal interchange formats they require no special encoding because the format supports unnormalized numbers directly.
Background
Denormal numbers provide the guarantee that addition and subtraction of floating-point numbers never underflows; two nearby floating-point numbers always have a representable non-zero difference. Without gradual underflow, the subtraction a−b can underflow and produce zero even though the values are not equal. This can, in turn, lead to division by zeroDivision by zero
In mathematics, division by zero is division where the divisor is zero. Such a division can be formally expressed as a / 0 where a is the dividend . Whether this expression can be assigned a well-defined value depends upon the mathematical setting...
errors that cannot occur when gradual underflow is used.
Denormal numbers were implemented in the Intel 8087
Intel 8087
The Intel 8087, announced in 1980, was the first floating-point coprocessor for the 8086 line of microprocessors. It had 45,000 transistors and was manufactured as a 3 μm depletion load HMOS circuit. The 8087 was built to be paired with the Intel 8088 or 8086 microprocessors...
while the IEEE 754 standard was being written. They were by far the most controversial feature in the K-C-S format proposal that was eventually adopted, but this implementation demonstrated that denormals could be supported in a practical implementation. Some implementations of floating point unit
Floating point unit
A floating-point unit is a part of a computer system specially designed to carry out operations on floating point numbers. Typical operations are addition, subtraction, multiplication, division, and square root...
s do not directly support denormal numbers in hardware, but rather trap to some kind of software support. While this may be transparent to the user, it can result in calculations which produce or consume denormal numbers being much slower than similar calculations on normal numbers.
Performance issues
Some systems handle denormal values in hardware, in the same way as normal values. Others leave the handling of denormal values to system software, only handling normal values and zero in hardware. Handling denormal values in software always leads to a significant decrease in performance. But even when denormal values are entirely computed in hardware, the speed of computation is significantly reduced on most modern processors; in extreme cases, instructions involving denormal operands may run as much as 100 times slower.Some applications need to contain code to avoid denormal numbers, either to maintain accuracy, or in order to avoid the performance penalty in some processors. For instance, in audio processing applications, denormal values usually represent a signal
so quiet that it's out of the human hearing range. Because of this, a common measure to avoid denormals on processors where there would be a performance penalty is to cut the signal to zero once it reaches denormal levels or mix in an extremely quiet noise signal. Since the SSE2 processor extension, Intel has provided such a functionality in CPU hardware, which rounds denormalized numbers to zero.
Further reading
See also various papers on William KahanWilliam Kahan
William Morton Kahan is a mathematician and computer scientist who received the Turing Award in 1989 for "his fundamental contributions to numerical analysis", and was named an ACM Fellow in 1994....
's web site http://www.cs.berkeley.edu/~wkahan/ for examples of where denormal numbers help improve the results of calculations.