What are the reasons for and against using normalized floating point?

Question

Floating point number formats can be normalized or not, meaning that ‘normal’ floating point numbers have an implicit (hidden) leading bit 1 in the significand. For example, the binary IEEE 754 formats are normalized, but the decimal IEEE 754 formats are not, i.e., they have an explicit leading bit (or number).

The advantage I see is the larger range or precision possible. (Larger range if compared to a format with one less exponent bit; greater precision if compared to a format with one less significand bit.)

However, the downsides I see are:

1. Dealing with a more complex format during computations, e.g., normalization steps, the existence of subnormals.

2. The loss of significance information. With, e.g., decimal IEEE 754 formats, the number of significand digits can be expressed. This would be a very useful feature, e.g., when encoding measurements. I would think that such information could also be exploited during computations.

On the whole, I feel that the larger range or greater precision that can be got with one extra bit does not justify the loss of a qualitative aspect, significance information. So my question is:

What are (other) reasons for or against using a normalized floating point format?

Perhaps the reason is historical and now the choice would be different; also this information is of interest to me. Perhaps the arguments I gave are not valid; it would then be interesting to know why.

Answer 1

Your terminology is non-standard. There are floating-point formats with an implicit leading mantissa bit, and floating-point formats with an explicit leading mantissa bit. I'm not aware that this property has a name, but if you say that the leading mantissa bit is implicit or explicit, then everyone knows what you are talking about.

Denormalized numbers, in standard use of the word, are floating-point numbers where the leading mantissa bit is not set, and the exponent has the smallest possible value. Floating-point numbers with explicit leading mantissa bit can have the leading mantissa bit set to zero even though the exponent does not have the smallest possible value; such numbers are called unnormalized (not denormalised).

Your question isn't about denormalized numbers in the standard use of the word, but numbers with implicit leading mantissa bit.

The extra bit made available by having an implicit leading mantissa bit is very valuable. It turns into an additional 0.3 digits of precision, which is useful for double precision, and very useful for single precision. You would need a very, very good reason to throw that away.

Implicit mantissa bit is not more difficult to handle. All that is needed is adding the mantissa bit if the exponent is not the smallest possible biased value, which is 0. That's simple. An explicit leading mantissa bit requires handling of unnormalized numbers, which is a pain.

Storing significance information: You could do that with unnormalized numbers. This has two problems: One, nobody cares about significance information. Two, try defining what the result of common operations should be. Especially considering that most people will just want the maximum possible precision. Enjoy.

It is reasonably simple to keep track of and estimating rounding errors, if that's what you're after. This will be much simpler than using unnormalized numbers for that purpose.

Answer 2

Binary IEEE format include both normal and denormal ('subnormal') numbers. The purpose of subnormals is to avoid a gap between the smallest normal numbers and zero larger than the smallest difference between consecutive normal numbers.

This gap make generate instability in some calculations dealing with small numbers,... Denormal have a performance and/or complexity cost in software and hardware. Sometimes denormals are not handled or a fast mode is provided where all denormals are considered as zero.

(Here is an historical perspective from William Kahan, one of the original creators of the P754 format: https://people.eecs.berkeley.edu/~wkahan/ieee754status/754story.html)