# Hypothetical question on floating point normalization

I've seen a few questions posted regarding the upsides/downsides of floating point formats that have explicit integrals vs. formats that have implicit integrals, but have not seen an answer that really handles my question about it. By integral, I mean the single digit to the left of the radix point.

Let's assume that the hypothetical formats in question both have adequate precision for every possible use they will be put to (so that we can avoid the obvious point that the implicit integral adds more precision in most circumstances).

In one format we have an implicit integral and the requirement that all operation results must be normalized. In the other format we have an explicit integral and do not have the requirement that operation results be normalized. What are the upsides/downsides to this second format in comparison to the first format?

My first observations are that: 1) The format with the explicit integral will have multiple valid representations for a large subset of representable numbers. The format with the implicit integral will have only one valid representation for any given representable number. I don't immediately see a reason why this would be a downside, but I feel like it could be.

2) The format with the explicit integral seems like it would have a marginally simpler hardware or software implementation. This doesn't look like much of an upside unless you are going for really intense performance optimizations, but it ought to be mentioned.

3) The big upside for the explicit integral is that it makes a special arrangement for subnormal numbers unnecessary. Every number that would have fallen into the not-quite-zero gap would just be another representable result with the same gap as between any other pair of results for a given exponent. It seems like this would also slightly simplify design implementations, but I don't take that as a certainty.

Is there anything obvious I'm missing? Anything non-obvious? I'm just trying to really understand this concept better from a purely theoretical CS perspective.

The IEEE 754 32 bit and 64 bit floating point formats have an implicit leading mantissa bit, while the 80 bit format has an explicit leading mantissa bit. "Implicit" means that we determine from other information whether that bit is 1 or 0 (for denormalised numbers, the implicit leading bit is zero).

80 bit numbers where the explicit leading is 0 where it would have been an implicit 1 in 32 or 64 bit are called "unnormalised" numbers (not "denormalised"). There are two ways to handle them, and I think an implementation is free to use either way: Either the unnormalised number is first converted to a normalised or denormalised number, or there is no requirement or guarantee how the number is treated at all. (It would also be Ok to raise an interrupt when unnormalised numbers are encountered, so the behaviour would be well-defined but sloooooow). It depends on what the implementation says. In no case is an implementation allowed to produce an unnormalised number as the result of an operation not involving unnormalised numbers.

So each number has only one canonical representation, and in many implementations only one legal representation. In practice, unnormalised numbers are just ignored. They would only come up if you use uninitialised memory which is undefined behaviour anyway. You can't use bytewise comparisons, but you can't use them anyway because of +0 and -0.

• Ah, yes, "unnormalised". That's the word I forgot! Oh, and by the way, as of 2008 the term is now "subnormal", not "denormal". – Pseudonym Aug 19 '18 at 23:24
• A subtraction with two very small positive operands can produce a denormal result. Multiplication can also produce a denormal result from two normal operands. So what do you mean by "In no case is an implementation allowed to produce an unnormalised number as the result of an operation not involving unnormalised numbers. "? Non-binary FP (e.g., hexadecimal and decimal) introduce other considerations. – Paul A. Clayton Aug 20 '18 at 1:38

So first off, the implicit digit to the left of the radix point is not called an "integral". That's a very confusing use of that word. But I digress.

Having an unnormalised representation for floating-point numbers may simplify the final step of any basic floating point operation, because you don't need to re-normalise the result. But in turn, the initial steps of just about every floating point operation become more complex.

Let's take division as an example. Most modern floating-point hardware use Goldschmidt's algorithm, which is based on this idea:

Suppose that we want to calculate $\frac{n}{d}$, where $d$ is close to (but not equal to) one. Then we multiply both the numerator and the denominator by $2-d$.

Suppose $d = 1 + \epsilon$, then the new division problem is:

$$\frac{n}{1+\epsilon} = \frac{n(2-d)}{(1+\epsilon)(1-\epsilon)} = \frac{n(2-d)}{1-\epsilon^2}$$

This results in a denominator which is quadratically closer to 1. Repeat this operation until it gets close enough.

So Goldschmidt's algorithm works as follows. To calculate the quotient of two floating point numbers:

$$\frac{m_1 \times 2^{e_1}}{m_2 \times 2^{e_2}}$$

where $m_1$ and $m_2$ are between 1 and 2, we use a lookup table to find an approximation to $\frac{1}{m_2}$, which we will call $R$. Then we calculate:

$$n_1 = m_1 R$$ $$d_1 = m_2 R$$ $$n_{i+1} = n_i (2-d_i)$$ $$d_{i+1} = d_i (2-d_i)$$

and when we have done enough iterations (say $j$), we return:

$$n_j \times 2^{e_1 - e_2}$$

The final step will involve normalisation; $n_j$ will not, in general, be between 1 and 2. So if we could have unnormalised floating point numbers, the final step will be a bit simpler.

But in return, the table lookup operation will be more complex. With normalised numbers, you can just use the first few bits of the significand. ("Significand", incidentally, refers to the part of the mantissa that is actually stored in a normal floating-point number, without the implicit leading 1.)

So in that sense, there is no advantage to having an unnormalised representation for floating-point numbers. What you gain in a simpler final step you lose by having more complex initial steps.