# What does normalizing with hidden bit really mean?

I have a question related to representing numbers in base 2 with floating point. For example, if I have such a number $$0.000011 \cdot 2^3$$ then is its normalized form this? $$1.1\cdot 2^{-2}$$ Generally speaking about normalizing, normalizing with hidden bit, does it imply that the first number of mantissa should always be zero, or does hidden bit just mean that a "one" should be taken left to the point?

Generally speaking, normalized means "put in scientific notation." That just means, the mantissa should never start with 0, and should be less than the base. In binary that means the mantissa must be "1". Since the mantissa of a normalized binary floating point number is always 1, we don't need to store the 1. The first mantissa bit is hidden in the sense that it always exists, but we don't actually store the bit, because we know its value is 1.

So your normalized result ($1.1 \times 2^{-2}$) is correct, and it is correct because you've moved the first 1 to the left of the binary point.

• How is it that "In binary ... the mantissa must be '1' "? I didn't understand this ? Can you please explain ? Thank you. – Ivan Gandacov Mar 2 '15 at 19:36
• The mantissa must start with "1", because leading zeros don't matter. Well, there is still the problem with representing zero, as there is no way to normalize it. – TEMLIB Mar 2 '15 at 20:36
• JohnG: when you write a number in scientific notation in base 10 the first digit is something in the range $[1, 9]$. In base 8 the first digit would be something in the range $[1, 7]$. In general, the first digit must be in the range $[1, (b-1)]$, where $b$ is the base. But in binary $b = 2$, so the range is $[1, 1]$. That is: the first digit of the mantissa is always "1". (Except in the case of representing 0, Nan, Inf, and denormals, as @TEMLIB points out.) – Wandering Logic Mar 3 '15 at 0:05

(Disclaimer: Sorry this answer got a bit long, but I had the same problem and also didn't find an easy explanation on IEEE754 on the internet so I'm posting this. This is answering the question of the OP but also a bit more than that.)

I'll explain this with an example. If you consider these three numbers and their binary values:

• 0.06 = 0.000011110101110...
• 3.14 = 11.0010001111...
• 31.72 = 11111.101110...

as you can see, the position of the radix point(the dot) is different in each of them. (it's in the 2nd place in 0.0000111101, in 3rd place in 11.0010001111, and in the 6th place in 11111.101110) So how can we store the position of the radix point in hardware? Remember we only have 0s and 1s there.

You might propose for example, we can put a number of 0s instead of it, so when we see those 0s we know it's our radix point! But that wouldn't work obviously. because there's no way to tell whether those 0s are really part of the binary number or our radix point. You can't use 1s either, and you can't use any mixed pattern of 1s and 0s as well, just for the same reason. So what do we do?

A better answer is: We can agree on some position to put the radix point there, and then specify how much it has to move towards left or towards right to give us the correct number(from now on we'll call this movement "offset"). If we use this new way, the above numbers will become(for brevity, I've only written 10 bits after radix point, for each):

• 0.06 = 0.0000111101 & radix point is in place, no movement to left or right is needed.
• 3.14 = 1.1001000111 & radix point must move 1 place towards right
• 31.72 = 1.1111101110 & radix point must move 4 places towards right

So now, we can keep these in our hardware(for now let's say we only want offsets as big as 7 at maximum (which means 3 bits for offset). Only so that all three below shapes fit in one line):

offset                        offset                          offset
|                             |                               |
|    0.06                     |    3.14                       |    31.72
v                             v                               v
+-+--------------+            +-+--------------+              +-+--------------+
|000|0000111101  |            |001|1001000111  |              |100|1111101110  |
+----------------+            +----------------+              +----------------+


(Notice #1: 4 is 100 in binary!)

(Notice #2: That vertical line after offset doesn't exist in hardware obviously, It's just put there as a vision helper so that you don't get ocular cancer when trying to distinguish the offset bits from the rest.)

So far so good, by only having these in place, we have achieved our initial goal. However, as always, there's still room for improvements! Two of such improvements are explained below.

• improvement #1: How about negative numbers? Easily enough, we can just add a sign bit at the beginning of our hardware representations. Usually 0 for positives and 1 for negatives.

• improvement #2: In our hardware representation of the first number(0.00001111...), we have so many 0s. That's an unforgivable waste of bits(i.e treasure!). To solve this, we can agree to begin every hardware representation from the first bit that is 1 and then work out the offset, so first representation becomes this(again I've only written 10 of the bits after the radix point):

• 0.06 = 1.1110101110 & radix point must move 5 places towards left.

please notice that we have lost no precision because those 0s were "really" 0s and they were not produced for example out of rounding. So it's completely safe to do this. But now, there's also another benefit along with this new convention: We can even skip another bit 1 when storing any numbers(sometimes called the hidden bit, as the OP asked), because with this new convention we can be sure that all* of our hardware representations will begin with a bit 1! So now we can also see why when converting ieee754 representations to decimal, we have the "1.M" in the formula:$$(−1)^{S}∗2^{E−127}*1.M$$. Because we hid it when writing the rest of the bits in hardware but we shouldn't forget it when interpreting the hardware stuff back to decimal. Anyway, here are our representations so far:

 offset                        offset                          offset
|                             |                               |
Sign|        M                Sign|        M                  Sign|        M
|  |        |                 |  |        |                   |  |        |
|  |  0.06  |                 |  |  3.14  |                   |  |  31.72 |
v  v        v                 v  v        v                   v  v        v
++--+--------+-----+          ++--+--------+-----+            ++--+--------+-----+
|0|101|1110101110  |          |0|001|1001000111  |            |0|100|1111101110  |
+------------------+          +------------------+            +------------------+


So far the OPs question has been answered, you can skip reading the rest of the answer. But the above representations still have one small (or big if you will) flaw. The first offset (101) is towards left, but the other two offsets (001 and 100) are towards right! But currently, we can't say that by looking at our representations! We have devised no ways to keep this important piece of information, namely, the direction in which the radix point must be moved.

We could consider another bit for that, but instead, here we'll do something else(and so does ieee754 similarly). Basically, what we need is a way to keep both positive and negative offsets, to represent movement towards left and right respectively(a radix point in a binary number moves towards left if the number is multiplied by positive powers of 2 and towards right if the number is multiplied by negative powers of 2). Now to have both positive and negative offsets, you may choose any notation, say two's complements or etc. But here we'll use Excess-N notation(and so does ieee754). To do this in our own case, let's take these steps:

1. We'll use 4 bits instead of 3 offset bits and we'll call these 4 bits E.

2. We'll agree on an arbitrary number between 0 and 15. Say 12. This is our N in Excess-N. So we're using Excess-12 now.

3. We'll subtract 12 from any number we read in the E.

Now we can have both positive and negative offsets! For example, if you fill the E with number 15 (all 1s), it will mean +3 offset (15-12), or if you fill the E with number 8, it will mean -4 offset (8-12). (Also you can see that if we had chosen either 0 or 15 as N in the second step, we wouldn't be able to produce either negative or positive offsets respectively). But in our case 12 isn't a good choice for N too, because we can have an offset of -12, but we can't even have an offset of +4, so a better N would be 7 or 8 because they make both positive and negative offsets representable as much as possible(of course in some special cases, you might need more positive or more negative offsets based on the problem and then there's nothing stopping you from choosing 'N's other than 7 and 8. but in our case, we have no assumptions about the type of the problem, so we'll keep it as much generic as possible). Now, assuming N is 7, our final version of the representations will become like this:

    E                             E                               E
|                             |                               |
Sign|        M                Sign|        M                  Sign|        M
|  |        |                 |  |        |                   |  |        |
|  |  0.06  |                 |  |  3.14  |                   |  |  31.72 |
v  v        v                 v  v        v                   v  v        v
+---+--------+-----+          +---+--------+-----+            +---+--------+-----+
|0|1100|1110101110 |          |0|0110|1001000111 |            |0|0011|1111101110 |
+------------------+          +------------------+            +------------------+


and to convert these to their decimal equivalents, one only needs to use this formula: $$(−1)^{S}∗2^{E−7}*1.M$$

(With the float data type defined in ieee754 things are very similar(except for special values and some technicalities). They're only different in bit lengths. E is 8 bits instead of 4, and M is 23 bits instead of 10, plus 1 sign bit which make up 32 bits in total, and the formula becomes: $$(−1)^{S}∗2^{E−127}*1.M$$ ).

Hope that helps.

* Of course not exactly all of them in ieee754. There are special values in ieee754 like denormalized numbers that are not like this.

• Alas, the question is not about IEEE754. (It sadly gives no indication of the intended definition of normalized.) – greybeard Dec 6 '19 at 5:48

The "hidden bit" refers to concrete representations, where you represent the -2 exponent as a bit field in memory storage, and the .1[00000...] as another bit field. For example in IEEE 32-bit format, it is:

00111110110000000000000000000000
+^^^^^^^^_______________________


With the sign, exponent (in bias format), and the hidden-bit mantissa.