# What does normalizing with hidden bit really mean?

I have a question related to representing numbers in base 2 with floating point. For example, if I have such a number $$0.000011 \cdot 2^3$$ then is its normalized form this? $$1.1\cdot 2^{-2}$$ Generally speaking about normalizing, normalizing with hidden bit, does it imply that the first number of mantissa should always be zero, or does hidden bit just mean that a "one" should be taken left to the point?

Generally speaking, normalized means "put in scientific notation." That just means, the mantissa should never start with 0, and should be less than the base. In binary that means the mantissa must be "1". Since the mantissa of a normalized binary floating point number is always 1, we don't need to store the 1. The first mantissa bit is hidden in the sense that it always exists, but we don't actually store the bit, because we know its value is 1.

So your normalized result ($1.1 \times 2^{-2}$) is correct, and it is correct because you've moved the first 1 to the left of the binary point.

• How is it that "In binary ... the mantissa must be '1' "? I didn't understand this ? Can you please explain ? Thank you. Commented Mar 2, 2015 at 19:36
• The mantissa must start with "1", because leading zeros don't matter. Well, there is still the problem with representing zero, as there is no way to normalize it. Commented Mar 2, 2015 at 20:36
• JohnG: when you write a number in scientific notation in base 10 the first digit is something in the range $[1, 9]$. In base 8 the first digit would be something in the range $[1, 7]$. In general, the first digit must be in the range $[1, (b-1)]$, where $b$ is the base. But in binary $b = 2$, so the range is $[1, 1]$. That is: the first digit of the mantissa is always "1". (Except in the case of representing 0, Nan, Inf, and denormals, as @TEMLIB points out.) Commented Mar 3, 2015 at 0:05

Shorter Answer (As requested by greybird in the comments) :

The gist of "normalizing" in scientific notation, is not to allow the same number to have multiple corresponding codes, i.e., we want each of the codes to be uniquely corresponding to a single number. This is not the case with floating point notation.

Both notations are in the form $$X * 10 ^ Y$$($$X$$ is called the significand or the mantissa, $$Y$$ is called the exponent), but with the difference that in the scientific form, the $$X$$ will be normalized (explained how in the example below), i.e., every number will be represented as a single code, but in the floating point notation every number will have multiple representations(so basically, the scientific notation is a special variant of the floating point notation).

Example: in base 10 scientific notation encoding which is in the form $$X * 10 ^ Y$$, if we do not agree on beginning all $$X$$s with digits in the range [1, 9], then every number will have many scientific notation codes corresponding to it, for instance, $$7.53$$ could be encodable as all these codes: $$75.3*10^{-1}, \ \ \ \ 7.53*10^0, \ \ \ \ 0.753*10^1, \ \ \ \ 0.0753*10^2, \ \ \ \ ... \ \ .$$

The same story is true about base 2 scientific notation, we do not want multiple codes for the same number, e.g, $$(0.011 * 10^{10})_{2}, \ \ \ \ (0.110 * 10^{01})_{2}, \ \ \ \ (1.10 * 10^{00})_{2}, \ \ \ \, ... \ \ .$$ So we have to make an agreement about the $$X$$, the same way we did above(in base 10). Now, we could agree on anything, for instance, beginning all $$X$$s with 00.01, or with 0.001, or etc. but these are obvious wastes of bits. So a better agreement is to begin all $$X$$s with "0.", because this way we won't be storing any insignificant bits. But you know what? There's still a 0 there... we don't like to consider it significant as well, is there still a better way? Well there is. We could begin the numbers with "1."(This is IEEE754, i.e., no more the scientific notation in its usual form, rather a variant of it). But, you might ask, how about representing the number 0.0? If all $$X$$s begin with 1., then normally the number 0.0 will not be representable. That's a very good question, but let me first explain what's meant by the "hidden bit 1" and after that, we'll get back to this question.

Let's say we have agreed on beginning all $$X$$s with "1.", then why storing this "1." too in memory at all?! We know it must be considered when decoding every code to a number, so we don't need to store it explicitly in memory at all. The benefit is obvious, we can add another bit to the fractional part of the $$X$$(after the dot) and thus make the number range both bigger and more precise (bigger for positive exponents, more precise for negative exponents).

Now back to the question on representing the number 0.0 in IEEE754. We can simply agree on some special bit pattern to represent it. When all exponent bits and all mantissa bits are 0 we will interpret the number differently, i.e, we'll consider it as $$^{+}_{-}0.0$$ (instead of $$^{+}_{-}1.0 * 2 ^ 0$$) (And... Yes, there are two codes for zero in IEEE754). There are also other special values for representing these symbols:

• +Infinity: {sign bit 0, all exponent bits 1, all mantissa bits 0}
• -Infinity: {sign bit 1, all exponent bits 1, all mantissa bits 0}
• NaN(Not a Number, e.g., $$\frac{1}{0}$$ and etc.): {all bits 1}

Also there are denormals (because they begin with a "0." instead of "1.") that fill the ugly gap around 0 (i.e. smaller absolute value than the smallest possible normal).

• denormals: {all exponent bits 0, at least one mantissa bit non-zero}.

Longer Answer (If you want to learn the IEEE754 more slowly) :

I'll explain this with an example. If you consider these three numbers and their binary values:

• 0.06 = 0.000011110101110...
• 3.14 = 11.0010001111...
• 31.72 = 11111.101110...

as you can see, the position of the radix point(the dot) is different in each of them. (it's in the 2nd place in 0.0000111101, in 3rd place in 11.0010001111, and in the 6th place in 11111.101110) So how can we store the position of the radix point in hardware? Remember we only have 0s and 1s there.

You might propose for example, we can put a number of 0s instead of it, so when we see those 0s we know it's our radix point! But that wouldn't work obviously. because there's no way to tell whether those 0s are really part of the binary number or our radix point. You can't use 1s either, and you can't use any mixed pattern of 1s and 0s as well, just for the same reason. So what do we do?

A better answer is: We can agree on some position to put the radix point there, and then specify how much it has to move towards left or towards right to give us the correct number(from now on we'll call this movement "offset"). If we use this new way, the above numbers will become(for brevity, I've only written 10 bits after radix point, for each):

• 0.06 = 0.0000111101 & radix point is in place, no movement to left or right is needed.
• 3.14 = 1.1001000111 & radix point must move 1 place towards right
• 31.72 = 1.1111101110 & radix point must move 4 places towards right

So now, we can keep these in our hardware(for now let's say we only want offsets as big as 7 at maximum (which means 3 bits for offset). Only so that all three below shapes fit in one line):

offset                        offset                          offset
|                             |                               |
|    0.06                     |    3.14                       |    31.72
v                             v                               v
+-+--------------+            +-+--------------+              +-+--------------+
|000|0000111101  |            |001|1001000111  |              |100|1111101110  |
+----------------+            +----------------+              +----------------+


(Notice #1: 4 is 100 in binary!)

(Notice #2: That vertical line after offset doesn't exist in hardware obviously, It's just put there as a vision helper so that you don't get ocular cancer when trying to distinguish the offset bits from the rest.)

So far so good, by only having these in place, we have achieved our initial goal. However, as always, there's still room for improvements! Two of such improvements are explained below.

• improvement #1: How about negative numbers? Easily enough, we can just add a sign bit at the beginning of our hardware representations. Usually 0 for positives and 1 for negatives.

• improvement #2: In our hardware representation of the first number(0.00001111...), we have so many 0s. That's an unforgivable waste of bits(i.e treasure!). To solve this, we can agree to begin every hardware representation from the first bit that is 1 and then work out the offset, so first representation becomes this(again I've only written 10 of the bits after the radix point):

• 0.06 = 1.1110101110 & radix point must move 5 places towards left.

please notice that we have lost no precision because those 0s were "really" 0s and they were not produced for example out of rounding. So it's completely safe to do this. But now, there's also another benefit along with this new convention: We can even skip another bit 1 when storing any numbers(sometimes called the hidden bit, as the OP asked), because with this new convention we can be sure that all* of our hardware representations will begin with a bit 1! So now we can also see why when converting ieee754 representations to decimal, we have the "1.M" in the formula:$$(−1)^{S}∗2^{E−127}*1.M$$. Because we hid it when writing the rest of the bits in hardware but we shouldn't forget it when interpreting the hardware stuff back to decimal. Anyway, here are our representations so far:

 offset                        offset                          offset
|                             |                               |
Sign|        M                Sign|        M                  Sign|        M
|  |        |                 |  |        |                   |  |        |
|  |  0.06  |                 |  |  3.14  |                   |  |  31.72 |
v  v        v                 v  v        v                   v  v        v
++--+--------+-----+          ++--+--------+-----+            ++--+--------+-----+
|0|101|1110101110  |          |0|001|1001000111  |            |0|100|1111101110  |
+------------------+          +------------------+            +------------------+


So far the OPs question has been answered, you can skip reading the rest of the answer. But the above representations still have one small (or big if you will) flaw. The first offset (101) is towards left, but the other two offsets (001 and 100) are towards right! But currently, we can't say that by looking at our representations! We have devised no ways to keep this important piece of information, namely, the direction in which the radix point must be moved.

We could consider another bit for that, but instead, here we'll do something else(and so does ieee754 similarly). Basically, what we need is a way to keep both positive and negative offsets, to represent movement towards left and right respectively(a radix point in a binary number moves towards left if the number is multiplied by positive powers of 2 and towards right if the number is multiplied by negative powers of 2). Now to have both positive and negative offsets, you may choose any notation, say two's complements or etc. But here we'll use Excess-N notation(and so does ieee754). To do this in our own case, let's take these steps:

1. We'll use 4 bits instead of 3 offset bits and we'll call these 4 bits E.

2. We'll agree on an arbitrary number between 0 and 15. Say 12. This is our N in Excess-N. So we're using Excess-12 now.

3. We'll subtract 12 from any number we read in the E.

Now we can have both positive and negative offsets! For example, if you fill the E with number 15 (all 1s), it will mean +3 offset (15-12), or if you fill the E with number 8, it will mean -4 offset (8-12). (Also you can see that if we had chosen either 0 or 15 as N in the second step, we wouldn't be able to produce either negative or positive offsets respectively). But in our case 12 isn't a good choice for N too, because we can have an offset of -12, but we can't even have an offset of +4, so a better N would be 7 or 8 because they make both positive and negative offsets representable as much as possible(of course in some special cases, you might need more positive or more negative offsets based on the problem and then there's nothing stopping you from choosing 'N's other than 7 and 8. but in our case, we have no assumptions about the type of the problem, so we'll keep it as much generic as possible). Now, assuming N is 7, our final version of the representations will become like this:

    E                             E                               E
|                             |                               |
Sign|        M                Sign|        M                  Sign|        M
|  |        |                 |  |        |                   |  |        |
|  |  0.06  |                 |  |  3.14  |                   |  |  31.72 |
v  v        v                 v  v        v                   v  v        v
+---+--------+-----+          +---+--------+-----+            +---+--------+-----+
|0|1100|1110101110 |          |0|0110|1001000111 |            |0|0011|1111101110 |
+------------------+          +------------------+            +------------------+


and to convert these to their decimal equivalents, one only needs to use this formula: $$(−1)^{S}∗2^{E−7}*1.M$$

(With the float data type defined in ieee754 things are very similar(except for special values, denormals, roundings and some other technicalities). They're only different in bit lengths. E is 8 bits instead of 4, and M is 23 bits instead of 10, plus 1 sign bit which make up 32 bits in total, and the formula becomes: $$(−1)^{S}∗2^{E−127}*1.M$$ ).

Hope that helps.

* Of course not exactly all of them in ieee754. There are special values like denormalized numbers as mentioned in the shorter answer.

• Alas, the question is not about IEEE754. (It sadly gives no indication of the intended definition of normalized.) Commented Dec 6, 2019 at 5:48
• i just signed up on this sub, to thank you. as you explained from where we got that 1. i have a request: please explain how LSB/MSB (Least/Most) Significant bit system works. Thanks in advance. i get some value in Hexadecimal as, 4800 to convert it to decimal, i have to write it down as "00" "48" -> 0048 and then convert it into decimal -> 72 Commented Jan 5, 2021 at 21:09
• @greybeard Thanks I've added a more explicit answer to the OP's question in an edit. Commented Feb 13, 2021 at 9:45
• @Usman you're welcome, I'm happy you've joined the network, please ask your question in a separate post(that's a rule) if you can't find it, and usually the community will answer it very soon. Commented Feb 13, 2021 at 9:47

The "hidden bit" refers to concrete representations, where you represent the -2 exponent as a bit field in memory storage, and the .1[00000...] as another bit field. For example in IEEE 32-bit format, it is:

00111110110000000000000000000000
+^^^^^^^^_______________________


With the sign, exponent (in bias format), and the hidden-bit mantissa.