# Addition errors in IEEE754 floating point representation

So in class, we were talking about the idea of floating point precision in IEEE754 format, and how, when some numbers are added, precision is lost. My professor then gave the following example of a large absolute error between what we think the answer might be, and what the answer actually is when our program (written in Java) actually compiles it:

$$0.4 \times10^{30} + 1.1 \times10^{30}$$

Compiling this in Java, i get $$1.5000001 \times 10^{30}$$

My question is, why does this happen? Printing out the IEEE754 format of the two numbers, I get:

$$\space\space\space0[11100010]10111100010010010101011=+1.10111100010010010101011\times2^{99}$$ $$+0[11100001]01000011000111100001000=+1.01000011000111100001000\times2^{98}$$

So, I suppose when you add these, you end up loosing precision, but exactly how? Moreover, how can I even tell, before adding, that the addition of two numbers will make them loose precision? Or is the only way to just plus into a program and see what happens? None of this was really covered in class unfortunately.

• Have you noticed that even before the addition, precision has been lost? – John L. Dec 19 '18 at 6:01
• How so? When I print the number in Java, it shows it precisely as $1.1E30$ and $0.4E30$. – BeepBoop Dec 19 '18 at 6:11
• $1.1=(1.0001100110011001100110011001100110011001\cdots)_2$. That is, 1.1 is an infinite fraction in base 2. – John L. Dec 19 '18 at 6:24
• When you ask Java to print, it prints $1.1E30$, trying to be intelligent anticipating that you want to see something simple. However, that is not the number that is actually stored in the memory. – John L. Dec 19 '18 at 6:26
• In Java, all floating point numbers are small integers (53 bit), multiplied by some power of two. 0.4 x 10^30 and 1,1 x 10^30 are not such numbers. – gnasher729 Dec 19 '18 at 9:12

Printing out the IEEE754 format of the two numbers, I get:

$$\space\space\space0[11100010]10111100010010010101011=+1.10111100010010010101011\times2^{99}$$ $$+0[11100001]01000011000111100001000=+1.01000011000111100001000\times2^{98}$$

Compare the exact values:

$$1.1011110001001001010101011000111111010110011111001110000010110110100111 \times 2^{99} \\ 1.0100001100011110000011111010111001101101011100100001011111001010101 \times 2^{98}$$

Observe that both of the numbers are already rounded to represent them in IEEE 754, and it happens that both of them have rounded up.

Also note that since the values have different exponents, they've been rounded at different places.

  1.10111100010010010101011 ×299
+ 0.101000011000111100001000×299
-------------------------------
10.01011101110110000101111 x299
11  1         1111


There's overflow, so the answer will have exponent $$2^{100}$$ and we have to round. Again, we're rounding up, to $$1.00101110111011000011000\times2^{100}$$.

So, I suppose when you add these, you end up loosing precision, but exactly how?

You will nearly always lose precision when adding two floating point numbers. If they're both positive and have the same exponent, the output will have a larger exponent and you lose some bits from the end. If they have different exponents, the smaller one will lose bits from the end. And if they have opposite signs and the same exponent, the output has a smaller exponent and you lose precision because you need bits beyond the end which were lost earlier. Essentially the only cases where you don't lose precision are when you add a number to itself, or when the input numbers and the result can all be represented exactly in the width of the type.

You were using single precision floating point numbers. Single precision floating point numbers look like this: You start with a mantissa, which is a number 1 ≤ m < 2 which is a multiple of $$2^{-23}$$, then you multiply by a power of 2, then you multiply by +1 or -1.

So when you write 0.4e30, that is not a possible floating-point number at all. Instead, you get the floating-point number closest to 0.4e30. So we have an error. And we have another error in 1.1e30. What you are adding is really the floating-point number closest to 0.4e30, plus the floating-point number closest to 1.1e30.

Now 0.4e30 has a smaller exponent than 1.1e30. If you add the two numbers, then the two last bits of 0.4e30 must be lost. So you add not quite the numbers you think you are adding, and you get an error during the rounding. If by coincidence the last two bits of 0.4e30 were both zeroes, then there's no rounding error.

There are some cases where we know there is no rounding error: If you add x+y, and x, y, and x+y are all integers up to 2^24, then you can prove that you will get the correct result with zero rounding error. If you subtract x-y where x and y are positive numbers and x/2 ≤ y ≤ 2x, then you can also prove that there is no rounding error. If you add a zero then there is no rounding error. But usually you just assume there will be a rounding error, and it will be up to one half of the value of the last mantissa bit of the result.