# How to compute relative error for the rounding of floating point numbers when the rounded number is 0?

I have asked this question on Stack Overflow, I am asking it here in the hope to get more traction.

The relative rounding error for a floating point number x is defined as

$$e_r = |\frac{(round(x) - x)}{x}| = |\frac{round(x)}{x} - 1|$$ (1)

Assuming that the rounding to nearest mode is used for $$round(x)$$, the absolute rounding error $$|round(x) - x|$$ is going to be less than 0.5 ulp(E(x)), where the ulp are units in the last place

$$ulp(E) = 2^E \cdot \epsilon$$

and $$E(x)$$ is the exponent used for $$x$$, and $$\epsilon$$ is the machine epsilon $$\epsilon=2^{-(p-1)}$$, $$p$$ is precision (24 for the single precision and 53 for the double precision IEEE formats).

Using this, the relative error can be expressed for any real number $$x$$

$$e_r = |\frac{(round(x) - x)}{x}| = \frac{|(round(x) - x)|}{|x|} < |0.5 \cdot 2^E \cdot 2^{-(p-1)}| / |2^E| < 0.5 \epsilon$$

For denormalized numbers $$0 < x < 2^Em \epsilon$$, where Em is the minimal exponent (-126 for single precision, -1022 for double):

$$0 < x \le 0.5 \cdot \epsilon \cdot 2^{Em}$$

the rounding always goes to $$0$$!

If the round(x) is 0, then by (1)

$$e_r =|\frac{(0 - 1)}{1}| = |1|$$ !

How is the relative error computed for such numbers? Should the relative error be even used for the numbers that are rounded to 0?

If x = 0 and is rounded to 0 then the error is 0, and the relative error is whatever you want it to be.

If x ≠ 0 and is rounded to 0, then the absolute error is x, and the relative error is 1 (or 100% if that is more intuitive).

If x = 0 and is round to y ≠ 0 then the absolute error is y, and the relative error is infinite or undefined, whatever you prefer.

• '0' is represented exactly by the IEEE floating point standard, so no rounding will take place. For $x\ne0$, if the relative error is really $1$ when $x<$ of the minimal denormalized number, then it is a special case that has to be handled in my program. – tmaric Oct 23 '18 at 16:44
• @tmaric That's for a single operation. A sequence of operations can have an exact result of zero and a rounded result != 0. – gnasher729 Sep 13 '20 at 8:54

Why answer would be preferable to 1?

Let's try abstracting a little: Say you have any small value $$c$$ for which 0 is the closest value that can be reliably represented. If rounding occurs, how much error occurs as a fraction of $$c$$? All of it. 100% of $$c$$. So the rounding error is 100% = 1.0.