I have asked this question on Stack Overflow, I am asking it here in the hope to get more traction.

The relative rounding error for a floating point number x is defined as

$e_r = |\frac{(round(x) - x)}{x}| = |\frac{round(x)}{x} - 1|$ (1)

Assuming that the rounding to nearest mode is used for $round(x)$, the absolute rounding error $|round(x) - x|$ is going to be less than 0.5 ulp(E(x)), where the ulp are units in the last place

$ulp(E) = 2^E \cdot \epsilon$

and $E(x)$ is the exponent used for $x$, and $\epsilon$ is the machine epsilon $\epsilon=2^{-(p-1)}$, $p$ is precision (24 for the single precision and 53 for the double precision IEEE formats).

Using this, the relative error can be expressed for any real number $x$

$e_r = |\frac{(round(x) - x)}{x}| = \frac{|(round(x) - x)|}{|x|} < |0.5 \cdot 2^E \cdot 2^{-(p-1)}| / |2^E| < 0.5 \epsilon$

For denormalized numbers $0 < x < 2^Em \epsilon$, where Em is the minimal exponent (-126 for single precision, -1022 for double):

$0 < x \le 0.5 \cdot \epsilon \cdot 2^{Em}$

the rounding always goes to $0$!

If the round(x) is 0, then by (1)

$e_r =|\frac{(0 - 1)}{1}| = |1|$ !

How is the relative error computed for such numbers? Should the relative error be even used for the numbers that are rounded to 0?


If x = 0 and is rounded to 0 then the error is 0, and the relative error is whatever you want it to be.

If x ≠ 0 and is rounded to 0, then the absolute error is x, and the relative error is 1 (or 100% if that is more intuitive).

If x = 0 and is round to y ≠ 0 then the absolute error is y, and the relative error is infinite or undefined, whatever you prefer.

  • $\begingroup$ '0' is represented exactly by the IEEE floating point standard, so no rounding will take place. For $x\ne0$, if the relative error is really $1$ when $x<$ of the minimal denormalized number, then it is a special case that has to be handled in my program. $\endgroup$ – tmaric Oct 23 '18 at 16:44
  • $\begingroup$ @tmaric That's for a single operation. A sequence of operations can have an exact result of zero and a rounded result != 0. $\endgroup$ – gnasher729 Sep 13 '20 at 8:54

Why answer would be preferable to 1?

Let's try abstracting a little: Say you have any small value $c$ for which 0 is the closest value that can be reliably represented. If rounding occurs, how much error occurs as a fraction of $c$? All of it. 100% of $c$. So the rounding error is 100% = 1.0.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.