I have a floating point number system and I have a number for which I need to calculate the exact relative error after rounding. The number is clearly an overflow. Does anyone know what I should do?

I've tried converting the number to from the decimal base to the base of the system, then just chopping of and rounding the excess and keeping the exponent within the bounds of the system.

For example, say I had .a0a1a2a3a4... and the exponent was say b4; I chopped off at a3 and rounded so now I have .a0a1a2a3 and I just changed the exponent to the bound which is b1

So, my new number is .a0a1a2a3 * b1 my answer need to be in base 10 so I take this number go back to decimal and calculate the exact relative error. I got something like 0.99... Is this what I'm suppose to do or is there something else to be done when it overflows?

  • $\begingroup$ If your system works in base $b$ (2, I presume) your computations should be in base $b$ as far as possible to avoid rounding errors. It will probably also turn out easier. $\endgroup$ – vonbrand Oct 9 '15 at 19:58
  • $\begingroup$ so was i correct in turn the exponent from 4 to 1? since in my system the exponent is bound by 1 and -1 $\endgroup$ – user1804234 Oct 9 '15 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.