Is it possible to have a rounding error when you convert a floating point number which can only be in increments of 0.01 to an integer by multiplying by 100 first? I would think that the lack of precision in representing floating point numbers is so small that it would never cause a rounding error when you only have to worry about 0.01 increments.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It depends on :
- the range of your input numbers
- the FP type, typically IEEE single (32bits, 23bits precision) our double (64bits, 52bits precision)
- The rounding mode, which is almost always "round to nearest even"
0.01 ≈ 7bits, add 3 bits for correct rounding, so around 13 bits remains with single precision, which give a range around ≈ ±10000.
[Well, I'm too lazy to give precise answers. The problem with FP is that every operation can slightly affect the precision, as well as the way the program is compiled and the target architecture, with or without FMA (fused multiply-add), for example.]
