Float type variable uncertainty

Question

I developed an image processing software and I need to do a numerical analysis of it, considering the error propagation associated to its operations and the uncertainty of float type variables caused by the inherent rounding up that happens with this type of variables.

Considering the IEEE 754 standard the machine epsilon for the float type variables is 1.19e-07. From what I understood, this value is the distance to the nearest representable float.

I did some testing to find if this is true by adding a float value to this epsilon as such: x + epsilon == x. This notion does not hold for every value of the float range, which is understandable since great values of floats have more uncertainty associated with them caused by the rounding and the limited number of bits used to represent them.

My question is what is the uncertainty associated to a float value in such a way that (x + y) || (x - y) == x being the float value x and the float uncertainty y.

It might be my lack of knowledge about the english language but I can not seem to understand the literature about this topic.

If someone could be as detailed as possible can you explain me the error in a simple operation such as the following?

float result = valA * 0.587f + valB * 0.331f;

If I knew the uncertainty of a float type variable this error could be simply calculated with this formulas, right?

Answer 1

The number 1.19e-07 is the difference from 1.0 to the next larger representable floating point number. Single precision floating numbers have the form $x = ± 2^k m$, where k is a small integer, and 1 ≤ m < 2 is a multiple of $2^{-23}$.

If you calculate a result x, and $2^k ≤ x < 2^{k+1}$, then x will be rounded to the nearest multiple of $2^{k-23}$, with a rounding error up to $2^{k-24}$.

Don't think of floating-point arithmetic to have "uncertainty". It gives a perfectly fine well-defined result, just not the one you might have wanted.

As far as your table is concerned, it doesn't make any sense to me, but I am very confident that there is no square root involved when you try to find upper bounds for floating point errors.

To analyse floating point errors: Multiplication and division may add the relative errors of their inputs, and then another rounding error is added (unless you can prove the result is exact, for example multiplying by 2.0). Addition and subtraction may add the absolute errors of their inputs, and then another rounding error is added (unless you can prove the result is exact, for example subtracting x-y where x/2 ≤ y ≤ 2x.

For tightest bounds, you analyse the error depending on the input values, and you can also improve the error by arranging operations in the right order.

Answer 2

The important thing to consider is than when using floating point numbers, everything is approximate. When you multiply x by 0.587, you don't actually multiply by 0.587, but by some other number within epsilon of 0.587. Usually the epsilon errors accumulate very slowly, but in specific cases even simple operations can yield answers with no significant bits in the result. Most code simply ignores the possibility, and the IEEE standard is designed to make that as OK as possible.

If you really need a bound on the error, you need a course in numerical analysis before you even start to look at your particular algorithms.