Increased rounding relative error when subtracting

Question

I'm reading the book "Lessons in Scientific Computing" by Schoerghofer and it says:

If x and y are real numbers of the same sign, their sum x + y has an absolute error that adds the two individual absolute errors, and the relative error is at most as large as the relative error of x or y. Hence, adding them is insensitive to roundoff. On the other hand, x−y has increased relative error.

I am not sure how to obtain the result for subtraction, i.e. how to show for $$ x(1+\epsilon_x)-y(1+\epsilon_y) = (x-y)(1+\epsilon) $$ that $\epsilon\ge\max(\epsilon_x,\epsilon_y)$.

Answer 1

On the other hand, $x−y$ has increased relative error.

The correct conclusion should be $x-y$ may have increased relative error. In other words, $x-y$ may have decreased relative error.

---

Let us review what is absolute error and relative error in scientific computing.

The absolute error in approximating $x$ by $\hat x$ is $e = e_x= e_{x, \hat x}= \hat x − x$. The relative error is $\epsilon =\epsilon_x=\epsilon_{x,\hat x}=\dfrac ex$, a dimensionless measure of error which is usually considered more informative.

Yuval has shown an extreme example where the relative error becomes 0. Here are a few more examples on the relative error of approximating $d=x-y$ by $\hat d =\hat x-\hat y$.

1. $x=2$, $\hat x=2.04$, $y=1$, $\hat y = 1.03$. Then $\epsilon_x = 0.02$, $\epsilon_y=0.03$, $d=1$, $\hat d=1.01$, $\epsilon_d=0.01$.
2. $x=4$, $\hat x=4.04$, $y=1$, $\hat y = 0.95$. Then $\epsilon_x =0.01$, $\epsilon_y=0.05$, $d=3$, $\hat d=3.09$, $\epsilon_d=-0.03$.
3. $x=21$, $\hat x=21.4$, $y=20$, $\hat y = 20.2$. Then $\epsilon_x \approx 0.019$, $\epsilon_y=0.01$, $d=1$, $\hat d=1.2$, $\epsilon_d=0.2$.
4. $x=2100$, $\hat x=2140$, $y=2098$, $\hat y = 2096$. Then $\epsilon_x \approx 0.019$, $\epsilon_y\approx0.00095$, $d=2$, $\hat d=44$, $\epsilon_d=22$.

We can make several observations.

The example 1 shows the relative error of the difference can be smaller than both original relative errors while example 2, 3 and 4 demonstrate the absolute value of the relative error can be bigger than one or both of the original relative errors. In particular, the example 4, where $\epsilon_d = 2310\max(\epsilon_x, \epsilon_y)$ illustrates the relative-error explosion where the relative error can go wildly larger when the two original values, which are 2100 and 2098 here, are near to each other.

---

Exercise 1. Let $x=cy$ for $x, y>0$. Let $\hat x=x(1+\epsilon_x)$, $\hat y = y(1+\epsilon_y)$. Let $d=x+y$ and $\hat d=\hat x+\hat y$. Then $$|\epsilon_d|\le\max(|\epsilon_x|,|\epsilon_y|)$$

Exercise 2. Let $x=cy$ for $x, y>0, c>1$. Let $\hat x=x(1+\epsilon_x)$, $\hat y = y(1+\epsilon_y)$. Let $d=x-y$ and $\hat d=\hat x-\hat y$. Then $$|\epsilon_d|\lt \frac {2c}{c-1}\max(|\epsilon_x|,|\epsilon_y|)$$

Answer 2

You can't prove it because it's not true. For example, $x(1+\epsilon) - x(1+\epsilon) = 0$. What you can say is that if all you know is that $x$ has relative error $\epsilon$ then the best bound on the relative error of $x-y$ is larger than $\epsilon$. This is because $$ \frac{x(1+\epsilon)-y}{x-y} = 1 + \frac{x}{x-y} \epsilon. $$