It's possible to represent a very large class of numbers called algebraic numbers exactly, by treating them as roots of polynomials.
This article as well as this mathoverflow question have more information, as well as many more resources about this topic. As far as I know, this method still fails to represent transcendental numbers such as $\pi$ or $e$, unless some generalization of it has been discovered. There are many other issues with this method, such as additional solutions to simple addition appearing out of "nowhere" (also known as the complex plane). It's a tradeoff, use your good judgement to tell if it's useful for you.