4
$\begingroup$

Due to finite precision of number representations, we face situations like:

In: 0.1+0.1+0.1==0.3                                             
Out: False

(on my machine, with Python 2.7 and Python 3 but this is not directly related to the language actually).

This is because there is just no exact representation of $0.1$ (for instance) in typical float formats:

In: "%f"%0.1                                                 
Out: '0.1'

In: "%.20f"%0.1                                                 
Out: '0.10000000000000000555'

If I understand correctly, this means that several decimal numbers may have the exact same internal representation, even with a finite number of decimals. This is true for instance with the decimal number $0.1$ and the one equal to the internal representation of $0.1$, as illustrated above.

Then, the standard practice that consists in sampling random decimal numbers by sampling integers and rescaling them raises questions: it seems to me that the obtained sample may not be uniformly distributed, even if the integer sampling is uniform. Worse, some decimals in the target set may never appear.

Indeed, a decimal number may be obtained from several integers. For instance, I may sample decimal numbers between $0$ and $1$ with $20$ digits precision by sampling integers between $0$ and $10^{20}$ and take their inverses with $20$ decimals. But, as shown above, the integers $10^{19}$ and $10^{19}+555$ lead to the same $20$ digit precision decimal number (on my machine). On the countrary, I cannot sample $0.10000000000000000000$ in this way.

Am I right?

Does anyone know any concrete situation where this may be a serious issue?

With all this in mind, I am having trouble understanding what generating a random float number between 0 and 1 even means, to mention only one of many such claims. It seems to me that such generators cannot generate $0.1$, which actually is not a float number (on my machine)... Does this mean that we sample a uniformly random decimal number among the ones that have an exact internal representation?

In my understanding, all the above means that, when uniformity is crucial, we should only work with integer or string representations of decimal number. Are there better ways?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Methods involving random numbers, such as stochastic simulation, usually compute gross approximations (say to four or fixe exact digits) which are completely unaffected by the non-uniformity you mention (only on the fifteenth significant digit or so). A situation where this matters would be pathological IMO. $\endgroup$
    – user16034
    Commented Dec 7, 2021 at 16:55
  • 1
    $\begingroup$ A much more serious issue is the use of the generators based on the rand function, returning a natural number up to RAND_MAX. On existing implementations, it only provides 15 significant bits ! $\endgroup$
    – user16034
    Commented Dec 7, 2021 at 17:04
  • $\begingroup$ If (0,1) was really continuous, the probability of getting a given exact number back becomes zero, so you would never expect to see 0.1 or any other particular real number. If we're limiting to some known precision, then random should work fine, but you may need to use stochastic rounding to avoid bias one way or the other (useful to speed up convergence for some deep learning algorithms, for example). $\endgroup$
    – C8H10N4O2
    Commented Dec 7, 2021 at 21:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.