# Questions tagged [floating-point]

Approximate representation of numbers as a fixed number of digits multiplied by a logarithmic scale.

222 questions
Filter by
Sorted by
Tagged with
59 views

### What are the use cases for the IEEE 754 inexact flag?

The IEEE 754 standard for floating point numbers defines a flag that is set when a result from floating point calculation isn't exact, i.e. has to be rounded. What algorithms are there that utilize ...
936 views

### Is significand same as mantissa in IEEE754?

I'm trying to understand IEEE 754 floating point. when I try convert 0.3 from decimal to binary with online calculator, it said the significand value was ...
33 views

### How does CPU determine Reserved Exponent cases?

Using IEEE 754 algorithm i assume, that it can be implemented in a branchless way. But how does CPU determine special cases (Reserved Exponent values): Exponent Significand is 11111111 000000000... ...
1 vote
29 views

### What does it mean unambiguously that a number is value 0 up to numerical precision?

I was reading that a quantity $x$ is $0$ upt to numerical precision. What does this statement formally mean -- especially in the context of numerical methods or real computers. I looked up in google ...
1 vote
22 views

### Standard for representing a float scaled to a particular range?

TL;DR Is there a "standard" way to represent a float scaled to a particular range, such that we get maximum precision for the given bit depth, within that range? I'll start with my general ...
50 views

### (Numerical Analysis) What is the largest double float represented for the gamma function and $n!$

Consider that \begin{align} \Gamma(n+1) = n! \end{align} for any integers. I then got the following two questions: What is the largest value of $n$ for which $Γ(n+1)$ and $n!$ can be exactly ...
58 views

### Bisecting Intervals of floating point numbers containing 0 and infinity fairly

It is seldom considered that floating points are not evenly distributed in the real number line. I've been working with interval arithmetic and noticed when bisecting $[a,b]$ on the real number line ...
1 vote
96 views

### Can Radix Sort be modified for signed ints and/or floats?

A few months ago I learned about the magic that allows radix sort to run in O(n) time and space. Most tutorials on radix sort say it is useful for very large ...
37 views

### IEEE 754 conversion

I'm trying to convert 3.2 into IEEE 754 format. We find that $(3)_2=11$ and we also find that $0.2*2=0.4 -0$ $0.4*2=0.8 -0$ $0.8*2=1.6 -1$ $0.6*2=1.2 -1$ and this cycle repeats so $.2=00110011...$ ...
53 views

### Uniformly random decimal numbers

Due to finite precision of number representations, we face situations like: In: 0.1+0.1+0.1==0.3 Out: False (on my ...
1 vote
21 views

### (Branchless) Bitonic Sorting Network for a Set of Floating Point Numbers

In the past I've implemented a branchless Bitonic Sorting Network on a gpu using CUDA, for integers. I am facing a related problem: In my Order Independent Transparency implementation, I would like to ...
1 vote
34 views

### How can vector angle comparison between lattice points be done without using floating-points? (Convex Hull)

Let's say I have a point $(x_0, y_0)$, and some other points $(x_1, y_1), (x_2, y_2) ... (x_n, y_n)$, such that all of them are lattice points; all have integer coordinates. Let's further assume that ...
1 vote
66 views

### How many Integers can be represent in Double-Precision floating-point form

How to calculate the number of Integers that can be represent in Double-Precision floating-point form?
1 vote
102 views

### Prove every number in double precision 32-bit floating-point format can be represented in 64-bit format

Theorem: Prove every number in double precision 32-bit floating-point format can be represented in double precision 64-bit floating point-format. 64-bit format: Attempt: Let $b = b_0 ,...,b_{31}$ ...
1 vote
40 views

### Why does floating point become less accurate as the powers of 2 increase?

https://fabiensanglard.net/floating_point_visually_explained/ I was reading this article where the exponent and the mantissa are explained as the window and offset respectively. As the gap between ...
39 views

### Is there a way to convert FLOPS to bit operation per second

My problem is the following: I have $N$ inner products to compute in parallel every second. Each of the vectors in those inner product is composed of $7$ bits. I want to know for which $N$ it starts ...
36 views

### Adding two numbers in half-precision IEEE754 standart

I have to add two numbers using IEEE754: -14.1875 and -7.4375. I managed to convert them to half-precision numbers in IEEE754: 1 10010 1100011000 (-14.1875) ...
109 views

### Unit conversion - Better to divide by an integer or multiply by a double?

I currently have a long timestamp measured in units of 100ns elapsed since January 1st, 1900. I need to convert it to milliseconds. I have the choice of either ...
1 vote
246 views

### Half precision floating point question -- smallest non-zero number

There's a floating point question that popped up and I'm confused about the solution. It states that IEEE 754-2008 introduces half precision, which is a binary floating-point representation that uses ...
1 vote
89 views

### Floating Point Arithmetic with 3 bits mantissa

Find all values of $x ∈ R$ such that x + 1 = 1 in floating point arithmetic with 3 bits mantissa. How do we represent number 1 in floating point arithmetic with 3 bits mantissa I wonder? After that, ...
106 views

### Python versus Matlab on the quantity 1/0

Python and Matlab seem to disagree on the division by 0. Python: ...
1 vote
522 views

### Negative Numbers in 32 bit Floating Point IEEE Numbers

So I understand the logic behind converting positive decimal numbers to IEEE 32 bit floating numbers but I'm not completely sure behind the negative one's. If for example we have a decimal number say -...
1 vote
87 views

### Adding two numbers in base 2(floating point) vs Multiplying two numbers in base 2(floating point)

Is it true that adding two numbers in base 2 is more complex than multiplying them? If so can someone please explain why this is the case?
1 vote
### Prove that $1^\text{nan} = 1.00$
I know that for most computation involve nan (not a number) the result is a nan itself except for some cases. For example, $1^{\text{nan}} = 1.00$ which proven by mathematicians to be true. I tried to ...