# Questions tagged [floating-point]

Approximate representation of numbers as a fixed number of digits multiplied by a logarithmic scale.

230 questions
Filter by
Sorted by
Tagged with
34 views

### Absolute difference between largest IEEE754 number and its predecesor

In simple precision format, the largest possible positive number is $A = 0 ~~~ 11111110 ~~~ 111\ldots 111$ Its predecessor is $B = 0 ~~~ 11111110 ~~~ 111 \ldots 110$ But what is the absolute ...
51 views

### Comparison of different algorithms for summing floating point numbers

I am exploring several approaches to summing floating point values, such as: Naive summation, for comparison Summing sorted values summing with numpy, again for comparison Kahan's algorithm Pairwise ...
1 vote
191 views

### Convert a rational number to a floating-point number exactly

We have two integers, $n$ and $d$. They are coprime (the only positive integer that is a divisor of both of them is $1$). They may be implemented as something that fits in a machine register, or they ...
175 views

### Multiplication of subnormal and normal numbers under IEEE 754

As far as I understand, when doing this operation, we first need to identify the subnormal value, normalise it, adjust the exponent, and then multiply the significands. Wouldn't it be advantageous to ...
216 views

### Floating-point modular multiplication algorithm

Is there a well-known algorithm for modular multiplication of floating-point numbers? I would like to multiply some large angle in single precision (6-7 significant digits) and wrap it back to 360 ...
1 vote
59 views

### FFT of logarithmic input data

Is there a reasonably accurate method of computing an FFT of logarithmically-represented input data (with a sign bit, that is $±2^{\text{double-precision value}}$)? The naive method (convert to linear ...
1 vote
29 views

### Floating-point rounding - bit patterns of values that are halfway between two possible results

I am working through the book Computer Systems: A Programmer's Perspective. The authors explain that round-to-even rounding can be applied for values that are halfway between two possible results. For ...
242 views

### Why XMM register width is 128-bit while double precision floating point is 64-bit?

As far as I know, XMM is used to store floating point. But Highest width floating point according IEEE754 standard is double precision (64-bit). But why XMM register width is 128-bit. Is another 64-...
39 views

### How much larger is the next representable value if 2^59 is stored in a double?

This is an exam question I couldn't solve If I store 2^59 as double, that would give me 1 * 2^58. Is the answer just 2? I.e. next value is 2^60??
50 views

### Best way to constrain a complex number to being within the unit circle?

What is the best way of implementing the following function, $$f(x) = \frac{x}{\max(1, |x|)},$$ where $x$ is complex, using a Cartesian representation of $x$ with IEEE 754 floating point ...
316 views

### What are the use cases for the IEEE 754 inexact flag?

The IEEE 754 standard for floating point numbers defines a flag that is set when a result from floating point calculation isn't exact, i.e. has to be rounded. What algorithms are there that utilize ...
2k views

### Is significand same as mantissa in IEEE754?

I'm trying to understand IEEE 754 floating point. when I try convert 0.3 from decimal to binary with online calculator, it said the significand value was ...
38 views

### How does CPU determine Reserved Exponent cases?

Using IEEE 754 algorithm i assume, that it can be implemented in a branchless way. But how does CPU determine special cases (Reserved Exponent values): Exponent Significand is 11111111 000000000... ...
1 vote
102 views

### What does it mean unambiguously that a number is value 0 up to numerical precision?

I was reading that a quantity $x$ is $0$ upt to numerical precision. What does this statement formally mean -- especially in the context of numerical methods or real computers. I looked up in google ...
1 vote
32 views

### Standard for representing a float scaled to a particular range?

TL;DR Is there a "standard" way to represent a float scaled to a particular range, such that we get maximum precision for the given bit depth, within that range? I'll start with my general ...
70 views

### (Numerical Analysis) What is the largest double float represented for the gamma function and $n!$

Consider that \begin{align} \Gamma(n+1) = n! \end{align} for any integers. I then got the following two questions: What is the largest value of $n$ for which $Γ(n+1)$ and $n!$ can be exactly ...
82 views

### Bisecting Intervals of floating point numbers containing 0 and infinity fairly

It is seldom considered that floating points are not evenly distributed in the real number line. I've been working with interval arithmetic and noticed when bisecting $[a,b]$ on the real number line ...
1 vote
247 views

### Can Radix Sort be modified for signed ints and/or floats?

A few months ago I learned about the magic that allows radix sort to run in O(n) time and space. Most tutorials on radix sort say it is useful for very large ...
50 views

### IEEE 754 conversion

I'm trying to convert 3.2 into IEEE 754 format. We find that $(3)_2=11$ and we also find that $0.2*2=0.4 -0$ $0.4*2=0.8 -0$ $0.8*2=1.6 -1$ $0.6*2=1.2 -1$ and this cycle repeats so $.2=00110011...$ ...
106 views

### Uniformly random decimal numbers

Due to finite precision of number representations, we face situations like: In: 0.1+0.1+0.1==0.3 Out: False (on my ...
1 vote
45 views

### (Branchless) Bitonic Sorting Network for a Set of Floating Point Numbers

In the past I've implemented a branchless Bitonic Sorting Network on a gpu using CUDA, for integers. I am facing a related problem: In my Order Independent Transparency implementation, I would like to ...
57 views

### How can vector angle comparison between lattice points be done without using floating-points? (Convex Hull)

Let's say I have a point $(x_0, y_0)$, and some other points $(x_1, y_1), (x_2, y_2) ... (x_n, y_n)$, such that all of them are lattice points; all have integer coordinates. Let's further assume that ...
1 vote
730 views

### How many Integers can be represent in Double-Precision floating-point form

How to calculate the number of Integers that can be represent in Double-Precision floating-point form?
1 vote
312 views

### Prove every number in double precision 32-bit floating-point format can be represented in 64-bit format

Theorem: Prove every number in double precision 32-bit floating-point format can be represented in double precision 64-bit floating point-format. 64-bit format: Attempt: Let $b = b_0 ,...,b_{31}$ ...
1 vote
45 views

### Why does floating point become less accurate as the powers of 2 increase?

https://fabiensanglard.net/floating_point_visually_explained/ I was reading this article where the exponent and the mantissa are explained as the window and offset respectively. As the gap between ...
123 views

### Is there a way to convert FLOPS to bit operation per second

My problem is the following: I have $N$ inner products to compute in parallel every second. Each of the vectors in those inner product is composed of $7$ bits. I want to know for which $N$ it starts ...
203 views

### Unit conversion - Better to divide by an integer or multiply by a double?

I currently have a long timestamp measured in units of 100ns elapsed since January 1st, 1900. I need to convert it to milliseconds. I have the choice of either ...
1 vote
875 views

### Half precision floating point question -- smallest non-zero number

There's a floating point question that popped up and I'm confused about the solution. It states that IEEE 754-2008 introduces half precision, which is a binary floating-point representation that uses ...
1 vote
483 views

### Floating Point Arithmetic with 3 bits mantissa

Find all values of $x ∈ R$ such that x + 1 = 1 in floating point arithmetic with 3 bits mantissa. How do we represent number 1 in floating point arithmetic with 3 bits mantissa I wonder? After that, ...
172 views

### Python versus Matlab on the quantity 1/0

Python and Matlab seem to disagree on the division by 0. Python: ...
1 vote
4k views

### Negative Numbers in 32 bit Floating Point IEEE Numbers

So I understand the logic behind converting positive decimal numbers to IEEE 32 bit floating numbers but I'm not completely sure behind the negative one's. If for example we have a decimal number say -...
1 vote
336 views

### Adding two numbers in base 2(floating point) vs Multiplying two numbers in base 2(floating point)

Is it true that adding two numbers in base 2 is more complex than multiplying them? If so can someone please explain why this is the case?
1 vote
149 views

### Prove that $1^\text{nan} = 1.00$

I know that for most computation involve nan (not a number) the result is a nan itself except for some cases. For example, $1^{\text{nan}} = 1.00$ which proven by mathematicians to be true. I tried to ...