The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [floating-point]

Approximate representation of numbers as a fixed number of digits multiplied by a logarithmic scale.

165 questions
Filter by
Sorted by
Tagged with
9 views

Is Decimal (correctly-rounded arbitrary precision decimal floating point arithmetic) fixed-point, floating-point or something else?

The Decimal data type I am referring to is GNU MPFR(https://en.wikipedia.org/wiki/GNU_MPFR), or libmpdec (http://www.bytereef.org/mpdecimal/doc/libmpdec/index.html). I have been searching for ...
38 views

33 views

How does normalised floating point binary work with two's complement?

I'm doing AQA a-level computer science, and the specification for which states that: Exam questions on floating point numbers will use a format in which both the normalised mantissa and exponent are ...
12 views

Where can I find some free benchmarks to evaluate a MCU? [closed]

At present, I'm designing a soft processor with single-precision floating point unit (FPU). I am going to put my soft core into an FPGA and do some performance evaluation. The benchmarks are supposed ...
63 views

Float number to binary

I would like to convert 0,347 and 0,9828 to binary, how can I do that? I know that sucessive multiplication by 2 can do this, but this method seems very painful and even ineffective since the size of ...
91 views

Stable and fast computation of the squared euclidean distance matrix

Let's say I want to compute the matrix $M$ of the squared euclidean distances between each pair of vectors $(x, y)$ belonging to two sets $X$ and $Y$ respectively. The sets of vectors $X$ and $Y$ have ...
171 views

Proof that (x-y)(x+y) is more accurate than x²-y²

I was carrying on my reading of What Every Computer Scientist Should Know About Floating-Point Arithmetic but got stuck on the proof of Theorem 2 (page 34). At some point it says: \begin{align} (x \...
49 views

Proof that a guard digit bound the error of subtraction

I was reading What Every Computer Scientist Should Know About Floating-Point Arithmetic, which is extremely interesting. But I have some troubles understanding the proof of Theorem 9 (page 33). First ...
54 views

Why isn't it necessary to store an integer part of significant in IEEE754 floating point notation?

We see that there is a sign, exponent, and mantissa part for the notation. But, there is no location for the significant bit. Why isn't it necessary to store an integer part of significant in IEEE754 ...
40 views

Doubt in definition of Float

Can anyone tell the meaning of the bold portion? Float: It is used to store decimal numbers (numbers with floating point value) with single precision.
953 views

Number of FLOPs (floating point operations) for exponentiation

What is the number of floating point operations needed to perform exponentiation (power of)? Assuming multiplication of two floats use one FLOP, the number of operations for $x^n$ will be $n-1$. ...
91 views

How to test for overflow when multiplying floats

I am trying to implement a 3-term recurrence relation: $$p_{n+1} = ap_n + bp_{n-1}$$ This can be implemented as ...
32 views

How does IEEE 754 decimal encoding work? [closed]

This may be a silly question, but if a computer works in binary how can you encript numbers using decimal?
97 views

What is the state of the algorithmic art for floating point arithmetic on complex numbers?

Most modern compilers and processors implement the IEEE 754 binary formats for floating point numbers. IEEE 754 guarantees that the addition, subtraction, multiplication, division, and square root ...
164 views

Binary Floating Point Range/Precision for 4-bit Mantissa and 4-bit Exponent

I'm trying to understand binary floating point and using just a 4-bit mantissa and a 4-bit exponent (both 2s compliment) to keep things simple. As far as I can tell, the largest denary number I can ...
41 views

Floating point precision from rearranging equation

When I run $x^2 - y^2$ with x=8.8888888888 and y=9.9999999999 in python, I get the following result: ...
37 views

Why do I get different results from two calculation methods?

I am wondering what the reason for the following is We know that , exponential has a taylor representation : $$exp(x)=1+x+\frac{x^{2}}{2!}+\frac{x^{3}}{3!}+...$$ Using the first n terms , in R , ...
59 views

Increased rounding relative error when subtracting

I'm reading the book "Lessons in Scientific Computing" by Schoerghofer and it says: If x and y are real numbers of the same sign, their sum x + y has an absolute error that adds the two ...
151 views

IEEE754 representation in hexadecimal?

In class, I've heard hexadecimal representation for IEEE754 mentioned and described in 32bit length as a format that consists of one bit for sign, normalized 6-digit fraction (with an implied leading ...
55 views

Addition errors in IEEE754 floating point representation

So in class, we were talking about the idea of floating point precision in IEEE754 format, and how, when some numbers are added, precision is lost. My professor then gave the following example of a ...
2k views

Understanding denormalized numbers in floating point representation

I am confused about how denormalized numbers work in floating point representation. I was referring to Stallings book and this article. The book initially explains floating point number format in ...
44 views

186 views

Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
83 views

Floating point Euclidean norm optimization

Past few days I've been struggling with floating point number exercises, trying to learn how they work and what are their limitations. The exercise I've encountered today is as follows: The ...