142

GPUs are bad at doing one thing at a time. A modern high-end GPU may have several thousand cores, but these are organized into SIMD blocks of 16 or 32. If you want to compute 2+2, you might have 32 cores each compute an addition operation, and then discard 31 of the results. GPUs are bad at doing individual things fast. GPUs only recently topped the one-...


79

"Back in the day" computers were defined more by their word size, for example the PDP-8 had 12-bit words composed of two 6-bit "bytes". A "nibble" was half a byte, or 3 bits in this case (and here the op codes were 3 bits). It is only in recent decades that 8-bit bytes became so prevalent as to make them the default. Calling the NES 8-bit is less ambiguous ...


77

As stated by user120366, 16 possible 2-input logic gates exist, I've tabulated them for you here: A|B||0|1|2|3|4|5|6|7|8|9|a|b|c|d|e|f -+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 0|0||0|0|0|0|0|0|0|0|1|1|1|1|1|1|1|1 0|1||0|0|0|0|1|1|1|1|0|0|0|0|1|1|1|1 1|0||0|0|1|1|0|0|1|1|0|0|1|1|0|0|1|1 1|1||0|1|0|1|0|1|0|1|0|1|0|1|0|1|0|1 A and B are the inputs, 0 through f ...


64

Branching One piece of hardware that pretty much no GPU has is a Branch Predictor. That's because their primary function is to compute simple functions over large sets of data. The only "branching" that a typical GPU does is the jump at the end of a loop body. CPUs, on the other hand, typically run code that executes branches quite frequently (about 20% ...


44

A) Historically, machines have been characterized by number of bits per 'machine word'. Why should NES be handled differently? B) Calling it a 'byte' is not as clear since historically a 'byte' has not always been composed of eight bits (e.g some early machines had six bits per byte). Admittedly this is not so strong a point anymore. C) On a side note: I ...


27

That means, one year of computation time on a single GPU (or half a year on two GPUs, or a quarter of a year on four GPUs, etc.). If you are thinking of using this term in your own writing, I encourage you to also specify what type of GPU you are using. One-GPU year on a Tesla V100 GPU is a lot more computation than one-GPU year on a K520 GPU. The notion ...


27

First, not every problem is easily amenable to a parallel solution. If it's not possible to formulate your problem as such, you might not gain anything from using a GPU or any parallel approach to begin with. Second, it takes time to move data from the host to the device (i.e., the GPU). You can waste time doing many such transfers and the potentially fast ...


25

It's easiest to think of $1$ representing a true statement and $0$ representing a false statement. The logic gates then act as truth functions. Say you put two statements, $p,q$, together to form a new statement, $r$. In the case of and (logical conjunction), both $p$ and $q$ must be true for $r$ to be true. In the case of or (logical disjunction), $r$ ...


23

GPUs are really good at doing the same simple calculation many times over in parallel. They're usually good at spawning millions of short-lived "threads" that perform the same instruction on multiple bits of data (Same Instruction, Multiple Data, or SIMD). They excel at SIMD situations. They have less memory than the CPU has access to and are not meant as ...


23

I think the questioner has it backwards. If we have a logical function such that A | B | result ---+---+------- 0 | 0 | 0 0 | 1 | 0 1 | 0 | 0 1 | 1 | 1 then we decide to call that function and because it is obvious that the result is 1 only when A and B are both 1. Similarly for or, exclusive-or, etc. There are 16 ...


15

Leaving aside specifics of CPUs vs GPUs, let's simplify the discussion to a single powerful processor (or a handful of them, i.e. multicore) vs an array of 1000s of "slower" processors. Some workloads are classified embarrassingly parallel since they consist of many tasks that are pretty much independent. These are ideal for GPUs. Some workloads have ...


11

You're asking a very broad question that isn't particularly easy to answer "correctly". What you are describing is what you learn if you follow through with a degree in computer science or maybe more precisely in electrical engineering and computer science. So I would recommend you to do exactly that: Go to you favorite university websites and look up ...


10

To augment @Juho’s answer a little: for workloads which are easy to multi-thread, there may also be significant instruction level parallelism that a multi-core superscalar CPU can take advantage of; current GPUs typically run at a lower clock speed than their CPU counterparts; GPU threads are not truly independent, implemented as 16 or 32-way SIMD, and ...


10

GPUs are bad at linear/quadratic programming: Gurobi is watching GPUs closely, but up to this point all of the evidence indicates that they aren't well suited to the needs of an LP/MIP/QP solver. Specifically: GPUs don't work well for sparse linear algebra, which dominates much of linear programming. GPUs rely on keeping hundreds or even thousands of ...


10

Perhaps a prime example would be cryptographic functions such as KDFs, which are specifically designed to benefit as little as possible from parallelization that GPUs and special cracking hardware offers. There's a whole class of so-called sequential memory-hard functions which make it difficult to benefit from parallel computing (a) in a single computation ...


9

Integer arithmetic GPUs are optimised for doing 3D rendering calculations. Following the history of OpenGL, these are traditionally done using 32-bit floating point numbers arranged as either vectors of four floats or quaternion matrices of 4x4 floats. So that's the capability GPUs are very good at. If you want to do floating point with more bits, or 64-...


9

The why of it actually comes from the development of logic, which is a philosophical study of what is true and what is not true. Logic was originally a study of human language with the assumption that if you can reason about how human language works you can maybe reason about how reason works. Since the language I'm answering you in English let's use ...


6

GPU works well or not mainly depends on computing instruction/IO instruction ratio. Here "IO instruction" includes any instruction that send/receive data through the boundary of the basic computation unit in GPU. "Basic computation unit" commonly have like 8-32 ALUs that need to execute instruction together and 16-128KB of registers/RAM and some instruction ...


6

Expanding a bit more on Juho's answer, it's true that GPUs are generally bad at branching, but it's also important to point out why. It's not just a matter of less die space dedicated to branch prediction (though that is part of it.) It's also a matter of less die space dedicated to instruction decoding and sequencing in general. What we call a "core" on a ...


5

Here I will complement the perfectly correct information that Paul gave with more gem5 specifics. One major use case for fast-forwarding is when you have to boot Linux/Android and only then run your content. The boot could take a very long time compared to your content, and so it is very important to find a way to not run it for every experiment, since ...


5

Three books: 1. Code: The Hidden Language of Computer Hardware and Software by Charles Petzold Using everyday objects and familiar language systems such as Braille and Morse code, author Charles Petzold weaves an illuminating narrative for anyone who’s ever wondered about the secret inner life of computers and other smart machines. 2. But How Do It Know ...


4

The answer provided by @Benjoyo is not accurate. The CPU clock can be lower than the system clock, at least in microcontrollers. As seen in this link, the PIC microcontrollers have the CPU clock, with a rate of $f_{cy}$ (also called instruction cycle frequency), and the system clock, with a rate of $f_{osc}$ (oscillator frequency). Physically, what ...


4

LSB (least significant bit) and MSB (most significant bit) apply purely to the values of an integer. The least significant bit is the bit with value 1, the second least significant bit is the bit with value 2, and so on. "Little endian" and "Big endian" are just artefacts from the fact that the bytes of a number can be accessed individually as they are ...


4

Real machines have access to a stack, and so can implement recursion. This is all that is needed to implement the Ackermann function. However, the Ackermann function grows very fast, so you would only be able to calculate a few of its values given realistic time and space constraints.


3

A sticky overflow bit means that the next operation that does not overflow (but would set the bit if it did) will not clear the bit. The value is sticky/persistent and must be cleared to detect newer overflows.


3

The phrase "The Ackermann function isn't implementable with for-loops" is shorthand for "The Ackermann function $A(m,n)$ cannot be implemented using bounded for-loops where an upper bound on the number of iterations in each loop is determined in advance from the values of the parameters $m$ and $n$". In other words the Ackermann function is not primitive ...


3

In current architectures, the CPU does not need to distinguish. Whatever is pointed by the program counter register will be interpreted as an instruction and executed. Everything else can be manipulated as "data". For example, you can have a program modify its own code (in fact, some computer viruses do this, with the aim of avoiding detection).


3

It doesn't. The CPU executes instructions beginning at the program's start address. After each instruction, the CPU either jumps to the address specified by that instruction or, if it's not a jump, moves on to the next instruction in memory. The start address is known to contain executable code because the program was just loaded there. The correctness of ...


3

A full answer to this question depends on the underlying computer architecture, as well as on layers of protection that the operating system may add on top of the basic architecture. In the Harvard architecture instructions and read/write data are held in physically separate memory stores accessed by separate data pathways (buses). It is physically ...


3

Consider a circuit having a single NOT gate, computing some function $f(x)$. We can write $f(x) = g(x,\lnot h(x))$, where $g,h$ are monotone functions. Consider now a sequence of inputs $x_0 < x_1 < \cdots < x_n$ (i.e., $x_0$ is the zero input, and $x_i$ is obtained from $x_{i-1}$ by changing one bit from 0 to 1). Since $h$ is monotone, it is ...


Only top voted, non community-wiki answers of a minimum length are eligible