Almost all advances in computer speed come from one of these areas:
Smaller transistors
Two things result from making transistors smaller:
- They are physically closer together, so the time it takes for an electrical signal to travel from source to destination is smaller. So although electrical signals don't travel any faster than 50 years ago, they often travel shorter distances now.
- More transistors can be included on a chip, which means more "work" can be done at the same time. The more transistors are added, the harder it is to find useful work for them to do, but many clever tricks are used (see below).
More "useful work" per instruction
For example, some processors lack instructions to multiply or divide integers; instead this task must be performed with slow software routines. Adding multiply and divide instructions speeds things up considerably. Adding floating-point instructions can speed up software that requires floating-point numbers.
An important way of doing more "useful work" per instruction is increasing the word size. CPUs that can perform operations on 32-bit numbers often require far fewer instructions to perform the same task as 16-bit or 8-bit CPUs.
Some processors support instructions that do several things at once, in particular instructions that do the same operation on multiple data items (SIMD).
More instructions per cycle
The "clock cycle" is how the processor goes from its current state to the next state. In a sense it is the smallest unit of work the processor can do at a time. However, the number of clock cycles a particular instruction takes depends on the processor's design.
With the advent of pipelined processors, it became possible for separate instructions to "overlap", i.e. one would start before the previous one finished. However, certain instructions can invalidate the next instruction, which will not be known until the next instruction has partially executed, so things can get complicated. (Pipelined processors include logic to make sure everything works out OK—but performance characteristics are more complex.)
Superscalar processors take this to the next level, literally allowing two instructions to execute at the same time, and out-of-order execution takes it one step further, allowing instructions to be executed out of order. These features require analysis of the instruction stream, working out which instructions don't clash with each other.
Although there are other such tricks (e.g. branch prediction, speculative execution), what's more important is the overall picture:
- every instruction takes a certain number of clock cycles to complete (not necessarily constant)
- but multiple instructions can be in progress at once
- so there is a measurable "instructions per cycle" which is >1 for high-end processors
- but it depends very strongly on the workload
More cycles per second
In other words, higher clock speed. Increasing clock speed not only increases heat generated, but also requires much more disciplined chip design, because there is a smaller time limit for the circuit to stabilise. We got a lot of mileage out of this one until the 2000s when we hit some practical limits.
Data in the right place at the right time
Although the components within the CPU have been getting closer and closer together due to shrinking transistors, the CPU and RAM are still a good 5-10cm apart. If an instruction needs something from RAM, that instruction won't take 5 or 6 cycles to complete, it'll take around 200. This is the von Neumann bottleneck problem.
Our main weapon against this is the cache. Recently-accessed data is more likely to be accessed again, so it is kept in special memory (called cache) that is within the CPU chip, making it much faster to access.
However, other techniques (such as pipelining and branch prediction) help by allowing the processor to do useful work while waiting for data to arrive, and also predicting which data might soon be needed.
Multiple and/or specialised processors
It's a lot easier to write software for a single processor than for multiple processors. However, sometimes the performance/cost/power consumption benefits make it worthwhile.
Also, certain processors are particularly well suited to certain tasks. For instance, GPUs are specifically designed for the calculations required for rendering 2D and 3D graphics and effects.
Multi-core processors are essentially multiple processors on a single chip.