Why we still need Short Term Memory if Long Term Memory can save temporary data?

Question

If RAM is a short term memory and SSD is a long term memory, why don't microarchitecture of computer nowadays use SSD or another long term memory for saving temporary data like hidden variable for programming?

If it's about speed, then SSD can improve its speed, is it possible that SSD will become faster than RAM at some point?

If SSD has address for memory location and data for opcode/instruction/operand like RAM, then will it possibly act like RAM?

Answer 1

There's two simple reasons, one fundamental and one related to our current technology. First the technical one: volatile storage is (generally) faster than non-volatile storage. It has fewer requirements - it only needs to store the data for a short while until it gets refreshed, so it's not a surprise that it often is faster.

But the fundamental reason is that memory gets slower to access the bigger it is. This is why modern architectures don't just have 'RAM' and 'disk', there's layers upon layers of increasing size memory, with only the topmost layer being non-volatile:

1. CPU registers
2. L1 cache
3. L2 cache
4. L3 cache
5. RAM itself
6. Cache on the disk micro-controller
7. The disk itself

Answer 2

@orlp is already discussed speed. There's probably more than could be added (e.g., about latency vs. bandwidth), but let's leave that aside for now.

Write Endurance

Instead, let's start by considering a completely separate point: write endurance. Most SSDs use Flash memory. Writing to Flash memory slowly wears it out. SSDs do wear leveling to help distribute writes evenly across available memory, to maximize the life of the device as a whole.

Main memory gets written drastically more often than storage though. As a simple rule of thumb, I'd figure 100 times as often (could be more or less, of course, but let's go with 100 for now).

So, if you used Flash as main memory, you'd be writing to it around 100 times as often as you do as an SSD. Expected lifetime of an SSD is typically around 5-10 years. So as main memory, that would be about 0.05 to 0.1 years. In other words, you'd expect your main memory to need replacement around once a month or so. Heavy computation might easily reduce that to less than a day.

Answer 3

Others have talked about different technologies, like L1 caches, etc. so I thought I'd give a more theoretical explanation.

Memory access time scales with square root of capacity, i.e. random access within a pool of N bits will take around $O(√N)$ time. This takes into account the various different forms of hardware we might use, from fast on-board caches up to distant datacentres. It is essentially due to the finite speed of light and how much we can fit within a certain radius, given the (roughly) Euclidean space we live in.

If we have $M$ bits of storage total, allocating it all to one big pool will subject all memory access to the slowest possible time, namely $O(√M)$

If we split $M$ into two equal-size pools of size $\frac{M}{2}$ (either on separate devices, or as known locations within a single device), this reduces to $O(√\frac{M}{2}) = O(\frac{√M}{√2}) \approx O(0.7 \times √M) = 0.7 \times O(√M)$; i.e. a $30\%$ speedup compared to a single pool.

We get a speedup because when we are accessing one pool, we are guaranteed that the other pool's data will not be needed. Hence we could, in principle, arrange for our desired memory to by physically closer when we issue the read. To make (physical) space for this, we can temporarily move the unwanted memory out of the way.

If we split $M$ into two unequal pools of size $P \times M$ and $Q \times M$, where $P + Q = 1$ and $P < 0.5$, their access times will be:

• $O(√(P \times M)) = O(√P \times √M) = √P \times O(√M)$
• $O(√(Q \times M)) = O(√Q \times √M) = √Q \times O(√M)$

If we choose a very small value for P, like $P = 0.01$, we get a factor of $√P = √0.01 = 0.1$, which is a $90\%$ speed improvement. The smaller we make $P$, the faster its (smaller) pool will be.

$Q$ will also get slower, but only a little since it's a small change relative to its previous size. For example, going from $Q = 0.9$ to $Q = 0.99$ multiplies its access time by $\frac{√0.99}{√0.9} \approx \frac{0.995}{0.95} \approx 1.05$ (about $5\%$ slower), whilst the pool for $P$ would get faster by $\frac{√0.01}{√0.1} \approx \frac{0.1}{0.32} \approx 0.31$ (about $69\%$ faster)

My current laptop has about 226GB of disk space and 16GB RAM, so $P = \frac{16GB}{16GB + 226GB} \approx 0.07$. Hence in theory, splitting up my laptop's memory between disk and RAM gives that RAM access speeds of $√0.07 \approx 0.26$ times what they would be if pooled (a $74\%$ speedup).

Answer 4

According to Google, the fastest SSD drive in 2020 has a 45 microsecond read access time. RAM access time is about 10 ns, that is 4,500 times faster. A factor of 4,500 is enough to make SSD without RAM completely unusable. We do come close to the point where it may make sense for low end computers to have less RAM and rely on swapping, since sequential reads/writes run at up to 6 / 4 GB per second, especially if multiple applications are run at the same time, but are not all used at the same time.

And there is the rather fatal point of write endurance. Although that would only matter if SSD write speeds were the same as RAM; as it is the time to destroy your SSD drive would still be quite long.

Answer 5

When we talk about speed, we need to distinguish between latency and throughput. Latency is fundamentally restricted by physics; for example DRAM can be accessed in about 10 ns, flash can be accessed in about 1 μs, HDD can be accessed in about 10 ms. Throughput can be achieved with engineering and economics; for example if one hard disk can access 100 MB/s sequentially, then 10 disks can achieve 1 GB/s.

You can easily make an SSD have the same throughput as RAM, but you can't match the latency or write endurance.

DRAM and SSDs are incomparable at a number of levels:

• DRAM latency is about 2 orders of magnitude less than flash memory.
• The CPU accesses DRAM directly using an address line. The CPU waits a fixed number of cycles for DRAM transactions. Whereas flash memory needs more sophistical protocols for access; they have controllers that remap addresses; they have very variable read/write times; they have to undergo unpredictable garbage collection.
• DRAM bandwidth is about an order of magnitude better than flash memory (something like 30 GB/s vs. 3 GB/s).
• DRAM can be rewritten an infinite number of times, whereas a flash memory cell can be rewritten 100~100k times depending on the technology (NAND, NOR, MLC, etc.).
• Almost any real-world algorithm will rewrite the contents of some memory locations thousands, maybe millions of times. DRAM can withstand this, but flash certain cannot.
• The minimum data access size for flash memory is around 4096 bytes, whereas for RAM it is around 64 bytes. Most algorithms have many components that access only a small piece of memory (like a single number), so flash memory would waste a lot of bandwidth transferring unused data.
• Writing and especially erasing flash cells are really slow. You can hide these operations behind a DRAM buffer... but then you're back to the characteristics of DRAM.

However, there are some exceptions:

• Some microcontrollers like those from Texas Instruments use FeRAM which is low-latency, non-volatile, and infinitely rewritable. These chips do not have separate DRAM and flash; they just use FeRAM for short- and long-term memory.
• Some computers/CPUs can execute instruction code directly from ROM or flash without having to load it into RAM.
• XPoint memory (a.k.a. Intel Optane) has characteristics in between DRAM and flash, in terms of cost, latency, and capacity.

Answer 6

In earlier days, when microprocessors were fairly slow internally, the Texas Instruments 9900 used external memory for its registers, making context changes as simple as pointing to another block in memory.

As has been said earlier, the balance of native speeds, endurance and cost of the various technologies decides what type of memory are used where.

If enduring memory gets fast enough to be used as the main memory of a CPU, that will lead to a revolution in OS design and how programs are distributed.

That is because the programs can be stored in memory as directly executable, as in the old days of MS-DOS and terminate-and-stay-ready (TSR) module code. That means that when installed, they are ready to run instantly. You would not start a program per se, but just resume it.

Basically, secondary memory would be subsumed into primary memory.

As well, the distribution media could just contain a memory image of a program so there would be no need for the slow translation and construction process involved in traditional installation processes, but a basic read-in directly to memory.

Conceivably, where some large programs are already being provided on SSDs for plug and play installation, they could be on enduring memory modules that are ready to run by just plugging straight into a system.

Of course, OSs would need to be modified to really take advantage of the new program topologies, as memory instructions would begin to be far more used than the IO instructions typically used for secondary memory devices like SSDs. Currently, the interface code to secondary devices are a major OS bottleneck and really slow down program execution, often substantially negating the quantum leap in speeds available from SSDs over HDDs.

In the end, while OS makers would be updating their products over time, the widespread merging of primary and secondary memory might be an opportune time to move to new OSs that are not bound by the 1960s, 70s and 80s architectures upon which current OSs are based. A brave new world awaits!

Answer 7

Even if the bytes of an SSD could otherwise compete performance wise with RAM and Cache, they have an advantage that SSD doesn't: physical distance between the CPU and memory is significant enough that the time it takes the electrons to travel between the two makes a big difference.

Of course, other answers about the current state of technology are also quite important. RAM is cheaper and chips are faster when smaller at the moment.

As this answer points out, speed declines as the square of the physical distance: https://superuser.com/a/1021891/306431

Answer 8

And also, programs designed today rather expect there to be a difference between 'things I want to persist' and 'things that I want to go away on exit'. If that suddenly changed to 'everything persists' then we'd have to code a little differently.

Not to mention concurrency. Suppose I write a program that has a variable 'foo'. Now I'm running 17 copies of that program. Clearly I don't want all of those variables to resolve to the same persistent storage location; we need process-local storage. So there are 17 different mappings of my address space to persistent storage. Now those 17 processes exit, but I suppose that the 17 instances of 'foo' still exist in this new 'everything persists' regime. Will they ever be useful again? I start a new instance of the program. Which of those 17 old instances does my new 'foo' map to?

We can imagine that the persistent copy is erased on program exit, but then we're using persistent storage to simulate ephemeral storage, which seems like adding complication for no gain.

Conclusion: As a programmer, I want ephemeral storage. And since I want that, there's little point in mapping it to persistent storage.