# Why we still need Short Term Memory if Long Term Memory can save temporary data?

If RAM is a short term memory and SSD is a long term memory, why don't microarchitecture of computer nowadays use SSD or another long term memory for saving temporary data like hidden variable for programming?

If it's about speed, then SSD can improve its speed, is it possible that SSD will become faster than RAM at some point?

If SSD has address for memory location and data for opcode/instruction/operand like RAM, then will it possibly act like RAM?

• This was supposed to be the premise behind HP's "The Machine" and memristors. – jamesdlin Feb 8 at 9:40
• en.wikipedia.org/wiki/Memory_hierarchy – Bergi Feb 8 at 12:46
• But why would you do that? What benefit would it have? Short-term memory will always be cheaper and faster; no matter how good your SSDs get, SRAM and DRAM will benefit from the same improvements and still be better and cheaper. And what would you get in return? Non-volatility? That's not exactly a universally good thing. And OSes already give you ways to suspend the computer's state and restore it later (while also taking care of all the things that changed on the outside - don't forget computers aren't just the CPU and RAM). – Luaan Feb 10 at 10:24
• The notion of SSD being as fast as RAM is a little weird: if we have technology to have an SSD interface that fast, we would have even faster RAM interface, so SSD will always be slower than RAM at the same technological level. What is possible is that eventually non-volatile RAM become cheap enough so we don't need HDD or SSD (look up "universal memory"), but todays OSs and software abstraction does not fit well with a diskless system. I.e. doesn't sound optimal to format a chunk of your RAM as NTFS so you can store your files in there. – lvella Feb 10 at 13:13
• "why don't microarchitecture of computer nowadays use SSD or another long term memory for saving temporary data like hidden variable for programming?" sounds like you are describing paging (a.k.a. "swapping"). It's (more than) 50-year-old technology. Is this what you intended? or were you thinking of something else? Most of the answers here are not responsive to this portion of your Question. – Eric Towers Feb 10 at 20:03

There's two simple reasons, one fundamental and one related to our current technology. First the technical one: volatile storage is (generally) faster than non-volatile storage. It has fewer requirements - it only needs to store the data for a short while until it gets refreshed, so it's not a surprise that it often is faster.

But the fundamental reason is that memory gets slower to access the bigger it is. This is why modern architectures don't just have 'RAM' and 'disk', there's layers upon layers of increasing size memory, with only the topmost layer being non-volatile:

1. CPU registers
2. L1 cache
3. L2 cache
4. L3 cache
5. RAM itself
6. Cache on the disk micro-controller
7. The disk itself
• It's amazing to think that this is now so normal (minus disk storage) that we don't think about it and just expect our CPU and OS to handle all this. Once a long time ago the idea of multiple levels of working memory (not storage) was so revolutionary that Digital named their OS after it: VMS (which happens to be the OS most of the early hackers from MIT and Berkeley learned to program on) – slebetman Feb 8 at 10:44
• @slebetman I worked for DEC at the time, and VMS was not popular among early hackers (IMHO because of the license costs, but they claimed otherwise). What was very popular with them was Berkeley Unix (and related) often running on VAXes. Later writers often assumed that all VAXes were running VMS, but actually almost 1/3rd of them were running some form of UNIX (though the fact that so many of them were was a big secret that DEC and ATT agreed to keep). – RBarryYoung Feb 8 at 16:14
• "But the fundamental reason is that memory gets slower to access the bigger it is." <-- I don't think that this is a 100% accurate statement. I can have a 16GB MicroSD card that's way slower than my PC RAM or my SSD, despite being way smaller. I believe that a more accurate statement could be that the memory is faster also based on how physically further away it is from the core(s). And yes, volatility impacts the speed too. – Ismael Miguel Feb 8 at 16:56
• @IsmaelMiguel: Perhaps it would be more accurate to say that the maximum realistically achievable speed of memory decreases the bigger it is, as a general trend. Of course, there are other tradeoffs (such as cost) so size is not the only determining factor. – David Feb 8 at 17:32
• @IsmaelMiguel, you need to compare apples to apples here. For example, L1, L2, and L3 cache usually use the same memory technology, and the few kilobytes of L1 have far lower latency than the megabytes of L3 (access time scales roughly as the logarithm of size). – Mark Feb 9 at 4:11

@orlp is already discussed speed. There's probably more than could be added (e.g., about latency vs. bandwidth), but let's leave that aside for now.

### Write Endurance

Instead, let's start by considering a completely separate point: write endurance. Most SSDs use Flash memory. Writing to Flash memory slowly wears it out. SSDs do wear leveling to help distribute writes evenly across available memory, to maximize the life of the device as a whole.

Main memory gets written drastically more often than storage though. As a simple rule of thumb, I'd figure 100 times as often (could be more or less, of course, but let's go with 100 for now).

So, if you used Flash as main memory, you'd be writing to it around 100 times as often as you do as an SSD. Expected lifetime of an SSD is typically around 5-10 years. So as main memory, that would be about 0.05 to 0.1 years. In other words, you'd expect your main memory to need replacement around once a month or so. Heavy computation might easily reduce that to less than a day.

Others have talked about different technologies, like L1 caches, etc. so I thought I'd give a more theoretical explanation.

Memory access time scales with square root of capacity, i.e. random access within a pool of N bits will take around $$O(√N)$$ time. This takes into account the various different forms of hardware we might use, from fast on-board caches up to distant datacentres. It is essentially due to the finite speed of light and how much we can fit within a certain radius, given the (roughly) Euclidean space we live in.

If we have $$M$$ bits of storage total, allocating it all to one big pool will subject all memory access to the slowest possible time, namely $$O(√M)$$

If we split $$M$$ into two equal-size pools of size $$\frac{M}{2}$$ (either on separate devices, or as known locations within a single device), this reduces to $$O(√\frac{M}{2}) = O(\frac{√M}{√2}) \approx O(0.7 \times √M) = 0.7 \times O(√M)$$; i.e. a $$30\%$$ speedup compared to a single pool.

We get a speedup because when we are accessing one pool, we are guaranteed that the other pool's data will not be needed. Hence we could, in principle, arrange for our desired memory to by physically closer when we issue the read. To make (physical) space for this, we can temporarily move the unwanted memory out of the way.

If we split $$M$$ into two unequal pools of size $$P \times M$$ and $$Q \times M$$, where $$P + Q = 1$$ and $$P < 0.5$$, their access times will be:

• $$O(√(P \times M)) = O(√P \times √M) = √P \times O(√M)$$
• $$O(√(Q \times M)) = O(√Q \times √M) = √Q \times O(√M)$$

If we choose a very small value for P, like $$P = 0.01$$, we get a factor of $$√P = √0.01 = 0.1$$, which is a $$90\%$$ speed improvement. The smaller we make $$P$$, the faster its (smaller) pool will be.

$$Q$$ will also get slower, but only a little since it's a small change relative to its previous size. For example, going from $$Q = 0.9$$ to $$Q = 0.99$$ multiplies its access time by $$\frac{√0.99}{√0.9} \approx \frac{0.995}{0.95} \approx 1.05$$ (about $$5\%$$ slower), whilst the pool for $$P$$ would get faster by $$\frac{√0.01}{√0.1} \approx \frac{0.1}{0.32} \approx 0.31$$ (about $$69\%$$ faster)

My current laptop has about 226GB of disk space and 16GB RAM, so $$P = \frac{16GB}{16GB + 226GB} \approx 0.07$$. Hence in theory, splitting up my laptop's memory between disk and RAM gives that RAM access speeds of $$√0.07 \approx 0.26$$ times what they would be if pooled (a $$74\%$$ speedup).

• At least to me, this does not look any more theoretical. The article that forms its basis is basically just graphing access times due to different levels of caching, drawing a line through it, and saying "that looks sort of like square root of N". – Jerry Coffin Feb 9 at 18:02
• @JerryCoffin It's much deeper than that: see en.wikipedia.org/wiki/Bekenstein_bound (discussed in part 2 of the linked article). I also recommend the article's FAQs (part 4). – Warbo Feb 9 at 20:14
• Bekenstein is functionally irrelevant to modern computing, and to this question. – Corey Feb 9 at 23:27

According to Google, the fastest SSD drive in 2020 has a 45 microsecond read access time. RAM access time is about 10 ns, that is 4,500 times faster. A factor of 4,500 is enough to make SSD without RAM completely unusable. We do come close to the point where it may make sense for low end computers to have less RAM and rely on swapping, since sequential reads/writes run at up to 6 / 4 GB per second, especially if multiple applications are run at the same time, but are not all used at the same time.

And there is the rather fatal point of write endurance. Although that would only matter if SSD write speeds were the same as RAM; as it is the time to destroy your SSD drive would still be quite long.

When we talk about speed, we need to distinguish between latency and throughput. Latency is fundamentally restricted by physics; for example DRAM can be accessed in about 10 ns, flash can be accessed in about 1 μs, HDD can be accessed in about 10 ms. Throughput can be achieved with engineering and economics; for example if one hard disk can access 100 MB/s sequentially, then 10 disks can achieve 1 GB/s.

You can easily make an SSD have the same throughput as RAM, but you can't match the latency or write endurance.

DRAM and SSDs are incomparable at a number of levels:

• DRAM latency is about 2 orders of magnitude less than flash memory.
• The CPU accesses DRAM directly using an address line. The CPU waits a fixed number of cycles for DRAM transactions. Whereas flash memory needs more sophistical protocols for access; they have controllers that remap addresses; they have very variable read/write times; they have to undergo unpredictable garbage collection.
• DRAM bandwidth is about an order of magnitude better than flash memory (something like 30 GB/s vs. 3 GB/s).
• DRAM can be rewritten an infinite number of times, whereas a flash memory cell can be rewritten 100~100k times depending on the technology (NAND, NOR, MLC, etc.).
• Almost any real-world algorithm will rewrite the contents of some memory locations thousands, maybe millions of times. DRAM can withstand this, but flash certain cannot.
• The minimum data access size for flash memory is around 4096 bytes, whereas for RAM it is around 64 bytes. Most algorithms have many components that access only a small piece of memory (like a single number), so flash memory would waste a lot of bandwidth transferring unused data.
• Writing and especially erasing flash cells are really slow. You can hide these operations behind a DRAM buffer... but then you're back to the characteristics of DRAM.

However, there are some exceptions:

• Some microcontrollers like those from Texas Instruments use FeRAM which is low-latency, non-volatile, and infinitely rewritable. These chips do not have separate DRAM and flash; they just use FeRAM for short- and long-term memory.
• Some computers/CPUs can execute instruction code directly from ROM or flash without having to load it into RAM.
• XPoint memory (a.k.a. Intel Optane) has characteristics in between DRAM and flash, in terms of cost, latency, and capacity.

In earlier days, when microprocessors were fairly slow internally, the Texas Instruments 9900 used external memory for its registers, making context changes as simple as pointing to another block in memory.

As has been said earlier, the balance of native speeds, endurance and cost of the various technologies decides what type of memory are used where.

If enduring memory gets fast enough to be used as the main memory of a CPU, that will lead to a revolution in OS design and how programs are distributed.

That is because the programs can be stored in memory as directly executable, as in the old days of MS-DOS and terminate-and-stay-ready (TSR) module code. That means that when installed, they are ready to run instantly. You would not start a program per se, but just resume it.

Basically, secondary memory would be subsumed into primary memory.

As well, the distribution media could just contain a memory image of a program so there would be no need for the slow translation and construction process involved in traditional installation processes, but a basic read-in directly to memory.

Conceivably, where some large programs are already being provided on SSDs for plug and play installation, they could be on enduring memory modules that are ready to run by just plugging straight into a system.

Of course, OSs would need to be modified to really take advantage of the new program topologies, as memory instructions would begin to be far more used than the IO instructions typically used for secondary memory devices like SSDs. Currently, the interface code to secondary devices are a major OS bottleneck and really slow down program execution, often substantially negating the quantum leap in speeds available from SSDs over HDDs.

In the end, while OS makers would be updating their products over time, the widespread merging of primary and secondary memory might be an opportune time to move to new OSs that are not bound by the 1960s, 70s and 80s architectures upon which current OSs are based. A brave new world awaits!

Even if the bytes of an SSD could otherwise compete performance wise with RAM and Cache, they have an advantage that SSD doesn't: physical distance between the CPU and memory is significant enough that the time it takes the electrons to travel between the two makes a big difference.

Of course, other answers about the current state of technology are also quite important. RAM is cheaper and chips are faster when smaller at the moment.

As this answer points out, speed declines as the square of the physical distance: https://superuser.com/a/1021891/306431

• (The essential measure is signal propagation time/speed. The average speed of available carriers of charge is ridiculously low, as is the distance travelled by any single one.) – greybeard Feb 11 at 8:25
• (Do RAM the same favour as SSD&CPU and type it as an acronym.) – greybeard Feb 11 at 8:26
• I wonder if Optical Computing can make this irrelevant one day? – Travis Wells Feb 13 at 5:21

And also, programs designed today rather expect there to be a difference between 'things I want to persist' and 'things that I want to go away on exit'. If that suddenly changed to 'everything persists' then we'd have to code a little differently.

Not to mention concurrency. Suppose I write a program that has a variable 'foo'. Now I'm running 17 copies of that program. Clearly I don't want all of those variables to resolve to the same persistent storage location; we need process-local storage. So there are 17 different mappings of my address space to persistent storage. Now those 17 processes exit, but I suppose that the 17 instances of 'foo' still exist in this new 'everything persists' regime. Will they ever be useful again? I start a new instance of the program. Which of those 17 old instances does my new 'foo' map to?

We can imagine that the persistent copy is erased on program exit, but then we're using persistent storage to simulate ephemeral storage, which seems like adding complication for no gain.

Conclusion: As a programmer, I want ephemeral storage. And since I want that, there's little point in mapping it to persistent storage.

• None of this makes sense to me. We've had swap space (which stores ephemeral data on non-volatile disks) for ages, and it has none of these problems. – Daniel Wagner Feb 8 at 16:06
• Running different instances of a programs can access different files to store its data. Nothing new here. – Paŭlo Ebermann Feb 10 at 0:48