In earlier days, when microprocessors were fairly slow internally, the Texas Instruments 9900 used external memory for its registers, making context changes as simple as pointing to another block in memory.
As has been said earlier, the balance of native speeds, endurance and cost of the various technologies decides what type of memory are used where.
If enduring memory gets fast enough to be used as the main memory of a CPU, that will lead to a revolution in OS design and how programs are distributed.
That is because the programs can be stored in memory as directly executable, as in the old days of MS-DOS and terminate-and-stay-ready (TSR) module code. That means that when installed, they are ready to run instantly. You would not start a program per se, but just resume it.
Basically, secondary memory would be subsumed into primary memory.
As well, the distribution media could just contain a memory image of a program so there would be no need for the slow translation and construction process involved in traditional installation processes, but a basic read-in directly to memory.
Conceivably, where some large programs are already being provided on SSDs for plug and play installation, they could be on enduring memory modules that are ready to run by just plugging straight into a system.
Of course, OSs would need to be modified to really take advantage of the new program topologies, as memory instructions would begin to be far more used than the IO instructions typically used for secondary memory devices like SSDs. Currently, the interface code to secondary devices are a major OS bottleneck and really slow down program execution, often substantially negating the quantum leap in speeds available from SSDs over HDDs.
In the end, while OS makers would be updating their products over time, the widespread merging of primary and secondary memory might be an opportune time to move to new OSs that are not bound by the 1960s, 70s and 80s architectures upon which current OSs are based. A brave new world awaits!