# Does having one large L1 cache instead of L1 and L2 cache makes computation faster?

Does having one larger L1 cache instead of L1 and L2 cache makes computation faster? Also will this make the CPU more expensive to make?

• The answer is most likely 'depends'; there's really not enough information here to say. What kind of computation? What are other parameters of the machine? And so on...
– Raphael
Dec 13 '15 at 0:44
• Of course. The only reason I settle for a smaller L1 and augument it with L2 is cost.
– ARi
Dec 13 '15 at 12:03
• @ARi, what about the manufacturability of a bigger L1 at the speed of the current L1? A design is a lot of trade-off between factors, some technical, some less so (price, positioning against other models, ...). You can't just modify the caches and expect to compare with something realistic. Dec 13 '15 at 12:17

Whether a shallower memory hierarchy provides better performance depends on the workload, the microarchitecture, and the implementation technology.

A workload that has a high miss rate for a "conventionally-sized" L1, when run on such a processor, would have more overhead in transferring cache blocks from L2 to L1 and (if writeback) dirty blocks from L1 to L2. In addition, if access to L2 was only started when a miss in L1 is determined, then latency for L2 would be larger (even if L1 in the shallow hierarchy processor used the same memory array implementation as L2).

A microarchitecture suited to modest-sized L1 caches would be biased (relative to one suited to huge L1 caches) toward speed-demon rather than brainiac design, exploiting the lower latency of a cache hit. (Workload, implementation technology, and other factors influence other tradeoffs with respect to speed-demon vs. brainiac.)

If the targeted workload can benefit from extensive hardware threading, providing a larger (and more associative) L1 can be advantageous as the thread-level parallelism can hide greater L1 access latency.

If the implementation technology reduces the latency penalty of a larger cache, the variety of workloads where the tradeoffs (in miss handling overheads and other factors) weigh in favor of a shallower memory hierarchy will increase. For example, in the future it is conceivable that 3D integration might be used for L1 cache and the memory used for this might have relatively high cell access latency (for low power to avoid thermal issues associated with 3D integration), so reduced distance (from 3D integration) and slower memory cells could increase the incentive for a larger L1 cache.

These types of questions become even more complex when one considers that the definition of L1 cache can get somewhat less clear. For example, if what would conventionally be called L2 cache is accessed in parallel with L1 but with higher latency, is it an L2 cache or part of a non-uniform cache architecture L1, especially if some cache blocks are never allocated to the smaller portion of the cache (cache bypassing has been proposed as a mechanism to better utilize capacity and bandwidth). (Even the NUCA L2 cache proposals allowed for transfers between slow and fast portions of L2.) Way prediction can also introduce variable latency for L1; if the prediction mechanism included consideration of expected criticality (and such was used for allocation to near or distant memory arrays within a NUCA design), one might reasonably consider the far memory arrays part of the L1 cache even though their inherent access latency is greater.

(Itanium 2 did not even probe the small L1 data cache for floating-point register loads, so for floating-point data one could almost consider the L2 cache as an L1 cache.)

• This is not a very good answer (yet?). Average memory access time would be a coarse measure of memory system effectiveness, but such treats all memory accesses as equally critical and perhaps slightly underestimates some benefits of a larger L1. Energy efficiency (in the core as well as the memory system) is also not a constant. (Data point: HP chose huge L1s for PA-RISC.) Dec 13 '15 at 1:42

Note: I first intended to make this a comment under Paul's answer, but it grew too big.

Does having one larger L1 cache instead of L1 and L2 cache... Also will this make the CPU more expensive to make?

The major issue is that just modifying the size of the L1 cache and removing the L2 one is no more a realistic design, it isn't just the costs of design and fabrication which will change but other design choices will be impacted. You probably can't make only that change, whatever the money you are able to spend out (increasing the size of the L1 cache without slowing it down or switching away from being fully associative is probably not an option, if it was possible it would probably have been done), you have thus now an unbalanced design, and it is not fair to compare a balanced design to an unbalanced one. As an example to show the effect on the design: how do we take the slower cache into account?

• slow down the clock? The pipeline can now and probably should be split differently.

• let the cache access take more pipeline cycles? Again, this means that a redesign of the pipeline is probably in order.

...makes computation faster?

Again lots of things missing here. Which computation? And when the computation (or workload which is a set of computations which may have different characteristics) is fixed, what other changes do you accept? A fixed program or do you accept changes (even algorithmic changes) to take into account the new size of caches? In the past I've preferred $O(N \log N)$ algorithms to $O(N)$ one because the $O(N)$ one had a working set too big for the cache, your change could speed it the computation if I change the algorithm, but not if I don't.