First the term MMU refers to many different thing (more on later below). For now let's defined MMU as "virtual to physical address translation".
The caches L1 and L2 are always managed in hardware, however note that for proper operation the OS still has to interact with the cache on some occasions (notably when using DMA). An important point to be aware of is that depending on whether the caches are virtually or physically indexed, they have a different relationship with the MMU. Indeed a virtually indexed cache (typically L1) will usually be accessed by the CPU without going through the MMU and it will not only contain the data but also the security bits associated with the MMU (privilege bit, read/write, cacheability, etc). On the other hand, a physically indexed cache must come after the address translation and does not care about the MMU at all. This also applies to higher levels (L3 and L4) which are always physically tagged (and sometimes outside the CPU die).
Thus a typical CPU will have the core with a memory bus connected to the L1 (which is typically split between data and instruction, but this is not important here). On cache miss, the L1 will issue read/write on a bus that goes through the MMU then L2, L3, L4 and finally DRAM.
The MMU itself may consist of different things. The simplest forms of MMU are just segmentation and PMU (protection memory unit). More advanced forms of MMU include a TLB so as to support paging. The TLB itself is always a hardware component (essentially just another cache). However there are two very different ways of managing the TLB:
- in hardware: the MMU has a "page table walker" that can issue read to the memory to load TLB entries on a miss (this includes x86, ARM, etc)
- in software: the MMU triggers a CPU exception on a TLB miss, and then the OS does the page table walk in software, fills the TLB with the entry and return from the exception (this includes MIPS)