The answer is actually none. This is not what your teacher expects in this homework assignment, but it's true.
The problem in process isolation is to protect against processes that try to access resources that they should not (resources that are used by other programs or by the operating system itself). Resources include memory and peripheral devices.
You can design an operating system that verifies all programs before running them, and only runs a program if it can prove to its satisfaction that the program will never attempt to access a resource that it is not allowed to access. Such operating systems are not very common, but they exist, for example some smartcards running Java Card work like this: only Java bytecode applications can be installed on the card, and they have to have been checked and passed by a bytecode verifier.
That being said, most operating systems enforce process isolation by leveraging hardware means. The most important part is the memory protection unit: a hardware component that assigns privilege levels to blocks of memory and ensures that only programs with the appropriate privilege level can access each block. Often memory protection is performed by a memory management unit which combines protection with virtual memory management, but the indirection afforded by virtual memory is not necessary for the protection aspect. There are processors with an MPU but no MMU. The uClinux project is an example of operating system which enforces process isolation and runs on processors with only an MPU.
In order to be of any use, the MPU has to be combined with some mechanism that assigns privilege levels to programs. Most CPU types that allow privilege isolation have at least two modes (sometimes more), which I'll call kernel and user, with kernel being the most privileged. At any time, the CPU is in one mode or the other. The CPU boots in kernel mode. Switching into kernel mode from user mode always causes a jump to a fixed address. That way, the initial program that is loaded at boot time remains in control of what can be executed in kernel mode. It is up to that program to set up the MPU/MMU so that other programs cannot overwrite kernel code or access peripherals directly, thus enforcing isolation between kernel code and user code. The kernel is also responsible for changing MPU/MMU settings when switching between programs so that programs can only access their own memory and not other programs'.
That's two mechanism. I don't know what your teacher had in mind for the third one.