Can a running interrupt handler be pre-empted by another interrupt handler? If this is possible, in which scenarios is this safe, and in which scenarios is it not? If this is not possible, why?
This depends on both the processor architecture and the kernel architecture.
Generally speaking, interrupt handlers start with interrupts disabled. This is necessary so that the interrupt handler has room for storage — at a minimum, the interrupt handler needs the exclusive use of a few registers, including the program counter, which the processor must save somewhere (in a set of banked registers).
On some processors, it's possible for an interrupt to be interrupted by another higher-priority interrupt. To give one example that I'm familiar with, on ARM processors, there are two interrupt levels: IRQ (normal interrupts) and FIQ (fast interrupts). Each has their own register bank for the program counter and stack pointer, and their own interrupt vector. An IRQ handler starts with IRQ disabled but FIQ enabled; an FIQ handler starts with both IRQ and FIQ disabled. Thus an FIQ can be triggered in the middle of handling an IRQ.
It's up to the operating system to decide when an interrupt handler will reenable interrupts of the same priority, or interrupts of a different type with the same priority. The longer interrupts are blocked, the longer the latency to handle an interrupt that comes soon after another. In particular, a real-time OS typically needs to reenable interrupts very fast. To give a couple of examples:
- FreeBSD's interrupt handlers can be preempted when they are performing a blocking operation (waiting for a lock).
- Linux encourages splitting the task of an interrupt handler into two parts: the top half is the actual interrupt handler, and runs with interrupts disabled; the bottom half runs with normal preemption for as long as it needs. Typically the top half exchanges data between hardware peripherals and a preallocated memory buffer, then the bottom half handles all the rest such as communicating with other kernel and userland threads, initiating new hardware I/O, etc.
At the point where an interrupt handler unmasks interrupts, it must be ready to cope: it must not be using any resource that another interrupt handler would thrash. Furthermore, since the interrupt handler hasn't finished its job, there must be some mechanism for the program counter to be saved somewhere; the mechanics of this depends a lot on both the processor and the kernel architecture.