I've been reading about how the kernel manages memory in x86 from this popular blog post(https://manybutfinite.com/post/how-the-kernel-manages-your-memory/) and had a few questions.

There is a part where the author describes page table entries and their associated fields. To quote "This lack of a PTE no-execute flag illustrates a broader fact: permission flags in a VMA may or may not translate cleanly into hardware protection. The kernel does what it can, but ultimately the architecture limits what is possible.".

I looked into the kernel source and found that the NX bit in modern systems is actually the most significant bit. Is this because modern hardware now supports NX?

If I tried to execute code on the stack with NX enabled, I crash, but what actually happens? Does the kernel check the page permissions on every execute? That seems really slow. Is it the hardware that knows if I can/cannot execute? If so, how does the hardware know. Does this have anything to do with the CS segment selector?


1 Answer 1


No, the kernel does not check the page permissions on every execute. If the CPU supports the NX bit, it checks that bit.

Exactly what the CPU does with it and when probably varies between CPU manufacturers and vendors.

Modern CPUs implement a split TLB, one for instructions and one for data. The Core i7, for example, has (ignoring 2MB and 1GB pages for the moment):

  • L1 data TLB: 4-way set associative, 64 entries
  • L1 instruction TLB: 8-way set associative, 64 entries
  • L2 unified TLB: 6-way associative, 1536 entries

I'm not a CPU designer, so this could be wrong, but if it were me, I would check the NX bit when inserting the page table entry into the L1 instruction TLB.

There are a few things you need to consider; for example, the TLB load might be caused by speculative execution, so an exception shouldn't be raised unless instructions on that page are actually executed. But this is nothing special; it's exactly the same behaviour as you would expect if the instruction execution would cause a page fault.

There are architectures which manage TLBs in software or which allow for that possibility, most notably MIPS, Itanium, and some variants of SPARC V9. On such a platform, it is perfectly feasible for the kernel to perform the equivalent of the NX check on every instruction TLB miss.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.