Why can't a compiler create universal N-bit code (e.g. 32-bit, 64-bit, etc.)

Question

It is my understanding from what I have read that you need to compile separate executables for 32 bit and 64 bit machines. So if I compile a C program for example I need to compile 2 different binaries for 32-bit and 64-bit machines.

However, it seems like intuitively a compiler should just be able to compile down to instructions that say "allocate N bits of space" and similar for other related instructions rather than "give me 32-bits of space" or "give me 64 bits of space"?

Why is this not the case (looking for some intuition at a high level)?

Answer 1

The instruction set for a 32-bit processor is different from the instruction set for a 64-bit processor. The 64-bit instructions can't be executed on a 32-bit processor. So, the program really does need to be compiled to something different: if you take a program compiled for a 64-bit processor, and try to run it on a 32-bit processor, the processor won't run it, because it doesn't understand the new 64-bit instructions.

Answer 2

However, it seems like intuitively a compiler should just be able to compile down to instructions that say "allocate N bits of space" and similar for other related instructions rather than "give me 32-bits of space" or "give me 64 bits of space"?

Only if those instructions exist, and that's the decision of the instruction set designers rather than the compiler designers. Besides, the width of the instructions varies between instruction sets, as do things like the number of registers (very important for an optimising compiler).

In principle it would be possible to write some clever code which executes correctly in two different instruction sets and jumps to different places. You could then use this as a kind of "loader", placing it at the start of the executable to pack a 32-bit and a 64-bit version into the same file. However, then you run up against a problem which is not easily worked around, which may be considered the "real" answer, although perhaps not at the level of abstraction you want.

Executables have headers with metadata, which includes a flag for the instruction set. For example, the Windows Portable Executable format has a COFF header which requires you to choose one of a list of machine types:

0x0000 IMAGE_FILE_MACHINE_UNKNOWN
0x01d3 IMAGE_FILE_MACHINE_AM33
0x8664 IMAGE_FILE_MACHINE_AMD64
0x01c0 IMAGE_FILE_MACHINE_ARM
0xaa64 IMAGE_FILE_MACHINE_ARM64
0x01c4 IMAGE_FILE_MACHINE_ARMNT
0x0ebc IMAGE_FILE_MACHINE_EBC
0x014c IMAGE_FILE_MACHINE_I386
0x0200 IMAGE_FILE_MACHINE_IA64
0x9041 IMAGE_FILE_MACHINE_M32R
0x0266 IMAGE_FILE_MACHINE_MIPS16
0x0366 IMAGE_FILE_MACHINE_MIPSFPU
0x0466 IMAGE_FILE_MACHINE_MIPSFPU16
0x01f0 IMAGE_FILE_MACHINE_POWERPC
0x01f1 IMAGE_FILE_MACHINE_POWERPCFP
0x0166 IMAGE_FILE_MACHINE_R4000
0x5032 IMAGE_FILE_MACHINE_RISCV32
0x5064 IMAGE_FILE_MACHINE_RISCV64
0x5128 IMAGE_FILE_MACHINE_RISCV128
0x01a2 IMAGE_FILE_MACHINE_SH3
0x01a3 IMAGE_FILE_MACHINE_SH3DSP
0x01a6 IMAGE_FILE_MACHINE_SH4
0x01a8 IMAGE_FILE_MACHINE_SH5
0x01c2 IMAGE_FILE_MACHINE_THUMB
0x0169 IMAGE_FILE_MACHINE_WCEMIPSV2

I suppose you might be able to get away with using a clever loader if you choose machine type 0x0000, but I wouldn't count on Windows agreeing to execute your program. And technically you really ought to implement support for every CPU on the list if you do that.