The trouble I have is: aren't all variables (be it primitives like int or composite objects) already represented by a sequence of bytes?
Yes, they are. The problem here is the layout of those bytes. A simple
int can be 2, 4 or 8 bits long. It can be in big or small endian. It can be unsigned, signed with 1's complement or even in some super exotic bit coding like negabinary.
If you just dump the
int binarily from memory, and call it "serialized", you have to attach pretty much entire computer, operating system and your program for it to be deserializable. Or at least, a precise description of them.
So what makes serialization such a deep topic? To serialize a variable, can't we just take these bytes in memory, and write them to a file? What intricacies have I missed?
Serialization of a simple object is pretty much writing it down according to some rules. Those rules are plenty and not always obvious. Eg an
xs:integer in XML is written in base-10. Not base-16, not base-9, but 10. It's not a hidden assumption, it's an actual rule. And such rules make serialization a serialization. Because, pretty much, there are no rules about bit layout of your program in memory.
That was just a tip of an iceberg. Let's take an example of a sequence of those simplest primitives: a C
struct. You could think that
has a defined memory layout on a given computer+OS? Well, it does not. Depending on current
#pragma pack setting, the compiler will pad the fields. On default settings of 32-bit compilation, both
shorts will be padded to 4 bytes so the
struct will actually have 3 fields of 4 bytes in memory. So now, you not only have to specify that
short is 16 bits long, it's an integer, written in 1's complement negative, big or little endian. You also have to write down the structure packing setting your program was compiled with.
That's pretty much what serialization is about: making a set of rules, and sticking to them.
Those rules can be then expanded to accept even more sophisticated structures (like variable length lists or nonlinear data), added features like human readability, versioning, backward compatibility and error correction, etc. But even writing down a single
int is already complicated enough if you only want to make sure you'll be able to read it back reliably.