0
$\begingroup$

I'd like to know how a computer can determine the beginning and end of certain file components (attributes, headers, frame/packet/segment headers etc.) when these components can be omitted or added in optionally, on top of being of variable length, and how it can even recognise files and protocol packets within an ocean of 1s and 0s at all.

Part of this questions stems from a theoretical about how you'd go about reading binary data from a storage medium that has been physically broken into multiple pieces. If, say, there was a text file stored within a broken piece, and if a computer could physically read the binary data successfully, could it find the start+end of this text file then read its contents?

I read over the answers provided in this question and this question, which (from my understanding) explains that data is stored in "blocks"/"sectors"/"clusters" of a minimum size based on the file system, and that the OS needs to know the sector associated with the data it needs (to pull up a file, load a program etc.), and the storage medium controller needs to know where this sector is physically loaded in order to read it and send it up the protocol stack. If there was only one file in each "sector" this would make sense, the OS would just read the sector from start to finish to read the file, and the remaining space left in the sector would be wasted space (all 0s) - the intuitive sense breaks down for me if there are multiple files per sector, though.

The other part of this question stems from the fact that an Ethernet frames (and other protocols, for that matter, I just have a background in Network Engineering, hence why I chose Ethernet for this question) can have varying header lengths - between 14 bytes at base, 18 if a single 802.1Q tag is used, and up to 22 bytes if 802.1ad tagging is used. Including or omitting certain optional headers actually changes the amount of bits a computer has to read when receiving the Ethernet frame (it changes the length of the frame header) - how would a computer know when to begin reading the Payload after the Source MAC address ends, if the 802.1Q tag between these two fields is optional AND of variable length?

If this question should be asked as two separate questions, and/or in different Stack Exchange Sites, please let me know.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ When fields have a fixed length/position, the knowledge of the length/position is embedded in the reading application. When they have variable length (including 0), this information is explicitly given, for instance in a field descriptor in a separate table or before the field itself. As there are zillion formats, only a general answer can be given. $\endgroup$
    – user16034
    Commented Oct 21, 2022 at 21:02
  • $\begingroup$ Thank you. So in the broken media example, would the computer/parser be looking for a certain value (Magic Number) based on the file it was told to look for and go "Ah-hah, text files in this format always start and end with these values, this must be a text file". But could it only do that if it knew where the start and ends of the sectors/blocks were physically located on the media (I'm assuming the start of a file is always at the start of a set of blocks)? Otherwise it could be reading random payload data and not protocol/file headers and potentially open a "corrupted" file? $\endgroup$
    – Inquisitive
    Commented Oct 22, 2022 at 2:58
  • $\begingroup$ It is a matter of common sense. There are file descriptors telling where the files reside. The descriptors are collected in a table. There is somewhere information that tells where the table resides. And so on. $\endgroup$
    – user16034
    Commented Oct 22, 2022 at 14:23

1 Answer 1

Reset to default
2
$\begingroup$

If you click on the link you provided you'll see that the word 802.1Q is itself a link and when you read that page you'll see that the field uses a special value, a magic number if you like for the first 16 bits.

Tag protocol identifier (TPID) A 16-bit field set to a value of 0x8100[b] in order to identify the frame as an IEEE 802.1Q-tagged frame. This field is located at the same position as the EtherType field in untagged frames, and is thus used to distinguish the frame from untagged frames.

So if the parser encounters this value it knows that it's an 801.1Q header, and if not it interprets the value as an EtherType/Size. Presumably that particular value is not a valid EtherType/Size.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for bringing my attention back to that. I found this section en.m.wikipedia.org/wiki/EtherType#Values that details the expected values. So basically, if the parser reads "0x0800" in the first 16 bits after the Source MAC, it'll go "Okay, the Ethertype is IPv4, I can start reading the Payload field now". But if the parser sees "x8100" it'll go "Okay, there's a 16 bit TCI field after this, THEN the Ethertype, THEN the Payload". Am I understanding that correctly? And assuming that the most significant bytes and bits are parsed/read first (eg. "left to right")? $\endgroup$
    – Inquisitive
    Commented Oct 22, 2022 at 2:23
  • $\begingroup$ And to clarify, I understand the transmission process "A data packet on the wire and the frame as its payload consist of binary data. Ethernet transmits data with the most-significant octet (byte) first; within each octet, however, the least-significant bit is transmitted first." I'm just trying to figure out the order in which computers read the bits and bytes once they're written to memory. $\endgroup$
    – Inquisitive
    Commented Oct 22, 2022 at 2:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.