Computer Science Stack Exchange is a question and answer site for students, researchers and practitioners of computer science. It only takes a minute to sign up.
Sign up to join this community
I am reading Programming Language Pragmatics by Michael Scott. He says that on a first pass, a compiler will break a program into a series of tokens. He says that it will check for malformed tokens, like 123abc or $@foo (in C).
What is a malformed token? A variable that does not meet the rules of variable-naming? An operator that does not exist (ex. "<-")?
Is this analogous to a misspelled word?
This question came from our site for theoretical computer scientists and researchers in related fields.
The basic syntactic units in any textual programming language are tokens, which are, if you wish, the words in the programming language. The compiler parses these tokens to build sentences and so forth.
A malformed token is a string of characters that is not a valid word for the programming language.
It is not a variable that doesn't match the naming rules – not all tokens are variables. It is not an operator that is unknown – sometimes operators can be defined by the programmer. It is one of the smallest pieces of syntax that will never correspond to something sensible.
