Every formal language is a subset of E*.
Let's say this formal language is python. If a program is syntactically correct, then the Python Automata accepts the "word", which is the program. If that is true, what are the atomic elements like "print" called?
Is this correct? Please correct my thought on this.
I think that your intuition is correct but the exact details of its explanation are fuzzy. Every syntactically correct program is a "word" or a "sentence" (depends on what you prefer) of some language $\mathcal{L}$.
In the context of lexer-full parsing, the "automaton" (intended as the program that prints "syntax error" in case of error) that accepts the language is composed of 2 distinct automata that cooperate to decide whether $\mathcal{p}\in\mathcal{L}$. They are the lexer and the parser.
The former is responsible for accepting the sequence of keywords and basic constructs of the language (int, for, while, print, +, -, numbers,...).
The latter is responsible for checking that sequences of non-terminal and terminal symbols (keywords and their sequences) form correct sentences.
Namely, the lexer is able to tell that "for" is a word of the language, but is unable to tell that "for while () }}}" makes no sense. That's the purpose of the parser.
Since both automata (lexer and parser) are accepting a language, all the sequences of characters or symbols accepted are words of their languages.
In general, informally, the lexer accepts words, while the parser accepts sentences. This is simply a handy way to distinguish sequences of characters from sequences of symbols read by the lexer. Still, there's no strict nomenclature.
int for the lexer these are three symbols. For the parser it already acts as a single symbol as it was decided by the lexer that this sequence represents a keyword. Summarizing, the alfabet changes.
$\endgroup$
Commented
Aug 24, 2021 at 23:00