All Questions

Filter by
Sorted by
Tagged with
1
vote
0answers
24 views

CYK algorithm - how to handle unknown terminals given in a sentence to parse?

There is a given treebank which we derive the Probabilistic context free grammar. I wonder how do one handles with a given sentence which includes terminals that don't exist in the derived rules? Is ...
3
votes
1answer
1k views

White-space and comment by lexical analyzer

Lexical Analyzer mostly deletes comments and white-spaces. One example where I think Lexical Analyzer might not be discarding white-spaces is in Python language, as indentation has a important role in ...
4
votes
2answers
492 views

What is the point of delimiters and whitespace handling

I see that language specifies reserved words, delimiters and whitespaces in the lexer section. Are delimiters just like reserved identifiers in the punctuation space or they have additioinal function ...
0
votes
3answers
2k views

What data is stored in the symbol table for a number token?

I'm reading the Dragon Book. The following is from the start of Section 3.1.3. When more than one lexeme can match a pattern, the lexical analyzer must provide the subsequent compiler phases ...