# How do I model line comments in a CFG?

Assume we want to define a context free grammar of say a programming language, where on each line everything after the character # until the end of line is considered a comment and should be ignored. How to express that in a context free grammar?

Without multi-line statements, it's rather simple; assuming line is the non-terminal for well-formed line, change its right-hand-side appearances to

line (# .*)?


(borrowing regular expression syntax for brevity).

Otherwise, if you explicitly handle line breaks, replace occurrences of the line-break token CRLF (or so) similarly by

(# .*)? CRLF


If line breaks remain implicit, one chance you have left is make it part of the lexer; don't only skip whitepace, but also (# .*)? [\r\n]. You can also create a token COMMENT for that and place it in all places of the grammar where it may appear. This strategy doesn't work for nested multiline comments, though, a common problem in older languages.

• thanks, sorry but I don't fully understand the syntax you're using. also I have to do it all using a context free grammar, even if I rely on a lexer, the lexer has to be expressed as a cfg as well. and unfortunately I have to support multiline statements that may contain comments each, just like // in C (btw I see that C lexer also had to do it using some kind of "hack" as in here lysator.liu.se/c/ANSI-C-grammar-l.html ) – Troy McClure Apr 13 at 9:17
• @TroyMcClure Are you a student facing an exercise task, or are you a practitioner traying to build a compiler? – Raphael Apr 13 at 17:40
• The first two approaches should be workable given the setting you describe. Note that, in practice, such issues are probably often taken care of by some preprocessing -- it's trivial to run a tool over a a set of files that removes all comment suffices from lines (unless your language has stuff like multi-line strings, you are interested in extracting the content of some comments, or any number of other complications, of course). Then you don't have to handle them in your grammar. – Raphael Apr 13 at 17:44
• @troy: the link you provide a quote is not "the C lexer", but rather a lexer put together by someone, not as part of any C compiler. Furthermore, it does not recognise line comments, which as Raphael says, are trivial, but rather C's multiline comment syntax. (This can also be represented as a regular expression, and you should be able to find a correct one easily enough a or with a little more thought write one.) The hack in the code snippet was probably pragmatic with lex, but flex works much better with regular expressions and any serious C implementation would use one. – rici Apr 13 at 19:49
• @Troy: anyway, you "model" a comment just like you model whitespace. In almost all languages, they're syntactically and semantically identical. – rici Apr 13 at 21:38

Can just treat comments as whitespaces.