1
$\begingroup$

I'm trying to write a context-free grammar for a comma-separated list of statements with comments. It is trivial if the comma appears at the end of a statement. It is trivial if the comment is ignored. But I don't know how to include a comma in the middle of a statement (before the comment).

This Python / Lark CFG:

start: def+

def: "{" statement ("," statement)* "}"

!statement: NAME "=" (
    | "Foo"
    | "Bar" "(" NAME ")"
    | "Tomato"
    | )

NAME: /[A-Z][a-z-]{0,15}/

COMMENT: "//" /.*/

%import common.WS
%ignore WS

parses this text:

{
    Mike = Foo,
    Fred = Bar(Chocolate),
    Joe = Tomato
}

producing this AST:

start
  def
    statement
      Mike
      =
      Foo
    statement
      Fred
      =
      Bar
      (
      Chocolate
      )
    statement
      Joe
      =
      Tomato

I want to parse the content of comments into the AST, not just ignore it:

{
    Mike = Foo,
    Fred = Bar(Chocolate),      // Yummy!
    Joe = Tomato
}

======= EDIT =========

The problem from which the question was distilled requires an optional description tied to each assignment. Arbitrary comments (such as on the lines with braces) aren't needed. So Yuval's answer (with the addition of a trailing comment) works like a charm. For completeness, the working grammar is:

start: def+
def: "{" statement ("," COMMENT? statement)* COMMENT? "}"

statement: NAME "=" (foo | bar | tomato)
foo:    "Foo"
bar:    "Bar" "(" NAME ")"
tomato: "Tomato"

and produces the AST:

start
  def
    statement
      Mike
      foo
    // Stuff
    statement
      Fred
      bar   Chocolate
    // Yummy!
    statement
      Joe
      tomato
    // aaaargh
$\endgroup$
  • $\begingroup$ This sounds like a coding issue more than a CFG issue $\endgroup$ – D. Ben Knoble May 20 at 17:17
  • $\begingroup$ Comments are usually handled differently – they are simply removed by the lexical analyzer. $\endgroup$ – Yuval Filmus May 20 at 17:36
  • $\begingroup$ Yes, as acknowledged in the question. I'm not asking what is usual, I'm asking whether the unusual is possible (within some threshold of additional complexity/usability/understandability). $\endgroup$ – Dave May 20 at 17:42
  • 1
    $\begingroup$ @D.BenKnoble: Using any CFG notation you like, parse the text at the bottom into an AST that includes the comment. The question has nothing to do with coding, even though a code example is shown for concreteness. $\endgroup$ – Dave May 20 at 19:53
1
$\begingroup$

Define a non-terminal COMMENT which generates comments (in your case, // followed by arbitrary non-newline characters and then newline), as well as the empty string. In the definition of def, add COMMENT after ,.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.