I'm looking to do natural language parsing and looking for how the CFG (or CSG) should be defined for English.

Surely one'd expect to find one from the internet already, but do you know where?

It needs to be in "formal grammar" form, not an implementation (e.g. a Python library).

It has to allow for as wide range of sentences as possible to be parsed correctly.

Something that I could be looking at:

S -> NP VP | VP
NP -> D N | N | N NP
VP -> V | V NP | V PP | V NP PP
PP -> P NP

But how sufficient is this?


  • 2
    $\begingroup$ You can find natural language process toolkit here: nltk.org - For Python. If I recall correctly, natural language (English) is far from context-free (observe the difference between "a trick" and "an element"), so I wouldn't expect a downloadable CFG. $\endgroup$ – DCTLib Dec 25 '15 at 13:16
  • $\begingroup$ @DCTLib No I'm looking for a CFG since what I'm doing is implementing what NLTK is doing, but for a more particular use and smaller scale. $\endgroup$ – mavavilj Dec 25 '15 at 13:18
  • 3
    $\begingroup$ So why do you expect such a CFG to exist? As already written, natural language is not exactly context-free. $\endgroup$ – DCTLib Dec 25 '15 at 13:19
  • $\begingroup$ @DCTLib Then what's NLTK using for parsing? $\endgroup$ – mavavilj Dec 25 '15 at 13:21
  • 1
    $\begingroup$ Nltk has a number of parser avaliable, some of which are context free, and some are more advanced. But in any case it's not comprehensive, it will certainly make mistakes. $\endgroup$ – jmite Dec 25 '15 at 18:24

It is uncommon to use a "closed set" context free grammar because the number of production rules is staggeringly large. The typical approach is to recover a probabilistic context free grammar from a treebank (e.g., the Penn Treebank) in order to do statistical parsing (via probabilistic CKY, Earley, etc).

The only time you might want a "closed set" is when working with toy data in which case you may define an overly simplistic subset of English grammar dealing with sentences, noun phrases, verb phrases, and so on. Such miniature grammars can be found in standard NLP texts such as Jurafsky and Martin, and Manning and Schuetz.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.