I'm wondering if there is a standard corpus that people use to evaluate their part-of-speech taggers. I am implementing my own tagger, and would like a meaningful way to measure its performance, which would ideally be against the same corpuses that other people are using.

  • $\begingroup$ Have you tried doing a literature search to look for published papers on part-of-speech tagging, and see what they did in their evaluation section? $\endgroup$ – D.W. Sep 23 '15 at 5:02
  • $\begingroup$ @D.W. A lot of the information I could find was about the tag sets people used, but not about training texts. $\endgroup$ – TylerAndFriends Sep 23 '15 at 5:09
  • $\begingroup$ This is not exactly what you are looking for, but may be helpful nevertheless: link.cs.cmu.edu/link/batch.html the list of all sentences properly parsed by link-grammar's parser. You could also ask on their mailing list for the corpus they use for testing (afaik it was archives of some American periodic). $\endgroup$ – wvxvw Sep 23 '15 at 21:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.