1
$\begingroup$

I'm doing a research on NLG systems. I need to annotate my corpus (~6 million words) automatically. My algorithm works well and I want to calculate Cohen's Kappa. What I cannot understand is the second annotator; I don't have the possibility to have another annotation. Is there any solution for this problem? Can I have a set of data as my sample and manually doing the annotation to calculate Cohen's Kappa? Is there any measurement for corpus analysis?

Thanks

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Welcome to CS.SE! What's a NLG system? (It might help to spell out the acronym.) Annotate your corpus with what? Why do you want to compute Cohen's kappa? Cohen's kappa is only relevant if you have two independent annotations; if you don't, then it's irrelevant, so I'm not sure why you want to compute it. What are you trying to achieve? What's your real end goal? $\endgroup$
    – D.W. ♦
    Aug 28, 2016 at 3:52
  • $\begingroup$ Welcome to Computer Science! The title you have chosen is not well suited to representing your question. Please take some time to improve it; we have collected some advice here. Thank you! $\endgroup$
    – Raphael ♦
    Aug 28, 2016 at 9:20

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.