For a bigram language model, I can calculate the perplexity of sentences of a test document. However, I'm not sure what would be the perplexity of the whole document. Should I get the average of the perplexities of sentences?

  • $\begingroup$ It’s up to you. You get to define the perplexity of a document as you please. $\endgroup$ – Yuval Filmus Mar 6 '19 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.