I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-forward neural language model with one linear hidden layer and a softmax activation. What is the purpose of the Kneser-Ney in such a neural network, and how can it be used for learning the conditional probability for the next word?


Authors mention that we may obtain different mulitple labels from n-1 words integrating Kneser-Ney smoothing. It is not clear for me what these labels are. As I know, the output of a LM could be the next word. How is it possible to get more labels with the Kneser-Ney smoothing ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.