# Why use maximum likelihood for word prediction?

According to https://www.tensorflow.org/tutorials/word2vec/, the standard approach for predicting the next word in a word sequence is maximum likelihood. The predicted next word is the word that maximizes

$$P(w_t|h)=\frac{\exp(\text{score}(w_t,h))}{\sum_{w'}\exp(\text{score}(w',h))}.$$

To me, it seems that the word predicted from this model is always the word with the maximum score, in which case we could just use the score function for prediction.

What does maximum likelihood provide that a simple score function does not?

That approach is using a score function (in a particular way). It is also using maximum likelihood. Both ways of describing the approach are valid.

In contrast, the maximum-likelihood approach gives us a principled way to choose the function. Here is how to derive that formula, given the maximum-likelihood principle. First, they have apparently decided to use the following approximation for the probability of seeing $w_t$ as the next word, given history $h$:
$$P(w_t,h) = \exp(\text{score}(w_t,h)).$$
\begin{align*} P(w_t|h) &= \frac{P(w_t,h)}{P(h)}\\ &= \frac{P(w_t,h)}{\sum_{w'} P(w',h)}\\ &= \frac{\exp(\text{score}(w_t,h))}{\sum_{w'}\exp(\text{score}(w',h))}. \end{align*}
• So in other words, the score function should be one that makes $P(w_t,h)=exp(score(wt,h))$ a valid approximation? – npCompleteNoob Jan 2 '17 at 16:55