# Understanding the geometrical interpretation of word2vec

I'm trying to understand how the $word2vec$ method actually nudges word vectors of similar semantic/syntactic content closer together in the word vector space.

I've read here (Quora answer) that it's because $word2vec$ tries to maximise the probability of target word $w$ given context $c$, and that this probability is roughly inversely proportional to the distance between $w$ and $c$ in the word vector space. I don't really see why the latter is the case though.

• After accepting I realised I'm still a bit confused: your explanation shows "Fuji" and "Gala" should have similar vectors, but not necessarily that "I", "really", or "love" should be closer to "Fuji" and "Gala", than to other words. I'm guessing it's because I don't exactly see how to connect $P(w|c)$ to the inner product between $w$ and $c$. – Timsey Sep 6 '17 at 15:18
• Hm, rereading the original paper, it seems that that's not actually true after all. What really happens is that maximising $P(c|w)$ (skipgram) forces the word vector $v_w$ of word $w$ embedded from input to hidden, and $v'_c$ the word vector of context $c$ embedded from hidden to output. Since these are not the same embeddings, the word and context vectors aren't necessarily close together in a single embedding. Clearly there is some relation between embeddings, but it's not obvious (I think) how that manifests itself in either vector space. – Timsey Sep 6 '17 at 16:19