# Why word embeddings are compared with cosine distance and not euclidean?

In most articles that compare word embeddings they use cosine distance to determine if words are similar. Why?

I guess that euclidean distance should work too. So, my question is: it doesn't? And why cosine distance doesn't fail?

• The inner product is how you measure correlation in a vector space. Dec 16, 2021 at 23:45

• Apply the linear transformation to the embeddings: $$v \to Av$$
• Predict the class by selecting the maximum coordinate: $$arg\max_i (Av)_i$$
In other words, you have vectors $$a_1, \ldots, a_n$$, where $$n$$ is the number of classes, and you select the max of $$\langle a_1, v \rangle, \ldots, \langle a_n, v \rangle$$, where $$v$$ is the embedding vector. Comparing two different products: $$\langle a_i, v \rangle > \langle a_j, v \rangle \iff \|a_i\| \cos(a_i,v) > \|a_j\| \cos(a_j,v)$$ Since $$\|a_i\|$$ and $$\|a_j\|$$ are fixed, the only things that actually depend on $$v$$ are $$\cos(a_i,v)$$ and $$\cos(a_j,v)$$. While it doesn't directly explain why $$\cos(v_i,v_j)$$ is a good similarity measure, it at least makes it look more natural than distances.
Both metrics are very closely connected. Let $$v,w$$ be two unit-length vectors (i.e., $$\|v\|=\|w\|=1$$), and let $$d(v,w)=1-v \cdot w$$ be the cosine distance between them. Then the Euclidean distance $$\|v-w\|$$ satisfies $$\|v-w\|^2 = 2 d(v,w).$$ In other words, if your embedding vectors are unit-length, there's effectively no meaningful difference between the two distance measures.