k-nearest neighbors (Euclidean distance): How to process multiple attributes?

Given this sample:

$$\begin{array}{|c|c|c|} \hline X1 & X2 (Kg/m^2) & \text{Class} \\ \hline 8 & 4 & A \\ \hline 4 & 5 & B \\ \hline 4 & 6 & B \\ \hline 7 & 7 & A \\ \hline 5 & 6 & B \\ \hline 6 & 5 & A \\ \hline \end{array}$$ we then calculate each of Euclidean distance values to the row:

with P = (7, 4)

$$\begin{array}{|c|c|c|} \hline X1 & X2 (Kg/m^2) & \text{Square distance to query point P} \\ \hline 8 & 4 & (8 - 7)^2 + (4 - 4)^2 = 1 \\ \hline 4 & 5 & (4 - 7)^2 + (5 - 4)^2 = 10 \\ \hline 4 & 6 & (4 - 7)^2 + (6 - 4)^2 = 13 \\ \hline 7 & 7 & (7 - 7)^2 + (7 - 4)^2 = 9 \\ \hline 5 & 6 & (5 - 7)^2 + (6 - 4)^2 = 8 \\ \hline 6 & 5 & (6 - 7)^2 + (5 - 4)^2 = 2 \\ \hline \end{array}$$

But what if each class has more than one attribute , for example: $$\begin{array}{|c|c|c|c|} \hline X1 & X2 (Kg/m^2) & \color{red}{X3} & \text{Square distance to query point P} \\ \hline 8 & 4 & \color{red}{2} & (8 - 7)^2 + (4 - 4)^2 = 1 \\ \hline 4 & 5 & \color{red}{5} & (4 - 7)^2 + (5 - 4)^2 = 10 \\ \hline 4 & 6 & \color{red}{1} & (4 - 7)^2 + (6 - 4)^2 = 13 \\ \hline 7 & 7 & \color{red}{4} & (7 - 7)^2 + (7 - 4)^2 = 9 \\ \hline 5 & 6 & \color{red}{6} & (5 - 7)^2 + (6 - 4)^2 = 8 \\ \hline 6 & 5 & \color{red}{2} & (6 - 7)^2 + (5 - 4)^2 = 2 \\ \hline \end{array}$$

How to calculate the Euclidean distance in this case?

1 Answer

Straight from definition:

$\sqrt{(q_1-p_1)^2 + (q_2-p_2)^2 + \cdots + (q_n-p_n)^2}$

In your particular case $n = 3$, so the query should also be 3D (e.g. {7, 4, 3}.

$\sqrt{(q_1-p_1)^2 + (q_2-p_2)^2 + (q_3-p_3)^2}$

So plugging in the third dimension: $$\begin{array}{|c|c|c|l|} \hline X1 & X2 (Kg/m^2) & X3 & \text{Square distance to query point {7, 4, 3}} \\ \hline 8 & 4 & 2 & (8 - 7)^2 + (4 - 4)^2 + (2 - 3)^2= 2 \\ \hline 4 & 5 & 5 & (4 - 7)^2 + (5 - 4)^2 + (5 - 3)^2= 14 \\ \hline 4 & 6 & 1 & (4 - 7)^2 + (6 - 4)^2 + (1 - 3)^2= 17 \\ \hline 7 & 7 & 4 & (7 - 7)^2 + (7 - 4)^2 + (4 - 3)^2= 10 \\ \hline 5 & 6 & 6 & (5 - 7)^2 + (6 - 4)^2 + (6 - 3)^2= 17 \\ \hline 6 & 5 & 2 & (6 - 7)^2 + (5 - 4)^2 + (2 - 3)^2= 3 \\ \hline \end{array}$$

• HI, may be I'm misunderstanding this , so in this case I would have (x, y ,z ) as input right? and not (x,y), right? – Plain_Dude_Sleeping_Alone Oct 3 '16 at 20:49
• Yes, exactly, {x, y, z}, so all distances are calculated on all dimensions, so given query point also. – Evil Oct 3 '16 at 20:51
• My neptune, I'm reading all unnecessary advanced papers, thank you some much!!! – Plain_Dude_Sleeping_Alone Oct 3 '16 at 20:53