$X_m$ and $Y_m$ are not shaded, because each of them *is* potentially the median of $A \cup B$. 

Consider two examples (Below I take the $\lfloor (n+1)/2 \rfloor$-th element as the median of an array of size $n$):

(1) $A = \{ 2, 4, 6 \}$ and $B = \{ 3, 5, 7 \}$.

(2) $A = \{ 2, 4, 6 , 8 \}$ and $B = \{ 3, 5, 7, 9 \}$.