I'm trying to implement a KNearestNeighbor model and came across the fact that many professional models use a K-D tree to index the K Nearest Neigbors. I also read that high-dimensional data makes a K-D tree less useful because you don't eliminate as many vectors for every branch in the tree when traversing it. All of that makes sense, but my question is at what ratio is there any benefit made of using a K-D tree?
For instance, I have a training set of shape (50000, 3072) but I am feeding only 10000 points to the model so the shape the model gets is (10000, 3072)... anyways, I assume these are the only two numbers which are relevant for the model to determine if a K-D would benefit it's runtime.
So in my fit method, I am making an if statement to either make a K-D model or not to make a K-D model for indexing, but I'm not sure what the condition should be. Given
n_data, n_dim = x.shape
I imagine the condition could be
if n_data / n_dim >= 1: pass
This is where my knowledge stops. I would be grateful if someone could propose a condition, and educate me about what makes a good condition.