The only problem is that the running time of $\Phi$ could be very long (probably hundreds of hours, even at the start). Ultimately, I want $F$ to be as close to infinite in size as I can get. So there needs to be some element of stochastic sampling of $F$ as well.

@D.W. Yes, the structure that I was trying to communicate is loosely $F_{x}(S) \approx F_{x}(T)$ when $S\approx T$ and $k_aF_a(S) \approx k_bF_b(S)$ for all $S$ ($k$ is constant).