I have a set of functions $F$.
The functions effectively take a set $S$ that is always a subset of a global set of all possible values $G$, where $|G|>1000$. (alternatively, they could take a $|G|$tuple of booleans, if that's easier). The functions return a real result. I want to minimize over $S$ the average value of these functions if all of them are given the same parameter $S$.
Now here's the catch:
The functions take a fantastic amount of time to run (say, a few hours), and I intend to have a huge collection of such functions (say $|F|>1000$ to start). Every so often, a new function will be created and be added to the set. There is no bound to the size of $F$.
Obviously, I can't just use closed form optimization (I'll be dead before it'll finish), so I'm interested in an algorithm that can find an approximate solution and progressively refine it over time as new function results and new functions come in (and hopefully converge faster than brute force).
We can have a better-than-brute-force because the functions $F$ behave similarly. Namely, $f(S) \approx f(T)$ when $S\approx T$ and $k_aF_a(S) \approx k_bF_b(S)$ for all $S$ ($k$ is constant). There are, of course, many exceptions.
Now, I am new in the CS field so I have no idea what things are called. AFAIK I haven't learned how to do this in Into to ML, but it has the feel of a Genetic Algorithm problem. If anyone could point me to a technique or a name, I'd be very happy.