2
$\begingroup$

I have a very expensive black-box function and at least 15 parameters that I want to explore (usually 5-6 at a time). So far I have tried Genetic Algorithms and Gaussian Process Surrogate Optimization (bayesian optimization). Both methods work fine and efficiently up to 5-6 parameters. However, I am having an issue on how can one be sure that a solution found by GA/GPSO is a good solution, without basically knowing nothing about the output space. I don't really want to go for an exhaustive grid search since even with 5 parameters, it requires incredible amounts of evaluations, yet I cannot come up with anything better.

I would also very much appreciate suggestions/keywords for literature reading, or event different methods for hyperparameter optimizations.

Thanks in advance

| cite | improve this question | |
$\endgroup$
0
$\begingroup$

The short answer: You can't. These are heuristics. There are no guarantees. They might fail badly; and there's no reliable way to detect that.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I think in one article, they were showing for GPSO that it converges %70 of times (with toy data set I believe), and making an argument on that. Are you aware of such a method? PS: I'll send a link to paper / explain it better $\endgroup$ – b.y Apr 2 at 17:01
  • $\begingroup$ @b.y, those are presumably empirical results. Also that presumably means that 30% of the time it didn't work well. Not very helpful if I have a single problem I want to solve; it will give me a proposed solution, but did that fall into the category where it worked well or where it didn't? No reliable way to know for sure. $\endgroup$ – D.W.♦ Apr 2 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.