I want to compare swarm intelligence algorithms in a optimisation problem. As far as I understand, a typical approach is to perform several runs of each algorithm (say 30 independent runs, with different random number generator seeds) and compute the average performance of best-of-run individual for each algorithm. Then one can conduct a statistical test, in order to compare the averages of the two variables (say m1 and m2, the average performance of best-of-run individual for algorithms 1 and 2).
Now imagine that in our optimisation problem we are only considering one objective function. What if this function is constrained and, for one run (or possibly more) of one algorithm the best solution it can find violates the function's constraints. If, in a given run, the algorithm is unable to compute a solution that lies within the problem's constraints how should I proceed in order to compare the performance of this algorithm with another one? Initially, I was considering assigning a NaN or a value like MAX_INT to all solutions that violate the problem's constraints, but if I do that then I won't be able to compute the algorithm's average performance nor perform the statistical tests.
What should I do in such a scenario? Do people tend to just perform more runs until they get 30 best-of-run solutions that do not violate the problem's constraints? How can I compare the performance of my two algorithms experimentally?
