0
$\begingroup$

I am developing a neural network that is trained using a genetic algorithm. The neural network is a multilayer perceptron using $\tanh$ as its activation function. Currently, the genotype of the neural network is by its weights. I used the method of making a connectivity matrix and linearizing it according to this paper: http://sci2s.ugr.es/keel/pdf/keel/articulo/NN-Garcia05.pdf

What is a good crossover method for this? I've tried uniform crossover but it is too disruptive as there is no improvement whatsoever. Single-point crossover is discouraged as I have read, so what should I use?

$\endgroup$
  • 1
    $\begingroup$ Training by backpropagation / gradient descent is likely to be much more effective than training with a genetic algorithm. $\endgroup$ – D.W. Apr 4 '18 at 22:45
  • $\begingroup$ I've actually successfully done backpropagation/gradient descent. This is just something more that I wanted to do. I know that there are better reinforcement learning methods but I just want to use genetic algorithms right now. $\endgroup$ – Lam Nguyen Apr 4 '18 at 22:50
  • $\begingroup$ Why are you doing crossover? For genetic algorithms, crossover is optional not a requirement. $\endgroup$ – Ray Apr 5 '18 at 16:44
  • $\begingroup$ @Ray Well, if you're doing genetic algorithms without crossover, you're actually doing a population based random search... crossover is the very essence of GAs. $\endgroup$ – Pål GD Jun 11 '18 at 18:20
  • $\begingroup$ @Pal GA with mutation is not random because of the selection operator; you only mutate the top fittest. It's more like random hill climbing. Adding a crossover step does nothing. If you think it adds something, I'd like to hear it. $\endgroup$ – Ray Jun 19 '18 at 19:05
1
$\begingroup$

I don't think you should.

But, supposing the structure is the same in all individuals, you could take a random set of nodes from one, and the rest from the other, and just keep all out edges from nodes you selected.

Another alternative is taking a random set of edges from one, and the rest from the other.

A third alternative is to take the first $\ell$ layers from the first individual and the last $L-\ell$ layers from the second individual.

As you can see, there are very many approaches, and I don't think any of them will be satisfactory, but I hope you try and see for yourself.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.