# Crossover operator in genetic algorithms in Neural Networks

I am developing a neural network that is trained using a genetic algorithm. The neural network is a multilayer perceptron using $\tanh$ as its activation function. Currently, the genotype of the neural network is by its weights. I used the method of making a connectivity matrix and linearizing it according to this paper: http://sci2s.ugr.es/keel/pdf/keel/articulo/NN-Garcia05.pdf

What is a good crossover method for this? I've tried uniform crossover but it is too disruptive as there is no improvement whatsoever. Single-point crossover is discouraged as I have read, so what should I use?

• Training by backpropagation / gradient descent is likely to be much more effective than training with a genetic algorithm. – D.W. Apr 4 '18 at 22:45
• I've actually successfully done backpropagation/gradient descent. This is just something more that I wanted to do. I know that there are better reinforcement learning methods but I just want to use genetic algorithms right now. – Lam Nguyen Apr 4 '18 at 22:50
• Why are you doing crossover? For genetic algorithms, crossover is optional not a requirement. – Ray Apr 5 '18 at 16:44
• @Ray Well, if you're doing genetic algorithms without crossover, you're actually doing a population based random search... crossover is the very essence of GAs. – Pål GD Jun 11 '18 at 18:20
• @Pal GA with mutation is not random because of the selection operator; you only mutate the top fittest. It's more like random hill climbing. Adding a crossover step does nothing. If you think it adds something, I'd like to hear it. – Ray Jun 19 '18 at 19:05

I don't think you should.

But, supposing the structure is the same in all individuals, you could take a random set of nodes from one, and the rest from the other, and just keep all out edges from nodes you selected.

Another alternative is taking a random set of edges from one, and the rest from the other.

A third alternative is to take the first $\ell$ layers from the first individual and the last $L-\ell$ layers from the second individual.

As you can see, there are very many approaches, and I don't think any of them will be satisfactory, but I hope you try and see for yourself.