0
$\begingroup$

I am developing a neural network that is trained using a genetic algorithm. The neural network is a multilayer perceptron using $\tanh$ as its activation function. Currently, the genotype of the neural network is by its weights. I used the method of making a connectivity matrix and linearizing it according to this paper: http://sci2s.ugr.es/keel/pdf/keel/articulo/NN-Garcia05.pdf

What is a good crossover method for this? I've tried uniform crossover but it is too disruptive as there is no improvement whatsoever. Single-point crossover is discouraged as I have read, so what should I use?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Training by backpropagation / gradient descent is likely to be much more effective than training with a genetic algorithm. $\endgroup$
    – D.W. ♦
    Apr 4, 2018 at 22:45
  • $\begingroup$ I've actually successfully done backpropagation/gradient descent. This is just something more that I wanted to do. I know that there are better reinforcement learning methods but I just want to use genetic algorithms right now. $\endgroup$
    – Dalop
    Apr 4, 2018 at 22:50
  • $\begingroup$ Why are you doing crossover? For genetic algorithms, crossover is optional not a requirement. $\endgroup$
    – Ray
    Apr 5, 2018 at 16:44
  • $\begingroup$ @Ray Well, if you're doing genetic algorithms without crossover, you're actually doing a population based random search... crossover is the very essence of GAs. $\endgroup$
    – Pål GD
    Jun 11, 2018 at 18:20
  • $\begingroup$ @Pal GA with mutation is not random because of the selection operator; you only mutate the top fittest. It's more like random hill climbing. Adding a crossover step does nothing. If you think it adds something, I'd like to hear it. $\endgroup$
    – Ray
    Jun 19, 2018 at 19:05

1 Answer 1

Reset to default
1
$\begingroup$

I don't think you should.

But, supposing the structure is the same in all individuals, you could take a random set of nodes from one, and the rest from the other, and just keep all out edges from nodes you selected.

Another alternative is taking a random set of edges from one, and the rest from the other.

A third alternative is to take the first $\ell$ layers from the first individual and the last $L-\ell$ layers from the second individual.

As you can see, there are very many approaches, and I don't think any of them will be satisfactory, but I hope you try and see for yourself.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.