Yesterday I've done some research how to optimize genetic algorithm and I've encountered a very interesting theory that we can use Lamarckian theory (adaptive theory) to optimize the neural network. But I still didn't understand how can it be done? What I've read from some source to optimize the genetic algorithm we must do some adaptation to the parents before we do the mating (crossover & mutation) but how is exactly the adaptation done? For example, if I have a genetic algorithm population to train a neural network, how can I alter the chromosome of the population to be adapted to the environment? Is it based on the error of the output or how?
$\begingroup$
$\endgroup$
6
Gilles 'SO- stop being evil'
44k8 gold badges118 silver badges184 bronze badges
asked Jan 12, 2013 at 9:40
-
$\begingroup$ I can certainly envision genetic algorithms in which the members of the population are somehow tweaked to make them "better". Perhaps the mutation technique of looking for "nearby" genotypes that are better should be described this way. As the whole GA area is really an extremely varied collection of ad hoc techniques with scanty theoretical underpinning, I'd just say try anything that looks reasonable. $\endgroup$– vonbrandCommented Jan 27, 2013 at 4:08
-
$\begingroup$ I see I see, I also thought about looking for "nearby" genotypes that are better and try to mutate to that way but I thought the population diversity will be decreased, what do you think about it? $\endgroup$– Niko Adrianus YuwonoCommented Jan 27, 2013 at 4:59
-
$\begingroup$ You are interested in the best in the population, so this isn't too bad. Perhaps do it as a fraction of the mutations? So you have yet another parameter to frob! $\endgroup$– vonbrandCommented Jan 27, 2013 at 5:02
-
$\begingroup$ Ah I see, I'll try to research further about it after I finish with my thesis examination, thank you vonbrand! $\endgroup$– Niko Adrianus YuwonoCommented Jan 27, 2013 at 7:55
-
1$\begingroup$ @NAYOSO it would be nice also if you put one or two links to the resources you have read and that prompted you to ask your questions. Those who will find your question later want to have something under their teeth ;-) $\endgroup$– Stephane RollandCommented Mar 6, 2013 at 12:32
|
Show 1 more comment
1 Answer
$\begingroup$
$\endgroup$
2
In fact, you're talking about Memetic Algorithms. In the case of a neural network, supposing you have training samples, the individual learning may be done through an iteration of backpropagation. If you don't have training examples, you may apply some noise to the weights and keep the modification only if it performs better (hill climbing). Actually, any local search algorithm could be used. Regarding the frequency of individual adaptation, this is currently a research topic, but you can find works that suggest applying the adaptation to all individuals at every generation as well as other works suggesting to increase this frequency gradually, since the last generations are the ones which need more fine tuning. Genetic algorithms are very good to find some good global solution, while local search is very good to find the optimal local solution. So the trick is to find the optimal solution region with GA and fine tune with local search.
-
$\begingroup$ I'm not following this answer yet. How can you tell whether it performs better if you don't have any training examples? $\endgroup$– D.W. ♦Commented May 28, 2015 at 5:56
-
$\begingroup$ @D.W. the training examples I'm refering to are per-step input-output pairs, which we use to train neural networks the conventional way (gradient descent). Without them, you still can evaluate your solution the same way you evaluate it with the genetic algorithm: run the network on some problem and extract a fitness value at the end according to its performance (it is a more coarse evaluation than using per-step input-ouput examples). $\endgroup$– rcpintoCommented May 28, 2015 at 16:57