0
$\begingroup$

I'm trying to create a genetic algorithm to train neural networks (because I'm to bad at back-propagation), and it works well until generation 18, where the loss stops to decrease and gets constant. The loss gets from 112 to about 90 in 18 generations and almost doesn't change:

Curve of the loss

Here is the genetic algorithm (I think the problem doesn't come from the neural network algorithm as it's very basic) in python :

# This is my reproduce function :
def reproduce(first,other,mutate=0):
    child = Entity() # Creates a new Neural Network (class Entity is the class of the Neural Networks)
    # Only two synapse matrices, each get the mean of its two parents
    child.syn0 = (first.syn0 + other.syn0)/2 
    child.syn1 = (first.syn1 + other.syn1)/2
    if mutate:
        # Add some mutation if it's enabled, but it didn't work without it
        child.syn0 += child.syn0*(np.random.random()*mutate-mutate/2)
        child.syn1 += child.syn1*(np.random.random()*mutate-mutate/2)
    return child

ent = [Entity() for _ in range(20)] # Creates 20 new Neural Networks with random weigths on their synapses

nb_dna = 1
gen = 0
scores = []
max_gen = 100

while gen < max_gen:
    np.random.seed(int(time.time()))
    # The Neural Networks goes forward.
    for i,e in enumerate(ent):
        e.set_input(int(np.floor(np.random.random()*nb_dna)))
        e.forward()
        # In the forward function, the score is calculated as the result of a squared error cost function

    if gen < (max_gen-1):
        # Do natural selection
        ent.sort(key=lambda x: x.score) # Sort my ascending score (the lowest the score is, the closest the NN is to the expected result
        ent = ent[:4] # Kill all entities except the five 1st

        # Reproduce the winners
        for i in range(4):
            for j in range(4-i):
                # 1st reproduces with 2nd, 3rd, 4th and 5th,
                # 2nd with 3rd, 4th and 5th,
                # 3rd with 4th and 5th,
                # and then 4th with 5th
                    new = reproduce(ent[i],ent[j],mutate=0.2) # Changing the mutation rate "mutate" will only change the convergence speed
                    ent.append(new)
    gen += 1

Can you tell me if I'm doing something wrong, or if I'm going the wrong way ? Else, is that the best I can get ? It's actually the first "real" genetic algorithm I create (previous ones were easy JS algorithm to try to find a target word, starting with random letters)

$\endgroup$
  • $\begingroup$ Sorry but "Please debug my code" isn't on-topic anywhere on Stack Exchange. $\endgroup$ – David Richerby May 21 '17 at 9:36
  • $\begingroup$ Maybe try to look at Neataptic for examples of neuroevolution. $\endgroup$ – Thomas W May 21 '17 at 18:28
2
$\begingroup$

Well, I'm not an expert at genetic algorithms, but here's what I would check, coming from a neural networks perspective:

1) Learning rate (in your case, I think this is your mutation rate? Please correct me if I'm wrong). When a neural network exhibits this type of 'convergence', often you need to decay the learning rate either as a step function after a fixed number of iterations, or smoothly through exponential decay. Is that an option for you?

2) Initialization. Have you tried initializing your algorithm from different points and seeing if the convergence behaviour changes?

3) Adding noise. Sometimes you can get stuck in a poor local minimum, or at a saddle point. Small neural networks are more likely to get stuck in local minima. Saddle points proliferate in high-dimensional space. When learning stalls like this, adding some jitter to your parameter values and trying to learn from that new location can sometimes move you from a saddle point or local min to a better location in error space, and learning can continue from there.

4) Hyperparameter selection. How do you know your network is the correct size? Have you done some cross-validation to determine the number of hidden units you need, for example? Have you tried any kind of regularization?

Hopefully that provides some inspiration, or at least a starting point. Best of luck!

$\endgroup$
  • $\begingroup$ Thank you for your answer. So for 1), I'm not sure that there must be a learning rate, as I'm not using gradient descent, but only the genetic algorithm to optimize the network... Or maybe did I miss something ? What I call the mutation rate is what you would call the "percentage" of noise to add, as you say in 3). For 2), I tried initializing with different random values (like random/10, random/100, random/1000) and took the one that had the less loss to start. For 4), I had a look at the hidden layer's size, but changing its size (according to some rules) didn't solve the problem. $\endgroup$ – paulolol May 20 '17 at 19:24
  • $\begingroup$ Well, increasing the size of the hidden layer did actually change the loss at the beginning, but the evolution still stops at a loss of 90... $\endgroup$ – paulolol May 20 '17 at 19:35
  • $\begingroup$ Hmm. Is there a way you could try a different optimization routine to see if that gives you better results? Then you would know whether there is a better solution than 90 to be found. Again, I'm no expert in genetic algorithms, but even though an initialization can look bad at the beginning, it's where you end up that matters, so maybe running the algorithm to convergence from different initial points would be a decent thing to try next. $\endgroup$ – StatsSorceress May 21 '17 at 18:03
  • $\begingroup$ Also, what is your activation function on your hidden units? If you're using sigmoid or tanh units, they may be saturating. Try ReLUs instead. $\endgroup$ – StatsSorceress May 24 '17 at 1:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.