How do I stop "cheating" in reinforcement learning (MLP+Evo. Algorithm)?

Question

I have a two hidden-layer MLP. I am trying to teach it classification of the sine function. For instance, if there is an [x,y] point above the sine function, the ANN should classify that point as a 1. Otherwise, it should be classified as a 0.

It should end up like this. Points above y=sin(x) should be blue. Points below should be red. The network is just outputting 1s or 0s, so we are plotting the point along with a binary value (0=red,1=blue).

Graphic done with Matplotlib.

I feedforwarded my ANN thousands of random points, calculated the error between desired and output, and used that as fitness for an evolutionary algorithm that would evolve the network weights. The lower the error, the more likely the weights would be bred and preserved for the next round.

Unfortunately, the algorithm found a way to "break" the network. It was able to minimize the error while outputting the wrong values. For instance, one network and weight iteration I evolved outputted 0 for all inputs I would give it, even though I got the error down to 0 when I evolved the weights. It's a beauty. The plot is below.

My Python code is pretty big, so I will spare how I wrote the MLP and show you just a few snippets of relevant code.

Here, we have lists of random points and are feeding them through the MLP. If it is trained properly, the error will be close to 0 because the MLP correctly classifies each point.

    i=0
    error = 0
    while i < 100:
        output1 = feedForward(neuralNet, [xValsAbove[i], yValsAbove[i]]) # feedforward a point that is above the sine function.
        output2 = feedForward(neuralNet, [xValsBelow[i], yValsBelow[i]]) # feedforward a point that is below the sine function.

        if output1 < 0.99:
            error += 1 - output1
        if output2 > 0.01:
            error += output2
        i+=1
    return error

So, let's start evolving. Generate a bunch of random weights. Feed them a few thousand random points each in a range of -50 to 50 on both axes. Select the ones with the lowest classification error. Continue. After 50 generations, my error is down to 2 or 3. Looks good. Let's feed it a bunch of test points and see how it did.

???

Can anyone help me here?

Answer 1

You are minimizing the wrong loss function. In other words, you have picked a poor objective function.

Look up the standard objective functions that are used for training neural networks for boolean classification tasks: e.g., mean-squared error, or cross-entropy loss. This is covered in standard tutorials. They don't use the function you show. The function you picked looks... odd, so I suspect it is the source of your porblem.