# How do I stop "cheating" in reinforcement learning (MLP+Evo. Algorithm)?

I have a two hidden-layer MLP. I am trying to teach it classification of the sine function. For instance, if there is an [x,y] point above the sine function, the ANN should classify that point as a 1. Otherwise, it should be classified as a 0.

It should end up like this. Points above y=sin(x) should be blue. Points below should be red. The network is just outputting 1s or 0s, so we are plotting the point along with a binary value (0=red,1=blue). Graphic done with Matplotlib.

I feedforwarded my ANN thousands of random points, calculated the error between desired and output, and used that as fitness for an evolutionary algorithm that would evolve the network weights. The lower the error, the more likely the weights would be bred and preserved for the next round.

Unfortunately, the algorithm found a way to "break" the network. It was able to minimize the error while outputting the wrong values. For instance, one network and weight iteration I evolved outputted 0 for all inputs I would give it, even though I got the error down to 0 when I evolved the weights. It's a beauty. The plot is below.

My Python code is pretty big, so I will spare how I wrote the MLP and show you just a few snippets of relevant code.

Here, we have lists of random points and are feeding them through the MLP. If it is trained properly, the error will be close to 0 because the MLP correctly classifies each point.

    i=0
error = 0
while i < 100:
output1 = feedForward(neuralNet, [xValsAbove[i], yValsAbove[i]]) # feedforward a point that is above the sine function.
output2 = feedForward(neuralNet, [xValsBelow[i], yValsBelow[i]]) # feedforward a point that is below the sine function.

if output1 < 0.99:
error += 1 - output1
if output2 > 0.01:
error += output2
i+=1
return error


So, let's start evolving. Generate a bunch of random weights. Feed them a few thousand random points each in a range of -50 to 50 on both axes. Select the ones with the lowest classification error. Continue. After 50 generations, my error is down to 2 or 3. Looks good. Let's feed it a bunch of test points and see how it did. ???

Can anyone help me here?

• I'm not sure what you're looking for. If it's help debugging your code, that's off-topic, here (and probably impossible without access to all the code). Are you looking for help with your general approach? Something else? (It's possible that I don't know what you're asking because I know nothing about machine learning.) May 9, 2016 at 6:07
• "outputted -1 for all" ​ error += output2 ​ ​ ​ ​
– user12859
May 9, 2016 at 12:14
• @Ricky Demer Sorry, that -1 should actually be a 0. May 9, 2016 at 14:54
• @ David Richerby No, I don't need help with the code. I am looking for a general solution to the problem and am thinking that some MLP programmers here might have experienced the same or similar. May 9, 2016 at 14:56
• The "general solution to the problem" is "debug your code". ​ ​
– user12859
May 9, 2016 at 20:24