I have a NN that has ten outputs. The output values range between 0 and 1. The elements in the target array are all zeros except one element, which is "one".

I am searching for a Fitness Function that will correctly evaluate (score) the Neural Networks.

Currently, I am trying to calculate the distance between the target and the output arrays. The issue is that at the beginning all my NNs return results that are very close to each other and I can't properly choose the fittest individuals.

For Example:

NN1 -> FF score:0.10030096 -> Rank?
NN2 -> FF score:0.09996143 -> Rank?
NN3 -> FF score:0.10015215 -> Rank?

Any suggestions?


The standard solution is: Use a softmax layer at the output of your neural network, and use the cross-entropy loss as your fitness function.

| cite | improve this answer | |
  • $\begingroup$ Please correct me if i am wrong. Using cross-entropy loss i will get very close ranks as well. NN1 -> FF score:0.10030096 -> 0.0459 NN2 -> FF score:0.09996143 -> 0.0457 NN3 -> FF score:0.10015215 -> 0.0458 How can i "space" the ranks (giving the better NNs more chance to be chosen)? $\endgroup$ – Max Z Jun 26 '18 at 20:19
  • $\begingroup$ @MaxZ, Sorry, I don't have time to check your calculations, and in any case you didn't give enough data to let someone do that anyway. But if you get very close cross-entropy scores, that is telling you something: that is telling you that the quality of those networks is very similar. That's just how it is. There are solid information-theoretic reasons to use the cross-entropy loss, so there are reasons it is the standard solution. You can read about the cross-entropy loss in many tutorials on neural networks, so I encourage you to do some reading and studying. It will probably help you. $\endgroup$ – D.W. Jun 26 '18 at 20:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.