I implemented my ANN using SKlearn module's class MLPClassifier. Fitting it on some data and testing it on a very specific subset of said training data, it gives a score of 1.0, but actually using the ANN to generate outputs for said testing data almost always gives incorrect output. What could be the possible reasons for this? I am using this ANN as a memory network and there is never a case when input is out of training data.
$\begingroup$
$\endgroup$
-
2$\begingroup$ Overfitting overfitting overfitting! $\endgroup$ – Tolga Birdal Apr 18 '17 at 20:17
-
$\begingroup$ My answer on a similar question might help you. $\endgroup$ – Kiritee Gak Apr 19 '17 at 3:38
$\begingroup$
$\endgroup$
You're testing your neural net with data from the training set, of course it will get you great results. But you're neural net is now trained for a specific data and is unable to make generalization. This is known as Overfitting.
You should either split your data into test/train or use another technique to estimate your NN performance. A good technique is K-fold Cross validation.
Also check Overfitting wiki page and Cross validation on sklearn if you want more details.
