I am solving a problem in the Kaggle learn section : https://www.kaggle.com/c/facial-keypoints-detection
The problem involves detecting facial keypoints in a 96x96 image, some 30 features are detected. It is a supervised dataset.
I am using a convolutional neural network with max pooling, 2 convolutional layers and 2 max pooled layers, and one fully connected layer.
The problem uses the rmse metric for scoring. So i used an ADAM optimizer to minimize the rms loss.
I just simply trained the model, i got an average training rms error of 11.2, and finally when the output is generated I clamp the values which are greater than 96 (It's a 96x96 image) to 96, i get a test rms error of 9.1.
I train the model itself with the clamped output values, i.e any output of greater than 96 gets clamped to 96 while training.
I thought CASE 2 would lead to faster and better convergence. I used the same hyper-parameters for CASE 2 and found that it is converging more slowly and gives a very high rms error of around 180.
Can anyone explain what is happening?