I am having a problem with my program of deep neural network using Theano. In my deep neural network, I have several layers of neural network to predict an output given a certain input. Because of an issue when compiling theano, I have to debug my program step-by-step. Since Theano gives me an error when updating weights of some layers, I perform a check by calculating the derivatives of only the last layer (using
tensor.grad if you know Theano). But if I do that then the network will not update properly and the cost does not decrease and the output is nowhere close to the true output. It seems like updating the weights of only the last layer does not seem to train the network to learn.
Is it mandatory to update all weights of all layers in a deep neural network to decrease the cost? What I understand is that if I fix the values of all weights except the weights of the last layer, it should still be able to predict correctly after many rounds of training (of course, the only layer that is learning is the last layer, but I accept that, I only needs the predicted values to be close to my true values while training).
I hope my explanation make sense to you. If you do not understand any part, I will gladly provide you more information.