Following Deep MNIST for Experts tutorial on Tensorflow, I realize I don't understand where the choice of numbers comes from when initializing weights.
In the tutorial, they first show the below function for creating random weights.
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
However, the shape
parameter has me confused when the tutorial shows the next line of code:
W_conv1 = weight_variable([5, 5, 1, 32])
The tutorial says the following:
The first two dimensions are the patch size, the next is the number of input channels, and the last is the number of output channels.
I don't understand the significance of the last two values, 1
and 32
.
What decides what values you input there? And what do they affect?
When I'm designing my own neural network, could I put any value I like for the last two, or if I were to change them in a neural network would it 'break' the entire network?