In the Deep MNIST Tutorial for Tensor Flow, (and in general), we create a convolutional layer with a weight tensor, say W, of shape (patch dims, #input channels, # output channels). However, when we define the bias tensor, b, its shape is only (#output channels).
When we convolve W and the input x, with correct padding, we will get a volume with depth = #output channels. How can we add a single bias vector to such a volume?
Are we supposed to expand the bias vector to take on the spatial dimensions of the input x?