# Backpropagation in multiple output neural networks

I want to understand how the backpropagation algorithm would work on a neural network with multiple outputs.

More specifically, I have a network with 21 binary (0/1) outputs and I want to minimize the number of outputs that I get correctly; in other words, I want to minimize the hamming distance between the output vector and the desired vector.

How does the loss function work here? How do I backpropagate the error and update the weights? I know this might be long to explain so I'm also happy with links to some good references that I could read on the matter.

• As far as I know, most real-world implementations use a continous output range $[0, 1]$ instead of the discrete set $\{0,1\}$. This allows to define a continuous cost function whose value can then be minimized. (The hamming distance is not continuous.) – Tobias Feb 26 '17 at 18:01
• You can use softmax for multiple output problem. – iLoveCamelCase Mar 8 '17 at 17:28