I am coding a neural network implementation, but a I have problems in the design. I was wondering about how to compare the output with the target, my neural networks has three outputs

groups = {'Iris-virginica':[0,0,1], 'Iris-setosa':[0,1,0], 'Iris-versicolor':[1,0,0]}

I know I must translate each output to 0 and 1.

I meant if my result is Iris-virginica and my output is more or less: [0.999979082561091, 0.9999918549147135, 0.9998408912106317], the subtraction would yield the following result:

[-0.999979082561091, -0.9999918549147135, 0.000159]

Is that correct, or I need to follow a different approach. Is possible train my net with 0, 1 and 2 values. Do I need to know any more?


The easy answer: Take the predicted class to be the argmax of your output vector.

The longer answer: Since you're doing multiclass classification you should probably be using softmax output units (if I had to guess, I would guess you're using sigmoid output units). If $\mathbf{x} \in \mathbb{R}^n$ is the input to your output units then the softmax for the $i$th output unit is

$$ y_i = \frac{\exp(x_i)}{\sum_{j}\exp(x_j)} $$

This output function is nice for a number of reasons.

  1. since your output vector sums to 1, you can interpret it probabilistically.
  2. the derivative is particularly nice in combination with a log loss function.

Note: the further information I've provided may seen redundant since clearly

$$ \text{argmax}\{\mathbf{x}\} = \text{argmax}\{\mathbf{\sigma(x)}\} = \text{argmax}\{\mathbf{\text{softmax}(x)}\}, $$

where $\sigma$ is the elementwise logistic sigmoid function. The problem with just taking the argmax of what you have is you will likely not be optimizing the function you are actually interested in internally.

On the other hand the choice of loss function for classification isn't as cut and dry as I've made it sound.

  • $\begingroup$ Can you write an example or a pseudocode? $\endgroup$
    – omar
    Aug 23 '12 at 19:39
  • $\begingroup$ @user12287, what part is giving you trouble? $\endgroup$
    – alto
    Aug 23 '12 at 19:45
  • $\begingroup$ I don't understand exactly how implement argmax function. $\endgroup$
    – omar
    Aug 23 '12 at 21:05
  • $\begingroup$ @user12287, sorry about that, perhaps I was a little sloppy. For the purposes here I simply meant the function that returns the index of the maximum value. So $\text{argmax}\{2, 5, 3\} = 2$, since 5 is the largest value and the second element. $\endgroup$
    – alto
    Aug 23 '12 at 21:16
  • $\begingroup$ What about the long answer? Can you give more details. $\endgroup$
    – omar
    Aug 23 '12 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.