# What values do $y_i$ in training data for binary classification problems attain?

In case of binary classification problem, what are the $y_i$ 's in the training data set $\{(x_1, y_1), (x_2, y_2), \dots (x_n, y_n)\}$?

I guess they are from $\{1,-1\}$. Now I see a method for finding a scoring function $f(x) = w^Tx + b$ by minimizing the squared error between the $f(x_i)$'s and $y_i$'s over $w$ and $b$. Now is it correct to minimize the error between $f(x_i)$'s and $y_i$? The latter is a sign while the former is a value? They seem incomparable to me.

If your prediction is $f(x) = \text{sign}(w^T x + b)$ then the 0-1 loss will simply be $$L_{\text{0-1}}(w) = \sum_{i=1}^n \mathbb{I}\{y_i \neq f(x_i)\},$$ where $\mathbb{I}$ is the indicator function, i.e., $\mathbb{I}\{a\} = 1$ if $a$ is true and $0$ otherwise. If you were to minimize this directly using subgradient descent you would get the perceptron algorithm. A more common approach is to upperbound the 0-1 loss with some other function, like the hinge loss, log loss, squared loss, etc and minimize that.
• But, then how to optimize expression which involves sign. I need to find $w$. – RIchard Williams Feb 2 '14 at 13:34