Suppose I'm performing machine learning on a simple dataset, and have a bunch of training data of the form:
x (feature) y (label)
-----------------------
1 0
2 1
3 1
4 0
5 1
6 1
...
Where the labels are values in two classes, $[0, 1]$. Clearly, this training data lends one to believe that it will be a classification task.
However, suppose I want to output instead the probability that a feature will take the class $1$. Then, my output is more of a regression task.
Consequently, when I'm designing a simple neural network with just a single input layer and single output layer, how many output units should I have? Should I have two output units, one for each class, and if so, how do I ensure that each pair of outputs will be a valid probability distribution (i.e. sum to one)? Or should I have only one output unit, and treat the entire problem as a regression task?
There are probably pros/cons to each approach... thanks for your help!