How do you determine the inputs to a neural network?

Question

I'm looking at this tutorial on neural networks. The data that is given from the UCI study includes various attributes, such as "mean x of on pixels", "total # on pixels" etc, which are taken as input to the neural network. However, it seems to me that this choice of input parameters is somewhat arbitrary, and, in this case, would only apply specifically to the black-and-white letter recognition problem, and maybe could be extended to some other image recognition problems.

As a result, my questions are:

1. How do you determine what inputs you should use for a neural network?
2. Are more inputs necessarily better?
3. Does it matter if inputs are linearly independent?
4. And, finally.. Could one build a neural network that could determine its own inputs for an arbitrary problem and raw data set?

Answer 1

How do you determine what inputs you should use for a neural network?

Experimentation. Initially you use a priori knowledge and intuition to guess the features will be useful for classification or prediction or whatever it is your network is doing. Then you test the network and observe its error rate. Based on the results you make adjustments; add/discard features, regularize, add/discard layers, adjust the learning rate, and so on.

Are more inputs necessarily better?

No. You're trying to find the features that matter to what you're trying classify/predict and discard unrelated features. Processing input/features that don't matter to the computation just wastes CPU time.

Does it matter if inputs are linearly independent?

If inputs are linearly dependent then you are in effect introducing the same variable as multiple inputs. By doing so you've introduced a new problem for the network, finding the dependency so that the duplicated inputs are treated as a single input and a single new dimension in the data. For some dependencies, finding appropriate weights for the duplicate inputs is not possible.

Could one build a neural network that could determine its own inputs for an arbitrary problem and raw data set?

This is essentially what neural networks that deal with raw data only must do. The hidden layers tease features out of the raw data and those features are gradually refined by the feedback that occurs during training. This is an approach you might use if you have no idea how to proceed given some classification problem. But usually you have some idea what features are both useful and difficult for the network to compute for itself, and you provide those features as additional input.

Answer 2

As far as I know:
0) If you provide different training set to similar problem it should work, but changing only this will probably result in bigish error rate.
1) in general case you cannot. The net with some structure was trained on some training data. If you have bunch of floating point values it might mean virtually anything. Inputs structure might hint that it was supposed to be applied to image. But in fact various nets with different purpose are not distingushable when they are trained. Moreover we do not exactly know in the most cases what net "knowledge" is, and reversing this problem or retreiving "actual knowledge" is open problem.
2) No. And to provide some insight - bigger inputs and longer training time results in overfitting, decreases speed (both of training and online work). It might be infeasible to train it from some point. There was experiment on several different sizes of input for ocr - success rate increased as long input was smaller than letters size, and dropped after that point. Not complete letters started to give false results.
3) Yes it does. Here we should be more specific, but taking small steps - linearly dependent data in e.g. Widrow-Hoff algorithm it will make problems - when there are not enough degrees of freedom the network cannot be trained with 0 error. So linearly independent data is preferable.
Also linearly dependent data provides less pieces of information. You can add some biased to salvage it, but how to choose them is arbitrary decision.
4) Since simpler problems are still open - no. Taking size of the network, structure and arbitrary data recognition I would not count on usable solutions anyway.