ANN when compared to biological neural systems have this common concept of reducing error in prediction over time (training) and becoming more good at predicting correctly. But there is one behavior of biological neural system where the prediction do become good over time but it also becomes fast and uses less energy, it seems like the system is able to do the prediction based on few input rather than requiring all the inputs. For example when you start to learn to read, you have to look at each character and then then brain predicts what the word is but over a period of time you don't even look at the whole word and the prediction becomes more fast and efficient by using just a small portion of the input.

Are there any ANN architectures with their corresponding learning methods that have this aspect of becoming more efficient in prediction along with more correct prediction?.


1 Answer 1


The machine learning term for this is dimensionality reduction (see here for a survey), which is used in a more general context than just neural networks.

Dimensionality reduction can be both supervised and unsupervised. An example for unsupervised dim. reduction is PCA, in which a projection onto a lower-dimension space is found, in a way that preserves distances / structure as much as possible. In the case of supervised dimensionality reduction, the labels of the training set are also used in order to reduce the dimension. This is, however, a more advanced research area. This paper and references therein would be a good starting point.

Hope this helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.