I developed an algorithm that can prune down neural networks while retaining most of the accuracy. For example, it can take a trained neural network (relu activation functions) with a hidden layer of 200 nodes and an output layer of 10 nodes and prune the 200 node large hidden layer down to 10 nodes while bringing the accuracy from about 97% down to an accuracy of about 82% with zero additional retraining after pruning. It does this by modifying the weights of the remaining nodes to preserve the information of the nodes that are being removed.

I have three questions for people who are familiar with this type of thing.

First, are there any algorithms that you know of that already exist that can do this? I couldn't find any when searching online. Second, is there any real practical use for this? Third, if it actually is original, and I can find some practical use for it, is it worth trying to publish somewhere?

  • $\begingroup$ Searching for "pruning neural" on Google Scholar turns up many research papers, from which you should be able to find many more. I suggest studying the state of the art in the research literature and comparing your scheme to standard ones that have been published. $\endgroup$ – D.W. Jan 9 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.