1
$\begingroup$

I am trying to solve this algorithm exercises (http://neuralnetworksanddeeplearning.com/chap1.html#exercise_513527) in Michael Nielson's online book: Neural Networks and Deep Learning (http://neuralnetworksanddeeplearning.com/chap1.html#complete_zero).

In another word, he says that there is a way to use four neurons to decide 10 digits on top of of the last hidden layer (which consists of 10 neurons and decides 10 digits separately with a high correct rate).

However I couldn't figure out the way. Do you have any idea about this? I look forward to your idea. Any help is appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You could use 4 neurons to produce a 4-bit output, which then represents the digit in binary. A digit in the range 0-9 can be expressed in binary as a 4-bit number.

However, classification accuracy might potentially be worse than the standard method of 10 output neurons with a softmax layer.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you @D.W.♦ Do you mean that each of the four neurons represents one bit (0 or 1) for each of the four positions? In another word, suppose neuron 1 represents position 1, neuron 2 represents position 2, neuron 3 represents position 3, and neuron 4 represents position 4. Then 0000 detects 0, and 1010 detects 10. Am I understanding you correctly? $\endgroup$
    – Counter10000
    Commented May 31, 2017 at 20:04
  • $\begingroup$ @LinguisticsStudent, yes. $\endgroup$
    – D.W. ♦
    Commented May 31, 2017 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.