I am attempting to learn neural networks using the Keras libary on the MNIST hand written digit dataset, using dense layers only. I am trying to figure out what the best shape for the network should be but I can't seem to find any literature or discussion on the subject.
By shape, I mean should the hidden layers have more nodes as I go deeper, less nodes, or the same? I've taken to calling them square, pyramid, and reverse-pyramid because I couldn't find a better name. See the end of this post for clarification on shape
I've tried all 3 and the reverse pyramid is giving me the best at 80% accuracy but I can't imagine why. I feel like I would just be losing data and introducing noise by expanding the number of neurons as I go deeper.
Is there any literature or discussions about this subject? Perhaps, I'm not using the right keywords to look, I just don't know the correct terminology. I've tried neural network shape, structure, hidden layer layout, etc. with very little luck.
Thanks in advance.
Square - Hidden layers stay same size until the end
Pyramid - less neurons at each additonial hidden layer (pretend they are all hidden layers)
Reverse Pyramid - more neurons at each additiona hidden layer