# How to reconstruct the image from a neural network output?

I am trying to use the genetic algorithm to optimise a multi-layered neural network for image classification (i am using a subset of the MNIST handwritten digit data set as my initial dataset, but eventually would like to make it more general).

my neural network is represented by connection weight matrices with inputs as the rows an outputs as the columns (so element (3, 4) is the weight of the connection between input 3 and output 4, etc) and have an extra column and row for the biases of the inputs/outputs

the system seems to work well, it takes an input vector, multiplies it by the first layer matrix, uses the sigmoid function to determine whether that neuron will fire or not, then uses the output vector of that as the input to multiply by the 2nd layer and so on to the final layer, and i am getting the results that i expect with test data that i know the expected output for.

i have an idea of how i will implement the genetic algorithm to search for the best solutions, but my problem is that im having a little bit of trouble understanding how neural networks can be used to reconstruct images so i can test for the error between the input image and the reconstruction to test each solution for fitness.

i understand that i need to first encode the image into a vector of pixels, and that that input needs to be passed through the network to the end layer, but i am not sure what that end data is supposed to represent..

the dataset i am using are images of 28*28 pixels, so the structure of my network is as follows (inspired by the Hinton paper here):

484 -> 1000 -> 500 -> 250 -> 30

so the output of the whole thing will be a vector 31 (30 neurons +1 for the bias column/row) elements long

how should i use this vector to reconstruct the image?

i read somewhere that to decode information from a neural network you need to multiply the output by the transpose of the transformation matrix, but that doesnt really make sense as (for example):

T = [ 1  2 ;
3  4 ]

A = [ 5  6 ]

A * T = [ 23  34 ] = B

B * T' = [ 91  205 ] != A


clearly this is not a very good reconstruction of the original data..

can someone give an explanation as to how neural nets can be used to reconstruct data, and if possible, point me to some good resources on the subject?

You can't necessarily use the output of a neural network to reconstruct the image, like you want. The output signals indicate how the neural network has classified the input image. Beyond that, they don't necessarily have any interpretation as a representation of the original image. There isn't necessarily any way to do what you want (to reconstruct the image given the output of the neural network), as this kind of neural network inherently throws away information.

Are you familiar with the notion of training "autoencoders"? One way to train multi-layer neural networks is by building a series of auto-encoders: e.g., to construct the weights for the first layer, you train a 2-layer autoencoder, then throw away the second layer of the autoencoder and use the first layer of the autoencoder as the first layer of your neural network. You repeatedly do this for each layer of your network. Autoencoders do reconstruct the original input, so if you're training each layer of the neural network in this way, then you automatically have a way to reconstruct (an approximation to) the original input from the signals at any layer -- if you saved the entire autoencoder at each stage. However, if your real end goal is to understand why an image was classified a certain way or debug your neural network, I expect this probably won't be helpful for that.

I'm not 100% sure what your real goal is. You mention "test each solution for fitness", but I'm not sure what you meant by that. However, my guess is that your best bet will be to figure out what your real goal is, and then ask about how to achieve that goal (which might not be by trying to reconstruct the original image).

• thanks for your response, does an autoencoder have the same number of input nodes as output nodes? so does that mean that for every layer that i have in my classification network, for pre-training i need to have a mirrored corresponding layer? so for training the first layer i would need to use a network that is 484 -> 1000 -> 484 and then second layer would use the output from the trained 1000 node layer to go 1000 -> 500 -> 1000? and so on? each time calculating the error between the input of and the output to determine fitness? Commented Aug 13, 2014 at 6:59
• also, if my inputs are scaled between 0 (black) and 1 (white), do you have any idea as to what would be a good range to generate initial random weights and biases? i tried between 0 and 1 but summing all the inputs meant that they would always fire, as the biases on the neurons were also between 0 and 1. is there a good rule-of-thumb for setting initial weights when training neural networks? Commented Aug 13, 2014 at 7:09
• @guskenny83, those are different questions. This is a question-and-answer site, where we do one question per question. There's lots written about autoencoders as a way of training deep neural networks; I suggest you do some research and learn about the subject, if you're interested.
– D.W.
Commented Aug 13, 2014 at 16:00
• thanks for your response, by "test each solution for fitness" i meant that the fitness function for selecting which members from the population would be some measure of the error between the original input and the reconstructed image from the autoencoder. i will go and do some more reading about training autoencoders and ask a separate question if i get stuck again. thanks for your help Commented Aug 14, 2014 at 4:47
• @guskenny83, please don't leave clarifications in a comment. Edit your question to clarify anything that is unclear. Comments exist only to help you improve your question / help me improve my answer, and the comments can disappear at any time. Anyway, this is not the way you test how accurate your neural network is. Instead, you need ground truth, and you should be using techniques like cross-validation for that.
– D.W.
Commented Aug 14, 2014 at 6:27