I was just wondering on a more technical side, if anyone could explain what Google does to create these amazing images from it's deep dream system.

Could anyone explain to me in a step by step way, what the program does and how it does it.

• watch the vedio on this page: coursera.org/course/neuralnets – Jake Jul 16 '15 at 21:31
• Google blog has some posts on this, for example googleresearch.blogspot.ch/2015/06/… and googleresearch.blogspot.com/2015/07/…. – Yuval Filmus Jul 16 '15 at 21:47
• What research have you done? There are multiple resources out there that explain what's going on at various levels of technical detail. What ones have you looked at? What didn't you understand? What did you find that was or wasn't suitable? We expect you to do a significant amount of research on your own before asking here, and to show us in the question what research you've done and tell us how they did or didn't meet your needs. Sharing your research helps us gives you better answers and helps others as well. See our help center for more. – D.W. Jul 16 '15 at 22:04
• "How did Google draw these pictures?" is not a computer science question per se. The first step is to ask Google (cf. above comments); then you can ask questions about the used concepts. – Raphael Jul 17 '15 at 0:03
• Just by looking at the question, I have no idea what it is about (I don't know what a/the deep dream system is). Can you somehow add more information to make the question stand on its own? Moreover, what kind of an explanation are you looking for? What in particular are you curious about? – Juho Jul 17 '15 at 13:56

Google's Deep Dream project is research being done to visualize neural networks' learning to understand more about them.

First, you have to understand a Deep Belief Network on a high level. In short, it's a way for machines to 'learn' about data (images, in this case). That works, essentially, by transforming an input by several matrices (layers) through the network to get an output. Deep Belief Networks are trained to reconstruct their original image. All of the images are generally of some type, like animals. After some training, they begin to recognize basic features that are common between members of the training set, such as edges or colors.

The images published for the Deep Dream project are different ways of visualizing the data from trained networks. For some of the images, they observe the state of the image in layers between the input and output, resulting in artistic looking sketches of the original. For other images, they start with one image that is not related to the training set, send it through the network, then send the output through the network repeatedly to accentuate features that the network found in the image.

there are several basic concepts going on here that are not all explained in a straightforward/ brief way in most references, of which there are now many, which tend to range from scientific articles to handwaving/ "glossing over" popular science accounts.

1. train a "deep" artificial neural network using large amounts of image data. sometimes this is collected from youtube videos, and there are other large standard "corpuses" (collections) of data used in machine learning. this is also usually done on a cluster supercomputer.

2. now you have weights and a network for this ANN. the weights are "frozen" and one can analyze what types of "stimulus" "most excite" certain "neurons" in this ANN. the ANN will have neurons that classify based on image types. one can choose different neurons that were assigned during training.

3. the stimulus is a 2d matrix of pixels, usually grayscale, but possibly in color. a test stimulus starts out as a random noisy array. then the array is slowly perturbed so that it increases the excitation of the target neuron. this is using an optimization algorithm that is basically "reverse engineering" the stimulus independent of the learning algorithm etc., using the fixed network & frozen weights.

4. eventually this optimization of the stimulus does not increase the excitation of the target neuron any more. this is a sort of "optimal stimulus". this is the "resulting image".

volumes more could be written. a key concept is that the ANNs are embodying a real world implementation of "feature detectors" which have been studied for decades in neurobiology and CS. more detail can be found in this nice recent blog post by Oygard "Visualizing google classes" which step-by-step replicates the results using open source/ open science.

So far nothing's been said about technical details of DeepDream. I'll fill the blank.

The procedure is the following: pick some layer from the network (usually a convolutional layer), pass the starting image through the network to extract features at the chosen layer, set the gradient at that layer equal to the activations themselves, and then backpropagate to the image.

Why does it make sense? Intuitively, it amplifies the features that are maximally activated in the network. By backpropagating this gradient, we'll make an image update that will boost any of the existing activations. If there's a cat-like detector in the layer and the image contains some patch that looks a bit like a cat, DeepDream boosts this activation by updating this patch to be even more cat-like. As a result, DeepDream is trying to find cats and dogs everywhere in the image (ImageNet dataset has lots of dogs, so the network has many dog-related neurons).

If you look at the code, the key part is objective_L2 function that makes this: $dx = x$ and then backpropagates.

Simply put, you:

1. Pick a layer
2. Forward propagate to that layer
3. Set the gradient at that layer to the activation at that layer.
4. Backpropagate to update the image

The general idea is that you want that layer's most activated neurons to change the image the most. So step 3 is actually just an easy hack to make sure the higher the activation of the neuron, the larger the gradient that gets backpropagated from that neuron and the more that neuron influences the change in weights.

As a result, you amplify the features in the image which those highest-activated neurons were already somewhat detecting in the beginning.

Some concise slides: http://www.cs.toronto.edu/~rgrosse/courses/csc321_2017/slides/lec13b.pdf

So, what happens in deep dream is that instead of minimizing the cost function in neural network, we try to maximize the output value of n-1/n-2 layer. And in this way we change our image, to maximize output value of n-1/n-2 layer.

Explanation

The higher layers(n-1/n-2) stores high level features information like buildings, ears of a cat etc. The lower layers (2nd/3rd) store information about edges, corners in a specific place etc.

So when we try to maximize the output of higher layers, we see more of higher level things like buildings, cats etc(this depends on what data is the model trained on earlier). In other way, we are saying we want more cats, buildlings. So anything that lightly resembles these things, it will be made more prominent.

Similarly, when we try to maximize lower layers output, we see more of edges like things.

You can watch the following video from computerPhile to understand more about Google's deep dream. "Deep Dream"!