The usual way one trains a neural network is to give it some input and provide the correct classification.

But what about letting the neural network produce its own inputs, and then classifying those?

Imagine the following "algorithm".

  • Use a neural network to generate 2 images (random at the beginning)
  • Let the user pick which one is more "interesting"
  • Update the neural network based on the choice

Do you think that as a general principle this might make sense, i.e. there might be an approach that generates more "interesting" images over time? Has something similar been done until now? What approaches/papers/libraries might best help one achieve such a goal?

  • 2
    $\begingroup$ This is similar to what neural networks like DeepDream do, although then automatically. They have a classifier (in your case the user), and then mutate the input to conform to the classifier. For a classifier with animal faces this causes the algorithm to enhance and hallucinate faces in many places in the input image. $\endgroup$
    – orlp
    May 4, 2017 at 18:50
  • $\begingroup$ as @orlp pointed out DeepDream is very similar to your approach. However, Nicola, i'm not sure your described algorithm would work, I can't really explain exactly why; but just imagine this: you want a perfect image, this image has multiple different perfect features. How can you guarantee that every time you run the network, it will show different good and bad features all the time? $\endgroup$ May 4, 2017 at 19:34

2 Answers 2


No, that doesn't seem likely to be a very useful direction.

Neural networks are a form of statistical machine learning. The premise behind all statistical machine learning schemes -- including neural networks -- is that the instances in the training set need to come from the same distribution as the instances that you will apply the classifer to (i.e., the instances in the test set).

If you artificially generate instances via some arbitrary procedure, they might not come from the right distribution. That will invalidate the core assumption behind statistical machine learning and may make the classifier perform poorly.

Your approach would only work if we knew that the neural network was able to generate images that come from the right distribution (i.e., the distribution on natural images that you'll want to classify). But there's no reason to believe that will be the case. In particular, I suspect it most likely won't be the case.

Or, to put it another way, your approach would only work if we had a way to characterize the probability distribution on natural images and sample from this distribution. However, this appears to be as hard as building a classifier, or even harder. So, there is no free lunch.

You might interested in the distinction between discriminative vs generative models. Standard neural networks are discriminative models, and thus don't provide any (direct) way to generate images. It is also possible to build generative models, which can generate images. However, building a generative model is believed to be at least as hard as building a discriminative model, and probably even harder. So, your proposal amounts to saying: "if I had a generative model, I could use that to build a training set to train a discriminative model". But if building a generative model is likely even harder than building a discriminative model, so this doesn't seem to help much. There's no free lunch.

Separately: You don't say how you plan to use a neural network to generate images. Neural networks are a classifier, so they can classify an existing image, but not generate a new image. You have to do something special to generate images from a neural network.

  • $\begingroup$ I think the same, but I just came across an article where they fine-tune a model on its own outputs (filtered by confidence) and report massive improvements: sciencedirect.com/science/article/pii/S1569843223001267. I find it really strange and feel like it conceptually is wrong. But since it has been peer-reviewed and published maybe I am missing something. Would you have a comment? I am genuinely curious $\endgroup$
    – Jan Pisl
    Apr 28, 2023 at 23:51
  • $\begingroup$ @JanPisl, it sounds like they're using semi-supervised learning. That's a different approach from the one listed in the question. $\endgroup$
    – D.W.
    Apr 29, 2023 at 5:40

I think that similar concept is actually used in Generative Adversarial Network. The entire neural network is like this:

             real image --> discriminator (neural network)
generator (neural network) -- fake image --->--┘

If you want to try your idea of "interesting image" generation, the Keras tool had a code example.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.