Often, I see people using networks like AlexNet, GoogLeNet, etc to train it on their dataset. These networks are trained or pretrained with a specific dataset, and people fine-tune it to their dataset.

However, is this really suitable? What if the custom dataset had a lot more images than the original or is detecting completely different classes?

In which ways could we modify the networks so they would fit our dataset better?

*one example is faster-rcnn, where lots of people have done this

  • 1
    $\begingroup$ I've edited it, hopefully it's clearer now. Basically I wanted to know if we can simply use those networks on our dataset that are possibly 1) having a lot more images or, 2) detecting completely different classes. And if we cannot simply do that, what in the network can we modify such that we train it better. $\endgroup$
    – User1915
    Commented Feb 15, 2017 at 16:38
  • $\begingroup$ @Evil "If people are using some networks, it is suitable to them already." Not necessarily. People use inappropriate tools and techniques all the time. When Google gives you a hammer, lots of people find that lots of things start looking like nails. $\endgroup$ Commented Feb 15, 2017 at 20:09
  • $\begingroup$ @DavidRicherby yes, that is true, couldn't agree more. But still using inappropriate tools is only possible within some range, it must yield acceptable results to be used, otherwise it simply won't be. And I would like to point the difference between "suitable", and "proper", where "suitable" I understand as possibility to convert a problem to nail and use this hammer, and the "proper" as the tool especially made for a given task that should be used. $\endgroup$
    – Evil
    Commented Feb 15, 2017 at 21:12

1 Answer 1


Is this really suitable?

Well, it works. At least, in some situations. And in machine learning, there's a sense in which that is the bottom line: is it useful? And yes, this approach does appear to be useful, in at least some situations.

Beyond that, I'm not sure what you mean by "suitable".

What if the custom dataset had a lot more images than the original or is detecting completely different classes?

Sure. There will cases where fine-tuning probably isn't what I'd recommend you try first. Look at it this way. Fine-tuning is one thing you can try. It's not a silver bullet. It may work well in some situations. No one is claiming it will work well in all situations. There are plenty of situations where it might work poorly.

Fine-tuning is probably most useful if your training set is small (or, at least, not very large), and if there is a pre-trained network that solves a task that is somehow similar to the task you're trying to solve. The classes don't have to be identical -- for instance, if you are solving some kind of object detection task, but your classes aren't identical to ImageNet, then starting with a pre-trained network that was trained on ImageNet can still be useful, as the image features learned for ImageNet are likely to be useful for other object detection tasks as well -- but the task probably needs to be somehow similar.

If your training set is very large, then fine-tuning might not be the best answer. You might do better to train a network from scratch, yourself.

Finally: Machine learning is ultimately a pragmatic field. Don't try to look for the one right answer that works in all situations, and don't expect there to be some theory that can predict what the most effective approach to your problem will be. Instead, understand that it's a bit of an art; trial and error is often needed; and the ultimate metric is whether it works well enough for the particular problem you are facing right now.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.