relatively new to Computer Vision.

Lets say for example, I have a sequence of images of a car driving away from a static camera into the horizon, and I want to use this image set for some bog standard computer vision experimentation (e.g to train a CNN to recognize a car). I label the car in each frame with a rectangular bounding box. At the start of the sequence, the area of annotation required to cover the visible car is approximately 1500x500 pixels. At the end of the sequence, the area is only 3x1 pixels. Does there come a point in this sequence of images, whether by some rule of thumb or by a derivable metric, in which it no longer becomes beneficial to label the car for training purposes (i.e it will harm model performance)? I remember reading somewhere recently that its not beneficial to train CV algorithms on objects that change size, but I'm not sure whether there are guidelines to setting acceptable tolerance thresholds.

  • $\begingroup$ I would consider sizes such that a human can recognize them as well, unless the application does not require to recognize them from a long distance. $\endgroup$
    – user16034
    Jun 16, 2022 at 11:47

1 Answer 1


Your labels should be based on what you want the model to output when used in deployment. If you use this in deployment, is it important to produce a 3x1 bounding box? If it is, you'd better include those images and label those cars. If it is important to not produce a 3x1 bounding box, you'd better include those images and not label those cars. If it doesn't matter whether your model produces a 3x1 bounding box for those tiny cars, then you might consider labelling those cars but marking them as "don't-care" so the model is neither penalized for producing a bounding box nor penalized for failing to produce a bounding box; or excluding those images from the training set.

How do you choose the threshold? Choose it based on your application needs.

How can you tell what is the impact of different values for the threshold? Probably you have to do this empirically. You can try an experiment where you set one threshold, train a model, and check to see what happens.

Is there a rule of thumb? Not that I know of specifically, but the usual way to figure this out is to find a few papers that tackle the same task, and check to see what threshold they use. A plausible starting point is to copy whatever they have used.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.