Text detection in computer vision

Question

I'm curious about the way text recognition works in machine learning(or more generally, the question of object vs not object) in computer vision.

How are systems trained when the not-object data set is so much greater in quantity and apparently lacks structure?

One approach is having the algorithm first searches for a text box and once it finds one applies character recognition. Thus the initial classification comes down to "text" or "not text". "Not text" doesn't have any particular structure though and in fact almost everything is "not text"...so how is this dealt with?

What would the "not text" training set be? Random images? Clearly you need negative examples.

Answer 1

Many research projects use something called "hard negative mining": instead of training on all of the positive instances (e.g., "text" or "object") and all of the negative instance (e.g., "not text" or "not object"), they train on all of the positive instances and a carefully chosen subset of the negative instances. In particular, they omit many of the 'obvious' negative instances and try to bias towards choosing negative instances that are more difficult to handle correctly. It's not clear to me whether this is truly necessary or beneficial with modern machine learning methods, but it is something you could try, if you don't want to use all of the negative instances.

Yes, to create negative instances, you would normally sample random patches of the image (at a random location), subject to the restriction that they not have too much overlap with the actual text/objects.

For learning more about how to recognize object vs objectness, read about "objectness" detection or saliency detection. For learning more about detecting text, you might be interested in the stroke width transform and the segmentation step in OCR.