Newbie to CV here so sorry of this is basic. Here's the deal, I have a program that I run many times. and each run I produce a screenshot. I need to compare screenshots from N-1 and N runs and make sure they aren't different in any dramatic way. Of course there are some minor changes like logos and pictures getting updated, etc.

SO far I've used something as simple as absdiff from opencv to highlight the difference regions and then use some sort of threshold to determine whether something passes or not. But I want to make it slightly intelligent but I'm not 100% sure how to proceed. Google hasn't yielded ghe best answers.

Essentially, I want to train the model on many different pairs of images and have the output be binary, yes or no depending on whether it should pass or not. In theory, I should be able to plug in 2 images and based on previous training, it should be able to tell me whether there is significant difference or not. What are some ways I might approach this, particularly with regards to what kinds of models to use. Thank you!

  • $\begingroup$ The requirements seem a bit amorphous. It's not clear what kinds of differences you want to consider minor and what you don't. All I see is you want it to be "slightly intelligent" but I'm not sure what that might mean specifically. How many pairs of images are you willing to label? $\endgroup$
    – D.W.
    Nov 30 '20 at 22:25
  • $\begingroup$ That's the problem, the differences could be, in theory, anything. I am hoping that there will be patterns between different images and that a model would pick up on that. Things like the name of a document is 045 instead of 056 or a logo is slightly updated. In terms of test data I would label, lets say a realistic number is 2-3 thousand? $\endgroup$
    – bumblyboi
    Dec 1 '20 at 0:18

I am skeptical whether computer vision is going to be the most effective approach, but you can certainly do it. There is lots of work on image similarity metrics using neural networks (Siamese networks, triplet loss, etc.) and you could certainly implement any of those architectures, acquire a few thousand labels, and train on it. I would suggest pre-training on some standard similarity dataset (e.g., start with LPIPS) and then fine-tune on your data. But be prepared that this might or might not be effective.

It might be more effective to extract features and then build a ML model that works with those features rather than with the raw images. For instance, OCR both images and compare them; compute L1 and L2 distance; etc.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.