Background: a set of 300 images are prepared, randomly pair compared with 60 other images, assigned a rating (from 1-star to 5-star) based on the pair comparison score (+1 if wins a pair comparison and +0 if loses a pair comparison), the score range for the rating assignment is 0-11 for 1-star, 12-23 for 2-star, 24-36 for 3-star, 37-47 for 4-star, and 48-60 for 5-star

Objective: want to rate 1000 more images based on the 300 rated images, again through pair comparison, but with some of the 300 rated images instead of between themselves


  • what images should be selected from the 300 rated images for the pair comparison, and how many?
  • how should the score function be defined for the pair comparison with the selected image?

Could somebody help this experiment design? Thank you very much for you attention and help!



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.