Background: a set of 300 images are prepared, randomly pair compared with 60 other images, assigned a rating (from 1-star to 5-star) based on the pair comparison score (+1 if wins a pair comparison and +0 if loses a pair comparison), the score range for the rating assignment is 0-11 for 1-star, 12-23 for 2-star, 24-36 for 3-star, 37-47 for 4-star, and 48-60 for 5-star

Objective: want to rate 1000 more images based on the 300 rated images, again through pair comparison, but with some of the 300 rated images instead of between themselves


  • what images should be selected from the 300 rated images for the pair comparison, and how many?
  • how should the score function be defined for the pair comparison with the selected image?

Could somebody help this experiment design? Thank you very much for you attention and help!



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.