Background: a set of 300 images are prepared, randomly pair compared with 60 other images, assigned a rating (from 1-star to 5-star) based on the pair comparison score (+1 if wins a pair comparison and +0 if loses a pair comparison), the score range for the rating assignment is 0-11 for 1-star, 12-23 for 2-star, 24-36 for 3-star, 37-47 for 4-star, and 48-60 for 5-star
Objective: want to rate 1000 more images based on the 300 rated images, again through pair comparison, but with some of the 300 rated images instead of between themselves
Problem:
- what images should be selected from the 300 rated images for the pair comparison, and how many?
- how should the score function be defined for the pair comparison with the selected image?
Could somebody help this experiment design? Thank you very much for you attention and help!
