I'm trying to derive an algorithm or procedure I would use to have a human interactively rank a series of items based on importance by comparing a random-pairing of items at a time.

So, given an array of let's say 10 "things", I want to present two of the things, and have the user specify which is the more important of the two. What I don't know is:

  • What is the proper name for this procedure/concept?
  • Would I use some form of sorting algorithm?
  • How could I determine when to stop? When is the ranking "done"?
  • And, importantly, I would prefer not to present the same "thing" to the user in a row. That is, if comparing Item B and Item K, user selects Item K as more important, the next iteration should not present either Item B or Item K.

What I'm after seems similar to this question: Nearest neighbour based on subjective human comparison - is this a thing?

... but I'm not interested in similarity, just relative importance. Think prioritizing a to-do list or subjectively ranking vacation destinations, etc.

The Elo Rating System may be similar to what I'm after, as well.


1 Answer 1


There are statistical models for relative ranking. I'd suggest taking a look at the Bradley–Terry model. The data is sometimes known as paired comparison data.

See also Analysis and classification based on data points, Peer grading design - choosing a graph, to get accurate rankings/ratings, https://stats.stackexchange.com/q/83005/2921.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.