I've written a "quiz" that prompts the user for comparisons between two items of subjective value, and once the position of all of the items is determined, displays an ordered list from most valuable to least. The number of items is fixed. The current implementation can be seen at https://dfdx.us/core-values-quiz/.
What matters most is minimizing the number of comparisons the user needs to input, or, more accurately, minimizing the amount of time the user requires to make each comparison times the number of comparisons.
Currently I offer two sorting methods: Quicksort and insertion into a binary tree. Compared to a self-balancing BST, Quicksort will always require more comparisons, but because each consecutive comparison is against a single item (the partition item), the brain doesn't need to make much of a context switch and can answer the questions quicker. The binary search tree will present fewer questions that take longer. I let the user decide which method they want to use, given the pros and cons of each.
Since the comparisons required to rebalance the tree are handled by the computer, and not the user, those comparisons don't need to be optimized at all, only the number of comparisons required to determine the relative position within the current tree. I believe this optimization is equivalent to, between each item's insertion, balancing the tree as perfectly as possible. That is, requiring the strictest balancing constraint.
The best algorithm I've tested is the AVL tree. While the AVL tree is pretty terrible for insertion/deletion times for most applications, it has a decent balancing constraint which results in great look-up speeds. The rebalancing between each insertion/deletion is what makes it unperformant, but since, compared to human input, those comparisons are negligible, it is very good for my situation.
My situation requires one more constraint: the user must never be presented with the same comparison twice. For the AVL tree, rebalancing breaks this constraint, so I've just memoized all of the user's answers in a dictionary. I can't imagine a situation in which this case couldn't be handled with that memoziation, but I figured I'd mention it nonetheless.
I really have two questions:
Am I right with my assertion that balancing the tree as perfectly as possible guarantees the fewest number of prompts to the user?
Is there a better balancing algorithm than an AVL tree?
Given that before the quiz begins, the items' order is randomized, even using a non-auto-balancing BST should result in a roughly balanced tree. I could see an argument being made that perfectly rebalancing after every insertion could actually produce a greater number of prompts, since each one will have as close to lg(n) prompts as possible; a slightly less balanced tree could potentially have a few "lucky" insertions that require a few less than lg(n), and a few "unlucky" insertions that require more, but that the lucky ones will, on average, outweigh the unlucky ones. (I hope that made sense.)