How do you protect an AI from a human doing "illogical" moves?

Question

Using a monte carlo approach and evalutation function. Some moves will deemed to be more advantageous than others.

As a computer plays itself, it will generally go for the best moves possible. And hence train itself to deal with these moves.

But what if a human does a few seemingly detrimental moves to throw the AI off track. And then using the human's expertise, go on to win the game?

How can one train an AI to account for this?

Or must we assume that it can generalise board positions from "perfect" games to board positions of imperfect games?

(Presumably AlphaZero has no problem with this? Has anyone ever tried to trick AlphaZero by doing a few silly moves?)

Answer 1

In principle, the problem of building an AI where this can never happen seems equivalent to the problem of building an AI that plays perfectly.

More precisely: If the AI can perfectly evaluate the value of each board position, then it is also able to play perfectly even if the adversary plays imperfectly. Conversely, if there is a situation where the AI can be tricked by playing an inferior move, then that means there is a board position that the AI evaluates incorrectly.

So avoiding this situation seems as hard as building an AI that never makes the wrong decision. If we knew how to do that, we would have a perfect solution to the game. There are some games (e.g., tic-tac-toe, Connect 4) where we do know how to achieve perfect play, but for more challenging games (e.g., chess), we don't know how to achieve perfect play. As I result, it seems likely there exists a board position where playing an inferior move helps you win where you otherwise wouldn't have.

This means that we need to focus on solutions that reduce the risk/likelihood of this bad situation, as we cannot eliminate it entirely.

When the AI is trained using statistical machine learning (as AlphaGo and AlphaZero are), you can think of this as avoiding overfitting -- you don't want to overfit to any one opponent strategy. One way that AlphaZero is trained is to play simulated games against multiple opponents -- not just against itself, but against multiple versions of itself, including older (and presumably inferior) versions of itself. This helps avoid getting stuck in a narrow portion of the space, e.g., where white always plays the Ruy Lopez opening and black always plays the Morphy defense. This may partly help make the AI more robust against inferior moves, but it is not a complete solution.

In Monte Carlo tree search (MCTS), the algorithm considers random moves for the opponent. These are drawn from a random distribution that is biased towards moves the algorithm thinks is better, but there is definitely a significant chance during any playout that we'll simulate what happens if the opponent makes a move we consider detrimental/inferior. This helps MCTS be robust against even inferior moves. It's not perfect, but it's already baked into the MCTS algorithm.