Say I'd like to build a Go AI. The Go AI takes in the board state and then predicts who's more likely to win from that state. When I want to make a move, I just test every next board state I could make, and play the move that makes me most likely to win. I'm interested in how such a model could be trained by self-play but it isn't clear to me what the loss function should be.
Say I use the model to play a batch of games against itself, or even just one game. Well the model both won and lost both games. How do I train it from there? Clear this setup is naive but somehow alpha Go zero was able to train just from self play and this question applies to arbitrary two player games of almost any kind. In fact the real game I'm interested in has iterated simultaneous moves.
One method I thought of was to train two separate models with the same topology, and pit them against each other. I'm not really sure how well this would work. Also at the end why do I have two models when I just want one? This also just feels inefficient.
How are models trained for self-play? What loss function is used in practice?