Neural network game players and incremental updates

Question

Neural networks in recent years have been successfully used for gameplaying. A difference between games and e.g. image processing is that the game boards get updated incrementally. Do any neural network implementations optimize for this fact?

For example, there are 361 squares on a Go board; in a typical turn, only one of those squares will change value. On the face of it, a program like AlphaGo could save time by not redoing the calculations from the other 360 squares each time it looks at a new board position.

On the other hand, if the calculations are being done on a GPU, this optimization involves moving away from the straight vector arithmetic that GPU's do so well, which would offset the efficiency gain.

On the third hand, the potential efficiency gain would become larger as you move to more complex games; how many variables are there in StarCraft?

Has anyone ever done this optimization, or studied it and given reasons for not doing it?

Answer 1

The AlphaGo algorithm does not benefit from this kind of optimization. It's hard to share an intuition for why without diving into the details of the algorithm, but at a coarse level: a single small change to the board can make a dramatic change to the strategic nature of the position, so it's not clear that one can reuse calculations effectively. In particular, it sounds like you are assuming that AlphaGo does some computation separately for each square of the board, so we could avoid re-doing the same ones for squares that haven't changed. However, that's not at all how AlphaGo works, so that kind of optimization is not applicable here.

Answer 2

Try changing your input in this case if you wanted to take only the difference of two states into consideration. Think of each rounds in a discrete manner, they will be of their own distinctive state if not encountered the exactly same configuration of the board beforehand.

What do you get? You get another set of states that takes differentiated states into account. So they are no different than the previous set of states, a raw states. It does not significantly change your problem but in a game where state changes are not substantial in each round, you might be able to get a slight advantage in input variable size.