In RL, in a game situation, usually the agent is trained by playing against itself.
When we should not depend on this self-training, and switch to train the agent with a real or different player agent other than the agent it self?
I mean is there any characteristics of the environment that would make self-play worse choice than playing against an existing player?
Ideally when training an agent you would reach an equilibrium between both player strategies strategies (hopefully a Nash equilibrium). When using only self-play this equilibrium might be different to the one you would reach when competing against a specific player, in particular, taking into consideration that the other player might not be very efficient, may have a different reward structure, etc.
Now, answering your specific question about the environment.. I would say that the critical aspect is the game being played. You should check if you are dealing with a zero-sum game, cooperative vs non-cooperative, etc.
