I am trying to apply MADDPG, a policy gradient algorithm that uses centralized training and decentralized execution, to a project. In the work of Lowe et al., the actor returns a pmf for a discrete actions pace. If I set it to Discrete(5) for each agent, it would return something like [0.1 0.15 0.05 0.34 0.46]. I am confused about how is this deterministic? Shouldn't it collapse to a specific actions [0 0 0 0 1]? Also, then it proceeds to apply these operations for the MPE (multi-particle-environment):
agent.action.u += action - action agent.action.u += action - action
Then in the environment, it seems this u represents the force:
p_force[i] = agent.action.u
Shouldn't it sample from that distribution to take an action? How is it that manipulating the elements of the action can get the force? What is action for?