I am trying to apply MADDPG, a policy gradient algorithm that uses centralized training and decentralized execution, to a project. In the work of Lowe et al., the actor returns a pmf for a discrete actions pace. If I set it to Discrete(5) for each agent, it would return something like [0.1 0.15 0.05 0.34 0.46]. I am confused about how is this deterministic? Shouldn't it collapse to a specific actions [0 0 0 0 1]? Also, then it proceeds to apply these operations for the MPE (multi-particle-environment):

agent.action.u[0] += action[0][1] - action[0][2]
agent.action.u[1] += action[0][3] - action[0][4] 

Then in the environment, it seems this u represents the force:

p_force[i] = agent.action.u

Shouldn't it sample from that distribution to take an action? How is it that manipulating the elements of the action can get the force? What is action[0][0] for?

  • $\begingroup$ Can you give a proper reference for Lowe et al? Include the paper title, the full list of authors, where it was published, and if possible, a link to a freely available PDF. Can you explain what MADDPG is? Perhaps give a citation to where it was introduced, or some background on what it is. Please ask only one question per post. You seem to be referring to some code without context; I have no idea what action[0][0] refers to, so I'm not sure it's going to be possible to answer this question without that context. $\endgroup$ – D.W. Apr 16 at 23:14
  • $\begingroup$ Thank you. This is the paper. arxiv.org/pdf/1706.02275.pdf It is a policy gradient algorithm that uses centralized training and decentralized execution. I want to use it but I cannot do that without understanding that specific issue. action[0] is [0.1 0.15 0.05 0.34 0.46] so action[0][1] is 0.15. for a specific agent. $\endgroup$ – Diego Benalcázar Apr 17 at 0:03
  • 1
    $\begingroup$ Thank you! Please edit the question to incorporate information into it, rather than leaving information in the comments -- we don't want people to have to read the comments to understand what is being asked. Don't just add to the question -- instead, revise it so it reads well for someone who encounters it for the first time. $\endgroup$ – D.W. Apr 17 at 1:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.