4
$\begingroup$

Andrew Ng, Daishi Harada, Stuart Russell published a conference paper entitled Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping.

There is a specific example there that I am extremely curious/interested about. It is in Figure 2(a) of the paper:

5x5 grid world with 5 subgoals (including goal state), which must be visit in order 1, 2, 3, 4, G

It is about a 5x5 gridworld with start and goal states in opposite corners. The catch is, the agent must learn to go from start to end BY VISITING specific subgoals 1,2,3,4 IN ORDER.

Has anyone seen/understood the code for this? I want to know how the reward function/shaping is given in this kind of problem.

I am interested to know how the flow of this modification to the grid world is written.

Improve this question
$\endgroup$
0
$\begingroup$

The subgoals are encoded in external rules to determine if a reward is given and in what amount. The creation of the rules and the used features is done manually by experts. There is no algorithm which calculates the reward function by itself.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The reason I asked because I am encountering a similar problem where there is a main goal, but I want the agents to learn first the subgoals. I'm not sure if the subgoals are hardcoded in this paper. But I definitely think that in order for the agent to learn the subgoal, then a reward must be given to reinforce the behavior. Is there any other way for the agent to go to the goal via the subgoals without explicity giving a reward while passing by each subgoal? $\endgroup$
    – cgo
    May 26 '16 at 9:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.