0
$\begingroup$

Learning a policy from sparse reward information (a reward function where a positive reward is only generated at the goal state) is challenging due to the resulting sparse feedback.

One solution is to shape the reward to enrich feedback during learning. Unfortunately, shaping the reward is challenging and often leads to unusual behavior or even situations where the learning agent just harvests intermediate rewards rather than achieving the task.

A proposed solution is to have an expert generate demonstrations of the desired task and have the learning agent learn a reward function that is consistent with the demonstrated behavior. The learning agent can then use this learned reward function to learn an optimal policy.

This last point is where I am confused. The whole motivation for IRL is learning, that is, finding a policy when the model is unknown, but why would the learning agent not know its own transition model? On the other hand, if the transition model is known, then finding the optimal policy from the learned reward function is not actually a learning problem.

$\endgroup$
0
$\begingroup$

Often the transition model is not known in advance. For instance, consider a complex system like flying a helicopter or surfing a surfboard. It's not easy to characterize in advance what the behavior of the system will be, in response to certain control inputs; the relationship may be highly nonlinear and complex.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.