0
$\begingroup$

I am having some trouble with answering the following question:

A rat is involved in an experiment. It experiences one episode. At the first step it hears a bell. At the second step it sees a light. At the third step it both hears a bell and sees a light. It then receives some food, worth +1 reward, and the episode terminates on the fourth step. All other rewards were zero. The experiment is undiscounted. Approximate the state-value function by a linear combination of these features with two parameters: $b · bell(s) + l · light(s)$. If $b = 2$ and $l = −2$, write down the sequence of $\lambda$-returns $v_t^\lambda$ corresponding to this episode, for $\lambda = 0.5$.

The provided answer is this:

$v^\lambda_1 = 0.5 (−2 + 0.5×0 + 0.5×1) = −3/4$

$v^\lambda_2 = 0.5 (0 + 1×1) = 1/2$

$v^\lambda_3 = 0.5 (2×1) = 1$

However, these are my calculations:

$v^\lambda_1 = 0.5 (.5^0×−2 + .5^1×0 + .5^2×1) = 0.875$

$v^\lambda_2 = 0.5 (.5^0×0 + .5^1×1) = 1/4$

$v^\lambda_3 = 0.5 (.5^0×1) = 1/2$

Any help is appreciated!

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.