In value iteration algorithm we update the utility of all possible states ("for each state update its new utility").

After we've updated all states we check to see if the delta is smaller than some value. If not we loop again over all sates.

If we have 2 adjacent sates say A, B - after updating A we can move on to updating since they are adjacent. Is it possible to go back to update A again since its value will be different again (since B just changed) ? or is the order should be to go over all states exactly once per the inner loop and go back to a state already visited only when we implement the inner loop again ?

  • 1
    $\begingroup$ Can you give a description or a reference to this 'value iteration algorithm in MDP'? Additionally, it would be useful if you can explain why this description does not already give an answer to your question. $\endgroup$ – Discrete lizard Feb 3 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.