0
$\begingroup$

I have searched a lot for this, but apparently there is no result on calculating any bound on the error $||Q-Q^*||$ when I stop Q-learning after say $N$ iterations ($Q$ is the vector of Q-values at current iteration and $Q^*$ is the optimal Q-value). Let's assume that you have visited each state-action pair at least once and you know/store the number of visits. I would really appreciate any pointer to any relevant reference.

$\endgroup$
  • $\begingroup$ I'm not sure it's possible to get any good bounds on this error. It's always possible that the next iteration will lead to some amazing reward you'd never seen before. $\endgroup$ – Draconis Jun 4 '18 at 17:16
  • $\begingroup$ I'll update my question: let's assume that you have visited each state-action pair at least once and you store the number of visits. $\endgroup$ – Perissiane Jun 5 '18 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.