I have searched a lot for this, but apparently there is no result on calculating any bound on the error $||Q-Q^*||$ when I stop Q-learning after say $N$ iterations ($Q$ is the vector of Q-values at current iteration and $Q^*$ is the optimal Q-value). Let's assume that you have visited each state-action pair at least once and you know/store the number of visits. I would really appreciate any pointer to any relevant reference.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ I'm not sure it's possible to get any good bounds on this error. It's always possible that the next iteration will lead to some amazing reward you'd never seen before. $\endgroup$– DraconisCommented Jun 4, 2018 at 17:16
-
$\begingroup$ I'll update my question: let's assume that you have visited each state-action pair at least once and you store the number of visits. $\endgroup$– PerissianeCommented Jun 5, 2018 at 15:30
Add a comment
|