# Q-Learning Error Bounds

I have searched a lot for this, but apparently there is no result on calculating any bound on the error $||Q-Q^*||$ when I stop Q-learning after say $N$ iterations ($Q$ is the vector of Q-values at current iteration and $Q^*$ is the optimal Q-value). Let's assume that you have visited each state-action pair at least once and you know/store the number of visits. I would really appreciate any pointer to any relevant reference.

• I'm not sure it's possible to get any good bounds on this error. It's always possible that the next iteration will lead to some amazing reward you'd never seen before. – Draconis Jun 4 '18 at 17:16
• I'll update my question: let's assume that you have visited each state-action pair at least once and you store the number of visits. – Perissiane Jun 5 '18 at 15:30