In the book Reinforcement Learning An Introduction,Chapter 8.5,there is an example that compares the efficiency of expected and sample updates:enter image description here

According to the author, "In this case, sample updates reduce the error according to $\sqrt{\frac{b-1}{bt}}$ where $t$ is the number of sample updates that have been performed (assuming sample averages, i.e., $\alpha = \frac{1}{t}$." I just wonder how to compute the sample-update error(i.e.,How to get the formula $\sqrt{\frac{b-1}{bt}}$?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.