1
$\begingroup$

In the book Reinforcement Learning An Introduction,Chapter 8.5,there is an example that compares the efficiency of expected and sample updates:enter image description here

According to the author, "In this case, sample updates reduce the error according to $\sqrt{\frac{b-1}{bt}}$ where $t$ is the number of sample updates that have been performed (assuming sample averages, i.e., $\alpha = \frac{1}{t}$." I just wonder how to compute the sample-update error(i.e.,How to get the formula $\sqrt{\frac{b-1}{bt}}$?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.