2
$\begingroup$

I'm using Q-Learning to find the values of states in on a gameboard. For example, something like:

-----------------
|   |   |   | 1 |
-----------------
|   | W |   | -1|
-----------------
|   | W |   |   |
-----------------
| S |   |   | 1 |
-----------------

I have the transition, and I can calculate the values of the states, but I can't tell when to stop computing. Normally I would just compute until a terminal state, for x-episodes; but this particular board has no terminal states. So I'm stuck to just calculating the values over say 1000 iterations (the more the more accurate obviously). I would like a way to stop though when I'm within an acceptable range.

How is this normally done for a non-terminal state space?

$\endgroup$
1
$\begingroup$

There are two ways that I know of: First, based on how many iterations, like you would do. Second, based on the difference between current Q value and previous Q value (i.e., insignificant difference means termination).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.