1
$\begingroup$

Last time I posted this question I got criticised for not being specific enough, hence this is my second attempt at trying to understand what it means to have a continuous action space.

Please refer to this paper, page 5 (A Survey on Policy Search for Robotics by Marc Peter Deisenroth, Gerhard Neumann and Jan Peters. Foundations and Trends in Robotics. 2013).

The authors say:

The RL algorithm has to manage (i) high-dimensional continuous state and action spaces, (ii) strong real- time requirements, and (iii) the high costs of robot interactions with its environment.

I do not understand how an action can be continuous, in the same way that a state-vector is continuous by virtue of having elements that are simply $x_k \in \mathbb{R}^D$ for a state vector $\mathbf{x} := [x_1,\ldots,x_K]^T$. But how can an action be quantified in the same way? It obviously can I simply do not understand how.

$\endgroup$
4
$\begingroup$

When you're driving of your car and you turn the wheel, is that a discrete or a continuous action? It's continuous, because you can control how much you turn the wheel. How much do you press the gas pedal? That's a continuous input. This leads to a continuous action space: e.g., for each positive real number $x$ in some range, "turn the wheel $x$ degrees to the right" is a possible action.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.