I'm working on a project, and have a candidate algorithm which I'd like to test. Before I go any further, I need to get the hang of how to code the "structure" of the environment in which my system is embedded, like what the relevant signals are, how they interact with each other, how they evolve with time, and so on.

I'm looking for resources on design of the environment, for artificial intelligence problems, as I don't want to reinvent any wheels. Any information would be welcome, especially introductory to intermediate level. Thanks!

  • $\begingroup$ This looks like a strange question. Surely you know more about your environment than we do. Especially with the amount of information you're sharing in this question. $\endgroup$
    – MSalters
    Apr 28 '16 at 13:52

In reinforcement learning, the "environment" is typically a set of states the "agent" is attempting to influence via its choice of "actions".

For example, in "Reinforcement learning design for cancer clinical trials" by Zhao, Kosorok, and Zeng (2009),

..."states" may represent individual patient covariates and "actions" can be denoted by various treatments or dose levels.

More abstractly, in "A brief introduction to reinforcement learning" by Murphy (1998),

The environment is a modeled as a stochastic finite state machine with inputs (actions sent from the agent) and outputs (observations and rewards sent to the agent).

So, the "structure of the environment," what signals are relevant, and how they interact is highly specific to the problem you're trying to solve, as is the choice of where to draw the boundary between the agent and the environment.

In "Reinforcement Learning: an Overview", Sutton and Barto (1998), give an overview of the Agent-Environment interface and note:

The agent-environment boundary can be located at different places for different purposes. ... In practice, the agent-environment boundary is determined once one has selected particular states, actions, and rewards, and thus has identified a specific decision-making task of interest.

"Reinforcement Learning: A Survey" by Kaelbling, Littman, and Moore (1996) also provides an overview of the problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.