0
$\begingroup$

I have a grid environment which contains in each cell a static agent. When my agent enters a cell, the static agent in this cell might take points away from me, give me points, or do nothing. My agent can not observe adjacent cells until it moves into one them, it can only move up, down, left or right.

This agent can not learn whilst it is exploring. It enters the grid from a particular corner, and may only leave from that corner. If the agent manages to successfully explore the environment and return back to the corner with positive health, then it can learn from its collected experience, which includes the (row, column) positions it visited, and the properties of the static agents located in those positions. If the agents' health drops to zero whilst exploring, game over. But I can restart the exploration as many times as I want.

Each static agent has one of three shapes, one of three colours, and one of two sizes. It also has an associated "reward", indicating how many points it adds/removes from me.

Each move in this environment costs me one point. I want to design an agent that correctly recognises the reward associated with each type of static agent in this grid.

Please may someone recommend a learning and/or evolutionary approach to solving this problem? I'm stuck at the moment because of the restriction that agent may not observe adjacent squares. I'm not sure how I can learn anything from this testing environment from the (row, column) and static agent properties encountered alone.

$\endgroup$
2
  • $\begingroup$ What have you tried? What is the context/motivation for this problem? Why do you say you cannot observe adjacent squares? What's the relation between health and points? At present this is a dump of a problem, not a question. If you have a specific question regarding the wording of the problem or about concrete steps in your own attempts at solving the problem, feel free to edit accordingly and we can reopen the question. See here for a relevant discussion. $\endgroup$
    – D.W.
    Mar 23, 2014 at 6:55
  • $\begingroup$ I feel your frustration, but I didn't know where to start, hence why there is no attempt. I now have a buzzword, Q-Learning, which I can try. Hopefully after this foray I will be able to pose more specific questions. Thank you for your patience. $\endgroup$ Mar 23, 2014 at 13:13

1 Answer 1

0
$\begingroup$

Obviously one of the best learning methods that matches this problem is the Reinforcement Learning and to be more specific Q-Learning.
You'll find a good starting point here.
Hints: Q-learning is quite well-known and I don't repeat others here, but I give you the key point here.

At each step keep track of what happens (reward/punishment - no need to agent types,... if the environment stays fixed and you want to learn this Env. not the agents) using for example an array (Fortunately you're not learning anything so far!). Keep doing this until the agent stops, that as you said can have two reasons :

  1. Agent's health drops to zero : Do nothing! (discard the array)
  2. Agent leaves from the other corner : Apply the following rule with the values in the array

enter image description here

$\endgroup$
1
  • $\begingroup$ @user3450881 You're welcome. $\endgroup$ Mar 23, 2014 at 13:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.