The tag has no usage guidance.

learn more… | top users | synonyms

2
votes
0answers
17 views

Why do we use expectation in reinforcement learning? [migrated]

In reinforcement learning we use lot of expectations, what's the logic behind them? Like in reward function, we use expected return instead of return, in value functions V and action-value functions ...
3
votes
0answers
28 views

How to best model multidimensional, continuous, non-convex “shape” as neural network?

I have: set of n-dimensional points that I know are inside of the shape n >= 18, range on all dimensions has upper bound and lower bound (no axis goes to infinity). shape is pretty large in this n-...
1
vote
1answer
34 views

Reinforcement Learning - Q Learning

I am having trouble understanding the following problem and Q learning in general. What I know so far about Q learning is that Q-learning is a model free method, i.e., it doesn’t need to learn P(s’|...
0
votes
1answer
27 views

Is TD-learning considered a model-based algorithm?

Differently from Sarsa and Q-learning, pure temporal difference learning (TD-learning) works with state value functions $V(s)$ and not state-action Q value functions $Q(s,a)$. It means that, in order ...
3
votes
0answers
20 views

Inconsistent results with Q-learning

I'm running a q-learning agent in tic tac toe against a minimax agent and measuring the tie rate (since you can never win against minimax). When I run 10,000 training games, I've found that there seem ...
0
votes
0answers
14 views

tips on how to define a state space for reinforcement learning

I have read some reinforcement learning model examples of various things and I was surprised by how varied unintuitive some of the state spaces are. For example: in the standard grid world, it is ...
1
vote
1answer
25 views

Difference between SNN RL and DNN RL?

In Reinfrocement Learning (RL) in Neural Networks (NNs), I've seen two approaches to Q-learning. The first is to tile the state space with basis functions using Spiking Neural Networks (SNN) to ...
1
vote
1answer
40 views

reinforcement learning in gridworld with subgoals

Andrew Ng, Daishi Harada, Stuart Russell published a conference paper entitled Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping. There is a specific example ...
1
vote
1answer
14 views

Using a combination of spatial and non-spatial inputs for convolutional neural networks

I'm working on training a game AI using deep reinforcement learning to achieve specific examples based on pixel input and some additional state information. Naturally, I'm using a convolutional ...
2
votes
1answer
32 views

How is the environment designed for testing a reinforcement learning algorithm?

I'm working on a project, and have a candidate algorithm which I'd like to test. Before I go any further, I need to get the hang of how to code the "structure" of the environment in which my system is ...
1
vote
0answers
25 views

Dyna-Q in non-deterministic domains

I've implemented the Dyna-Q reinforcement learning algorithm and it works perfectly on a discrete deterministic environment, the cliff. However, when applying it to a continuous environment (mountain ...
3
votes
1answer
43 views

Apprenticeship vs. imitation learning - what is the difference?

There are many sub-domains of reinforcement learning, two of which are apprenticeship and imitation learning - but that appears just to be two different names for the same thing?
1
vote
0answers
21 views

Middle ground between model-based and model-free approaches

In the context of the reinforcement learning domain, the dichotomy between model-based (learn a model and used it to determine a controller) and model-free (learn a controller without learning a model)...
0
votes
1answer
25 views

What does it mean to have a continuous action space w.r.t. to reinforcement learning?

Last time I posted this question I got criticised for not being specific enough, hence this is my second attempt at trying to understand what it means to have a continuous action space. Please refer ...
2
votes
1answer
30 views

single agent vs multiple agent reinforcement learning

I am confused about 'single' vs 'multiple' agent reinforcement learning. Let's say that I have 1 hunter who I am training to hunt 1 static prey, so that only the hunter is moving around. This is ...
4
votes
0answers
63 views

Reinforcement Learning - Agent training

In RL, in a game situation, usually the agent is trained by playing against itself. When we should not depend on this self-training, and switch to train the agent with a real or different player ...
0
votes
0answers
14 views

Memory storage capacity of Bienenstock-Cooper-Munro rule

I would like to know the memory storage capacity of the BCM learning rule when it is implemented on a Hopfield network. I understand that it will be a function of n where n is the number of neurons.
1
vote
1answer
34 views

Combining Production Rules using Reinforcement Learning

Production systems have been used to solve puzzles such as the Tower of Hanoi for years with hard-coded production rules. However, has there been any research in using reinforcement learning to ...
5
votes
1answer
74 views

The meaning of discount factor on reinforcement learning

After reading of the google deepmind achievements on Atari's games, I am trying to understand the q-learning and q-networks, but I am little bit confused. The confusion arise in the concept of the ...
1
vote
0answers
35 views

How to determine convergence when using Q-learning?

I'm using Q-Learning to find the values of states in on a gameboard. For example, something like: ...
2
votes
1answer
53 views

How does SARSA handle episode termination

When applied to domains that are episodic and have a "final" state but no final action, like a game, how does SARSA update the Q-values? e.g. A game agent would receive this series: $$ s_0,a_0,...
6
votes
1answer
225 views

What's the difference between Adaptive Control and Hierarchical Reinforcement Learning?

After watching Travis DeWolf presentation on scaling neural computation, I'm a bit confused about the difference between Reinforcement Learning (whether hierarchical or not) and Adaptive Control. They ...
3
votes
0answers
49 views

Reinforcement learning - state space and action space

I am working on a reinforcement learning strategy for parameter control of a local search heuristic. The complete state for this RL problem can be defined as $S = \{s, p\}$, where $s$ and $p$ ...
3
votes
1answer
140 views

Why does ε-greedy $Q$-learning not oscillate?

I have a intuitive question on the convergence of $Q$- learning. In $Q$ learning for each step a $Q$- value is learned for the state-action pair where the action is selected according to the $\epsilon$...
2
votes
1answer
249 views

Q-learning in a Dynamic environment

I am new to reinforcement learning. Lately, I have learned Q-learning using the following tutorial. Is Q-learning still possible if the environment is dynamic. Using the environment of the tutorial ...