# Questions tagged [reinforcement-learning]

The tag has no usage guidance.

37 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
531 views

### Machine learning approach to auction game

I am newbie with machine learning. In order to learn more I decided to try solving a specific problem/game that I have in mind. The problem is the following: I have a list of $N$ items which are ...
276 views

### reinforcement learning in gridworld with subgoals

Andrew Ng, Daishi Harada, Stuart Russell published a conference paper entitled Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping. There is a specific example ...
528 views

### Reinforcement learning - state space and action space

I am working on a reinforcement learning strategy for parameter control of a local search heuristic. The complete state for this RL problem can be defined as $S = \{s, p\}$, where $s$ and $p$ ...
64 views

### Relationship between dynamic programming and reinforcement learning

I wasn't sure whether to post this here or in the ai stack exchange - please let me know if i need to move my post elsewhere) I have been learning about how dynamic programming can be used as a tool ...
29 views

### Policy dependent on initial state distribution in finite horizon MDPs

Consider an MDP defined as the tuple $\langle S,A,R,P,\mu,\lambda\rangle$ where $S$ is the state space, $A$ the action space, $R:S\times A\times S\to\mathbb{R}$ the reward function, $P$ the transition ...
127 views

### Q-learned policy differs from double-Q-learned policy

I implemented Q-learning and double-Q-learning as presented in Sutton's "Reinforcement Learning: An Introduction". I test the algorithms on the OpenAI cliff walking gym and analyze the resulting ...
66 views

### How to best model multidimensional, continuous, non-convex “shape” as neural network?

I have: set of n-dimensional points that I know are inside of the shape n >= 18, range on all dimensions has upper bound and lower bound (no axis goes to infinity). shape is pretty large in this n-...
10 views

### Reinforcement learning with 0 rewards and costs

Suppose we have a hallway environment, i.e, $N$ nodes from left to right, and we can either move left or right. Moving left at the leftmost node does nothing and reaching the right most node gives you ...
20 views

30 views

### Why is the objective in a multi-armed bandit problems to minimize (cumulative) regret?

In a multi-armed bandit, the objective is to maximize the expected cumulative reward. This objective is usually (equivalently?) stated in terms of expected cumulative regret. Question: Why not just ...
17 views

### How to compute the sample update error?

In the book Reinforcement Learning An Introduction，Chapter 8.5，there is an example that compares the efficiency of expected and sample updates: According to the author, "In this case, sample updates ...
42 views

### What does actually happens on tile hashing

I am going through Richard Sutton book about Reinforcement Learning and I just encountered the tile coding method. I understood pretty well the principles, however, at the very end of the section, ...
62 views

### Multi Arm Bandit (MAB) — Increasing reward function

In the general stochastic MAB model, the reward obtained at each trial is generally assumed to be independent of previous trials and obtained from some fixed (but unknown) distribution. However, if ...
29 views

### Markov Decision Process Optimal Policy

Consider the setting of finite MDPs. I will be using the notation in Chapter 2 of http://rll.berkeley.edu/deeprlcourse/docs/ng-thesis.pdf. Say we have already computed values for the optimal $Q$-...
71 views

would be really thankful if someone could clear this up. In reinforcement learning, When we use basis like the fourier or say polynomial basis, and say we have 4 actions, theta will then be a N x4 ...
75 views

### Inconsistent results with Q-learning

I'm running a q-learning agent in tic tac toe against a minimax agent and measuring the tie rate (since you can never win against minimax). When I run 10,000 training games, I've found that there seem ...
39 views

### Middle ground between model-based and model-free approaches

In the context of the reinforcement learning domain, the dichotomy between model-based (learn a model and used it to determine a controller) and model-free (learn a controller without learning a model)...
4 views

### Inferring reward function and transition model from optimal policy

Consider an MDP where the transition model and the reward function are unknown. Consider an optimal policy $\pi^*$ generated from this MDP (say by some oracle who does know the transition model and ...
19 views

### Motivation for Inverse Reinforcement Learning

Learning a policy from sparse reward information (a reward function where a positive reward is only generated at the goal state) is challenging due to the resulting sparse feedback. One solution is to ...
20 views

### How to get best path from a set of path using Q-Learning?

I have 10 data sets (lat and long) of the same path. I started from point A and stopped in point B and did this 10 times for a single route to get the data. While collecting the data, there were some ...
28 views

### Questions on AlphaZero Implementation

So I've been implementing AlphaZero for Chess from scratch and there were a few things the papers mentioned that I'm not sure how to implement. I'll reference both the original AlphaGoZero paper and ...
17 views

### Is MDP Considered as The Model-based Value Iteration in/of Reinforcement Learning?

Is MDP Considered as The Model-based Value Iteration in/of Reinforcement Learning? If no, then Reinforcement Learning is all about being Model-free learning. Right?
82 views

### Transition Function in MDP

I got a question about who and how sets the transation function values in markov decision processes? I mean when some says that an agent, in real world grid, is going to step up by %80 and left/right ...
35 views

### Is there a difference in the convergence analysis/proof of the chaotic learning automaton compared to the standard LA?

We have recently presented an article entitled Improving learning ability of learning automata using chaos theory. In this article, a new type of learning automaton called chaotic Learning Automaton (...
19 views

### Minimizing the length a Boolean Algebra Expression in disjunctive normal form

I'm looking to minimize the length of an expression in boolean algebra that has been given in disjunctive normal form and is free from redundancy. To remove redundancy from the original expression I ...
17 views

### Path planning agent in a grid with compulsory states to visit with Reinforcement Learning

Is it possible to make a Reinforcement Learning agent for path planning in a 2D grid where visiting certain intermediate states is mandatory? Please give an insight if it is indeed possible as to ...
29 views

### Use ML to create a graph

I'm currently looking for literature/papers on machine learning techniques to create structures. In detail, I want to generate finite automata (NFA, DFA), which are useful for student-exercises. So I ...
133 views

### Converting Turing machine into the source code in industrial programming language?

Are there methods how to convert Turing machine (e.g. neural Turing machine or other rigorous Turing machine) into the source code/program that is written in some industrial programming language like ...
29 views

### Why is bias not defined for non-stationary MDP policies?

The Handbook of Markov Decision Processes 1 defines bias of a stationary deterministic policy as follows: $h(x, \phi) = \sum_{n = 0}^{\infty} E[r(x_n, \phi(x_n)) - w(x_n , \phi)]$ where $x$ is a ...
86 views

### Hessian in reinforcement learning

The Hessian of multi-layered network exhibits known behaviour at critical points as shown in . The tools of random matrix theory allow  to deduce the asymptotic distribution of the eigenvalues ...
I have searched a lot for this, but apparently there is no result on calculating any bound on the error $||Q-Q^*||$ when I stop Q-learning after say $N$ iterations ($Q$ is the vector of Q-values at ...