Questions tagged [reinforcement-learning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
31 views

How to distinguish between Unrepresentative Train Dataset and minimal overfitting from a learning c

Hi I saw a picture in a group and I asked to myself what exactly is this? Is this a minimal over fitting or like unpresensitive train dataset? However I read it on the net but still have problem in ...
Amirreza Hashemi's user avatar
1 vote
0 answers
35 views

DeepMind Alphadev: How did it use Reinforcement Learning to reduce the search space?

Google DeepMind recently published a new paper which describes how they used a reinforcement learning to discover faster sorting algorithms. A summary of the paper is here and the paper is here. It ...
equis's user avatar
  • 11
0 votes
0 answers
22 views

How to solve MAB by linear program?

To solve multi-armed bandit problem, the common approaches are UCB or TS and there are many variants of these algorithms. I am wondering if it is possible to model and solve this problem as a linear ...
Amin's user avatar
  • 101
0 votes
0 answers
9 views

Role of State Dynamics Target Network in DDPG

I am trying to create a variant of DDPG in MATLAB that has no action-value $\langle Q \rangle$ net, but that instead works with networks $\langle V \rangle, \langle f \rangle, \langle r \rangle$, and ...
Vera Leighton 's user avatar
0 votes
0 answers
16 views

Bellmann Error with Time Step

I define the return $G$ as the discrete time integral of the reward $G = \Delta t \sum_{t = 0, ..., T}r_t$. In supposing $\Delta t = 1$ throughout, my instructor wrote the following formula for the ...
Vera Leighton 's user avatar
0 votes
2 answers
78 views

Multi-Armed Bandit - Reward Probabilities

I am new to reinforcement learning, and recently came across the following issue. When implementing a multi-armed bandit algorithm, we assume we have k machines with reward probabilities [p_1,..., p_k]...
alextan's user avatar
1 vote
0 answers
136 views

Tabular Meta-Learning in RL

There are various meta-learning algorithms in RL that are proposed for settings when we have a (deep) neural network and the policy (or the value function) are parameterized as such. Can these methods ...
Perissiane's user avatar
0 votes
1 answer
54 views

Reinforcement Learning Reward Function for Optimizing Golf Aim?

I read this article, mentioning that either here, or StackOverflow would be the best places to ask generic machine learning questions, however, if the question isn't programming specific with a ...
wildcat89's user avatar
  • 103
1 vote
0 answers
28 views

Long and short memory in reinforcement learning Connect 4 AI

I'm writing an AI based on reinforcement learning to play Connect 4. That's my second bot and attempt to RNN and AI (first was copy a code of snake RNN AI from youtube) and I'm looking for some ...
Saguro's user avatar
  • 11
1 vote
0 answers
13 views

Evaluating the safety of deep RL algorithms

Is there any free/open-source environment, tasks, or dataset for evaluating deep RL algorithms in terms of safety? all available environments (like openAI's) are environments for simple games. These ...
mac179's user avatar
  • 161
1 vote
0 answers
36 views

Deep RL for healthcare: existing benchmarking datasets or environments

I am currently a Ph.D. student in the computer science department, I was given the subject of Deep RL for Healthcare. However, after lots of research on the internet, I could not find any free dataset ...
mac179's user avatar
  • 161
2 votes
0 answers
29 views

Where to find the current state of the art performance of Deep RL algorithms?

Recently, I had an idea of a novel Deep RL algorithm that might perform better than existing algorithms such as DQN, TRPO, PPO, etc. However, I do not know of a website or a research paper that might ...
mac179's user avatar
  • 161
2 votes
1 answer
65 views

What makes Deep RL "fundamentally/mathematically" advantageous?

Note: I consider myself to be a beginner in the field of Deep RL. Deep RL has proven tremendous success in recent years like playing atari and beating go champion. Therefore, considerable interest for ...
mac179's user avatar
  • 161
3 votes
2 answers
813 views

How to setup the Bellman Equation as a linear system of equation

I was watching a video on Reinforcement Learning by Andrew Ng, and at about minute 23 of the video he mentions that we can represent the Bellman equation as a linear system of equations. I am talking ...
krishnab's user avatar
  • 151
0 votes
1 answer
26 views

What is the relation between the compatibile features and the state features in Actor Critic Algorithm?

According to Actor-Critic algorithm, $\psi_{\theta}=\nabla_{\theta}\ln \mu_{\theta}(s, a)$ where $\mu_{\theta}(s, a)$ is the policy followed by the actor and $\psi_\theta$ is the compatibile features ...
Roshan Jacob's user avatar
1 vote
0 answers
13 views

Reinforcement learning with 0 rewards and costs

Suppose we have a hallway environment, i.e, $N$ nodes from left to right, and we can either move left or right. Moving left at the leftmost node does nothing and reaching the right most node gives you ...
Just_A_Doubt's user avatar
0 votes
0 answers
8 views

Inferring reward function and transition model from optimal policy

Consider an MDP where the transition model and the reward function are unknown. Consider an optimal policy $\pi^*$ generated from this MDP (say by some oracle who does know the transition model and ...
Erik M's user avatar
  • 183
0 votes
1 answer
70 views

Motivation for Inverse Reinforcement Learning

Learning a policy from sparse reward information (a reward function where a positive reward is only generated at the goal state) is challenging due to the resulting sparse feedback. One solution is to ...
jonem's user avatar
  • 371
0 votes
0 answers
26 views

How to get best path from a set of path using Q-Learning?

I have 10 data sets (lat and long) of the same path. I started from point A and stopped in point B and did this 10 times for a single route to get the data. While collecting the data, there were some ...
mnijhum's user avatar
1 vote
0 answers
26 views

Resources on Dynamic Programming with Indefinite Recursion

I am trying to explain the value iteration method that is used in reinforcement learning. The method is used to estimate a solution to a recursive equation like: $ Return(state_t,action_t) = Reward(...
farhanhubble's user avatar
2 votes
0 answers
150 views

Discrete action space for MADDPG

I am trying to apply MADDPG, a policy gradient algorithm that uses centralized training and decentralized execution, to a project. In the work of Lowe et al., the actor returns a pmf for a discrete ...
Diego Benalcázar's user avatar
1 vote
0 answers
19 views

Where can I learn theoretical aspect of Reinforcement learning

Every RL paper has a section with some finite sample analysis/error bound/ convergence proofs. I have difficulty fully understanding such proofs and coming up with my own analysis for my personal ...
NPN's user avatar
  • 11
1 vote
0 answers
50 views

Proving facts in inverse reinforcement learning [closed]

I was going through paper titled "Algorithms for Inverse Reinforcement Learning" by Andrew Ng and Russell. It states following basics: MDP $M$ is a tuple $(S,A,\{P_{sa}\},\gamma,R)$, where ...
Rnj's user avatar
  • 225
1 vote
1 answer
185 views

Reinforcement learining basics - is it possible to deal with environment with random events

As part of data science course I would like to solve particular problem with reinforcement learning algorithm. I believe I understand general concept, however the problems I had read about up till now ...
White_Raven's user avatar
1 vote
0 answers
59 views

Reinforcement learning and Graph Neural Networks: Issue with entropy [closed]

I am currently working on an experiment to link reinforcement learning with graph neural networks. This is my architecture: Feature Extraction with GCN: there is a fully meshed topology with ...
Micha Christ's user avatar
2 votes
0 answers
282 views

Relationship between dynamic programming and reinforcement learning

I wasn't sure whether to post this here or in the ai stack exchange - please let me know if i need to move my post elsewhere) I have been learning about how dynamic programming can be used as a tool ...
quest ions's user avatar
0 votes
1 answer
41 views

What is softmax in reinforcement learning?

There are multiple very complicated articles about softmax online. I just want to know a few things about it: Why do we need softmax? What is the "problem"? What does it do? How does it do ...
Kuroi Ryū's user avatar
0 votes
1 answer
498 views

Transition Function in MDP

I got a question about who and how sets the transation function values in markov decision processes? I mean when some says that an agent, in real world grid, is going to step up by %80 and left/right ...
Aki's user avatar
  • 1
3 votes
1 answer
229 views

Policy dependent on initial state distribution in finite horizon MDPs

Consider an MDP defined as the tuple $\langle S,A,R,P,\mu,\lambda\rangle$ where $S$ is the state space, $A$ the action space, $R:S\times A\times S\to\mathbb{R}$ the reward function, $P$ the transition ...
jonem's user avatar
  • 371
0 votes
0 answers
106 views

Minimizing the length a Boolean Algebra Expression in disjunctive normal form

I'm looking to minimize the length of an expression in boolean algebra that has been given in disjunctive normal form and is free from redundancy. To remove redundancy from the original expression I ...
Fergus Kavanagh's user avatar
1 vote
1 answer
75 views

What solution to apply for finding the optimal parameters?

For a study, I have a system (black-box) that requires an input in the form of an array with 4 values (input_array) and depending on their values it produces an ...
Cristian M's user avatar
1 vote
2 answers
38 views

Time efficient way to implement Multi-Armed-Bandits?

I'm doing a research on Multi-Armed Bandit (MAB) problem with approx. 1 million arms. In contrast, the number of iterations is of course much larger, about 10-20 million. Most MAB-algorithms require ...
D. B.'s user avatar
  • 21
0 votes
0 answers
18 views

Path planning agent in a grid with compulsory states to visit with Reinforcement Learning

Is it possible to make a Reinforcement Learning agent for path planning in a 2D grid where visiting certain intermediate states is mandatory? Please give an insight if it is indeed possible as to ...
fearKing's user avatar
1 vote
1 answer
50 views

What are the basics of CS i should know,before I start my journey into machine learning

I am myself a non-cs graduate and would love to be a machine learning engineer. I have learned to code and know the basics of <...
Fasty's user avatar
  • 111
0 votes
0 answers
35 views

Use ML to create a graph

I'm currently looking for literature/papers on machine learning techniques to create structures. In detail, I want to generate finite automata (NFA, DFA), which are useful for student-exercises. So I ...
Timo Bergerbusch's user avatar
0 votes
0 answers
258 views

Converting Turing machine into the source code in industrial programming language?

Are there methods how to convert Turing machine (e.g. neural Turing machine or other rigorous Turing machine) into the source code/program that is written in some industrial programming language like ...
TomR's user avatar
  • 1,381
1 vote
0 answers
28 views

Value flow (and economics) in stacked reinforcement learning systems: agent as reinforcement environment for other agents?

There is evolving notion of stacked reinforcement learning systems, e.g. https://www.ijcai.org/proceedings/2018/0103.pdf - where one RL systems executes actions of the second RL system and it itself ...
TomR's user avatar
  • 1,381
1 vote
0 answers
9 views

Is there any intersection in the applications of multi-agent reinforcement learning and more traditional branches of machine learning?

From my limited understanding, it seems like the structure of problems that multi agent reinforcement learning attempts to attack is quite different from the structure of problems in more traditional ...
Joebevo's user avatar
  • 41
0 votes
0 answers
29 views

Why is bias not defined for non-stationary MDP policies?

The Handbook of Markov Decision Processes 1 defines bias of a stationary deterministic policy as follows: $h(x, \phi) = \sum_{n = 0}^{\infty} E[r(x_n, \phi(x_n)) - w(x_n , \phi)]$ where $x$ is a ...
Mathias_Sinner's user avatar
1 vote
0 answers
180 views

Is MCTS an appropriate method for this problem size (large action/state space)?

I'm doing a research on a finite horizon decision problem with $t=1,\dots,40$ periods. In every time step $t$, the (only) agent has to chose an action $a(t) \in A(t)$, while the agent is in state $s(t)...
D. B.'s user avatar
  • 21
1 vote
0 answers
34 views

Why is the objective in a multi-armed bandit problems to minimize (cumulative) regret?

In a multi-armed bandit, the objective is to maximize the expected cumulative reward. This objective is usually (equivalently?) stated in terms of expected cumulative regret. Question: Why not just ...
jonem's user avatar
  • 371
1 vote
0 answers
18 views

How to compute the sample update error?

In the book Reinforcement Learning An Introduction,Chapter 8.5,there is an example that compares the efficiency of expected and sample updates: According to the author, "In this case, sample updates ...
z4z5's user avatar
  • 11
1 vote
1 answer
196 views

Help needed for understanding proof of No Regret Multi Armed Bandit Algorithm

I was reading Elad Hazan's book on Online Convex Optimization(http://ocobook.cs.princeton.edu/OCObook.pdf) and am facing difficulty understanding the proof given for the No regret algorithm for MAB (...
csTheoryBeginner's user avatar
1 vote
0 answers
51 views

What does actually happens on tile hashing

I am going through Richard Sutton book about Reinforcement Learning and I just encountered the tile coding method. I understood pretty well the principles, however, at the very end of the section, ...
naifmeh's user avatar
  • 125
0 votes
1 answer
761 views

Understanding On-policy First Visit Monte Carlo Control algorithm

I am going through the Monte Carlo methods, and it's going fine until now. However, I am actually studying the On-Policy First Visit Monte Carlo control for epsilon soft policies, which allows us to ...
naifmeh's user avatar
  • 125
3 votes
1 answer
163 views

(DROP) Data Reduction Algorithm - How it works?

I am studing a PHD framework which the propose is to reduce the dataset with the most representative samples for training a classifier. Maybe I am loosing something, but I could not undestand a ...
rej's user avatar
  • 31
0 votes
2 answers
98 views

Is the credit assignment problem a well-posed one?

Credit assignment is the process of assigning credit (or blame) to a particular move in a sequence of moves (temporal credit assignment) or to a particular node (structural credit assignment) among ...
Joebevo's user avatar
  • 41
1 vote
1 answer
447 views

Is deep learning appropriate to approximate dynamic programming problems?

I have a problem which can be completely solved using dynamic programming, but in a very intractable way (On^4, where n is around 1000). I won't get into the details of the problem since it's a bit ...
lightning's user avatar
  • 225
0 votes
0 answers
117 views

Hessian in reinforcement learning

The Hessian of multi-layered network exhibits known behaviour at critical points as shown in [1]. The tools of random matrix theory allow [2] to deduce the asymptotic distribution of the eigenvalues ...
user88903's user avatar
0 votes
1 answer
312 views

Existence of Optimal Policy for infinite-state MDPs

It is well-know from Puterman's book (1994) that in any finite-state MDP, if there exists an optimal policy, then that policy is stationary and deterministic. How about MDPs with continuous state ...
Perissiane's user avatar