# Transition Function in MDP

I got a question about who and how sets the transation function values in markov decision processes?

I mean when some says that an agent, in real world grid, is going to step up by %80 and left/right by %10 each. How did he know that? Do we have to determine the transition probability for each step or agents have to? Simply, it should be non-deterministic process, so how could someone tell the probability of that transition?

It should be %25 for each direction!!

• Suppose that I toss a coin which is heads 2/3 of the time. Who gets to decide these probabilities? Shouldn't it be 50% heads? MDPs are supposed to model some situations. The person using MDPs gets to decide which MDPs they use. Nov 15, 2020 at 10:44

Another scenario that could explain the probabilities you asked about is the agent could learn a stochastic policy, meaning instead of using a deterministic policy $$\pi:S\to A$$ The agent uses a stochastic policy $$\pi:S\to prob(A)$$ which maps states to probability distributions over actions. In this case, even if the environment is deterministic the agent may choose to move up with probability 0.8 and left or right with probability 0.1 each.