Consider a partially observable Markov decision process (POMDP), see here for a complete definition.

My question is in relation to the conditional observation probabilities (denoted by $O(o|s',a)$ in the above link). This represents the probability of seeing observation $o$ if the current state is $s'$ and the action was $a$. Why is the conditional probability defined in this way?

Is there any issue with defining the conditional observation probability in terms of the previous state and action, $O(o|s,a)$, or in terms of the current state, action, and previous state as $O(o|s',a,s)$?

  • 1
    $\begingroup$ A current observation is supposed to be based on the newest state, which is $s'$ in your case. Due to this, the observation need not be conditioned on the past state $s$. This makes sense because given you want to know where you are now, you just care about your current state and not where you were before. Now if your problem couldn't depend on the Markov Property, this could be different. $\endgroup$ – spektr Apr 27 '17 at 23:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.