https://arxiv.org/abs/2001.00805 is the article which is pointing towards the direction that the optimal policy of the reinforcement learning can be expressed as the Bayesian/probabilistic program (whose inference can lead to the optimal action for the each state). Inference in such bayesian programs/probabilistic logic if far more resource intenstive than in the crisp (standard, binary/few-value/discrete... logic) logic/programs. That is the argument from the resource side. But there are more philosophical arguments - experience shows that most rules of nature, most conceptualizations in the humanitarian/social sciences are in the crisp logic. E.g. if there is some uncertainty involved around some concept in humanitarian sciences (e.g. in psychology, ethics, aesthetics, etc.) the researchers usually prefer to derive new, more nuanced concepts with the more exact definition and not to stay with the very limited set of concepts for which the probability is assigned.

So - both arguments point to the direction that crisp programs/logics are very suitable to describe rules that can govern worlds in which Markov decision processes happen and so crisp logics/programs can be more conscise and appropriated for the "RL as inference" than bayesian logics/programs. Of course, the observations are probablistic and so, the bayesian programming should be used for the analysis of the observations, but crisp programs could be the ultimate goal and they can be the basis for the inference of the next action, for the represenation of the optimal program.

So - my question is - are there research how to induce crisp programs from the Bayesian programs? Such crisp programs can be generalization or approximation of the Bayesian programs. Are there such research?

I tried to google, but without results. I don't know even the terms for the right query - e.g. is it normally to call the usual discrete programs as crisp program or are the better terms that are used in the case of the dichotomy of crisp/probabilistic programs/logics.

I am talking about programs/logics at the same time, because the collection of rules/statements (Horn clauses) in some logic can be considered as the program too (logic program).

Cited Arxiv article is quite sceptial about RL as inference due to high complexity of reasoning with probabilistic programs. If crisp programs could be used in such setting then RL as inference could be far more efficient.

https://en.wikipedia.org/wiki/Defuzzification is approach in the context of the fuzzy programs. I am searching for the similar procedure for the Bayesian/probabilistic programs.

https://openreview.net/pdf?id=SygbjU6iBS is in the right direction but still - it is model counting which should not happen for crisp programs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.