Questions tagged [markov-chains]

The tag has no usage guidance.

27 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
256 views

68 views

Form of conditional observation probabilities in a POMDP

Consider a partially observable Markov decision process (POMDP), see here for a complete definition. My question is in relation to the conditional observation probabilities (denoted by $O(o|s',a)$ in ...
161 views

Cheeger constant of a graph versus conductance of a Markov chain

Given some graph $G$ with vertices $V$ and edges $E$, its Cheeger constant $h(G)$ is well defined as $$h(G) = \min_{S\subset V,0<|S|\leq|V|}\frac{|\partial S|}{|S|}.$$ Given some doubly-...
59 views

Calibrating a Markov Chain with little data

I am trying to calibrate a Markov Chain. Usually you would have the amount that "moves" from one state to the other and then the resulting value, with data at any given time with the following ...
98 views

What is the meaning of the output weights of a Conditional Random Field (CRF) model?

Problem When train my linear chain CRF with annotated observations, I feed it with a number of sequences containing observation values and a "ground-truth" label for each observation. I'm currently ...
13 views

Concrete practical examples for discrete time markov chain with transition rewards

Can anyone show examples for discrete-time markov chains with associated transition rewards? I am searching for examples of industrial usage of markov chains where their expect and variance or ...
29 views

188 views

probability matrix from digraph adjacency matrix

All I have in hand is a adjacency matrix of a digraph with equal weight on every edge, is there a very simple way to convert this to a state change probability matrix?
203 views

What's the relationship of max-flow-min-cut and Markov Random Fields?

I am trying to follow this paper [1]. There is a relationship between Markov Random Fields (MRF) to max-flow-min-cut. An MRF can be represented as an undirected graph, and you can find flow through it,...