I'm stuck in trying to compute an overall entropy calculation with an agent.

Let me first introduce some background of the problem. Basically, I'm doing some work with the contextual bandit problems. Let's say I have an agent with $m$ actions to select from and in different contexts (I use states in the following), it will choose different actions according to the actions' probabilities. For example, for state $1$, the action probability distribution is $[0.7, 0.1, 0.2]$ and the agent will select an action based on this distribution. So, in other words, the agent selects an action based on the probability distribution associated with a specific state. With the selection probability distribution of actions under each state, we can compute the entropy of an individual state by $SE(s)=\sum_{i}{p(a_{i}) \log p(a_{i})}$. This is kind of the entropy of a random variable of action selection at a certain state, and I call this state entropy. State entropy is supposed to represent the uncertainty of action selection at an individual state.

Suppose an agent has $n$ states. I'm wondering if there is a way to represent the overall uncertainty or entropy of the agent? I've found this equation in a paper $H=\frac{\sum _{j=1}^{n}SE(s_{j})}{n\log m}$ where $n$ and $m$ is the number of states and number of actions respectively. The paper didn't give any explanation regarding this equation though. Is this a well-proved equation for the entropy of such a situation? Please can anyone give their opinion about this or can anyone guide me the way to find the answer? Appreciate your help!

  • 1
    $\begingroup$ Can you please edit your question to define all concepts you are using? What is meant by "entropy of the system" or "entropy of an individual state"? The entropy is defined on a random variable (or probability distribution), so what is the random variable / probability distribution you want to compute the entropy of? I don't see how the actions are relevant. Can you compute the distribution on states? Perhaps it would also be helpful to give us some context: where you encountered this task, or motivation for why you want to solve it. $\endgroup$ – D.W. Mar 15 at 18:37
  • $\begingroup$ Thank you for the comment. I'll edit the question to make it more understandable. $\endgroup$ – joeW Mar 15 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.