Suppose $\Omega$ is a set of mutually exclusive outcomes of a discrete random variable and $f$ is a utility function where $0 < f(\omega) \leq 1$, $\sum_\Omega f(\omega) = 1$, etc.
When $f$ is uniformly distributed over $\Omega$ and $f$ is a probability mass function, the Shannon entropy $H(\Omega) = \sum_{\Omega}f(\omega)log\frac{1}{f(\omega)}$ is maximized ($=log|\Omega|)$, and when one element in $\Omega$ has all of $f$'s mass, the Shannon entropy is minimized ($0$, in fact). This corresponds to intuitions about surprisal (or uncertainty reduction) and outcomes and uncertainty (or expected surprisal) and random variables:
- When $f$ is uniformly distributed, uncertainty is maximized, and the more outcomes there are for mass to be uniformly distributed over, the more uncertain we are.
- When $f$ has all its mass concentrated in one outcome, we have no uncertainty.
- When we assign an outcome a probability of $1$, we gain no information (are "unsurprised") when we actually observe it.
- When we assign an outcome a probability closer and closer to $0$, the observation of it actually occurring becomes more and more informative ("surprising").
(This all says nothing about the much more concrete -- but less epistemic -- coding interpretation of Shannon information/entropy, of course.)
However, when $f$ has the interpretation of a utility function, is there a sensical interpretation of $log\frac{1}{f(\omega)}$ or $\sum f(\omega)log\frac{1}{f(\omega)}$? It seems to me that there might be:
- if $f$ as a PMF represents a uniform distribution over $\Omega$, then $f$ as a utility function corresponds to indifference over the outcomes that could not be greater*
- a utility function where one outcome has all of the utility and the rest have none (as skewed of a utility as there could be) corresponds to very strong relative preferences -- a lack of indifference.
Is there a reference expanding on this? Have I missed something about the limitations on comparing probability mass functions and normalized, relative utilities over discrete random variables?
*I am aware of indifference curves and do not see how they might be relevant to my question for a variety of reasons, starting with my focus on a categorical sample space and with the fact that I am not interested in 'indifference' per se, but rather how to interpret utilities as probabilities and how to interpret functionals on probabilities when the (discrete) 'probability distribution' in question actually or (additionally) has the interpretation of a utility function.