Suppose that I am wondering about whether an event will happen (e.g. whether it will rain tomorrow in a particular location). Suppose that my beliefs about whether the event will happen can be represented by a probability $p_t \in [0, 1]$; and that I know that I may receive new information about this in period $t + 1$. After I receive this new information, I will form a new belief $p_{t+1} \in [0, 1]$ using Bayes Rule.

My understanding is that, under certain assumptions, $$ \mathbb{E}[p_{t+1}] = p_t $$ That is, my beliefs should be a martingale. The intuitive idea is quite clear: if I expected my belief to go up tomorrow, then my belief should already go up today! However, I am not quite clear on when this should hold, and appreciate an explanation or some useful links.

To be a bit more concrete, there are a few things which confuse me:

  • In this helpful looking document, I read that "expected value of the posterior . . . is just the prior." I guess that this is just a generalised version of the claim I am discussing here? (Generalised because it allows for any number of states, not just two states as in my question.)

  • It doesn't seem that this result can be true in general. For example, suppose that, unbeknownst to me, somebody wants to change my mind. If I believe that the event is unlikely to occur, they will provide me with evidence suggesting that it is likely to occur; and if I think it is likely to occur, they will provide me with evidence suggesting that it probably won't occur. In this case, it seems that the belief is not a martingale; rather, it can be expected to move towards $0.5$ in the next period. If this is right, what assumption is violated here? [Note: for this example to be interesting, I think it's important that I don't know that the `data generating process' takes the perverse form outlined above.]

Many thanks in advance for any ideas, references, etc.

  • 3
    $\begingroup$ Random walks and martingales are not the same thing. $\endgroup$ Oct 4 at 22:13
  • $\begingroup$ With regards to the second bullet point, the question is, expected by whom? Your interlocutor has access to information that you don't have (namely, that they are trying to change your mind and presenting evidence to you selectively), so their prior over your future beliefs can be different from your prior over your future beliefs. They might expect your posterior to change systematically, but you shouldn't! $\endgroup$
    – Nathaniel
    Oct 5 at 7:34
  • $\begingroup$ @MichaelGreinecker thanks for catching this error, now fixed. $\endgroup$
    – afreelunch
    Oct 5 at 7:58
  • $\begingroup$ @Nathaniel Yes I do agree with this -- whether the belief is 'expected' to change depends (in the example) on whose expectation we are talking about. $\endgroup$
    – afreelunch
    Oct 5 at 7:58

The expected posterior is the prior. But for this to hold, you have to take the expectation with respect to the prior. In your example, you describe a Bayesian but ev aluate their update with respect to your belief.

Ann and Bob might be perfect Bayesians and their expected posteriors might be exactly their priors. But that does not mean that Ann's expectation of the posterior of Bob is the prior of Bob.

  • $\begingroup$ Thanks, this is very helpful. Does this require any assumptions to hold, or is it always true that A's expectation of A's posterior is A's prior (if A is Bayesian)? $\endgroup$
    – afreelunch
    Oct 5 at 8:01
  • 1
    $\begingroup$ That is always true. Sometimes, this is essentially how one defines posteriors in some cases. $\endgroup$ Oct 5 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.