Bayesian Reasoning and Prediction Markets
John Horgan's piece on Bayes Theorem for Scientific American. Bayes
Theorem and Bayesian reasoning are highly applicable when thinking about
forecasting and prediction markets; indeed, one
prediction market built a Bayes Net into its platform. In this post
I'll explain what Bayesian reasoning is, why it matters to prediction
markets, and give a concrete (but semi-fictitious) example of how it's
The logical Bayesian reasoning is basically as follows:
Assign each of your beliefs a
probability value somewhere between 0 and 1
Upon encountering new evidence
supporting or opposing a belief, adjust your beliefs in the direction that
the new evidence suggests
That's basically it. Says Horgan: "It reminds me of the theory of evolution, another idea that seems tautologically simple or dauntingly deep, depending on how you view it." (A couple other ideas I might slide into Horgan's description as both simple and deep: the Coase Theorem and the Nash Equilibrium)
Bayes Theorem provides precise math for how much to adjust beliefs. It's worth studying the math in some depth, because it leads to some counter-intuitive results, but for now I'll just use approximations.
So let's say I want to predict whether the New England Patriots will win this year's Super Bowl. My thought process might go as follows:
There are 12 teams in the NFL
playoffs, so my initial belief is that the Patriots have a 1/12 change, or
But wait, the Patriots also have a
first round bye, which means they only have to win three games, whereas some
teams have to win four. This seems important, and so I adjust my estimate up
SportsCast, which says the Patriots have an 11% change of winning the
Super Bowl. Close, but a little lower than my estimate. I adjust my estimate
downward to 12.2%. I also notice that the Super Bowl question includes 11
forecasters. Who are these people, and what do they know?
Looking at the chart, I notice that
the Patriots were recently as high as 18% on SportsCast. Has something
happened recently to lower their odds, or is this random noise? I suspect
it's random noise, so I adjust my estimate up to 12.8%.
I see that user SneakyPete
recently forecasted the Patriots downward from 15.1% to 14.8%. But a few
weeks earlier, user ben forecasted upward from 10.6% to 16.4% and left
a comment about Gronk coming back. Who is Gronk, I wonder, but I also wonder
about SneakyPete and ben. ben is a level 10 forecaster,
so he must be pretty smart, right? Whereas SneakyPete is only level 1,
so maybe he doesn't know what he's talking about (or maybe he does?).
I adjust my estimate up to 13.5%.
I check Football Outsiders,
which says the Patriots are only 9.6% to win the Super Bowl. Five Thirty
Eight says 13%. Hmmm, both estimates are lower than mine, so I adjust it
downwards to 12.5%.
- Now that I've done some research
and applied Bayesian reasoning, I decide to make a forecast, and adjust the
market probability to 11.8%. I also leave a comment on SportsCast to alert
fellow user's that I'm applying Bayesian reasoning.
(editorial note: at this point, the example becomes fictional)
A few hours later, user
StannisB_forGOP makes a forecast adjusting the Patriots' Super Bowl odds
up to 35%, leaving a comments that says "Pats totally gonna win; they
super goood" Who is this new user and why are they so confident in
the Patriots? I notice this is StannisB's first forecast on SportsCast
and suspect they might be suffering from overconfidence bias. Also, their
comment suggest they're probably not some insider with insight into how
plan to cheat this year. I adjust my estimate only to 12.6% (from 12.5%)
and make a fairly large forecast on SportsCast, moving the probability back
down to 13%.
- There's a rumor spreading on Twitter that Tom Brady will miss the playoffs. It hasn't shown up in other media, so I'm cautious, but I nonetheless adjust by estimate down to 11%, and make the same forecast on SportsCast, noting the rumor in my comment.
- A SportsCast user named MonkeySoup points out that the source of the rumor is a notorious Twitter prankster and forecasts the Patriots back up. Yikes, it looks like I've been suckered by Twitter again. Soon after the Patriots confirm that Brady is fine. I adjust my estimate back up to 12.6%.
- In their first playoff game, the Patriots score a touchdown on their first drive. I adjust my estimate up to 12.9%.
As you can see, there are many reasons to adjust your estimate. Lots of events can qualify as evidence, and the challenge is to identify what the evidence is saying, and how credible it is. Bayes Theorem provides a framework for making these adjustments, assigning greater mathematical weight to evidence that is more credible. What's also clear from this example is that participating in prediction markets is a great way to obtain more evidence to improve your forecasts. As you interact with other users, you learn things, question the reliability of different sources, and constantly reevaluate your own beliefs. In addition to the competition aspect, it's the potential for learning that keeps me excited about using prediction markets.Ben Golden/@BenGoldn is an Engineer at Cultivate Labs.