Bayesian Reasoning and Prediction Markets

By Ben Golden on January 08, 2016

I enjoyed John Horgan's piece on Bayes Theorem for Scientific American. Bayes Theorem and Bayesian reasoning are highly applicable when thinking about forecasting and prediction markets; indeed, one prediction market built a Bayes Net into its platform. In this post I'll explain what Bayesian reasoning is, why it matters to prediction markets, and give a concrete (but semi-fictitious) example of how it's applied.

The logical Bayesian reasoning is basically as follows:

1. Assign each of your beliefs a probability value somewhere between 0 and 1
   
2. Upon encountering new evidence supporting or opposing a belief, adjust your beliefs in the direction that the new evidence suggests
   

That's basically it. Says Horgan: "It reminds me of the theory of evolution, another idea that seems tautologically simple or dauntingly deep, depending on how you view it." (A couple other ideas I might slide into Horgan's description as both simple and deep: the Coase Theorem and the Nash Equilibrium)

Bayes Theorem provides precise math for how much to adjust beliefs. It's worth studying the math in some depth, because it leads to some counter-intuitive results, but for now I'll just use approximations.

So let's say I want to predict whether the New England Patriots will win this year's Super Bowl. My thought process might go as follows:

• There are 12 teams in the NFL playoffs, so my initial belief is that the Patriots have a 1/12 change, or about 8.3%.
  
• But wait, the Patriots also have a first round bye, which means they only have to win three games, whereas some teams have to win four. This seems important, and so I adjust my estimate up to 12.5%.
  
• I check SportsCast, which says the Patriots have an 11% change of winning the Super Bowl. Close, but a little lower than my estimate. I adjust my estimate downward to 12.2%. I also notice that the Super Bowl question includes 11 forecasters. Who are these people, and what do they know?
  
• Looking at the chart, I notice that the Patriots were recently as high as 18% on SportsCast. Has something happened recently to lower their odds, or is this random noise? I suspect it's random noise, so I adjust my estimate up to 12.8%.
  
• I see that user SneakyPete recently forecasted the Patriots downward from 15.1% to 14.8%. But a few weeks earlier, user ben forecasted upward from 10.6% to 16.4% and left a comment about Gronk coming back. Who is Gronk, I wonder, but I also wonder about SneakyPete and ben. ben is a level 10 forecaster, so he must be pretty smart, right? Whereas SneakyPete is only level 1, so maybe he doesn't know what he's talking about (or maybe he does?). I adjust my estimate up to 13.5%.
  
• I check Football Outsiders, which says the Patriots are only 9.6% to win the Super Bowl. Five Thirty Eight says 13%. Hmmm, both estimates are lower than mine, so I adjust it downwards to 12.5%.
  
• Now that I've done some research and applied Bayesian reasoning, I decide to make a forecast, and adjust the market probability to 11.8%. I also leave a comment on SportsCast to alert fellow user's that I'm applying Bayesian reasoning.
  (editorial note: at this point, the example becomes fictional)
• A few hours later, user StannisB_forGOP makes a forecast adjusting the Patriots' Super Bowl odds up to 35%, leaving a comments that says "Pats totally gonna win; they super goood" Who is this new user and why are they so confident in the Patriots? I notice this is StannisB's first forecast on SportsCast and suspect they might be suffering from overconfidence bias. Also, their comment suggest they're probably not some insider with insight into how the Patriots plan to cheat this year. I adjust my estimate only to 12.6% (from 12.5%) and make a fairly large forecast on SportsCast, moving the probability back down to 13%.
  
• There's a rumor spreading on Twitter that Tom Brady will miss the playoffs. It hasn't shown up in other media, so I'm cautious, but I nonetheless adjust by estimate down to 11%, and make the same forecast on SportsCast, noting the rumor in my comment.
• A SportsCast user named MonkeySoup points out that the source of the rumor is a notorious Twitter prankster and forecasts the Patriots back up. Yikes, it looks like I've been suckered by Twitter again. Soon after the Patriots confirm that Brady is fine. I adjust my estimate back up to 12.6%.
• In their first playoff game, the Patriots score a touchdown on their first drive. I adjust my estimate up to 12.9%.
• etc.

As you can see, there are many reasons to adjust your estimate. Lots of events can qualify as evidence, and the challenge is to identify what the evidence is saying, and how credible it is. Bayes Theorem provides a framework for making these adjustments, assigning greater mathematical weight to evidence that is more credible. What's also clear from this example is that participating in prediction markets is a great way to obtain more evidence to improve your forecasts. As you interact with other users, you learn things, question the reliability of different sources, and constantly reevaluate your own beliefs. In addition to the competition aspect, it's the potential for learning that keeps me excited about using prediction markets.

Did you know that you can sign up for free at SportsCast and AlphaCast? If you liked this post, you can follow us on Twitter at @cultivatelabs, and sign up for our monthly newsletter.

prediction markets crowdsourced forecasting