Aug 20 2008
The Uses and Limits of Prediction Markets in Forecasting
Hmm. As the 2008 White House race hots up, we’re going to be hearing more and more – and then even more – about who prediction markets forecast to win, so it’s time to put down a thought or two about uses and limitation of this forecasting tool.
First, what’s if all about? If you already know, skip this section. Let’s start with the example in yesterday’s Telegraph: “Predicting the future – with the power of betting” Paul Parsons, August 19, 2008. As Parson’s reports, the University of Iowa is running a market where investors can buy “shares” in the two major US election candidates, each priced between $0 and $1. On election day, traders holding stock in the winner – Obama or McCain – receive $1 per share while the others lose their money. Investors can buy and sell their shares along the way, and as they do this the candidate more people will want to own (because they think he will win) will get more expensive. In other words, market forces will drive up the price of the outcome more people think more likely. As of August 19, the trading value of the Obama, at $0.62, suggests participants expect a 62 percent chance he will win. (Another prediction market site, midasoracle.org, has the figure currently at 59.8 percent.)
Prediction markets mimic stock market and deploy much the same software. Where a real market trades shares in an underlying asset, in a prediction market it is future outcomes which are “securitized”. The key principle at work is the sage market wisdom that “the price of a stock captures all the information known about it” – that is, all information is factored into the price (notwithstanding that some may have more or better information than others; some may be acting more wisely on their information). Therefore price is our guide to the cumulative knowledge of all participants and, in prediction markets, this “price discovery” allows us to know what most people think the future holds. They allow the “the wisdom of crowds” to be turned to a future problem, and tapped.
Serious Success
What’s exciting about all this is its success rate. Prediction markets are amazingly accurate in many circumstances, and by all accounts consistently beat more conventional quantitative and extrapolative methods. Prediction markets have consistently out-predicted election opinion polls and exit polls. Of course the predictive potential goes way beyond polling. Forecasting markets can and have been set up to predict the dollar movements to the success of same-sex marriage legislation, to who will win best actor Oscar. At one point there was even a US government market in future terror targets (trying to elicit public predictions of likely targets so as to plan accordingly) but this was deemed inappropriate and taken down.
As it has become clear that this method outstrips conventional forecasting methods, prediction markets have taken root in forward-looking businesses. Companies such as Google and Hewlett-Packard routinely use (internal) prediction markets to forecast sales figures, customer preferences, product adoption, and so on. HP is on the record as saying prediction markets consistently outperform their official forecasts.
The method has other advantages too. First, it requires no special techniques or expense. There are no fancy models to apply or complex algorithms to … to do whatever one does with such things. Second the forecasts are available in real time, all the time, and constantly update themselves. There’s no waiting for data collectors to collect, or statisticians to emerge with their answers.
The Limits
In my book, Future Savvy, I show how and why humans are poor at predicting, for dozens of reasons. The record of predicting is littered with failure. But, is that now all in the past? Do prediction markets solve the perennial problem of predicting the future, or at least get us closer? Yes and no.
Yes where prediction markets are appropriate. They work best under two conditions: first where there is a clear view of the options and operating conditions; second (related) where the time frame predicted is relatively short, usually under 18 months depending how fast things are moving. Where predicting the future means choosing between known alternatives, such as an election winner, or anticipating a point along a known continuum, for example the level of next year’s sales, prediction markets are great.
Where prediction markets run dry is in dealing with unfamiliar conditions, or unknown variables, or potential game-changing disjunctures in the world. Where the future is seriously fuzzy, where there are many variables, and the way they interact unknown, and drivers, blockers, and lags are hidden, prediction markets are of limited use because the outcomes can’t be framed adequately so that people can bet on them or against them. A prediction market for US president in 2012 would be far less useful than 2008. Similarly, while a market for the oil price in 2009 would be helpful, by 2010 or beyond factors driving the price may be so different (viz. developments in sustainable energy or geopolitics) that the result of a prediction market conducted in 2008 would be undependable.
So while prediction markets sort out probabilities between known likelihoods, they are not adequate to the task of investigating complex situations where we cannot frame the likely outcomes, or at least can’t know if we’ve framed them right. Also while prediction markets do help us, on aggregate, avoid some perceptual/cognitive fallacies, they are as likely as any other predictive tool to fall into the Zeitgeist effect. More on this soon…
A good list of articles on prediction markets is available here: http://www.midasoracle.org/best/
3 Responses to “The Uses and Limits of Prediction Markets in Forecasting”
Leave a Reply
Additional comments powered by BackType
adam.gordon (at) futuresavvy.net
Early on in the article you point out that markets aggregate all the available information about the value of an asset, but then you say that long term questions are harder to predict. The examples you give: president in 2012 or gas prices in two years aren’t ones where I’d use a prediction market, but not because they’re long term. No one knows much about the 2012 race, so there isn’t much useful info to aggregate, and the best predictions for the price of oil are the financial futures. The predictions two years or more out aren’t that great, but they’re the best indicator available if you need a guess on which to make a decision.
There are other questions on which long term claims can be useful. If people are interested and knowledgeable, then the best projection would probably come from a market. Want to know hotel occupancy levels in 2015 at some vacation destination or the price per gigabyte of memory three years from now? I don’t think you’d get a better answer by paying a research group to do a survey than by funding a market to answer the question. And as you said, the value will be constantly updated and will incorporate any information released by market research groups.
Very interesting article. Just wondering who would benefit from this tool the most?
Interesting : In the paragraph about the limits of prediction markets, you could easily change the words “prediction markets” by “data mining”. It would stay correct.
The advantages : prediction markets are like data mining, but with the advantage of random forests of natural neural networks : a lot of models, each with another sample of observations and predicting variables. And with the second advantage that you do not need to calculate all those little models : they are in the heads of the participants and come to you to form the global prediction.