Posted by Adam Gordon on Mar 16, 2011 in all, decision-making, forecast filtering, leadership, management, managing uncertainty, risk management, strategic foresight
Fukushima plant, Japan. Picture: digitalglobe.com
At the time of writing, Japan is battling a nuclear meltdown and radiation emergency, and Fukushima could become a word suddenly the whole world knows, like Chernobyl.
Bloomberg News has called the whole tsunami crisis Naoto Kan’s “Katrina moment,” and one can only hope and pray for all concerned that the Japanese prime minister is a more competent leader than Bush was at this moment of human catastrophe.
As to the nuclear meltdown: If ever we have been warned about anything in the future, we have been warned about nuclear plant catastrophes. Not only have there been, as it were, verbal warnings going all the way back to the 1950s, but real-world events such as Three-Mile-Island and Chernobyl have fully fleshed out the scenario of nuclear reactor failure or near failure in populated areas.
If nuclear-generated electricity makes sense anywhere, it makes sense in Japan, which famously has no coal or gas reserves. But these are nuclear plants … built right on the Pacific Ring of Fire? Japan is a small island with 125 million people densely packed into urban areas. As we face the possibility of this many people put at risk, however the next few days play out it’s clear the risk and reward of nuclear energy here is out of alignment.
This is hardly news. The question is, why are the plants are there? And the answer is not a simple one of collusion or corruption of government, or shenanigans of power companies, although there may be some of that. It comes down to a misapprehension of probability and risk among leaders and decision-makers such that it appears that risk and reward are in balance, when in fact they are not.
Year 869AD
To think about this, consider yesterday’s BBC Story: Japan tsunami ‘could be 1,000-year event,” saying last week’s tidal wave was equivalent to a giant wave that hit the Sendai coast in 869AD. The report says: ”It is not unusual for undersea earthquakes to generate tsunamis in this part of Japan. Offshore quakes in the 19th and 20th centuries also caused large walls of water to hit this area of coastline. But previous research by a Japanese team shows that (only) in the 869 ‘Jogan’ disaster, tsunami waters moved some 4km inland, causing widespread flooding.”
The point is, tsunamis are common, but “the big one” is a one-in-thousand year event — an extremely low probability outcome.
Here I’m strongly reminded of the days following the depth of the Credit Crunch, Bear Stearns’ collapse, and general world financial system meltdown of 2008. If bankers said one thing sensible through the whole period it was: “this was a one-in-ten-(hundred, etc.)-thousand probability outcome, and extreme ‘outlier’ event!”
A low-probability event means we can relax, right? Wrong. The problem is probability says zilch about impact. “Wild Cards,” or now more famously in Nassim Taleb’s terms, “Black Swan” events are low probability but of game-changing impact.
Taleb’s point, made repeatedly across his various books and articles, is that standard probability theory and Gaussian statistics lull analysts into thinking that because an event is low probability – an outlier in a normal bell-curve distribution – it is of low or lower consequence.
Ignoring the tail of the Bell Curve is okay if events are genuinely assessed as low impact. If they are high-impact aka “fat-tailed” events, they are the most important events we face in the future, in building or maintaining any system or organization.
A probabilistic framework misleads decision-makers because it degrades their attention to crucial events (by tagging them low-probability,) which means next thing they are betting banks on mortgage-backed securities, or building nuclear plants on earthquake fault lines.
read more
Posted by Adam Gordon on Aug 20, 2008 in all, failed predictions, foresight tools & methods, Future Savvy, prediction markets
Hmm. As the 2008 White House race hots up, we’re going to be hearing more and more – and then even more – about who prediction markets forecast to win, so it’s time to put down a thought or two about uses and limitation of this forecasting tool.
First, what’s if all about? If you already know, skip this section. Let’s start with the example in yesterday’s Telegraph: “Predicting the future – with the power of betting” Paul Parsons, August 19, 2008. As Parson’s reports, the University of Iowa is running a market where investors can buy “shares” in the two major US election candidates, each priced between $0 and $1. On election day, traders holding stock in the winner – Obama or McCain – receive $1 per share while the others lose their money. Investors can buy and sell their shares along the way, and as they do this the candidate more people will want to own (because they think he will win) will get more expensive. In other words, market forces will drive up the price of the outcome more people think more likely. As of August 19, the trading value of the Obama, at $0.62, suggests participants expect a 62 percent chance he will win. (Another prediction market site, midasoracle.org, has the figure currently at 59.8 percent.)
Prediction markets mimic stock market and deploy much the same software. Where a real market trades shares in an underlying asset, in a prediction market it is future outcomes which are “securitized”. The key principle at work is the sage market wisdom that “the price of a stock captures all the information known about it” – that is, all information is factored into the price (notwithstanding that some may have more or better information than others; some may be acting more wisely on their information). Therefore price is our guide to the cumulative knowledge of all participants and, in prediction markets, this “price discovery” allows us to know what most people think the future holds. They allow the “the wisdom of crowds” to be turned to a future problem, and tapped.
Serious Success
What’s exciting about all this is its success rate. Prediction markets are amazingly accurate in many circumstances, and by all accounts consistently beat more conventional quantitative and extrapolative methods. Prediction markets have consistently out-predicted election opinion polls and exit polls. Of course the predictive potential goes way beyond polling. Forecasting markets can and have been set up to predict the dollar movements to the success of same-sex marriage legislation, to who will win best actor Oscar. At one point there was even a US government market in future terror targets (trying to elicit public predictions of likely targets so as to plan accordingly) but this was deemed inappropriate and taken down.
As it has become clear that this method outstrips conventional forecasting methods, prediction markets have taken root in forward-looking businesses. Companies such as Google and Hewlett-Packard routinely use (internal) prediction markets to forecast sales figures, customer preferences, product adoption, and so on. HP is on the record as saying prediction markets consistently outperform their official forecasts.
The method has other advantages too. First, it requires no special techniques or expense. There are no fancy models to apply or complex algorithms to … to do whatever one does with such things. Second the forecasts are available in real time, all the time, and constantly update themselves. There’s no waiting for data collectors to collect, or statisticians to emerge with their answers.
The Limits
In my book, Future Savvy, I show how and why humans are poor at predicting, for dozens of reasons. The record of predicting is littered with failure. But, is that now all in the past? Do prediction markets solve the perennial problem of predicting the future, or at least get us closer? Yes and no.
Yes where prediction markets are appropriate. They work best under two conditions: first where there is a clear view of the options and operating conditions; second (related) where the time frame predicted is relatively short, usually under 18 months depending how fast things are moving. Where predicting the future means choosing between known alternatives, such as an election winner, or anticipating a point along a known continuum, for example the level of next year’s sales, prediction markets are great.
Where prediction markets run dry is in dealing with unfamiliar conditions, or unknown variables, or potential game-changing disjunctures in the world. Where the future is seriously fuzzy, where there are many variables, and the way they interact unknown, and drivers, blockers, and lags are hidden, prediction markets are of limited use because the outcomes can’t be framed adequately so that people can bet on them or against them. A prediction market for US president in 2012 would be far less useful than 2008. Similarly, while a market for the oil price in 2009 would be helpful, by 2010 or beyond factors driving the price may be so different (viz. developments in sustainable energy or geopolitics) that the result of a prediction market conducted in 2008 would be undependable.
So while prediction markets sort out probabilities between known likelihoods, they are not adequate to the task of investigating complex situations where we cannot frame the likely outcomes, or at least can’t know if we’ve framed them right. Also while prediction markets do help us, on aggregate, avoid some perceptual/cognitive fallacies, they are as likely as any other predictive tool to fall into the Zeitgeist effect. More on this soon…
A good list of articles on prediction markets is available here: http://www.midasoracle.org/best/
read more