Unexpected prediction modesty highlights problems of timing and impact

Posted by Adam Gordon on Oct 29, 2009 in all, economy & finance, failed predictions, forecast filtering, foresight tools & methods, managing uncertainty, Perils of Prediction, risk management, strategic foresight

Continuing the theme of financial types talking to each other about predictions and predictability, this ‘Tea with the Economist’ interview of Stephen Roach, Chairman, Morgan Stanley Asia by Economist New York Bureau Chief Mathew Birk, carries interesting lessons about the limits of prediction.

Birk commends Roach for being one of the few to have predicted the Credit Crunch problems, to which Roach demurs in saying he was “too early”. He then furthers his modesty in saying that the “breakage” in the financial system was “in excess of anything I envisioned.”

Self-deprecation in assessing one’s predictive abilities will endear anyone to me. Even Roach, who later in the interview burns this hard-won credibility by laying the blame for the credit crunch at the door of regulators, forgetting how hard financial institutions lobbied regulators for greater freedoms in the 1990s.

But I digress. The predictive issues the interview raises are as follows. Issue one: it’s not enough (as any stock short-seller will confirm) to get the direction of a future change right. One must get the timing right too. Issue two: it’s not enough to anticipate a change. One must be able to judge it’s impact. Getting either timing or impact wrong is effectively to have missed the future.

.

Probability

On the latter topic — the problem of impact — Nassim Taleb is unrelenting, and he is right. Analysts routinely mix up probability and impact. They think that because an event has a low probability (‘it would be a 10-sigma event!’) it can be marginalized in the predictive number crunching. Of course, it can’t. The low-probability of a wildcard or black swan event is irrelevant because when it happens it will change the game, and that’s why, in every predictive situation of reasonable complexity and uncertainty, using statistical extrapolations (regressions and so on) to predict, is to dangerously paper over the cracks. It is precisely the cracks that businesses and policy makers need to worry about.

Determining the direction of change is hard enough. Assessing timing or extent of impact — a ‘total future impact index’ — is wickedly difficult. It’s a task not to be underestimated, and to simply extrapolate current trends (= assuming the trend’s timeline and impact stay the same as in the past) is the royal road to underestimating it.

This is the reason foresight for complex, uncertain, changing situations can only be grasped by NOT predicting (quantitatively or otherwise) but by exploring the limit-conditions of the plausible (What would happen if the timing of the change accelerated, or was significantly delayed? What if  the impact was 10x or one tenth of what we expect? And so on.)

.

A look back on how people look forward, and the need for ‘futuriography’

Posted by Adam Gordon on Jul 3, 2009 in all, failed predictions, forecast filtering, foresight tools & methods, Future Savvy, history, lifestyles & values, Perils of Prediction, politics of the future, social change, strategic foresight, technology change

Samuel, L., Future: A Recent History, University of Texas Press, 2009

I recently received a copy of Future: A Recent History to review. True confession: what hit me first on picking up the book was (a) “wow, the title Future is not already taken!? And (b) what a fabulous job the University of Texas Press has done producing this book. It is beautifully designed, with an understated Art Deco motif, and carefully laid out with enough text on the page, on delightfully solid paper stock.
It may seem odd to go on about text on the page, but it’s much easier to read like an adult, in paragraphs. So many books, particularly business books, these days appear produced at 14-point, double spacing, like pre-school readers. Makes you wonder…

Anyway, author Larry Samuel’s project is to investigate the history of views of the future from 1920 to the present. (The book has an acknowledged US-centric focus, partially defended by the notion that future-mindedness is “a principle strand in America’s DNA.”) He organizes the book chronologically into six periods between then and now, and shows, with interesting examples, how each period had its own views of the future, and how the views shifted from period to period.

In tracing the history of “tommorowism,” in this way, Future is on a similar track to the classic book in this field: I.F. Clarke’s The Pattern of Expectation 1644-2001 (Jonathan Cape, 1979). It ultimately makes similar points, although Samuel’s argument is obviously drawn from more recent examples. As Samuel puts it: “A look back on how people looked forward reveals that while it possesses certain common themes … the future is not a fixed idea but a highly variable on that reflects the values of those who are imagining it.”

Happily I can say this chimes exactly with the argument of Future Savvy, particularly Chapter 4 “Zeitgeist & Perception,” where I argued how heavily the nature of the present and its topical issues frames how the future is seen (what is forecast, what is aspired to or feared, what counts as a valid method for thinking ahead, and so on). Which means the framing conditions of the present  should be carefully analyzed in assessing the validity of any future view.

Historiography

Historiography – investigating the meta-conditions surrounding what is recorded and how it is interpreted by historians – what counts as “history” and for whom –  is a well-understood part of doing good history. Unfortunately, there is no equivalent standard “futuriography” in the foresight field, despite it being absolutely fundamental to understanding the value of our own predictions as, similarly, highly determined by the epistemic configurations of their production. It is here that Samuel very competently fills a much needed gap.

The practical implication of this, which Future does not get into – it’s not that kind of book – is that to make better predictions (or make valid assessments of others’ predictions) we need to ask stiff questions as to how much of what we foresee is determined by the perspectives of today, and expect the answer to be “very much.” Understanding the limitations and biases of our own perspective is the sine-qua-non of a robust view of what tomorrow will actually bring.

read more

The luxury good sector gets humble about forecasting – but knows what follows “bling”

Posted by Adam Gordon on Mar 30, 2009 in 2015, all, economy & finance, leadership, lifestyles & values, management, managing uncertainty, social change, strategic foresight, trend tracking

The International Herald Tribune (New York Times Global Edition / Reuters Business) last week ran an interesting foresight story headlined ‘Crisis complicates forecasting by luxury brands,’ reporting from the International Herald Tribune’s eighth conference on luxury in New Delhi. The gist was that although most of the famous brands continue to do well despite the recession, luxury sector executives are very uncertain about the future.

Christian Blanckaert, Executive Vice President at Hermès International was quoted as saying: “We have absolutely no visibility into 2009!”

On the one hand, fair enough. This economic downturn is steeper than previous down cycles, and the basic viability of the financial sector has been tested. Access to credit is normally easier in a recession, but in this one it is not. All of which makes luxury spending harder to predict.

No doubt the most unlikely prediction of all would have been that Hermès, Burberry, LVMH, Moët Hennessy, Louis Vuitton, and PPR (Gucci , Yves Saint Laurent) have all recently reported better-than-expected results.

Nevertheless luxury industry leaders have declined to provide investors and analysts with any official outlook. What’s curious, from an industry foresight point of view, is how executives such as Blanckaert thought they really had more “visibility” into any previous year, or that they will somehow gain it again when the financial crisis is over. They will not. The world will continue to surprise them and us. What they will gain, certainly, is a greater likelihood that the standard business-as-usual future assumptions they make will not be upset by reality.

Meanwhile, judging by the conference, the luxury goods industry has a very decent grip on current social and moral trends, and clear insight into the bigger picture of change in its industry over the next five to ten years. As they know from before, what happens in a recession is that luxury goes out of fashion. Conspicuous consumption wanes, or retreats further behind secluded walls. This is a basic pendulum swing that tracks the economy (witness how the early 1990s recession stimulated a return to “values” era after the “me, me, me” 1980s.)

Sustainable luxury

So we are again in a swing to modesty. But we also know that each swing of the pendulum also carries with it the specific issues of its time. Current key issues for consumers in this segment are sustainability, global warming, business ethics, and globalization (or fear thereof).

Therefore the luxury brands will be looking for ways of making, transporting, and displaying goods in an energy-efficient and socially conscious way, including a renewed emphasis on local artisans and traditional craftsmanship that speaks sustainability in both natural and human resources. This will be the basis of the “sustainable luxury,” positioning that the famous houses will define and compete in. Fabulous and renewable  – now there’s something you can charge top dollar for.

Credit crunch: the foresight was there, the problem was elsewhere

Posted by Adam Gordon on Dec 15, 2008 in all, decision-making, failed predictions, forecast filtering, Future Savvy, managing uncertainty, Perils of Prediction, scenario planning, strategic foresight

One of the questions I’m asked a lot is whether Future Savvy would have helped to predict the credit crunch. My response, as in this INSEAD interview, has been that the book gives readers the tools to judge the merits of predictions, so wouldn’t have directly helped predict the financial crisis, but it would have been a key resource in drawing attention to the poor view of the future that bankers and regulators were acting on.

In many ways, focusing on whether “this” or “that” is predicted, or not predicted, is to put the cart before the horse. The horse is the adequacy of our approach to anticipating outcomes and the quality of our foresight as a whole. When this is good, the cart – not missing important changes – will follow.

Credit: http://www.lewrockwell.com/blog

In this, it’s important to realize that many did predict the financial crisis (as many predicted 9/11 in various ways). Sticking with the financial crunch for now: it has generally been portrayed it as a “why-didn’t-anyone-see-it-coming” event. It wasn’t. Hats off to The Times for their October 12 piece: “10 People Who Predicted the Financial Meltdown.”(Summary here). Allowing for a fairly loose definition of “predicted,” the article shows that among those who foresaw the crunch were: Vince Cable, deputy leader of the Liberal Democrats (2003); US congressman Ron Paul (2003); Stephen Roach, senior executive at Morgan Stanley (2004); Christopher Wood – chief strategist of a broking firm in the Asia-Pacific Market (2005); and Nouriel Roubini, economics professor at NYU (2006)… and there were many others.

A different problem

So this reframes the problem entirely. It’s not that the predictions were not there. It was that not enough people believed them and, particularly, important decision-makers didn’t believe them or didn’t have the institutional capacity to respond. So there are two halves to the problem: the ability to see the full spectrum of what may happen, including unexpected outcomes; and the ability to act on what we see. Quality in foresight work – the raison d’etre of Future Savvy – makes it possible to see more outcomes more clearly, and to act with more confidence in choosing what to prepare for. (In the real world we can’t prepare for every outcome.)

There was a good letter published in the FT from eminent futurist Peter Schwartz on December 2, which describes this very well. It shows predictions for what they are (one-horse scenarios), and how decision-makers are typically bound into inaction or wrong action not only by working on the basis of a wrong prediction, but by the predictive mindset itself. This mindset – the habit or culture of picking “one right answer” in the face of a complex situation with many competing outcomes, prematurely closes alternatives and leaves us open to surprise. As Schwartz says, as scenario planners have always said (and he was one of the people who defined the field in the first place), a compelling set of alternative future scenarios encourages decision-makers to recognize unlikely and unpopular outcomes, along with expected outcomes, and therefore to be able to respond earlier and more effectively whatever happens.

Scenarios also contribute to the “act” side of the problem. In a well-done set for the banking industry, a financial-meltdown scenario would at least have been in play, institutionalizing the consideration of less unlikely, less popular outcomes in company and government forums, forcing serious consideration of necessary strategies and contingencies, and therein creating the ability to act early and effectively without having predicted the crisis.

The letter is well worth quoting in full:

Sir, The real question regarding the financial crisis is not, as the Queen asked: “Why did nobody see this coming?” In fact, any number of thoughtful people in academia, politics and business had been compiling the data and sounding warnings for several years.
The question we should be asking is: “Why didn’t decision-makers believe that a global financial meltdown was increasingly likely and then act on that belief?” Or, to put it another way: “What would it take to make decision-makers both believe and act?”

The problem is that decision-makers believe that they are forced to pick one right answer: the most likely scenario. Their approach to decision-making does not afford them the opportunity to consider apparently low probability but highly consequential scenarios. The answer, therefore, to the “believe” half of the question is a decision-making process that considers several scenarios: compelling stories about alternative futures that incorporate the analysis of “outliers” and describe three or four plausible paths forward.
Good scenarios force decision-makers to challenge their own assumptions and reconsider what is possible. As a result, they can take seriously those scenarios that seemed less likely at first, but whose plausibility increases over time.

The second part of the question – “What would it take to act?” – is much harder to address. Suppose that Ben Bernanke or Hank Paulson had come to believe a year or two ago that the house of cards was about to collapse and trigger cascading, global failures. What would they have done, given the realities of the complex interconnected systems at the heart of the problem? Perhaps if they had good scenarios with appropriate indicators to start with, they could have rehearsed different strategies and contingencies. Importantly, these decision-makers could have used these scenarios to persuade others on all sides of the issue also to recognise the complexity of the impending crisis in a more timely way. It’s never easy to convince everyone around you that the game they have been playing to their great benefit is about to change. But with a shared recognition of the magnitude of the risks and the ways they might unfold, they could have acted far earlier to prevent some of the dire consequences that have occurred, let alone what is to come.

The Wheel Turns on the Same Old Future for Drug Policy

Posted by Adam Gordon on Nov 20, 2008 in all, failed predictions, foresight tools & methods, Future Savvy, lifestyles & values, policy, social change, systems dynamics

The foresight news story of the day is undoubtedly the breakthroughs in stem cell use in facilitating human organ donation. Instantly one can add decades to the human lifespan in places where this class of treatments will be available and affordable. But that’s a topic for another time. What I’ve been mulling over is a Columbian government media tour in the UK, aimed at drawing middle class “recreational” drug consumers’ attention to the environmental cost of drug production, particularly cocaine. As reported in the Guardian yesterday (November 19), Columbian VP, Santos Calderón told a police conference that 300,000 hectares destroyed each year in Columbia for coca plant, that is, 4sq meters of rainforest  for every gram of cocaine produced. (Savvy says: what’s the validity of these numbers and who do they favor? Place a question mark there.) But it’s probably safe to assume the profit motive behind drug production overrides Green sensitivities, and the environmental cost is severe.

The environmental pitch is a new salvo in the old “war on drugs,” which has been waged backwards and forwards, over decades now, without being won. It’s worth stopping to think why it has not been won, because it’s a salutory lesson in thinking about the future. It has nothing to do with the morals of “pushers” or willpower of “addicts” or the “the youth of today.” It is perfectly explained by the reinforcing loop (aka viscious/virtuous cycle) that dominates the drug-prevention system. This can be diagrammed as follows:

Alternatively the identical idea may be represented as a “fixes that fail” archetype, as defined in “The Fifth Discipline Fieldbook” (Peter Senge et al, Doubleday, 1994, p125).

For more on systems thinking see The Systems Dynamics Society. The role of systems thinking in improving our understanding of change (or non-change) is also the topic of Chapter 8 of Future Savvy.

So, yes, these are simple charts. We could make them more complex by filling in details of all agents and institutions at work in drug supply, demand, and prevention – but this would only elaborate, not alter the logic of the system. Either way, the chart allows us to see the wood for the trees, which is that drugs and their prevention are in a reinforcing loop. While it appears that preventive laws and their enforcement will lower drug use, in fact law enforcement constrains availability, pushing up the price, which makes production more attractive, which creates incentives to farm (incl, in rainforests), which raises supply, which leads to drug pushing (marketing by another name), which leads to drug trial, usage and addiction, therefore social concern, and therein renewed pressure for stricter legislation and crackdown, which sends the loop round once again. (There are many side effects of this main loop, including increased street crime – funding drug habits; the creation and enrichment of gangs and warlords; and so on.)

Scratching doesn’t help

Nobody in their right mind wants this to happen. But even a kindergarden child can see that policing and jailing, like a good scratch, feels good in the short term but just drives the wheel of the problem in the long term. What are the alternatives? From time to time pressure is brought to bear on production, for example, trying to obliterate coca or poppy fields, or disrupt supply chains. But this is also hopeless because as long as there is a good price to be had, the systemic reality is that drugs will be grown, produced, and shipped. What shows great promise is tackling price. The legalization lobby is all about capping price by making drugs legal, supervised, available, and free (or low-price), removing the superprofits from the industry and thereby blunting the primary interest of drug bosses and warlords. (This is what happened when Prohibition was repealed).

From a systemically informed viewpoint, only a solution that changes the system (interrupts the reinforcing cycle) can change the future. In other words nothing significant will occur in the future until the system changes, and removing drug barons’ price interest is the only way to do it. Until this happens the savvy forecast must be: no change.

Educating consumers

But the public is not ready for such policies. So we are left with the holding pattern we are in. And this includes exhorting the consumer, as Vice-President Calderón is doing. (The same story and interview was featured earlier this week on Radio 4′s “Today” Show.) He’s targeting the middle class, occasional, and recreational drug users who, he says, otherwise recycle, and compost, and “drive a hybrid” and buy fair trade coffee, and so on, and so should be desist from drug use because of it’s environmental impact.) This is not the first time that consumers have been “educated” – school and public education programs consistently target, inform, and discourage consumers and would-be consumers (including, of course, in the laughable “Just Say No” campaign.) All good or at least harmless work, in a good cause.

Into this Calderón has added a new-to-the-industry category of demotivator – the environment. Sure, this should work in giving middle-class consumers pause. But if environmentally sensitive cocaine customers are a big part of the market — and it’s hard to tell if they really are — expect producers to just respond with Green reassurance, real or fake: “No trees were ploughed under in the creation of your snort.”

“Future Savvy” prescribed for Masters Program in Strategic Foresight

Posted by Adam Gordon on Nov 4, 2008 in 2025, all, forecast filtering, foresight tools & methods, Future Savvy, horizon scanning, managing uncertainty, strategic foresight

My aim for this blog is not particularly to champion success stories for Future Savvy. I’m happy to let the book talk for itself. I’d prefer to look at forecasts and foresight work out there and think about how well it is working, and/or who it may be working for. However it’s nice to be able to report, inter alia, that the book has been quickly picked up and prescribed as a required resource in the Masters Program in Strategic Foresight, at the School of Global Leadership & Entrepreneurship, Regent University (VA).

Future Savvy is intended to be a book for business and policy professionals, not academics. But it does speak to students and scholars who need to assess and evaluate foresight work. In any event, professors Jay Gary and Dennis Walters have included it alongside works by Wendel Bell, Jerome Glenn, and Ted Gordon (no relation) — household names in the foresight field — so the comparison is of course very happily accepted.

The following is from the course outline. I’ve included the full bibliography, which is in itself a valuable collection of sources in the futures field, and merits attention all the way down the list.

Course Description:
Surveys traditional forecasting theory and methods. After a consideration of forecasting in general, students learn how to conduct research using both qualitative (secondary sources, interviews and questionnaires) and quantitative (data analysis, numerical forecasting and trend decomposition). They also apply critical thinking skills to existing forecasts. [Learning objectives:] 1. Managing: understand the principles and applications of operational forecasting within organizations. 2. Assessing: decide when to use statistical or judgmental methods in strategic forecasting, and how to combine foresight methods to generate 10 to 20 year outlooks. 3. Evaluating: gather information in a specific domain that can be used to forecast baseline as well as alternative futures. 4. Researching: construct a long-term strategic forecast for a client organization that draws upon both quantitative and qualitative sources.

Required Resources
* Bell, Wendell. 1996. Foundations of futures studies: History, purposes, and knowledge. (Human Science for a New Era), vol. 1. New Brunswick, NJ: Transaction. ISBN: 0765805391
* Carlberg, C. G. (2005). Excel sales forecasting for dummies. Hoboken, NJ: Wiley. ISBN: 0764575937
* Glenn, J. C., and Gordon, T. J. Futures Research Methodology V2.0 CD-ROM American Council for the UNU. ISBN: 097220511X
[This item is available through http://www.acunu.org/millennium/FRM-v2.html]
* Gordon, A. (2009). Future Savvy: Identifying trends to make better decisions, manage uncertainty, and profit from change. New York: American Management Association. ISBN: 0-8144-0912-1
* Jain, C. L. ed. (2001). Practical guide to business forecasting. Flushing, NY: Graceway. ISBN: 092126758

Recommended and supplemental resources:
*  Coates , Joseph F 2025, John B. Mahaffie, and Andy Hines. 2025: Scenarios of US and Global Society Reshaped by Science and Technology. Oak Hill Press. ISBN: 1886939098, also available in .pdf files via http://www.josephcoates.com/2025_PDF.html
* Armstrong, J. S. (1985). Long-range forecasting: From crystal ball to computer (2nd ed.). New York: Wiley. ISBN: 0471823600, also available in .pdf files via http://www.forecastingprinciples.com/Long-Range%20Forecasting/contents.html
* Armstrong, J. Scott. 2001. Principles of Forecasting . Kluwer. ISBN: 0792374010.
* Caplow, T., Hicks, L., & Wattenberg, B. J. (2001). The first measured century: An illustrated guide to trends in America , 1900-2000 . Washington , DC : AEI Press. Download chapters at: http://www.pbs.org/fmc/book.htm
* Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571-582.
* Duberley, J., & Johnson, P. (2000). Understanding management research: An introduction to epistemology. Thousand Oaks, CA: Sage.
* Einhorn, H.J. (1986). Accepting error to make less error. Journal of Personality Assessment, 50, 387-395.
* Fischoff, B. (1994). What forecasts (seem to) mean. International Journal of Forecasting, 10, 387-403.
* Gawiser, Sheldon R., and G. Evans Witt. 1994. A Journalist’s Guide to Public Opinion Polls . Praeger. ISBN: 0275949893.
* Gillham, Bill. 2000. The Research Interview. Continuum International. ISBN: 082644797X.
* Hetman, F. (1969). Le Langage de la prévision, the language of forecasting: With a French-English-German vocabulary. Paris: S.ÉD.ÉI.S. http://www.cnam.fr/lipsor/eng/data/langageprevision.pdf
* Jantsch, E. (1967). Technological forecasting in perspective. Paris: OECD. http://www.cnam.fr/lipsor/recherche/laboratoire/data/prevtech_en_final.pdf
* Makridakis, S. G., Wheelwright, S. C., & Hyndman, R. J. (1998). Forecasting: Methods and applications (3rd ed.). New York: John Wiley & Sons.
* Molitor, G. T. (2003). The power to change the world: The art of forecasting. Potomac, MD: Public Policy Forecasting.
* Moore, D. A., Kurtzberg, T., Fox, C. R., & Bazerman, M. H. (1999). Positive illusions and forecasting errors in mutual fund investment decisions. Organizational Behavior & Human Decision Processes, 79, 95-114.
* Orrell, D. (2007). The future of everything: The science of prediction. New York, NY: Thunder’s Mouth.
* Rescher, N. 1998. Predicting the future: An introduction to the theory of forecasting. Albany: SUNY Press. ISBN: 0-7914-3553-9
* Salant, Priscilla, and Don A. Dillman. 1994. How to Conduct Your Own Survey . Wiley. ISBN: 0471012734.
* Seidensticker, R. B. (2005). Future hype: The myths of technology change. San Francisco, CA: Berrett-Koehler.
* Schnaars, S. P. (1989). Megamistakes: Forecasting and the myth of rapid technological change. New York: Free Press
* Sherden, William A. (1998). The fortune sellers: The big business of buying and selling predictions. New York: John Wiley.
* Wood, G. (1992). Predicting outcomes: Sports and Stocks. Journal of Gambling Studies, 8, 201-222.

Debates in forecasting Euro’s status vs. Dollar, 2025

Posted by Adam Gordon on Sep 23, 2008 in 2015, 2025, all, decision-making, failed predictions, forecast filtering, foresight tools & methods, managing uncertainty, Perils of Prediction, strategic foresight

A recent forecast-and-critique exchange between economists is worthy of attention from a forecast assessment and evaluation point of view.

The forecast is the recently published academic research paper: Chinn & Frankel (2008), “The Euro May Over the Next 15 Years Surpass the Dollar as Leading International Currency,” Faculty Research Working Paper RWP08-016 (Cambridge, MA: Harvard University, John F. Kennedy School of Government) available here. Frankel is a Professor of Economics at the Kennedy School.

The critique, “Forecasting the Euro’s Future,” by Benjamin Cohen, is here

The argument of the Chinn & Frankel paper, which is also summarized here is that the euro may surpass the dollar as the leading international reserve currency as early as 2025. The authors use econometrically-estimated determinants of the shares of major currencies in the reserve holdings of the world’s central banks. Significant factors include: size of the home country, rate of return, and liquidity in the relevant home financial center (as measured by the turnover in its foreign exchange market). The analysis predicts a narrowing in the gap between the dollar and euro over the period 1999-2007, and forecasts this trend to continue.

Cohen has technical issues with the forecasts, saying, “the analysis addresses just one specific function of the two rival monies – their use in central bank reserves – ignoring all the many other roles that international currencies play. But the essence of his critique is deeper. He says, “By concentrating purely on economic factors, (the forecast) ignores the politics involved, which in practice could prove to be far more decisive… key considerations include both the quality of governance in a currency’s home economy and the nature of relationships between countries. Is the issuer of a currency capable of assuring effective political stability at home? Can it project power abroad? Does it enjoy strong inter-governmental ties – perhaps a traditional patron-client linkage or a formal military alliance? Though it is by no means easy to operationalise many of these factors for purposes of empirical analysis, it is hard to deny their importance (for an accurate forecast.)”

Cohen’s agenda is not merely to tackle possible shortcomings of Chinn & Frankel’s study, but to critique economic forecasters far-and-wide that analyze the technical data, while ignoring political (or social) factors that are hugely influential on outcomes, yet harder or impossible to quantify, and which are therefore conveniently ignored.

Coming to grips with politics
Says Cohen: “Chinn and Frankel are not alone in this shortcoming, of course. Many economists, perhaps even most, have a hard time coming to grips with the intricacies of politics, which can seem so messy and indeterminate when compared with the pristine parsimony of formal economics. When it comes to the analysis of public policy, few even bother to try to address political factors systematically.

“The result, though, is sadly predictable. By ignoring the role of politics, economists often get it wrong. How many trade specialists were prepared for the recent breakdown of the Doha trade talks, despite the obvious gains to be had on all sides from a new round of liberalisation? How many can explain the unprecedented accumulation of reserves in China or other East Asian countries, the widespread distrust of multinational corporations or the failure of the international community to do a better job at combating global warming? Politics is clearly critical to all these questions, and more… (Yet) conveniently, Chinn and Frankel set all these considerations aside in order to build a parsimonious model that they can use for forecasting purposes. Only three independent variables are highlighted in their regressions: country size (relative income), foreign-exchange turnover (representing the depth of competing financial markets), and trend exchange-rate changes (representing the rate of return on currency balances).”

Cohen offers potential political and ideological blockers to the particular forecast: “Japan, for instance, has long relied on a formal security umbrella provided by the United States to protect it against external threats; and the same, less formally, is true for Saudi Arabia and other Gulf states as well. Can we really imagine any of these nations, all very large dollar holders, casually jeopardising their ties to Washington for the sake of a few basis points of return on their reserves?”

To be fair to Frankel, the nature of his analysis is consistently political – see his blog at http://content.ksg.harvard.edu/blog/jeff_frankels_weblog/
One can’t imagine that Frankel or Chinn would dispute that politics will strongly influence the accuracy of their forecast. (What they clearly imply in their data-centered model is that the economic data is backed up by political shifts towards Europe, or at least there is nothing in the political realm that would counter their technical analysis.)

Yet the problem remains that these contextual factors are not built into the model. The technical stuff is quantifiable and gets forecasted quantitatively. The rest is a kind of political/social/ideology soup that we flounder in, and the best we can apparently say is “it’s going in the same direction” or “ceteris paribus”.

International Political Economy
Going with Cohen, one may well ask: what is the value of the forecast that ignores the context, or separates it in this way? Surely very little. As impressive as the economics or the modeling is, the results are are circumscribed by the larger questions that are not in the model, and that affect everything.

As an alternative, Cohen offers International Political Economy (IPE), which explicitly combines political analysis with economic theory, saying, “part of what IPE offers is a critique policy choices as ‘rational calculus by unitary actors responding to well-defined structural constraints and incentives – in effect, an approach akin to the analysis of atomistic firms in a setting of perfect competition.’” IPE suggests three levels of political analysis: the systemic level (macro-international politics); the domestic level, revealing competition of domestic interest groups and institutions; and the cognitive level, ideas that legitimate governmental policy making. If one is not thinking at all three levels of politics, any prediction will surely fail.

Whether IPE succeeds in mitigating the shortcomings of technical analysis or not, one can only say amen to the principle – and that, additionally, there’s surely even more to factor in. Beyond politics, there are issues of technology change, changes in culture, values, ideologies and perceptions that shape the future. Truth is, we don’t know how to quantify all this – and it’s certainly not tractable to quantitative measures for anything but the short term. Using the technical analysis to predict the euro’s status vs. the dollar in 2025 must return a result which (while even possibly correct) is one we cannot rely on.

The Uses and Limits of Prediction Markets in Forecasting

Posted by Adam Gordon on Aug 20, 2008 in all, failed predictions, foresight tools & methods, Future Savvy, prediction markets

Hmm. As the 2008 White House race hots up, we’re going to be hearing more and more – and then even more – about who prediction markets forecast to win, so it’s time to put down a thought or two about uses and limitation of this forecasting tool.

First, what’s if all about? If you already know, skip this section. Let’s start with the example in yesterday’s Telegraph: “Predicting the future – with the power of betting” Paul Parsons, August 19, 2008. As Parson’s reports, the University of Iowa is running a market where investors can buy “shares” in the two major US election candidates, each priced between $0 and $1. On election day, traders holding stock in the winner – Obama or McCain – receive $1 per share while the others lose their money. Investors can buy and sell their shares along the way, and as they do this the candidate more people will want to own (because they think he will win) will get more expensive. In other words, market forces will drive up the price of the outcome more people think more likely. As of August 19, the trading value of the Obama, at $0.62, suggests participants expect a 62 percent chance he will win. (Another prediction market site, midasoracle.org, has the figure currently at 59.8 percent.)

Prediction markets mimic stock market and deploy much the same software. Where a real market trades shares in an underlying asset, in a prediction market it is future outcomes which are “securitized”. The key principle at work is the sage market wisdom that “the price of a stock captures all the information known about it” – that is, all information is factored into the price (notwithstanding that some may have more or better information than others; some may be acting more wisely on their information). Therefore price is our guide to the cumulative knowledge of all participants and, in prediction markets, this “price discovery” allows us to know what most people think the future holds. They allow the “the wisdom of crowds” to be turned to a future problem, and tapped.

Serious Success

What’s exciting about all this is its success rate. Prediction markets are amazingly accurate in many circumstances, and by all accounts consistently beat more conventional quantitative and extrapolative methods. Prediction markets have consistently out-predicted election opinion polls and exit polls. Of course the predictive potential goes way beyond polling. Forecasting markets can and have been set up to predict the dollar movements to the success of same-sex marriage legislation, to who will win best actor Oscar. At one point there was even a US government market in future terror targets (trying to elicit public predictions of likely targets so as to plan accordingly) but this was deemed inappropriate and taken down.

As it has become clear that this method outstrips conventional forecasting methods, prediction markets have taken root in forward-looking businesses. Companies such as Google and Hewlett-Packard routinely use (internal) prediction markets to forecast sales figures, customer preferences, product adoption, and so on. HP is on the record as saying prediction markets consistently outperform their official forecasts.

The method has other advantages too. First, it requires no special techniques or expense. There are no fancy models to apply or complex algorithms to … to do whatever one does with such things. Second the forecasts are available in real time, all the time, and constantly update themselves. There’s no waiting for data collectors to collect, or statisticians to emerge with their answers.

The Limits

In my book, Future Savvy, I show how and why humans are poor at predicting, for dozens of reasons. The record of predicting is littered with failure. But, is that now all in the past? Do prediction markets solve the perennial problem of predicting the future, or at least get us closer? Yes and no.

Yes where prediction markets are appropriate. They work best under two conditions: first where there is a clear view of the options and operating conditions; second (related) where the time frame predicted is relatively short, usually under 18 months depending how fast things are moving. Where predicting the future means choosing between known alternatives, such as an election winner, or anticipating a point along a known continuum, for example the level of next year’s sales, prediction markets are great.
Where prediction markets run dry is in dealing with unfamiliar conditions, or unknown variables, or potential game-changing disjunctures in the world. Where the future is seriously fuzzy, where there are many variables, and the way they interact unknown, and drivers, blockers, and lags are hidden, prediction markets are of limited use because the outcomes can’t be framed adequately so that people can bet on them or against them. A prediction market for US president in 2012 would be far less useful than 2008. Similarly, while a market for the oil price in 2009 would be helpful, by 2010 or beyond factors driving the price may be so different (viz. developments in sustainable energy or geopolitics) that the result of a prediction market conducted in 2008 would be undependable.
So while prediction markets sort out probabilities between known likelihoods, they are not adequate to the task of investigating complex situations where we cannot frame the likely outcomes, or at least can’t know if we’ve framed them right. Also while prediction markets do help us, on aggregate, avoid some perceptual/cognitive fallacies, they are as likely as any other predictive tool to fall into the Zeitgeist effect. More on this soon…

A good list of articles on prediction markets is available here: http://www.midasoracle.org/best/

Future Savvy: What’s Under the Hood

Posted by Adam Gordon on Jul 9, 2008 in 2015, 2025, all, decision-making, failed predictions, forecast filtering, foresight tools & methods, Future Savvy, horizon scanning, leadership, managing uncertainty, Perils of Prediction, scenario planning, strategic foresight, strategic planning, systems dynamics, technology change, trend tracking

The book Future Savvy shows readers how to critically judge forecasts for themselves. These are the chapters that take the reader there:

Chapter 1: Recognizing Forecast Intentions, deals with considerations of how forecasts come about, who makes them, and with what intention. Those who research and produce forecasts, those who invest in understanding trends and drivers of change, and those (including the media) who bring the forecasts and their implications to our attention, inevitably have reasons for doing so – to benefit from the knowledge by seizing opportunities or avoiding threats or by affecting outcomes in the world. Understanding a forecast’s “return on investment” gives us an important vantage point in assessing the merits of a forecast.

Chapter 2: The Quality of Information, shows how a forecast communicates information between forecaster and reader subject to the same standards of accuracy, truth-telling, and bias-control by which one would judge any communication. Forecasts can be very different in methods and goals, but all forecasts lay claim to factual truth, particularly truth in the data, and the argument deals with the various ways in which data can be less solid than it looks, even with the best intentions.

Chapter 3: Interpretation and Bias, considers how data – whether good or bad in itself – can be interpreted or misinterpreted in forecasting, that is, the “political” aspects of forecasting. Just as there is no value-free look at history, so too there is no value-free look to the future and asking the right questions allows us be ready to mentally rebalance forecasts that are presented.

Chapter 4: Paradigms and Perception, investigates how predictive statements are exposed to a broader form of interpretive bias that has to do with the forecaster’s mental model or “paradigm,” and the “zeitgeist” (spirit of the times) when the forecast is made. This chapter investigates situations where forecast failure is caused by failure to escape society’s current mental models – which often do not hold through the forecast period.

Chapter 5: The Utility Principle, considers economic and market forces, and the role of consumers, in promoting or resisting the future. Without reigning in creative thinking, some simple economic filters inevitably apply direction or timing realism to futurist flights of fancy.

Chapter 6: Drivers, Blockers, and Trends, consider drivers and blockers of change, and how viewing these dynamics improves forecast assessment. It identifies the roles of Drivers, Enablers, Friction, and Blockers acting on events to cause change or resist it, and problems in dumbly projecting current trends.

Chapter 7: The Limits of Quantitative Analysis, discusses the role of statistical analysis and quantitative modeling in predicting the future – where this is possible and useful and where it is not, and why not.

Chapter 8: The Systems Perspective, investigates “system effects,” which occur whenever different elements or variables that may appear isolated are in fact linked together, such that changes in one element cause changes in others. Anticipating future behavior of any variable hinges on identifying the broader systemic elements influencing it and failing to do this is a big part of what causes forecasts to fail.

Chapter 9: Living with Alternative Futures, investigates non-predictive ways of approaching change – where the tone is more about managing uncertainty than predicting the future. It acknowledges unfathomable complexity of most future questions and provides perspectives that raise chances of success in an inherently unpredictable future.

Chapter 10: Forecast Filtering in Action, illustrates the processes of the book by applying them in case studies to real-world sample forecasts that decision makers in business and policy areas might find themselves interacting with. This demonstrates how real everyday predictive material may be probed and critically evaluated, following the principles developed in previous chapters.

Chapter 11: A Forecast Filtering Checklist, is a cross-cutting checklist which summarizes the principles of the book in one convenient, thematic list.