Future Savvy: Quality in Foresight

A recent forecast-and-critique exchange between economists is worthy of attention from a forecast assessment and evaluation point of view.

The forecast is the recently published academic research paper: Chinn & Frankel (2008), “The Euro May Over the Next 15 Years Surpass the Dollar as Leading International Currency,” Faculty Research Working Paper RWP08-016 (Cambridge, MA: Harvard University, John F. Kennedy School of Government) available here. Frankel is a Professor of Economics at the Kennedy School.

The critique, “Forecasting the Euro’s Future,” by Benjamin Cohen, is here

The argument of the Chinn & Frankel paper, which is also summarized here is that the euro may surpass the dollar as the leading international reserve currency as early as 2025. The authors use econometrically-estimated determinants of the shares of major currencies in the reserve holdings of the world’s central banks. Significant factors include: size of the home country, rate of return, and liquidity in the relevant home financial center (as measured by the turnover in its foreign exchange market). The analysis predicts a narrowing in the gap between the dollar and euro over the period 1999-2007, and forecasts this trend to continue.

Cohen has technical issues with the forecasts, saying, “the analysis addresses just one specific function of the two rival monies – their use in central bank reserves – ignoring all the many other roles that international currencies play. But the essence of his critique is deeper. He says, “By concentrating purely on economic factors, (the forecast) ignores the politics involved, which in practice could prove to be far more decisive… key considerations include both the quality of governance in a currency’s home economy and the nature of relationships between countries. Is the issuer of a currency capable of assuring effective political stability at home? Can it project power abroad? Does it enjoy strong inter-governmental ties – perhaps a traditional patron-client linkage or a formal military alliance? Though it is by no means easy to operationalise many of these factors for purposes of empirical analysis, it is hard to deny their importance (for an accurate forecast.)”

Cohen’s agenda is not merely to tackle possible shortcomings of Chinn & Frankel’s study, but to critique economic forecasters far-and-wide that analyze the technical data, while ignoring political (or social) factors that are hugely influential on outcomes, yet harder or impossible to quantify, and which are therefore conveniently ignored.

Coming to grips with politics
Says Cohen: “Chinn and Frankel are not alone in this shortcoming, of course. Many economists, perhaps even most, have a hard time coming to grips with the intricacies of politics, which can seem so messy and indeterminate when compared with the pristine parsimony of formal economics. When it comes to the analysis of public policy, few even bother to try to address political factors systematically.

“The result, though, is sadly predictable. By ignoring the role of politics, economists often get it wrong. How many trade specialists were prepared for the recent breakdown of the Doha trade talks, despite the obvious gains to be had on all sides from a new round of liberalisation? How many can explain the unprecedented accumulation of reserves in China or other East Asian countries, the widespread distrust of multinational corporations or the failure of the international community to do a better job at combating global warming? Politics is clearly critical to all these questions, and more… (Yet) conveniently, Chinn and Frankel set all these considerations aside in order to build a parsimonious model that they can use for forecasting purposes. Only three independent variables are highlighted in their regressions: country size (relative income), foreign-exchange turnover (representing the depth of competing financial markets), and trend exchange-rate changes (representing the rate of return on currency balances).”

Cohen offers potential political and ideological blockers to the particular forecast: “Japan, for instance, has long relied on a formal security umbrella provided by the United States to protect it against external threats; and the same, less formally, is true for Saudi Arabia and other Gulf states as well. Can we really imagine any of these nations, all very large dollar holders, casually jeopardising their ties to Washington for the sake of a few basis points of return on their reserves?”

To be fair to Frankel, the nature of his analysis is consistently political – see his blog at http://content.ksg.harvard.edu/blog/jeff_frankels_weblog/
One can’t imagine that Frankel or Chinn would dispute that politics will strongly influence the accuracy of their forecast. (What they clearly imply in their data-centered model is that the economic data is backed up by political shifts towards Europe, or at least there is nothing in the political realm that would counter their technical analysis.)

Yet the problem remains that these contextual factors are not built into the model. The technical stuff is quantifiable and gets forecasted quantitatively. The rest is a kind of political/social/ideology soup that we flounder in, and the best we can apparently say is “it’s going in the same direction” or “ceteris paribus”.

International Political Economy
Going with Cohen, one may well ask: what is the value of the forecast that ignores the context, or separates it in this way? Surely very little. As impressive as the economics or the modeling is, the results are are circumscribed by the larger questions that are not in the model, and that affect everything.

As an alternative, Cohen offers International Political Economy (IPE), which explicitly combines political analysis with economic theory, saying, “part of what IPE offers is a critique policy choices as ‘rational calculus by unitary actors responding to well-defined structural constraints and incentives – in effect, an approach akin to the analysis of atomistic firms in a setting of perfect competition.’” IPE suggests three levels of political analysis: the systemic level (macro-international politics); the domestic level, revealing competition of domestic interest groups and institutions; and the cognitive level, ideas that legitimate governmental policy making. If one is not thinking at all three levels of politics, any prediction will surely fail.

Whether IPE succeeds in mitigating the shortcomings of technical analysis or not, one can only say amen to the principle – and that, additionally, there’s surely even more to factor in. Beyond politics, there are issues of technology change, changes in culture, values, ideologies and perceptions that shape the future. Truth is, we don’t know how to quantify all this – and it’s certainly not tractable to quantitative measures for anything but the short term. Using the technical analysis to predict the euro’s status vs. the dollar in 2025 must return a result which (while even possibly correct) is one we cannot rely on.

Tweet This Post

In the MBA elective “Industry Foresight and Scenario Planning” that I teach, toward the beginning of Day One, I ask participant some very basic questions – basic questions being, of course, the hardest. One of them is: Can We Predict the Future, Yes or No?

Being graduate students, they’ve learned to prevaricate, and they do. It’s either No, with a bit of yes; or Yes with a bit of no.  Both are correct or course. Clearly nobody can see the future perfectly, but there do seem to be times and/or situation where some see it much more clearly than others. (And therefore make better forecasts. How one can recognize this is a core topic of “Future Savvy.”)

Anyway, I’m reminded of this because I stumbled on two Web links, one after the other, that are salvos in this debate. The first is at:           
http://picasso.rediffiland.com/blogs/2008/09/08/Predicting-the-Future-We-veall-hear.html
This is very much the standard, smirking, “look-see-bigCheese-got-egg-on-his-face” testimonial, of which there are many. Bloggers are, in many ways, journalists, and all journos like to see a big-shot egg-faced.

The other link is a fun 3-min video, posted on the Disney blog, see

which is at
http://thedisneyblog.com/2008/09/08/how-good-was-disney-at-predicting-the-future/

The clip argues, possibly slightly tongue in cheek, that the Disney forecasts as portrayed “Horizons,” EPCOT in 1983 – 25 years ago – were, in fact, not bad predictions. (Context is Disney’s tomorrow visions have, generally, been discredited.)

Back to the smirk site, above, which bears further thinking about. This one is hardly original (why do they all have the same 20 quotes? They also normally start with the Yogi Berra-ism “Predicting is hard, especially about the future.” Yawn) but at least they all correctly put us on our guard as to the poor future thinking of industry experts. In fact, the record of future prediction is littered with the most astounding mistakes. From underwater cities never built to rocket mail that never flew to Y2K disasters that never materialized – the list of laughable errors is a mile long. Experts aside, all of us are liable to confidently anticipate things that wont happen while missing what is brewing right under their noses.

Prediction-skepticism
Fair enough. But, the predictive nihilism behind these smirk sites is is dangerous in a number of ways.

First it promotes the skepticism that “we cant know anything” about the future. If the experts were so wrong – let’s all just give up. And therein we get the problem of many people, including highly-paid managers, justifying ignoring or under-funding future thinking. Sometimes managers, not wanting to look unprepared, suggest resources and expertise be channeled into “fast response” so that when the future becomes clear they can move rapidly to profit. This view is soundly rubbished in Hamel & Prahalad’s classic HBR 1994 article “Competing For The Future,” and I don’t think more needs to be said. 

Second, there’s obviously no science behind the smirk. They pointedly do not show the number or extent of incorrect forecasts *in context of the total forecasts made*. We don’t know, in other words, how many people got it right or at least right enough to have profited or avoided losses. Wherever you have significant success, it is likely that there is a good-enough forecast behind it.

Finally the failed-forecast smirk lists also miss the fact that many forecasts are not meant to be an accurate anticipation of events. Many are trying to influence the future, that is, talk a particular outcome into being or shape it, or stop it from happening. People make predictions to sway an audience, or get a response from authorities or opposing forces. When Gates said: “640K should be enough for anyone,” who was he talking to, and what was he trying to achieve …? A real prediction of the future? I think not. Microsoft did not stop at 640, and nor did anyone think it would. And nor did Gates think anyone else would. Forecasts are often salvos in the games of power and influence, flagrantly used to marshal situations or promote self-interests, in situations where accuracy is not the point.

Tweet This Post

In my previous post, below, I threatened myself with the penance that I’d have to come back and think about the zeitgeist effect as a further limit on the use and validity of prediction markets. (Once again, I’m basically sold that prediction markets are a fabulous way to think about the short-term and/or contained system future, but it’s worth being clear about the limits of this tool.)

Zeitgeist, German for “spirit of the times,” refers to the often unconscious spectrum of intellectual views, analytical approaches, political and social concerns, etc., that people in any era share. Evidence from the checkered history of predicting the future shows that forecasters have been very heavily biased by their then-current conditions, current issues, and current state of the world, that is by their zeitgeist. They see reality and therefore the future through the lens of their times. The key marker of this effect at work is when many forecasts are not only wrong, but are wrong in the same way.

The zeitgeist effect, by the way, is not a peculiar condition. It is part of the many common human and social cognitive-perceptual biases that exist. (For a longer list and their effects in forecasting, see Future Savvy.) Zeitgeist is in fact sometimes more broadly known as “situational bias,” where situations that people find themselves in frame what they see and how they interpret events. In a depression, famously, it is difficult to see any source of upturn; in boom times it is hard to see the crash.

The Zeitgeist of the 1990s
Just one example of zeitgeist bias in forecasting can be seen in predictions made in the 1990s, when the fall of Soviet Union and the end of the Cold War provoked new hope about global interaction and growth, and this, combined with the rise of digital technologies, the Internet, WTO agreements, and the dot.com market boom, fueled a new zeitgeist of optimism. It was very common, in this era to predict global prosperity, international peace and harmony, rising standards of living, enhanced personal freedoms, and a better environment. By the end of 2001, the NASDAQ bubble had burst, Al-Qaeda had struck buildings that symbolized U.S. power, the “War on Terror” had begun, and the entire rosy 1990s and all the forecasts that went with it, were finished. Zeitgeist or situation has, evidently, a very strong pull on what people think is possible and likely in the world at large, and in their own sector or industry. It frames the questions people ask, the topics they think are important, the outcomes they expect, and how they interpret signals of change.

The point is, do prediction markets somehow counter this well-known perceptual-cognitive bias in forecasting? No. The outcomes people think more plausible – that they will “buy up” in a prediction market – are deeply affected by situational conditions. This is sure to be a factor when, for example, Google sets up a market to tap its employees for forecasts. The people involved, smart as they are, will be strongly by situational factors – and this will affect which future outcomes look more likely.

Herd Effects
Moreover, they will be pulled in the same direction. As prediction markets, like Delphi studies, are particularly a consensus-based method (forecasters drawing predictive results from the study/game are going on what most people say/do), they are by definition deeply vulnerable to the zeitgeist effect. This dovetails with another well-known social cognitive bias, the “herd effect” or “bandwagon effect,” or “groupthink,” where people to believe or do things because many other people are doing or appear to be doing them. The only way to avoid groupthink is to isolate people from each other (as Delphi studies attempt to do by not disclosing others’ responses while the study is live.) But in prediction markets the player can, of course, see what others are doing. When buying a “stock” that is getting more expensive they may be reassured that others are buying it too — leading them to buy more, producing a classic herd-effect situation.

These biases – and various others, as detailed in Future Savvy, are among the many perceptual and cognitive biases that people bring to the world. They are part of (and evidence of) perception being active and constituent in our understanding of the world, and therefore of the future. We can chip away at our perceptual framwork, particularly by questioning our assumptions and investigating the basis of our knowledge (possibly through scenario planning.) However, no matter how astutely we look and how consciously we try to eliminate them, our paradigms exist, and they color our forecasts, and prediction markets do not make them go away.

Tweet This Post