Future Savvy: Quality in Foresight

Continuing the theme of financial types talking to each other about predictions and predictability, this ‘Tea with the Economist’ interview of Stephen Roach, Chairman, Morgan Stanley Asia by Economist New York Bureau Chief Mathew Birk, carries interesting lessons about the limits of prediction.

Birk commends Roach for being one of the few to have predicted the Credit Crunch problems, to which Roach demurs in saying he was “too early”. He then furthers his modesty in saying that the “breakage” in the financial system was “in excess of anything I envisioned.”

Self-deprecation in assessing one’s predictive abilities will endear anyone to me. Even Roach, who later in the interview burns this hard-won credibility by laying the blame for the credit crunch at the door of regulators, forgetting how hard financial institutions lobbied regulators for greater freedoms in the 1990s.

But I digress. The predictive issues the interview raises are as follows. Issue one: it’s not enough (as any stock short-seller will confirm) to get the direction of a future change right. One must get the timing right too. Issue two: it’s not enough to anticipate a change. One must be able to judge it’s impact. Getting either timing or impact wrong is effectively to have missed the future.

.

Probability

On the latter topic — the problem of impact — Nassim Taleb is unrelenting, and he is right. Analysts routinely mix up probability and impact. They think that because an event has a low probability (‘it would be a 10-sigma event!’) it can be marginalized in the predictive number crunching. Of course, it can’t. The low-probability of a wildcard or black swan event is irrelevant because when it happens it will change the game, and that’s why, in every predictive situation of reasonable complexity and uncertainty, using statistical extrapolations (regressions and so on) to predict, is to dangerously paper over the cracks. It is precisely the cracks that businesses and policy makers need to worry about.

Determining the direction of change is hard enough. Assessing timing or extent of impact — a ‘total future impact index’ — is wickedly difficult. It’s a task not to be underestimated, and to simply extrapolate current trends (= assuming the trend’s timeline and impact stay the same as in the past) is the royal road to underestimating it.

This is the reason foresight for complex, uncertain, changing situations can only be grasped by NOT predicting (quantitatively or otherwise) but by exploring the limit-conditions of the plausible (What would happen if the timing of the change accelerated, or was significantly delayed? What if  the impact was 10x or one tenth of what we expect? And so on.)

.

Tweet This Post

I’ve been quite careful not to use this blog as a “brag wall” for Future Savvy. I can say reviewers have all been glowing, without exception. But this review, below, which recently appeared in the St Andrews Management Institute’s Vector Magazine, I felt was worth reposting here because – more than just saying nice things – it also captures the essence of what the book is trying to do. Here it is:

Book reviews by SAMI fellows and associates
“Future Savvy” by Adam Gordon (American Management Association, 2009)

“Forecasts and predictions are ubiquitous. We are bombarded with views of the future on a plethora of subjects from myriad sources, with a diverse set of motivations and self-interests. Adam Gordon seeks to provide a practical users guide to the assessment and interpretation of all things about the future, with special emphasis on the cautions and ‘health warnings’ that need to be applied, so as not to be misled by forecasts. However, the author is careful not to veer towards over-cynical dismissal of all future projections; rather, he seeks to provide guidance to the reader on how to apply the necessary caveats, and in the author’s words “profit from change”.

The book covers a very broad field, from the basic issues of the misuse of data and statistics, covering the quality and validity of data as well as their misinterpretation, through technology forecasting, trend and horizon scanning to quantitative modelling and scenarios. The one theme common to all these activities is the need to be alert to bias, whether it be a deliberate motive to influence behaviour through a dire prediction; or a bias inherent in futurologists needing to see rapid and pervasive change in all areas of society – if it exists or not – and evangelising it.

The track record of much futurology is mixed. Well-known examples are quoted: television did not lead to the end of the cinema industry. Nor has space exploration led to people taking foreign holidays on other planets – yet! Bias may also lie in the beholder. The ‘Zeitgeist’ tendency, whereby we are all influenced by contemporary perceptions, affects not only how “experts” and professionals see the world, but also how the audience receives the views of the future – often with unprepared minds. The internal “official future” of an organisation can pose a real blind spot to its progress.

The weaknesses of much quantitative modelling are highlighted, with such forecasts only being as good as the assumptions on which they are based, but which are often not overtly stated. In contrast to the conceptual and practical errors inherent in much futures output, the role and advantages of scenario planning are emphasised as a tool for challenging assumptions and developing alternative futures: “It’s better to be vaguely right than precisely wrong”.

The penultimate chapter takes examples of relatively recent forecasts from a range of organisations, whose subjects range from US agricultural production to UK dementia sufferers. These are subjected to a form of ‘retro wind-tunnelling’ to illustrate the deficiencies in their construction and how they would have benefited from the application of methodologies described earlier in the book. The final chapter provides a summary checklist, or framework, to apply in evaluating forecasts and future predictions.

Adam Gordon has written a shrewd and perceptive book that deserves a wide readership, especially among managers in both the private and public sectors, as well as the familiar ‘general reader’. Those wishing a more detailed technical guide to the various forecasting and futurist methodologies will need to consult other standard works. Professionals in the fields of management and strategy consulting and scenario practitioners might well be familiar with many of the points made in the book. However, those with some savvy might do well to recommend the book to their clients.

Michael Owen, 20 April 2009

Tweet This Post