Future Savvy, as viewed by ‘Info-Savvy’ Peter Stoyko (SmithySmithy)

Posted by Adam Gordon on Jul 29, 2010 in all, decision-making, failed predictions, forecast filtering, Future Savvy, managing uncertainty, scenario planning, strategic foresight, trend tracking

I was lucky enough to have Future Savvy included in a lengthy review of critical thinking in forecasting & foresight, done on the SmithySmithy “info-savvy” blog. The post also included Nassim Taleb’s ‘The Black Swan’ (2007) and ‘Fooled By Randomness’ (2005); Kenneth Posner’s ‘Stalking the Black Swan’ (2010), and Chris Luebkeman’s Drivers of Change (2009).

As Stoyko’s is head-and-shoulders the most insightful and thorough assessments of the book itself, and the book in context, I’m reposting it here, with thanks. There are also fabulous graphics added, such as these (see more below):

“My search led to Adam Gordon’s Future Savvy. Like Posner, Gordon challenges Taleb’s blanket dismissal of forecasting. Gordon does not deny the existence of Black Swan events. And his book is a giant compendium of all of the things that usually go wrong with predictions. Moreover, Gordon offers a sceptical discussion of the subject that chastises simple-minded futurists, tech enthusiasts, and various other prophets of doom and boom. The difference between Taleb and Gordon is that Gordon doesn’t dismiss out-of-hand the usefulness of structured thinking about the future. Many important decisions require us to speculate about what the future might hold. Gordon wants us to be savvy in the way we anticipate the future instead of flying by the seats of our pants, so to speak.

“To set the stage, Gordon talks about how the forecasting industry is rife with problems. There are no standards, no accepted methods, no standard terminology. There are no penalties for failure given that people tend to forget forecasts by the time they can be proven wrong. And when dealing with the forecasts offered by pundits, stakeholders, and activists, Gordon reminds us, “we are knee deep in predictive wishful thinking, scare-mongering, or blatant self-promotion.” (p. 5) Buyer beware.

“Then there are the data problems. Forecasters use data from the past to project trends into the future. They rely heavily on data gathered for other purposes, not gathered for the task at hand. Availability is patchy. The data comes from multiple sources and is created using different methods. Important statistical caveats get lost. The context of the original studies gets forgotten. Variables are often defined loosely … and change over time … and are measured differently in different places. Data gathering methods often change over time in ways that exaggerate or obscure a trend. Sensationalist “newsy” data often commands the most attention. Some things are inherently difficult or impossible to measure accurately. All sorts of assumptions get embedded in data projected into the future. Furthermore, Gordon talks about the ways in which numbers can be finessed in an underhanded way. He advocates “number scepticism”, warning: “But no matter how scientific the data appears, choices have been exercised at every point about what to observe, what to count, how to measure it, and how to report it. … But numbers are not bedrock. There is no bedrock.” (p. 59)

“As an aside, statisticians have a snide nickname for analysts who mix’n’match statistics from a hodgepodge of sources to create complicated models or story-lines. That nickname is junk-yard dog. Gordon gives the impression that the forecasting business is, by necessity, heavily populated with these collectors.

“The sources of potential error don’t end with data. Our biases cause us to misinterpret and misreport the data.

“Some bias is intentional manipulation. Rascally analysts ignore or downplay countervailing evidence. They give evidence less scrutiny if it confirms the desired result. Emotionally charged language and associations are used. Terms are defined in leading ways. Extreme cases are used to represent the norm. Forecasts that don’t accord with an agenda get ignored, especially if the forecast is sponsored by a powerful interest. Organisational incentives can cause those being scrutinised to fudge the numbers. When forecasts are presented to the media, the most extreme trends get attention and important caveats remain unreported. Gordon is particularly critical of the so-called futurists who use “stretch thinking” and “big-picture thinking” to imagine a world full of only big changes. Many have a technophile bias, or the assumption that technology is the sole motive-force of large-scale societal change. Gordon’s advice is to keep your guard up and be wary of motives.

“Setting aside the thinness of this advice, Gordon has a strange attitude when talking about manipulation. He makes a distinction between forecasts that attempt to be accurate and forecasts that attempt to influence. Employee-prodding managers, partisan policy wonks, and alarmist activists use loaded forecasts to move minds. Humility, qualification, and tentativeness don’t have a place in these circles. There may be a legitimate reason for using leading forecasts, such as communicating the art-of-the-possible or giving someone an ambitious target to strive for. However, leading forecasts without full disclosure are instruments of underhanded manipulation. Gordon is eerily agnostic. His advice and tone of voice suggests that he is oblivious to the ethical problems posed by the manipulative use of forecasts. It’s a strange contrast with Gordon’s advice about being careful and pragmatically sceptical. [Editor's note: Agnostic? Moi? Hardly, but perhaps the chill of my irony was not chilly enough.]

“Back to the sources of error.

“Gordon itemises a number of cognitive biases that are inherent to the way we think. We often miss Black Swan events and abrupt changes in prevailing wisdom (“paradigm shifts”), he argues, because we are always filtering information based on perceived relevance. This “inattentional blindness” causes us to not notice important influences on the future. We also overemphasize recent happenings over older events (the recency effect). We’re susceptible to herd thinking and faddish ideas. A few chance events are often mistakenly interpreted as a trend or other pattern. Gordon places particular emphasis on how our current context frames the way we see and think (situational bias), especially how the prevailing mindset and preoccupations of an era skew the way we think about the future (Zeitgeist bias). For example, nuclear-powered airplanes may have seemed inevitable to someone living in the 1950s, a time preoccupied with thoughts of nuclear technology, suggests Gordon. That notion seems absurd today. To counter this problem, he argues for the need to extract the assumptions underpinning our expectations. Those assumptions need to be questioned and tested. And one good test is to reverse the assumption; that is, consider how the future would be different if the opposite (or very different) assumption were used.

I would add that people habitually rely on lazy assumptions about the future in general. As Howard Segal points out in his book Technological Utopianism in American Culture (2005), late-19th and early-20th-Century intellectuals assumed a technological plateau when describing the future. Even today, we assume our arrival at some destination—a future steady state—instead of a world of on-going change that is unevenly distributed and erratically paced, as exists now.

Gordon invites us to consider the utility people derive from a particular technology before jumping to conclusions about how it will revolutionise everyone’s lives. Tech-happy futurists are too quick to assume broad public acceptance of a new technology while ignoring the trade-offs of adoption. There are costs to be considered. In many cases, the price is too high and existing technologies do a good enough job. Or old technologies have an inertia, such as when users are “locked in” to a particular technology. Or social values change. Or switching creates undue inconvenience and aggravation. Or the technology has uneven appeal across diverse groups in society. Or, or … Gordon reminds us that simple technological domino effects almost never happen. The pace of change is usually slower than anticipated. A variety of factors determine how successful an innovation will be.

That leads us to the dynamics of change. I’m not going to describe each dynamic in detail. Gordon devotes a lot of space to them. Instead, I’ve listed them iconographically in the following diagram. Note that the darker lines signify consequences (and consequences of consequences; a.k.a. second-order and third-order events).

“A trend observed today may not continue onward along a straight-forward path. Trends peter out … change course … hit limits … get caught in reinforcing loops … have side-effects … provoke reactions … et cetera. The same goes for underlying causes. Trends can be particularly difficult to track within the complex systems that govern our lives. Thus, Gordon offers a chapter on system analysis.

“As someone who studies organisations, I’m often seeing policies and strategies change with sadly predictable pendulum swings. Gung-ho leaders push in one direction with gusto only to get a lesson in humility. Their efforts hit limits and opposition. Their assumptions hit reality. Subsequent leaders see wreckage everywhere and push in the opposite direction, looking for balance. Balance alludes them and they go to far. Another pendulum swing begins. Some swings happen from season to season. Others happen over decades. These swings may be predictable, but their exact timing certainly isn’t.

“Gordon rounds out Future Savvy with a utilitarian survival-guide of sorts. His big advice is that “it’s better to be vaguely right than exactly wrong.” Success is being alert to important changes and being prepared to cope, not with having accurate predictions. Narrowing down the things that need to be prepared for is an important practical benefit. In that spirit, Gordon talks about the strengths and weaknesses of using multiple scenarios instead of pat forecasts. He steps the reader through the analysis of some forecasts while looking for weaknesses. A chapter-long battery of questions is offered to guide the analysis. These questions do a good job of summarising the book.

“All told, Future Savvy is an excellent textbook for those who want to discipline the way they think about the future. I disagree with Gordon’s tangents about the inherently subjective nature of truth. I also have a few qualms about his take on scepticism. But these tangents rarely get in the way of his stock-taking exercise. That exercise has led me to be even more suspicious of forecasting, especially forecasts in volatile industries where data is patchy and assumptions are legion. I’d love to know the success rate of high-tech cheer-leaders … er, research firms that peddle forecasting numbers. Gordon dismisses the tracking of forecast failures as “smirk lists”. I’m with Taleb and his tsk tsking. If these numbers are just part of the hype machine and have a dismal track-record, then what good are they? Validation for reckless investment strategies? Fodder for misleading Power­Point slides? Numbers that give a false sense of being in-touch with the market? Tsk tsk.

“That said, Future Savvy has increased my interest in foresight more generally. Gordon’s guide left me wondering how I can better prepare groups of decision-makers to think about the future. How do we get them to see the many changes afoot with greater foresight?”

‘We need to find a way to make futurism dull,’ says Mr Foresight

Posted by Adam Gordon on Mar 11, 2010 in all, Future Savvy, policy, strategic foresight, trend tracking

Paul Saffo is always good value, and doesn’t shy from polemic. In this talk at the Foresight Institute 2010 conference, Saffo, emeritus and alumnus of the the foresight industry for over 20 years has a full swipe at ‘futurists’ who participate in ‘future-entertainment’ or profess to ‘see into the future;’ but calls for the broad infusion of foresight into public debate, including the restitution of the Congressional Office of Technology Assessment (OTA) scrapped by Newt Gingrich in 1995.
.

“Profiles of the Future” at the Foresight Institute 2010 conference

Says Saffo: “Futurists today are talking to the wrong people, don’t have good methods (for the most part,) and are still doing the kinds of silly things we did (or they did) when futurism got started…We should have an instant prohibition on anyone who writes an article titled: ‘Top 10 Trends to Watch’… We’ve got to get rid of this ‘future entertainment’ stuff and ‘top-10 trends’ stuff, and get serious.”

Part of the raison d’etre of Future Savvy, of course, is to demythologize exactly this kind of self-promoting infotainment foresight, and give real-world managers a way to see through it. Thinking long-term is too important to allow it to be tainted by snake-oil salesmen. Saffo admits he’s ranting on this topic (as I do too.) In a less ranting mode, he would probably admit there are also many high-quality thinkers doing exemplary foresight work. Certainly he’s all in favor of thinking long-term, and doing it better.

Saffo’s solution? “Move foresight to the masses; make policy conversations cool; engage powerful myopics (short-term thinkers on Wall Street and other financial institutions); engage politicians (incl. via the OTA). But he doesn’t say how, and of course therein lies the rub.

.

The turkey problem in trend work: is your prediction robust to Thanksgiving?

Posted by Adam Gordon on Nov 24, 2009 in all, failed predictions, managing uncertainty, Perils of Prediction, risk management, scenario planning, strategic foresight, trend tracking, wildcards & black swans

We owe a debt to Nassim Taleb for memorably encapsulating the demerits of predicting by extrapolating trends as “The Turkey Problem,” and now seems the moment to reiterate it:

Imagine you are a turkey. Every day someone comes to feed you. Every day you get bigger. Your portion sizes get bigger too, brought by a nice man at regular intervals. You extrapolate the trend and you confidently predict a bigger you, with more to eat. Regularly too.

But what happens is … Thanksgiving. Or Christmas

Taleb, N., The Fourth Quadrant: a Map of the Limits of Statistics, Edge Foundation, September 2008

The hard reality for those who predict the future by extrapolating trends (and those gullible enough to believe them) is that even if our turkey had excellent data points (carefully observed and accurately recorded in, for example, a time series analysis) and, moreover, even if our turkey was a mathematically sophisticated — not merely simply projecting trends, but applying all the latest modeling techniques, from moving averages to compound regression — he is still going to be wrong about the future. Dead wrong.

All the data analysis in the world, all the fancy computer software, all the consulting time paid for, and he is still a dead duck.

Ouch. The lesson: there may be (or, vexingly, may not be) something outside the trend, a framing condition, which where it does exist is invisible within the trend projector’s mental model. The only way to get a view of the future that is “robust to Thanksgiving” is (a) to question assumed framing conditions, for example through properly done scenarios, and (b) to hold a view of the future which assumes fundamental ‘game-changing’ surprises can and will occur.

If, as they say, “the trend is your friend” it is assuredly only your fair-weather friend.