Posted by Adam Gordon on Oct 29, 2009 in all, economy & finance, failed predictions, forecast filtering, foresight tools & methods, managing uncertainty, Perils of Prediction, risk management, strategic foresight
Continuing the theme of financial types talking to each other about predictions and predictability, this ‘Tea with the Economist’ interview of Stephen Roach, Chairman, Morgan Stanley Asia by Economist New York Bureau Chief Mathew Birk, carries interesting lessons about the limits of prediction.
Birk commends Roach for being one of the few to have predicted the Credit Crunch problems, to which Roach demurs in saying he was “too early”. He then furthers his modesty in saying that the “breakage” in the financial system was “in excess of anything I envisioned.”
Self-deprecation in assessing one’s predictive abilities will endear anyone to me. Even Roach, who later in the interview burns this hard-won credibility by laying the blame for the credit crunch at the door of regulators, forgetting how hard financial institutions lobbied regulators for greater freedoms in the 1990s.
But I digress. The predictive issues the interview raises are as follows. Issue one: it’s not enough (as any stock short-seller will confirm) to get the direction of a future change right. One must get the timing right too. Issue two: it’s not enough to anticipate a change. One must be able to judge it’s impact. Getting either timing or impact wrong is effectively to have missed the future.
.
Probability
On the latter topic — the problem of impact — Nassim Taleb is unrelenting, and he is right. Analysts routinely mix up probability and impact. They think that because an event has a low probability (‘it would be a 10-sigma event!’) it can be marginalized in the predictive number crunching. Of course, it can’t. The low-probability of a wildcard or black swan event is irrelevant because when it happens it will change the game, and that’s why, in every predictive situation of reasonable complexity and uncertainty, using statistical extrapolations (regressions and so on) to predict, is to dangerously paper over the cracks. It is precisely the cracks that businesses and policy makers need to worry about.
Determining the direction of change is hard enough. Assessing timing or extent of impact — a ‘total future impact index’ — is wickedly difficult. It’s a task not to be underestimated, and to simply extrapolate current trends (= assuming the trend’s timeline and impact stay the same as in the past) is the royal road to underestimating it.
This is the reason foresight for complex, uncertain, changing situations can only be grasped by NOT predicting (quantitatively or otherwise) but by exploring the limit-conditions of the plausible (What would happen if the timing of the change accelerated, or was significantly delayed? What if the impact was 10x or one tenth of what we expect? And so on.)
.
read morePosted by Adam Gordon on Jun 24, 2009 in all, decision-making, foresight tools & methods, Future Savvy, management, managing uncertainty, risk management, strategic foresight
Peter Bernstein, the author of “Against the Gods: The Remarkable Story of Risk,” died recently at the age of 90. In memoriam McKinsey Quarterly reposted this recent Bernstein interview. I put it up here because it’s a timely and timeless lesson in thinking about uncertainty and threats, and avoiding simplistic (quantitative) approaches to managing them – one of core themes of “Future Savvy.” Bernstein offers and endorsement of real options and explains why sophisticated Long Term Capital Management (LTCM) mathematical models to control risk created “a math dependency” that was blind to, among other things, unexpected systemic feedback to its own emergence:
One of the first things Bernstein says is that risk implies that we don’t know what will happen, which could be good things happening too. Risk management, as it is currently understood, gets executives to look at what could go wrong in the uncertain future of the enterprise. (Somehow threats are easier than opportunties to get departmental budget for.) The standard approach is to break risks down into commonly understood threat categories: a typical analysis would illuminated risks posed by technology failure, communications failure, security failure, natural disasters, accidents, or market/reputation risk, liability risk, financial/credit risk, and so on. This negative-outcome identification is typically followed by strategies to monitor, minimize, or control the risk event or its impact.
Doing all this is great, BUT it is just a narrow part of enterprise and industry foresight. Why? First, industry foresight or futures studies for business is focused as much on the opportunities change offers as on threats. Second, foresight tools (when correctly applied) set themselves the task of enlarging perspectives or mental maps so that we can see more things, or more possibilities than the generally expected set (whether good or bad). Set against this, risk management is little more than the catalog of known threats. The unknown or poorly understood threat, or unseen opportunity missed (and grabbed by others) is likely to be more damaging to the enterprise.
read morePosted by Adam Gordon on Feb 18, 2009 in all, foresight tools & methods, horizon scanning, managing uncertainty, prediction markets
It’s the week of the 81st Academy Awards and this means my automated Internet searches for future predictions are bunged up with blogger & media pundits predicting whether it’s going to be Brad Pitt or Sean Penn; Kate Winslet over Angelina Jolie; Slumdog Millionaire or The Reader, etc. This is just the fun-of-the-fair forecasting of course. But, turns out there are some significant things to talk about from a Future Savvy point of view.
First, there is the prediction game on offer from ABC, taglined: “The Oscars Live Challenge: Think you can Predict a Winner? Make Your Picks Now!”
It’s all part of the marketing drive of course, but, nevertheless how would one play it best and what might that tell us? Let’s assume there is something at stake, like you’re really going to sit in front of the TV and mark off your right vs. wrong predictions, and compare your score with that of your spouse for year-long bragging rights – now there’s pressure – how would you predict? Would you think (a) “this is the best movie so I predict it will win”? Hardly. You would think: (b) “this is the one that I think most people will pick, so that’s the one I think will win.”
You would be making a meta-prediction – going with what you think most are going to choose. In this particular case you would also know that that Oscar winners are chosen by balloting the 6,000 members of the Academy of Motion Picture Arts and Sciences. So your more exact question would be: who is this special group likely to choose in each category?
What’s going on? In future situations that are heavily dependent on aggregate human choices – which is very many situations – the savviest predicting strategy is to figure out the choices most people are going to make. Oscars aside, figuring out the choices most people will make on any issue – hybrid cars, tighter securities legislation, public health care, etc. – is an excellent guide to what will really happen. It’s a mass market-led view of the future to be sure, but that’s exactly what makes it dependable in mass-opinion situations. (Not all situations are determined by mass-market choices – predicting a presidential election winner is; predicting a superbowl winner is not.)
Playing the game
I had a shot at the Oscar prediction game, joining the alleged 1,680 other “players” who were then online. From what I could tell via the rather gristly Flash interface is that the game is not (yet) “social” in that you can’t see what other people are predicting – there is no access to aggregate opinion. No matter. One can instantly get this in hundreds of prediction market forums right now, for example Intrade, where the price of each outcome in each Academy Awards category directly reflects how strongly players as a whole have bid up that outcome.
At Intrade, at time of writing, Slumdog Millionaire is at $87.30 (max is $100; the other 4 movies share the remaining $12.70). When used as a prediction this means that the aggregate opinion of people staking real money has been effectively captured: it is that Slumdog Millionaire is 87% likely to be the choice of the Academy members in its category.
This is a guide to Oscar night that I would not bet against if I wanted to hold onto my bragging rights. Even in situations less overwhelmingly agreed on by players, it has been shown that prediction markets, tapping the aggregate “wisdom of crowds” (working like “Ask the audience” on Who Wants to be a Millionaire) are a fabulous tool for capturing what most people think will happen, resulting in excellent predictions. Caveat Emptor: prediction markets are poor at predicting long-term, open-ended situations, particularly where the outcome alternatives are unknown or can’t be clearly bounded, as blogged a few months back.
Posted by Adam Gordon on Feb 12, 2009 in all, design, emerging technologies, innovation, lifestyles & values, managing uncertainty, systems dynamics, technology change
It’s an auspicious time for those of us long convinced that design and future studies are fields with significant overlap whose coordination is helpful in addressing both social and commercial problems and/or future opportunities.
Conference: March 19-21
Tim Brown of IDEO, the the industrial design firm, recently published a Harvard Business Review piece Design Thinking – investigating designer-methods in business innovation. At Davos last month there was a “Global Agenda Council/ Design,” featuring Newsweek’s Bruce Nussbaum and built-environment design firm ARUP’s head of foresight, Chris Leubkeman. (The general agenda may be found here.) Next month, the Association of Professional Futurists are having a “Futures by Design” conference in association with The Art Center College of Design in Pasadena, CA.
And so on. I’m going to be blogging more about this. But for now I wanted to put out a note-to-self I wrote on the issue about five years ago, trying to briefly define how the fields relate to each other, and what the crossover is. Here goes:
The tools of design and planning dovetail closely with those of industry foresight. The overlap and interaction between these two disciplines is not commonly understood, and so the methods and process insights from design professions that could augment the range of strategic foresight tools is often ignored.
1. Beyond aesthetics
Sunday supplements and glossy magazines often use “design” to mean style and fashion. While aesthetics is important, good design means much more than how products appear. It is about creating better processes, interactions and solutions for human benefit. This often involves experimenting with new technologies, envisaging possibilities under conditions of uncertainty and complexity, exploring and comparing alternatives, and determining the best and most durable solution for the long term.
2. Future focus
Whether planning a building, or redesigning a product, or innovating a process, the designer is called on to anticipate a solution that caters to future needs often responding to futures issues, for example environmental-sustainability pressures and changing social values. In other words, design methods, like futures tools in general, form the bridge between current products, systems and practices and what it will be required and desired in the future.
In achieving this future focus, designers, like good “futurists,” must use techniques of imagination, creativity and intuition to generate and evaluate future outcomes. Like futures professionals, designers are called on to practice original thinking, imagine the world differently and see possibilities that others don’t. They are required to take risks, negotiate change and challenge the status quo under conditions of ambiguity and uncertainty. And like good foresight work, design succeeds only if it finds the right tradeoffs between technology possibilities, economic realities, and social needs.
3. Rendering
More than merely anticipating the future, designers and planners are practical agents of visual imagination, creating the blueprints for the objects and experiences of tomorrow. From product creation to urban renewal, designers and planners have tools and experience translating abstract future concepts and ideals into visible or tangible form - “making the invisible visible.” Through this rendering function they are primary agents in articulating the future, and therefore in helping us see and negotiate (or refuse) the transition.
4. Systemic innovation
Design is about systems and practices as much as products: better-designed systems improve utility, cut costs, and improve resource use. Designers play a key role in the organizational innovation process as a whole, including the development of integrated product and services, or inventing new types of value chains, alliances, and collaborations.
In sum, much of what foresight professionals are trying to do every day is already being done by design professions. Their methods and process insights should be integrated into the foresight field as a whole.
read morePosted by Adam Gordon on Oct 22, 2008 in all, failed predictions, forecast filtering, horizon scanning, lifestyles & values, social change
They say a definite cure for romantic notions about any previous era of human existence is to think about the dentistry. That fixes any nostalgia. However it’s safe to say that no one will be nostalgic for all prior eras of working with data which was – when findable (pre-search engines) – a matter of scouring through tables of figures in heavy books.
No longer. The paradigm was broken by the Hans Rosling (Gapminder) video “Debunking Myths About the Third-World,” 2006. By Rosling’s own admission, his analysis is not based on new or better data. The (UN) data has always been there (yes now it’s becoming more available). But the seachange is new software which makes it easy to filter and present it in dynamic, graphic form. And, no surprise, this is popular. According to Gapminder, this video has seen by 500,000 people, not bad for a 20-minute treatise on perceptions of developing world countries.
Data turned into dynamic moving pictures is, one might say, required in our era (trends: visual literacy, short attention span, computing power) so thankfully we can expect more of this. What’s important, for forecast evaluation purposes, is the power of explanation and mental-model challenge that the improved communication provides. As Rosling says of his Swedish graduate students: “Their problem was not lack of data, it was preconceived ideas” (an outdated world view of “1st world” vs “3rd world.”) An endless amount of poring over dusty tables of figures would be unlikely to change that. But it’s hard to watch Rosling’s moving bubbles and not have one’s paradigm shaken.
Another site, in a similar vein, is worldmapper, a University of Sheffield initiative. Worldmapper communicates hundreds of world indicators, from infant mortality to military spending and so on, by manipulating the size of territory of each country to indicate presence or absence of the variable in question, as the following maps show:
Prisoners as percentage of population
Girls not at secondary school
Strikes and lockouts, 2002
Again it is basically UN data that is being sourced, but now presented in a way that cuts through the obscurity tells and the story much more vividly. As we know, humans “get it” better and faster via images than via words or figures. It challenges our perceptions in a way that figures in dusty tables cannot. They payoff is it’s harder to miss what’s really going on. So we have a better view of the world: our mental model aka ‘paradigm” more closely approximates reality. That means we will make better assumptions going forward which will, on balance (no guarantees of course), convert into better predictions.
read morePosted by Adam Gordon on Jul 18, 2008 in 2015, 2025, all, foresight tools & methods, horizon scanning, strategic foresight, trend tracking
More on the Media Futures Conference – having yesterday got sidetracked into pushing back at misconceptions about citizen journalism (based on lousy forecast filtering) – now I’m actually getting to what I intended to talk about…
Early in the day, as a warmup I think (but for me this was the juice) there was a “Research in the Real World” section. It started with a presentation by Alex McKie reporting on a tour she made across the UK, where she interviewed people asking them what their “three wishes for the future” were. This was followed by Gill Wildman and Nick Durrant of Plot, who presented interviews where consumers were asked where and how they used media, and what they wanted from it.
The research is anthropological, no more or less than a customized field trip: going out, seeing what people do, and how they live, and what’s important and meaningful to them – and then thinking how one’s own area of interest (e.g. product) fits into this, or could fit into it in the future. That gives some clues as to what people will adopt and/or buy – what the market will “pull”.
The futures field lingo for this type of work is “a learning journey,” a process usually omitted in the helter-skelter of tracking new technology capabilities and other apparently more profitable lines of research. There are some good writeups of future learning journeys: one that comes to mind is “The Moen Story” Johnston, R. & Douglas Bate, J., The Power of Strategy Innovation, Amacom Press, 2003, Chapter 5. Another is “Conduct Reconnaissance into the Future,” Sull, D & Wang Y, Made in China, Chapter 3, HBS Press, 2005. I recently saw that Christus CEO Tom Royer said his medical institution had conducted learning journeys (to Canada and India) as part of its Futures Task Force II scenario building process.
Tuning in
No question this type of research is often tedious. You have everyday people umming and aahing inarticulately and often unimaginatively about their preferences and problems, and hopes for the future. In fact the conference audience were impatient about having been presented with the interviews in raw form. But it is precisely in the careful listening that much about the real future is revealed. It is a vital ingredient in thinking about the future, and reigning in poor forecasts.
In the event, the consumers (in Plot’s terms “the people formerly known as … users”) were revealed as media wise, but often their savvy to screen out the information firehose. Although media types were thinking about the cutting edge, real people were articulating the need to be informed in a way they could manage – not too much or too little – and to be able to trust the news source, and be exposed to stories that move or inspire them.
Learning journeys are a very dependable way to think about the future by checking our industry insider preferences against the preferences of real people out there. Any prediction that makes assumptions about the market without this perspective is heading for failure. But there is a wrinkle, and it is this: market research – even this deep market field trip research which is much better than focus groups – is seldom enough to adequately anticipate the next new thing. It tells us what lab fantasies or executive business model fantasies will not fly. But it doesn’t help us make the jump either. Experience is that consumers want what they already have, maybe a bit better, maybe a bit cheaper. Market research did not see the Walkman. And as as Hal Sperling of Chrysler said: “In all the time we spent developing the Minivan, not once did we have a soccer mom come and ask us for one.”
read morePosted by Adam Gordon on Jul 9, 2008 in 2015, 2025, all, decision-making, failed predictions, forecast filtering, foresight tools & methods, Future Savvy, horizon scanning, leadership, managing uncertainty, Perils of Prediction, scenario planning, strategic foresight, strategic planning, systems dynamics, technology change, trend tracking
The book Future Savvy shows readers how to critically judge forecasts for themselves. These are the chapters that take the reader there:
Chapter 1: Recognizing Forecast Intentions, deals with considerations of how forecasts come about, who makes them, and with what intention. Those who research and produce forecasts, those who invest in understanding trends and drivers of change, and those (including the media) who bring the forecasts and their implications to our attention, inevitably have reasons for doing so – to benefit from the knowledge by seizing opportunities or avoiding threats or by affecting outcomes in the world. Understanding a forecast’s “return on investment” gives us an important vantage point in assessing the merits of a forecast.
Chapter 2: The Quality of Information, shows how a forecast communicates information between forecaster and reader subject to the same standards of accuracy, truth-telling, and bias-control by which one would judge any communication. Forecasts can be very different in methods and goals, but all forecasts lay claim to factual truth, particularly truth in the data, and the argument deals with the various ways in which data can be less solid than it looks, even with the best intentions.
Chapter 3: Interpretation and Bias, considers how data – whether good or bad in itself – can be interpreted or misinterpreted in forecasting, that is, the “political” aspects of forecasting. Just as there is no value-free look at history, so too there is no value-free look to the future and asking the right questions allows us be ready to mentally rebalance forecasts that are presented.
Chapter 4: Paradigms and Perception, investigates how predictive statements are exposed to a broader form of interpretive bias that has to do with the forecaster’s mental model or “paradigm,” and the “zeitgeist” (spirit of the times) when the forecast is made. This chapter investigates situations where forecast failure is caused by failure to escape society’s current mental models – which often do not hold through the forecast period.
Chapter 5: The Utility Principle, considers economic and market forces, and the role of consumers, in promoting or resisting the future. Without reigning in creative thinking, some simple economic filters inevitably apply direction or timing realism to futurist flights of fancy.
Chapter 6: Drivers, Blockers, and Trends, consider drivers and blockers of change, and how viewing these dynamics improves forecast assessment. It identifies the roles of Drivers, Enablers, Friction, and Blockers acting on events to cause change or resist it, and problems in dumbly projecting current trends.
Chapter 7: The Limits of Quantitative Analysis, discusses the role of statistical analysis and quantitative modeling in predicting the future – where this is possible and useful and where it is not, and why not.
Chapter 8: The Systems Perspective, investigates “system effects,” which occur whenever different elements or variables that may appear isolated are in fact linked together, such that changes in one element cause changes in others. Anticipating future behavior of any variable hinges on identifying the broader systemic elements influencing it and failing to do this is a big part of what causes forecasts to fail.
Chapter 9: Living with Alternative Futures, investigates non-predictive ways of approaching change – where the tone is more about managing uncertainty than predicting the future. It acknowledges unfathomable complexity of most future questions and provides perspectives that raise chances of success in an inherently unpredictable future.
Chapter 10: Forecast Filtering in Action, illustrates the processes of the book by applying them in case studies to real-world sample forecasts that decision makers in business and policy areas might find themselves interacting with. This demonstrates how real everyday predictive material may be probed and critically evaluated, following the principles developed in previous chapters.
Chapter 11: A Forecast Filtering Checklist, is a cross-cutting checklist which summarizes the principles of the book in one convenient, thematic list.
read more