Superforecasting

[Amazon Link]
(paid link)

Science fiction readers Of A Certain Age will remember Asimov's Foundation books fondly, which prominently featured Hari Seldon's "psychohistory", a subversive science that allowed the accurate prediction of the (dismal) future of the Galactic Empire. The brave effort to minimize the inevitable barbarism gave rise to many stories and novels.

This book says: we ain't there yet. We're unlikely to ever get there. But it is an excellent overview of the best current efforts to (at least) make predictions about the near future.

The book's primary author, Philip Tetlock, is a UPenn prof (in Psychology, Political Science, and the Wharton business school, impressive). The subject has been the primary focus of his research for most of his career. The secondary author, Dan Gardner, is a journalist, and probably punched up the prose and ironed out some of the academese. The result is excellent, very readable even for the layman. (As long as the layman doesn't seize up at an informal presentation of Bayes' Theorem.) It is full of insights, wittily presented.

Most popular "pundit" forecasting is sloppy: full of weaselly qualifications and vague time scales. (NYT columnist Thomas Friedman is used as an example.) Worse, pundits don't usually get called on their failed predictions. (Example here is from "our" side: Larry Kudlow, CNBC superstar, who was consistently, disastrously wrong about the 2007-2008 recession. Yet, he's still in the lucrative business of TV punditry.)

So it's easy to despair. Yet, Tetlock approached the issue as a rigorous science: let's ask for predictions precisely, with unambiguous language, and specific timescales. (Example: will Kim Jong-Un vacate his office by June 2015?) And ask for probabilities rather than yes/no: ("My forecast for Kim Jong-Un vacating by June 2015: 20%.")

Tetlock assembled a raft of volunteers who threw their brains into judging the likelihood of such outcomes. (Still ongoing. There's a website.) Result (although the book's title is kind of a spoiler): some forecasters did a lot better than others, even better than a dart-throwing chip would have. (High praise, as Tetlock shows.)

Then the interesting question becomes: What did the superforecasters have in common? A lot of things, as it turns out. Humility. Skill in breaking down problems into more-easily analyzed parts. Knowing where to get information is relatively easy; knowing what to get is vital. Math literacy helps, although few superf'ers applied math rigorously in making their predictions. A determined non-ideological approach is also a plus; if you "know" that the right answer is determined by your faith in capitalism/socialism/bureaucracy/democracy/etc. then you are likely to be way too confident in your guesses. And more.

So, highly recommended. If you don't believe me, and you shouldn't, the back cover has High Praise blurbs from some people you may have heard of: Daniel Kahneman, Steven Pinker, Robert Rubin, Tyler Cowen, and Jonathan Haidt.


Last Modified 2024-01-26 4:17 PM EDT