Forecasters Needed.  No Skill Required

Wouldn't it be helpful if we knew when another big El Niño was about to hit so we could board up the windows, bring in the cat, and cancel our cable television contract?

Like general circulation modelers seeking to predict future climate, climatologists worldwide are working on complex predictive models that say when and where the next El Niño might occur. One small problem: When you make a forecast for an event that occurs within your lifetime, someone just might hold you accountable.

Along come Chris Landsea and John Knaff, who, in the Bulletin of the American Meteorological Society, decided to do just that. They compared the 1997–1998 forecasts produced by 12 different El Niño models, some dynamical (i.e., expensive) and some statistical (i.e., less expensive). Their approach? CLImatology and PERsistence (CLIPER), a model that uses past events (Climatology) and the recent history of the current event (Persistence) and is commonly used to evaluate tropical cyclone predictions.

In a sense, CLIPER is the simplest possible model. Given the data, it's the kind of model that an undergraduate statistical climatology class could develop between the end of the football game and the start of the fraternity mixer using your average home computer.

That's not a criticism. In fact, Landsea and Knaff argue that if these more sophisticated forecast models are to have any useful predictive ability (or "skill") they should, at a minimum, be better than CLIPER.

Figure 1 compares what actually happened ("verification") with the 6- to 8-month advance forecasts of a variety of models (with a few more simple statistical forecasts thrown into the mix). Some forecast verifications differ since some models make predictions for slightly different regions of the tropical Pacific.

Figure 1.  A comparison of 12 ENSO forecasts with observations reveals these models to be largely inaccurate.

How did they fare? In a word, terrible, but with varying degrees of terrible-ness. Note that CLIPER outperforms nearly all of the more "sophisticated" dynamical models. As the authors point out, national meteorological centers may do well to note that "the current best tools are the relatively cheap statistical systems," not the expensive but ineffective dynamical models that have yet to produce a reliable forecast."

Toward the paper's conclusion, the authors offer a remarkably candid (for science) perspective:

[It is] disturbing that others are using the supposed success in dynamical El Niño forecasting to support other agendas. As an example, an overview paper by Ledley et al. (1999) to support the American Geophysical Union's "Position Statement on Climate Change and Greenhouse Gases" said the following: "Confidence in [comprehensive coupled] models [for anthropogenic global warming scenarios] is also gained from their emerging predictive capability. An example of this capability is the development of a hierarchy of models to study the El Niño-Southern Oscillation (ENSO) phenomena....These models can predict the lower frequency responses of the climate system, such as anomalies in monthly and season averages of the sea surface temperatures in the tropical Pacific.

"On the contrary," the authors state, their own results suggest we should have "less confidence in anthropogenic global warming....The bottom line is that the successes in ENSO forecasting have been overstated (sometimes drastically) and misapplied in other arenas."

In the end, they report, "There were no models that provided both useful and skillful forecasts for the entirety of the 1997–98 El Niño. This is a conclusion that remains unclear to the general meteorological and oceanographic community."

References:

Landsea, C.W., and J.A. Knaff., 2000, How much skill was there in forecasting the very strong 1997-98 El Niño? Bulletin of the American Meteorological Society, 81, 2107–2119.

Ledley, T.S., et al., 1999, Climate change and greenhouse gases, EOS, Transactions of the American Geophysical Union, 80, 454–458.

Books: Satanic Praise

Kudos to WCR Chief Editor Patrick J. Michaels, Ph.D., and co-author Robert C. Balling Jr., Ph.D., a WCR contributing editor, on a recent review in Nature of their book, The Satanic Gases: Clearing the Air About Global Warming.

In the Sept. 28 review of the book, which was published May 1 by Cato Institute Press, Hans von Storch calls the book "well written and easy to read."

Furthermore, he writes, "the authors should be applauded for their bold predictions for the future, which will be proven right or wrong by the course of history."

It is that degree of boldness that unnerves the paradigm-followers Thomas Kuhn identified in his 1962 classic The Structure of Scientific Revolutions (see "The Way Science Works").

Watch these pages for frequent updates of just what course history is taking. What matters is not whether the authors are accurate to the last detail, but whether their self-described "mainstream skeptic" viewpoint is in fact right.

The climate models are, for the most part, neither.