"For instance, EIA, by its own admission, states that they had overestimated crude oil production 62 per cent of the time; they had overestimated natural gas production 70.8 per cent of the time; and they had overestimated natural gas consumption 69.6 per cent of the time. Not the best track record by anyone's estimation, except perhaps EIA's."

The EIA also overestimated the energy intensity ratio (a measure of total energy consumption and GDP) "a whopping 96.5 per cent of the time."

Concluded Rogers: "In short, EIA is not very good at forecasting. But what is even more interesting is that Dr. Montgomery, the lead author of this new study, was once in charge of models and forecasts at EIA."

Renewable mistakes

The experts have also failed to come to terms with modest gains made in renewable sources of energy. In 2009, Germany's Renewable Energy Agency evaluated 50 energy forecasts made over the last couple of decades. Almost all underestimated the growth of renewables in Europe. In fact, solar, wind and biogas (made from corn or trees) reached predicted values "several years earlier than planned, and surpassed these by several hundred per cent."

Nor did anyone predict that Germany's success with renewable forms of energy would pose great environmental challenges.

German's top magazine, Der Spiegel, recently noted that the country's largest solar park sprung up in a bird sanctuary and that industry now wants to plant huge windmills in protected conservation areas.

Meanwhile, more than half of the trees felled in Germany find their way to the wood pellets and biofuel industry, threatening the status of the nation's ancient beech trees. Industrial corn monocultures have also industrialized farming communities in order to provide feedstock for the biogas industry.

In many respects failed energy forecasts remain an enduring long-term trend. In the 1970s, one well-known European prognosticator, Cesare Marchetti, declared that new energy sources would replace one another in a well-ordered transition "as though the system had a schedule, will and a clock."

Just as coal replaced wood, nuclear power, solar and natural gas would eventually displace oil, reasoned Marchetti. But 40 years later Marchetti's model has shown no real schedule.

Instead of nuclear and renewables replacing oil, the world has experienced a stable (although slightly declining) consumption of petroleum as well as steady consumption of coal and natural gas in fairly equal proportions.

Rules for those who would forecast

What Marchetti proved, however, was just how off base long range forecasts could be.

Nearly 30 years ago, the nuclear physicist Alvin Weinberg noted that the failure of energy forecasting taught two lessons: "We must try to formulate energy policies that finesse uncertainties and that are resilient to surprises."

But Tetlock's research combined with the sorry record of energy forecasters offers a few disquieting truths worth pondering.

1. First and foremost, regard all energy forecasts, whether boom or gloom, with radical skepticism. Recognize that many are paid for by dominant energy players in a world where fewer and fewer corporations now control energy flows the way 18th century slave traders once did.

2. Search out humble foxes. They draw their information from a diversity of sources and are accountable for their actions. They tend to be geologists and physicists and not economists.

3. Accept that our energy future will not only be uncertain but largely unpredictable. Unanticipated surprises will shock a highly complex system into unforeseen directions.

At the end of Future Babble, Dan Gardner's lively examination of failed expert predictions, the Ottawa-based journalist offers some sound advice from the late essayist Alistair Cooke.

"In the best of times our days are numbered anyway. And so it would be a crime against nature for any generation to take the world's crisis so solemnly that it put off enjoying those things for which we were designed in the first place. The opportunity to do good work, to fall in love, to enjoy friends, to hit a ball, and to bounce a baby."

And that may be the only reliable energy forecast available.

Next week, the Shale Gale: A bridge or a retirement party?