Join our community of smart investors
Opinion

How wrong are forecasters?

How wrong are forecasters?
December 21, 2009
How wrong are forecasters?

My table gives us a clue. It compares actual GDP growth to the consensus forecast made in the previous December; that is, forecasts in, say, December 1998 for growth in 1999. You can see that the average error is large. Ignoring the sign, GDP growth has on average been a full percentage point away from the forecast.

The consensus' forecasting record
December forecasts for next year's GDP growth 
Made in:ForecastOut-turnError
19980.83.52.7
19992.93.91.0
20002.62.5-0.1
20011.82.10.3
20022.32.80.5
20032.63.00.4
20042.52.2-0.3
20052.12.90.8
20062.42.60.2
20071.90.6-1.3
2008-1.5-4.5(e)-3.0
 Average error=1.0
Source: H.M. Treasury & ONS

If we interpret this average error as a standard error, we can say that there's a two-thirds chance that growth next year will be within a percentage point of forecast - that is, between 0.3 and 2.3 per cent.

This interpretation, though, is too hasty. Forecast errors are not evenly spread. Mostly, they are small, except for two years - 1998 and 2008 - when economists got things badly wrong. If we exclude those two years, the average error is just 0.5 percentage points.

Now, in one sense, we shouldn't exclude those two.

But there is a reason why we might. Those two errors came in response to financial crises - the Russian default and the collapse of LTCM in 1998 and the banking crisis of 2008. (Intriguingly, the errors were of opposite size; economists overestimated the damage the 1998 crisis would do, but underestimated the damage the 2008 crisis would do.)

This suggests that forecasts go especially wrong when economists are faced with the question: how will the economy cope with a big shock?

There's a mathematical fact that - partially - explains why this should be so. GDP growth in any year depends quite a lot upon growth in the final quarter of the previous year; if the forth quarter (Q4) is strong, it gives a higher platform from which the economy can grow the following year. And in December, economists don't yet know how the economy performed in Q4. This means that when there is big uncertainty about Q4 - as there was in the aftermath of the crises of 2008 and 1998 - there will be big errors in GDP growth the following year.

Which brings us to the outlook for 2010. Because there hasn't been any big shock in recent weeks, we might be able to discount, at least partially, those big errors made in 1998 and 2008. This narrows down the margin of uncertainty.

But it still leaves it quite wide. I would regard GDP growth of anywhere between around 0.8 per cent to around 1.8 per cent as being no great surprise.

The chances are, though, that the bigger surprises will be elsewhere.