When diversification fails

Your UK equity holdings are probably not as diversified as you think, because shares rise and fall together to a greater extent than is generally realised.

To see this, let’s consider the correlations of monthly returns between the main 25 FTSE sectors since 2003. Every single one of these 300 pairwise correlations is positive. This means that if you take any two sectors, they are more likely than not to rise and fall together in the same month. In fact, the average of these 300 correlations is 0.43, implying a high likelihood of any two sectors moving in the same direction; my table provides a sample of these correlations.

Even shares in apparently very different industries tend strongly to rise and fall together. For example, the correlation between banks and engineering is 0.57; that between construction and media stocks is 0.65; and that between IT and chemicals is 0.58.

Correlations are especially high among and between financials and cyclical stocks. That between banks and life insurers for example is 0.78, as is that between chemicals and engineers, suggesting that these pairs move almost in lockstep with each other from month to month.

By comparison, correlations tend to be lower between defensive sectors such as tobacco or pharmaceuticals and others. Even these, however, are positive, implying that these sectors have a greater than 50-50 chance of falling if (say) banks or oil stocks fall.

Correlations, however, change: there’s nothing immutable about them. For one thing, they tend to be higher in bad times. If the All-share falls only one per cent there’s a high chance that some sectors will move in opposite directions. If it falls 10 per cent however almost all will fall.

It’s for this reason that correlations between sectors have generally been higher in the volatile times of last three years than they were in the previous three; the average correlation has been 0.45 compared with 0.32 in 2016-2018.

In particular, the recession caused by the pandemic has caused huge co-movement between cyclical stocks. In the last three years construction stocks have had a correlation coefficient of 0.77 with transport and 0.82 with retailers and support services.

Such heightened cyclical risk (both downside and upside) has also caused previously lightly correlated sectors to move together – such as retailers and travel stocks or transport and general finance. This has happened because a factor that causes co-movement between stocks (cyclicality) has increased in importance relative to things that might drive shares in opposite directions such as perceptions of management quality or valuations.

You might think that if correlations can rise they can also fall. True – and this is especially likely if the market remains reasonably stable. Which brings us to a paradox. Equity diversification is most likely to work in quiet markets – but that is when we least need it. On the other hand, when we most need diversification – when prices are tumbling – it works less well. The times we’ve most needed diversification in recement years have been September-October 2008 and February-March 2020 when equities fell sharply around the world. But on these occasions all the main FTSE sectors fell sharply. Diversification across equities thus failed. Time diversification – simply holding on in the hope of better returns later – worked better.

It’s easy to think that if we are investing in different businesses then we are diversifying well. We’re not. In bad times almost all stocks tend to fall simply because all are sensitive to market risk and fears of recession. In such circumstances the best that equity diversification can do is simply lose you slightly less money. Which isn’t much of an achievement.

Now, there is a massive caveat to all this. Although sectors rise and fall together in the short term this is not the case in the long. In the past 10 years, for example, some sectors such as IT or beverages have tripled in price while others such as oil, banks and tobacco have fallen. This tells us that diversification across stocks works better in the long-run than in the short.

Whether you have the mindset that allows you to see through months in which all your holdings do badly and focus only on the long term is, however, another matter. If you want to diversify short-term risks you must hold non-equity assets such as bonds, gold, foreign currency and of course cash. Yes, these offer lower likely returns than equities – but insurance comes at a price.