Using smart beta

My table gives a flavour of what I mean. It shows the sensitivities of annual changes in some sectors to one percentage point changes in various risk factors since 1998. It shows, for example, that the oil and gas sector (on average) rises 0.76 per cent for each 1 per cent rise in the All-Share index; rises 0.17 per cent for each one percentage point that Aim outperforms the All-Share index; but falls 0.35 per cent for each percentage point that small caps outperform the All-Share. These effects control for the other factor in my table, and all the effects are statistically significant; the blank spaces represent insignificant responses.

Of course, there are many possible risk factors. My list isn't complete, but it captures some big risks. The obvious one is simple market risk - a fall in the All-Share index - but there are others:

• Inflation risk, as measured by the difference between five-year conventional and index-linked gilt yields. Expectations of higher inflation are associated with miners and retailers doing well, but utilities and insurers doing badly.

• Sentiment risk, as measured by the relative performance of Aim stocks. Improved sentiment is good for IT and miners, which have a speculative element, but it is bad for defensives such as utilities and tobacco.

• Small-cap risk, measured by the FTSE small-cap index relative to the All-Share. General retailers tend to do well when small caps do, but miners and oil do badly.

• Value risk, as measured by the performance of the FTSE 350 high-yield sector relative to the low-yield index. Construction stocks and banks do well when value does, but IT does badly.

• Commodity price risk, as measured by the S&P/GSCI. As you might guess, this is good for miners but bad for retailers.

• Overseas risk. Tobacco and construction stocks tend to do well if the MSCI world market (in dollar terms) outperforms the UK while miners and pharmaceuticals do badly.

You might wonder why I've omitted recession risk. I haven't. It's captured by the other factors. For example, in recessions we are likely to see inflation expectations fall and small and value stocks do badly. Banks, construction and general retailers, being exposed to these factors, thus carry recession risk.

Although I've used only a few risk factors, they explain a lot of sector returns. On average across the 27 main FTSE sectors, they account for 71.9 per cent of the variation in annual returns since 1998; the proportion ranges from 43.2 per cent (food retailers) up to 89.7 per cent (support services).

This table explains why correlations between sectors vary over time. For example, if the market falls we'd expect all sectors to fall because all are significantly correlated with the All-Share index. This gives us positive correlations between them. However, moves in other risk factors can generate negative correlations. For example, a rise in inflation expectations (other things equal) would cause miners and insurers to move in opposite directions; a rise in small caps would cause tobacco and retailers to diverge; a rise in value stocks would cause construction and IT to move in opposite directions; and so on.

We can use all this in two different ways.

It could be an aid to stock picking. For example, if you think inflation expectations will rise - say, because the world economy grows more strongly than expected or because deflation fears are overdone - then you might favour miners or retailers at the expense of insurers or pharmaceuticals. Or if you envisage investor sentiment improving, you might prefer miners or IT to utilities.

But there's another possible use - and one I prefer. We can use this as a way of managing risk.

For example, if you're worried by the possibility of inflation - say because you have a flat-rate annuity - you might prefer shares that do well if inflation expectations or commodity prices rise because these help to insure you against higher inflation. And you might avoid those that do badly in such cases because they add to a risk which you are already taking. Similarly, if you are exposed to cyclical risk - say, because you own a small business that could suffer in recession - you might want to avoid especially cyclical shares as these would add to your risks. This would mean preferring shares that are negatively correlated with inflation expectations or smaller stocks such as telecoms or utilities.

Whatever use this process has, though, one thing is clear. A handful of systematic risk factors can explain a massive proportion of equity returns. The investor who picks stocks only by looking at company accounts is therefore ignoring a large chunk of what determines share price moves.