Join our community of smart investors

When irrationality works

Investors sometimes misjudge probabilities and so have portfolios that are more balanced than experts would recommend. But this is a good thing
February 27, 2020

One oddity of many investors’ portfolios is that they are split quite evenly across assets, equities or funds. In a classic paper in 1999, Shlomo Benartzi and Richard Thaler showed that many pension fund investors divided their money evenly across the funds available to them, something they called naïve diversification. We often see something similar in our readers’ portfolios. This behaviour conflicts with portfolio optimisation, which often recommends a big holding of a few low-volatility assets. And it also conflicts with the core-satellite approach, which calls for a big holding of tracker funds and lighter investments elsewhere.

Which poses the question: what explains such behaviour? One answer lies in a new paper by Thomas Graeber and Benjamin Enke at Harvard University. They distinguish between two types of uncertainty. One type is inherent in the fact that the world is an unpredictable place, and would be even if we were rational and fully informed. But there’s another type, which they call cognitive uncertainty. This is the fact that, sometimes, we don’t know our own minds. This could be because we are unsure of our attitudes to risk. Or it can be that we don’t trust our abilities to estimate probabilities, either because we don’t trust our own judgment or because we aren’t sure how reliable our information is.

And here’s the thing. People who feel lots of cognitive uncertainty do something they call probability compression. They treat small probabilities as if they were larger than they really are, and large ones as if they were smaller. We see this in sports betting: people bet too much on outsiders and not enough on favourites – treating low probabilities as if they were higher and high ones as if they were lower.

This fits with research by Baruch Fischhoff and Waendi Bruine De Bruin at Carnegie Mellon University. They asked students what they thought were the chances of rare events such as being burgled. And they found that people were much more likely to estimate these at around 50 per cent than in the ranges 30-39 or 60-69 per cent.

This could be related to what psychologists call the oblique effect. While we are good at spotting whether a picture on a wall is hanging straight or not, we are bad at estimating angles that are some way from 90 degrees. In the same spirit, we are comfortable with 100, zero or 50 per cent chances, but not so good at coping with, say, 20 per cent probabilities.

This leads naturally to naïve diversification. If uncertainty causes you to think that all outcomes are more or less equally likely, why not spread your investments evenly?

Which raises the question; is this an expensive mistake?

It certainly can be. For years, we’ve seen something like the favourite-longshot bias in stock markets. Speculative stocks, which have low probabilities of coming good, have generally been overpriced while safer stocks have been underpriced – which is why Aim stocks have badly underperformed the All-Share index since their inception.  

And people who, during the 20th century, overweighted the small probability of disaster avoided equities and so missed out on massive returns.

But, but, but. Naïve diversification might not be so costly after all. Victor DeMiguel at London Business School and colleagues have shown that it often works as well or better than fancy optimisation.

What’s going on here is an example of how irrationalities can cancel out. Recovering alcoholics sometimes say that their next drink will kill them. This overrates its danger. But it has the healthy effect of keeping them sober. And a gambler at the roulette table might think 'red is on a roll' and then think 'black must be due soon', thereby arriving at a correct estimate of the odds via two errors.

Similarly, naïve diversification can protect us from mistakes. One is that sophisticated optimisation can lead us badly astray. As the Bank of England’s Adam Brinley Codd and Andrew Gimber say, it can give us “false comfort”. This is because it requires good data on expected returns, volatilities and correlations. If we rely on the past as a guide to the future, however, the same disaster can strike us as would befall a motorist who tries to drive by using only his rear-view mirror. When Goldman Sachs’ David Viniar famously said during the financial crisis that “we were seeing things that were 25-standard deviation moves, several days in a row” he was making the error of assuming that the past was a guide to the future. But it wasn’t. Naïve diversification can protect us from this overreliance on misleading numbers.

It also protects us from something else – overconfidence. This can lead us to believe that events are either certain or impossible. Believing that the chances are evens can help correct this silly but widespread error.

Not all irrationalities are expensive. Given that irrationality is ubiquitous, this is just as well.