Although the consequences are nowhere near as grave, the investment industry is arguably sometimes a culprit of publication bias, when only flattering data is provided to private investors. It could also be termed presentation bias, where the manner of the data's presentation is similarly geared to a positive result. Take back-testing, often used by fund managers to launch a new investment strategy. It is incredibly difficult to convince investors to back a strategy before the three-year performance mark, or sometimes the five-year point: while legally required marketing insists that past performance is no guide to the future, it is all clients have to go on.
When a strategy is brand new, the manager will often 'backtest' it, on a given asset allocation, against the market. There are obvious ways to engineer results: firstly, take the most flattering time period. Secondly, tinker with the strategy to get the best result for a given time period (read the FT's John Authers on smart beta here). Not to mention the criticism from academics that such backtests are often too short to take into account a full business cycle, while short enough to be skewed by changing valuation trends.
A study published four years ago by Cass Business School measured a series of alternative indices between 1968 and 2011. All of them delivered higher risk-adjusted returns than the market capitalisation-weighted index. But, crucially, so did the vast majority of 10m equity indices constructed randomly; "by monkeys", the researchers added to make the point. Had the same set of research been carried out purely for fund-selling aims, it is difficult to imagine that the randomly generated indices would have been included in the findings.
Readers heading for asset managers' publicly traded stock, rather than their funds, do not have to look far to see another potential area of misinformation: the proportion of a company's funds that are top-quartile for performance. Again, at first sight this seems a fairly objective, and clearly important, measure. But there are areas that can be tweaked: as it is usually a measure of performance calculated since launch, or since a new manager was appointed, so there are two obvious levers that can be pulled to improve the output number here: shut or reboot underperforming funds or replace managers.
There are obvious ways to improve information: giving comparative figures based on the entire life of the strategy would be a small one. Backtests could be presented over a large series of different time periods, or even letting the investor set their own period through an online modeller.
Journalists are hardly immune to the vice of presenting data to fit the story, and leaving out that which contradicts it, but there are examples of good objective practice. My colleague Algy Hall's regular stock screens are a good example of publishing the data come what may: the screens live and die by their own construct, and do not get buried if they underperform an index.
"You can't break out of a circle/That you never knew you were in," sang US folk artist Conor Oberst on 'Moab'. Presentation bias leaves private investors at risk of a similar confinement. There will be countless more examples, in the corporate as well as the fund management sphere, to which readers could point. One of the advantages of the open sharing of data should be the ability to scrutinise these claims. More data, please.
