Quant Theorists Are Paid to Delude Themselves, Cam Harvey Says

(Bloomberg) — A hedge fund pursuing a trading strategy based on fantasy goes broke. A market researcher who does it, on the other hand, is apt to get tenure.

Most Read from Bloomberg

That in a nutshell is the argument in a new paper by Duke University theoretician Cam Harvey, who says that way too many academic projects that go looking for trading edges succeed in finding them. In reality, only a handful stand up outside the walls of academia.

By Harvey’s tally, more than 400 factors — strategies that slice and dice stocks by things such as size, volatility or valuations, and which are supposed to beat the market — have been published in top journals since the 1960s, with roughly half of them discovered in the past decade.

“It just didn’t make any sense to me. It’s really hard to find something that outperforms a market,” Harvey said in a phone interview. “How many factors can there credibly be? Well, to me, it’s maybe up to a couple of dozen.”

Harvey, who’s also a partner at Research Affiliates and serves as an investment strategy adviser to Man Group, has pushed back against aspects of quant investing for years. Underpinning the seemingly widespread breakthroughs, he says, is an incentive culture that encourages researchers to manipulate the data and see what they want to see. As a result, many factors that look promising on paper fail to work in real life.

“The incentive problem, along with the misapplication of statistical methods, leads to the unfortunate conclusion that roughly half of the empirical research findings in finance are likely false,” Harvey wrote in the paper titled “The Pitfalls of Asset Management Research.”

Add on top implementation costs that eat into performance and the fact that some factors generate too small extra returns, and the number of true gems in the quant world is likely significantly lower, he says.

The essay is the latest broadside against an area of research that has come to dominate the financial world and underlie the rise in both quantitative investing and smart beta exchange-traded funds. It joins a growing body of literature that suggests people looking for a trading edge through market chaos are often prejudiced, and sometimes confuse performance with luck.

At the center of the controversy is the ever-popularized idea that certain stock attributes, from time to time, hold the key to picking winners and losers. For instance, a company’s profit potential or size may give clues to its future returns.

Readmore: Why Not Everyone Thinks Smart Beta’s a Smart Idea: QuickTake Q&A

The latest proliferation of such factors, Harvey says, is likely inflated because of an incentive system that ties the number of publications to promotions or salary raises. To get “positive” results and achieve statistical significance, researchers resort to various tactics of data mining, such as choosing different sample starting dates or excluding certain influential periods that might diminish the strength of the results.

Known as p-hacking, these deliberate choices for how to exclude outliers have cultivated a large pool of theoretical factors that claim to have an advantage over the market.

It’s not the first time that Harvey has criticized the quality of finance research. In a paper published in 2021, he warned that the industry faces the same “replication crisis” as the broad scientific field where many papers don’t pass the test when they’re duplicated.

The subject is far from clear-cut. A study conducted by scholars at Copenhagen Business School and AQR Capital Management looked at more than 150 factors and concluded they could largely be grouped into a dozen themes.

The argument is that the deluge of identified factors Harvey sees as a warning sign is not an exercise in data mining but a natural outcome of a decentralized effort where contributions overlap. The same paper also found the majority of factors could be replicated, albeit with some degree of performance decay after they were published.

To Harvey, replication should not only happen on paper but in real world as well. Granted, the propensity to bend data to one’s will is less severe in the practice of finance simply because their research sometimes forms the foundation of a product. Without repeatable performance, money would flee.

Still, the performance of theoretical factors doesn’t take into account costs related to transactions or short selling. And evidence of data overfitting exists in asset management, Harvey says, pointing to a few recent ETF studies by other researchers that showed stellar returns during the years leading up to their launch only to languish afterward.

“That is the external validation of my thesis that is really hard to argue against,” he said. “So you can do all of these academic exercises without transactions costs and say, ‘no, there’s no replication crisis.’ And then what about the external validation? That is super powerful in my opinion.”