Imagine someone shows you a strategy with a superb ten-year backtest. The numbers are real. Nobody faked anything. The question worth asking is not whether the figures are honest — it is what those figures are evidence of.
Two researchers took 97 published predictors of stock returns and measured what happened to each one after the study that discovered it. Returns were about 26% lower in the period after the original sample but before publication, and about 58% lower after publication.[1]
Those two numbers describe two different problems, and separating them is the whole lesson.
The 26% is the cost of choosing. Nobody had read the paper yet, so no competitor had arrived. The edge shrank anyway, because the rule was selected partly for fitting the noise in one particular stretch of history. Test enough ideas against one dataset and some will look brilliant by luck alone.
The remaining gap — roughly 32 percentage points more — is the cost of being known. Once the idea is published, capital arrives to trade it, and the prices that produced the edge move.
It is like a shortcut through a quiet side street. Some of the time you saved was never really there — you happened to hit green lights the week you timed it. The rest was real, right up until everyone else found the same street.
- In-sample: the window the rule was chosen from. Always flattering.
- Out-of-sample: a later window nobody had seen. Roughly a quarter of the edge is typically gone here already.
- Post-publication: after the idea is public and competed against. Roughly half is gone.
- The honest number is the one earned after the choosing stopped.
None of this means research is worthless or that every backtest is a lie. It means a backtested figure is a hypothesis about the future, and the only evidence that settles it is a record built after the rule was fixed. That is why the date a strategy was written down matters as much as its numbers.
Finisdom’s Fund Foundry makes this measurable rather than theoretical: publishing a fund freezes its rule and stores the backtested claim permanently, and from that day the Decay Board compares the claim against what the identical rule actually delivers. A fund under about six months old is reported as too new to read rather than given a flattering score.
Does every backtested strategy decay?
Not every one, but the average decline across published predictors is large — around 58% after publication. Some edges are compensation for a real risk somebody has to bear, and those survive better than edges that were mispricings, because a risk premium does not disappear just because people know about it.
How long does a live record need to be before it means anything?
Longer than most people assume. A few months of live returns is mostly noise. Roughly six months before the numbers say anything at all and a couple of years before they say much is a reasonable rule of thumb — and shorter windows should be reported as unreadable rather than scored.
Can I avoid decay by not publishing my strategy?
Keeping it private avoids the competition half, but not the selection half. The roughly 26% that vanished before publication happened with nobody watching. The only defence against that portion is to fix your rule first and judge it on data you did not use to build it.