Researchers showed that if you try many strategy variations, you’re almost guaranteed to find one with a beautiful backtest that has zero real skill.[1]
- Overfitting — tuning a strategy until it fits past noise, not real signal.
- Look-ahead bias — accidentally using information you wouldn’t have had at the time.
- Survivorship bias — testing only the winners still around, ignoring the ones that died.
Flip enough coins and someone gets ten heads in a row. They look like a genius — until the next flip. A dazzling backtest can just be that lucky coin.
Defences: keep it simple, test on data you didn’t tune on, be suspicious of perfect curves, and expect real results to be worse than the backtest.
Where these numbers come from
Finisdom uses honest walk-forward testing and real crash periods, and never hides the bad years — because a believable backtest beats a flattering one.