Banks delete the crash, then sell you the chart.

The Hook

My backtest handed me a Sharpe ratio of 9.

When I saw the Sharpe of 9, or close to 9, I thought that is probably not possible. There must be a mistake in the backtest, and we need to verify that during live trading with money.

A Sharpe ratio is your return divided by how much that return bounces around. Above 2 is considered excellent.

The Context

A backtest is role-playing on historical data. You have the financial data from the past and you basically ask, what if I would have done this and that. Then you take notes on which trades went well and which didn’t, and you keep track of that.

And you always set a chunk of that historical data aside, in order to test whatever you found on the other part.

Using future information in the present is called look-ahead bias. Take the SpaceX IPO. If you already know how SpaceX performs on the first day once it has IPO-ed, you will just go long. If you know that on the second day it dropped massively, you would just go short, and then just make a profit on both legs. The thing is, on day zero, a few minutes before the IPO actually drops, you don’t really know how it is going to perform. If you run a backtest with historical data, you can actually have a peek. That is the difference.

The other failure is overfitting. You calibrate a strategy to the past data and you do so really well, because during training it saw the entire dataset already. Once you hit a part of the market you haven’t seen, the performance drops massively.

A Sharpe of 9 means you found a bug, not an edge.

The Approach

The defence is a walk-forward. Train on three days, test on the fourth. Train on four, test on the fifth. Six folds, and never a test day the model has already seen.

Mine survived that: nine days of five-minute windows, 531 simulated trades at a dollar each, 6.4 cents of expected profit per dollar staked.

So where does a Sharpe of 9 come from? Arithmetic. The daily figure was 0.46, which is unremarkable. Annualising nine days of it multiplies by roughly nineteen. The number is a unit conversion, not a discovery.

The honest version of that same result is the interval around the 6.4 cents. It runs from minus 1.1 cents to plus 14.1 cents. It contains zero. Three of the nine days lost money. My backtest does not know whether this makes money. Nor do my live trades. Same answer, twice, from opposite ends.

Now the part I am allowed to be angry about. Banks sell QIS notes, Quantitative Investment Strategy, and I will say allegedly throughout, because I cannot see inside their models. They make the most money on the fees: a fee when people go in, a fee when people want to exit, and the spread on whatever they are selling on top. The incentive is a bit skewed and not really aligned with their customers. To generate more fees they need more notes, so they run backtests and make the backtest look really nice, with a nice return-to-volatility ratio and so forth. If you have more of those good-looking strategies, a client might be inclined to get more of them.

Take a QIS strategy running on a basket of foreign exchange pairs, one of which is the Turkish lira. Erdogan was doing his inflation thing, because he thought inflation is not really driven by the central bank interest rate. He forced the man to cut, the lira started to devalue massively, and everyone who could went out of their lira and into the dollar. One pair takes the whole basket down with it.

Now the performance looks terrible on paper. The client sees that, so the bank cannot sell it, so there are no fees. What did they do? They dropped the Turkish lira pairs, backtested the entire thing again, and voilà. It looks amazing again, so they were able to sell it.

The difference between them and me is the incentive. They have this corrupt fee structure. I was just not disciplined enough. The blame sits with me.

And I have the receipt. I had a hunch my execution was off, so I tested resting a bid instead of paying the ask. The backtest came out okay. Live, it was horrible. The resting arm filled 51 times and gave back 26% of everything it staked. On its worst session it filled 42 times and lost every single one. On the same day, paying the ask made 19%. I stopped. Then I rebuilt the backtest properly, and this time it agreed with the live money.

The Takeaways

1. Read the Sharpe ratio before you celebrate it. Nine days annualised multiplies by nineteen, and the multiplication is doing all the work.
2. Treat the backtest as the first step only. The final judge is always running it live with money. If it holds up, good. If not, back to the drawing board.
3. Don’t run it live too fast, unless you are happy to lose some money in order to expensively buy some execution data.

What comes next

There are two frontiers. One is finding a signal, and a backtest of it, that convinces me. The other is that the execution has to be worked out too.

Assume you did everything right. You found the signal that holds up in the backtest, you built a strategy around it that also holds up, and you deployed that thing live. It is only half of the story.

Next: the best strategy in the world earns nothing if you never get the trade.

Which chart in your deck deletes the crash?