Look Ahead Bias and its Impact on Trading Results

As most system traders, my mind is constantly thinking of ideas that I think can help me beat the markets comprehensively without too much of a effort (no day trading types for example). So, today as I was doing what people do in bathrooms (having a bath), I suddenly had this idea for a new test – it wasn’t really new since I had tested parts of the ideas yesterday and this was more of a build on top of the model I had tested.

Given that so many such wonderful ideas end as failure, I knew there was no point in calling out EUREKA and rushing to the system to test it out pronto though I did kind of write down the general thesis of the idea before I forgot about it and went off to meet up with a few friends.

But as soon as I was back, it was to test out the system and man, had I hit the jackpot. The logic was simple and yet it seemed to work very well, in fact too well. Draw-downs were minuscule and the equity curve was pretty smooth. I was so excited that I even wondered for a second about posting this chart out on Twitter (how else do you think we system traders massage our ego’s :D)

System-1 and System-2 are both from same system though System-2 had a additional trend filter which seemed to clean out even better. At that point I was already thinking of how much equity to allocate to this bugger.

The only fly in the ointment happened when I then went ahead with bootstrapping the results. The output (p-value) was not something I anticipated (and regardless of anything else I found later would have killed the system). One thing experience has taught us is that if its too good to be true, its most likely not true.

So, the next step was to recreate in fresh all the steps I took from clean data and was the output a revelation. Here is the chart with both the original equity curve and the new ones (Realty-1, Realty-2)

Realty-1 was still okay though way below the originals while Realty-2 was a disaster. Do note that all of the above is before any slippage / brokerage / STT, etc. Add that and you will question the need to trade.

So, what went wrong?

For starters, the system had two Independent variables wherein decision was taken based on how Variable 1 acted and where Variable 2 was placed. Now, theoretically the idea was that if Variable 1 ended the day positively, I would look whether Variable 2 traded above a certain parameter and then bought Nifty 50 at end of the day.

Now, this meant that I should look at changes in the next day of Nifty 50 of the next day (holding period was of 1 day) and take that change as either my Profit (if it ended the day positively) or as Loss (if it ended in Negative zone). But while coding I had made a small error which meant that rather than take the next day’s figure, it was taking the current day’s figure.

In essence, it looked ahead by taking the returns of the day when the system was triggered rather than the next day. Remember, when the day opens, I have no clue as to how Variable 1 or Variable 2 will close and yet the system conjured that I was already long from previous night and would based on the decision close out my position.

Look ahead bias is one of the frequent biases you shall come across as a system trader for this one bugger makes a bad system look great and a okay system look like you have hit the jackpot. Realty-1 is still nice and may be worth working upon, but for now, sleep was calling.

And rather than Rinse-repeat-Rinse, I shall post the following pic on Look Ahead Bias from the website Quantstart.com