Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the restrict-user-access domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home1/portfol1/public_html/wp/wp-includes/functions.php on line 6114

Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the google-analytics-for-wordpress domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home1/portfol1/public_html/wp/wp-includes/functions.php on line 6114

Deprecated: preg_split(): Passing null to parameter #3 ($limit) of type int is deprecated in /home1/portfol1/public_html/wp/wp-content/plugins/add-meta-tags/metadata/amt_basic.php on line 118
Backtest | Portfolio Yoga

Tag: Backtest

Worst Rank Held – Return Variations

In Momentum strategies, one of the ways to reduce churn is to implement a worst rank held strategy. Let’s assume that the strategy picks the top 30 stocks. In the coming month if one of the stocks that is currently part of the portfolio (previous month being ranked in the top 30) is now ranked 31? Should you remove the stock and replace it with a stock that is currently in the top 30?

Stocks are ranked based not just on their own behavior but also the behavior of others. What this means is that even a stock that is doing good can get deranked if another stock does slightly better than this.

One way to avoid such unnecessary churn is to implement a worst rank held feature. Instead of throwing out a stock that is ranked 31, we can by having a feature such as worst rank held, decide to hold the stock until the rank falls below say 60 (or any other rank that you may be comfortable with).

The biggest advantage of having this is the reduction in churn. But nothing comes free and there are always trade-offs. For example, when you retain a stock that is currently ranked say 50, you are also  not selecting a stock that could be ranked in the top 10. 

As the chart below shows, top winners outperform the next set of winners who outperform the set below them and so on and so forth

So, when we choose to retain a stock that is currently placed at rank 35 for instance, are we in a way dragging down our own returns?

We decided to test it out and see for ourselves.

First, the equity curves plotted against each other. 

The No Worst Rank held beats others comprehensively. But lets dig a bit further

The broad statistics

CAGR is best when there is no worst rank held and goes down as we increase the ranks til which a stock is allowed to continue.

But look at 60th rank as the cut off. While the CAGR is lower than for no worst rank, there are large improvements on every other score.

Median holding goes up from 2 months to 4 months, number of trades reduces by 36% and we get to ride on to one of the biggest trades (for those curious, its Eicher Motors – bought in Feb  2012, sold in December 2015).

Rather than a running score, how about looking at rolling 3 year returns to smoothen out the edges?

Once again, not having the worst rank comes on top. 

Every decision has trade offs. More action can lead to a slightly higher return though the invisible costs can reduce some of the outperformance. This is the same issue when we look at rebalancing on a weekly mode vs on a monthly. 

While the worst rank held has value, using it blindly can lead to trouble. Understanding what the objective of the methodology is and being willing to be flexible is a way to go.  

Backtesting Asset Allocation

Asset Allocation is the cornerstone of every investor. A good asset allocation is something that allows you to comfortably sleep at night. A bad asset allocation on the other hand gives you heartaches with higher volatility while not exactly meeting your goals either.

When it comes to basic asset allocation, you can go in two ways – the fixed method where you decide on an split between equity and debt and keep it that way by re-balancing once a year or go the tactical way with your asset allocation dependent on other factors – how cheap or expensive the market currently is for example. The Portfolio Yoga Allocator is a tactical asset allocator.

These days I find asset management companies pushing towards Hybrid funds as a way to smoothen the volatility. A few years back, the same funds were being pushed as a superior investment owing to their continuous dividend paying policy which provided cash flow to the end customer. When the markets change, I change my Narrative. What do you do, Sir 

Across fund houses, as on date we have 180+ schemes that match the Hybrid / Balanced fund category managing over 3.8 Lakh Crore. The wonderful part of the whole equation is that they charge asset weighted around 1.65% for the pleasure of providing you some level of asset allocation split.

While the choices are many, just 28 of the 180+ schemes account for 80% of the total assets under management with HDFC Balanced Advantage Fund being the biggest of them (11.6% of total assets).

In the United States, Vanguard offers portfolios with fixed asset allocation split with expense ratio being below 0.10%. Interestingly you can devise your own fixed asset allocation portfolio here in India with the end cost being just around what Vanguard is providing its US clients.

SBI ETF that tracks Nifty 50 for example has an expense ratio of just 0.07%. While there has been Liquid Bees that has been in existence for long, Axis Liquid with assets under management of nearly 30 thousand crores and expense ratio of 0.11% is just as suitable.

Our goals are dependent on the future growth of our portfolios and while fund managers talk about high expectation of return, I for one felt that looking at the past data should provide us with a clue on what we should expect in the future.

For this exercise, I have chosen Nifty Bees as proxy for Equity and HDFC Liquid Fund as proxy for Debt. Portfolio’s was created in January 2002 and rebalanced once every year. The rebalance was subject to a variation of 5% from our target allocation. For instance if Equity shot up during the year and we ended the year with 65% Equity and 35% debt vs starting point of 60% Equity and 40% Debt, the additional equity is sold off and used to buy Debt to bring it back to the 60/40 ratio.

60 – 40 (Equity – Debt) Rolling CAGR

First up is the predominant asset allocation split. 60 / 40 is universal & the simplest asset allocation with a slight tilt towards equity. Do note though that since we have 60% in equity which is of much higher volatility than debt, if you were to measure the asset allocation in terms of volatility, this would seem more like a 80 – 20 split.

As on date, the 3 Year CAGR for this mix comes to 11%, its 7% for 5 years and 9% for 10 Years.

We measure risk by looking at the composite draw-down of the portfolio from the peak. As the chart below shows, most of the time the draw-down was limited to 10% or lower with the maximum being seen as expected in 2008 when markets dropped more than 50% and this portfolio dropped around 25%

Inverting the Ratio – 40 – 60 (in favor of Debt)

What if we tilted the ratio in favor of Debt vs Equities. How would that impact the returns and the Risk?

The chart below plots the same and as can be observed, the difference is minor. But do note that compounding on the long term can meaningfully change the end values for a 60/40 split vs a 40/60 split if the equity premium continues to remain strongly positive.

With Equity taking a back-step, we see a slight reduction in the draw-down as well. 14% is something one can live with given the rarity of such events.

The Aggressive Investor – 80 – 20 Split

If you are young with limited savings and a long road ahead, you may opt to be aggressive when it comes to equity allocation. While you can always go with a 100% allocation to equities, given the uncertainties in life, having a small allocation to debt can help during the bad days.

There is no free lunch and a 80 – 20 Asset Allocation also has a higher draw-down and higher volatility. But at the right time, high allocation to equity can boost your returns significantly.

The Reluctant Investor, 20 – 80 Split

But what if on the other hand you are retired and dependent on the savings for the rest of your life. You wouldn’t want to take unnecessary risks. Yet, with dropping interest rates, it does help to have a small allocation to Equities that can push up the overall returns in good years while not dragging them down substantially in bad years.

Thanks to the fact that our equity exposure is just 20% and one that is re-balanced regularly to keep in shape, the draw-downs is something you can sleep comfortably with even when the rest of the world is seemingly jumping off the ledge.

Asset Allocation is specific to each individual, this goals and his risk taking capacity. While we all wish for the largest gains, we cannot easily digest the large draw-downs that occasionally come with it.

Panic is certainty for everyone of us, but with the right allocation rather than panic we can take advantage of the opportunities provided by the market.

Before I conclude, lets take a look at the Equity Curve of all the above allocation strategies. It goes without saying that higher the equity exposure, better the end result.

One observation from the data is that returns are trending down for a long time now. While the future maybe bright, this data to me provides a perspective on what is possible and what is not.

Compared to fixed, I continue to believe that tactical can provide you with a larger advantage but since its requires monitoring of a regular nature and one can go wrong as well, its not everyone’s cup of tea.

Hope this post provides you some food for thought on how to approach your asset allocation from the perspective of future returns. If you have any queries, do drop me a mail or comment below.

The Value of a Back-Test and Momentum Investing

I recently saw the reality sitcom “Shark Tanks” {Season 10 is available on Amazon Prime}. As an entrepreneur, this is an amazing show highlighting the tough path of bringing an idea to reality. While I kind of dislike the behavior of the Sharks who act pretty cruel, this being television and if they are really investing real money, they have all the right to ask the tough questions.

The two main deal breakers for those who couldn’t get a deal were 

  1. The person has a great idea but is yet to execute or has failed at execution. 
  2. He or She is not fully committed to his enterprise (it being either a part time venture or one of the multiple he is trying to run)

The gap between an idea and the ability to execute it so wide that the vast majority of those with ideas are never really able to execute successfully. You may have heard that google wasn’t the first search engine, but did you know that even the famous logic of Google thata set it apart – ranking of page – wasn’t new. It was invented by Robin Li Yanhong who later went onto found Baidu.

In the Shark Tank, of the few ideas that seemed to be vetted among the thousands that come up for a chance to pitch. But even among the few we are able to see, it’s interesting to see how vast the difference is between those who are just ideating or have had a one off viral sales and one who has worked round the clock to ensure consumer connect and is trying to fulfill a requirement.

The second and what has been generally a deal breaker is the commitment of the entrepreneur to his baby. Most if not all are fully committed and often have invested big money of their own savings before coming to pitch to sell an equity stake. The Sharks seem to hate those who are still currently employed elsewhere while trying this out in their free time (there have been exceptions, but since we don’t have data on all those who apply and get rejected, I would assume a vast majority would fall in this category).

When it comes to finance, it’s easy to talk about how great one’s stock picks are or how great one’s strategy is. The spin can be narrative on the company, it’s great products, it’s wonderful management, it’s enormous scope among plenty others or the quantitative spin.

The quantitative spin refers to the ability to showcase back-tests that show how you could have multiplied your money using a particular strategy that has been discovered by the strategy seller. But most such back-test fail when they are implemented in real time with real money.

The reason for the failure lies in the fact that rather than start with a hypothesis that has a strong fundamental backing, most back-tests are based out of testing hundreds or even thousands of rules till one stumbles across the one that seems to meet every characteristic one was looking out for.

For example, take David Leinweber‘s totally brilliant discovery: that butter production in Bangladesh, U.S. cheese production, and sheep population in Bangladesh and the U.S. together “explained” (in a statistical sense) 99% of the annual movements of the S&P 500 between 1983 and 1993. (Link)

Unfortunately post the discovery, the whole correlation fell apart. But that is what data mined back-test is all about.

Back-testing today has been made very easy thanks to the availability of data and processing power that can run through millions of combinations to find the optimal ones with relative ease. A small investor or a trader sitting on a laptop can find good correlations combining multiple strategies using tools such as Amibroker or R.

A good back-test is built on a hypothesis that is of a sound nature. The reason for testing the data of a historical era is to try and understand if certain strategy that sounds interesting and worthwhile to follow has in the past delivered returns of the nature we seek.

While years of history can be tested and plotted in a few minutes, the reality is much different even if history repeated itself as much of back-test assumes.

Assume for instance a strategy that delivered great returns. This despite the fact that it did not reach its all time high for 6 years after a very great run. 6 years of being under-water is something no investor relishes and one that is a huge career risk to a fund manager. 

Would you be willing to bet on such a strategy. Remember, this strategy beats the hell out of any other comparable index in the long run, but how long is something that is left undefined. If you are willing to invest, how much of your equity holding would you be willing to allocate?

On smallcase, I am finding more and more advisors launch momentum strategy. This is good for more advisors mean better appreciation and understanding of a strategy. But it comes with a caveat – most of the back-tests are extraordinarily good. That in itself is not a bad thing if the strategy is properly tested but with not much data available publicly to validate the same, it sounds a bit too good to be true.

A second disappointment is the fact that many advisors offer packages that come as low a time frame as a month. This when testing would have shown them that even after 3 years, there is always a probability of the client being in the red. 

Back-tests should always be taken with a bag of salt unless you have been able to replicate it yourself. But that is generally not possible due to the proprietary nature of the strategy even though if you were to look at the portfolio’s, more than 80% of them would overlap with each other. As the famous quote from Kungu Panda goes, “There is no secret ingredient”

But beware of strategies that try to do too much – looks great in testing, can be sloppy when it comes to real time. The ideal way to improve your returns always lies with you – allocating right. Tinkering to show better returns is easy, but it always comes with a cost to the end user. More variables, more prone are the results to data mining bias.

Momentum Investing is no different from other factor investing strategies when it comes to recovery periods from draw-downs. They suffer like everyone else, the only advantage being that when they perform, they outclass most others. 

Do not venture unless you are committed for the long run for the short run is mostly messy and one that is unlikely to provide favorable results other than in unusual periods such as 2017. 

Hypothetical performance and real world problems

Recently, a US based fund – F-Squared Investments was in the news for “falsely advertised a successful seven-year track record for the investment strategy based on the actual performance of real investments for real clients”. Even the hypothetical results they provided via a back-test of the investment strategy was bumped up due to a error in the way it was calculated.

Here is a quote from the magazine Fortune

They had developed the model’s hypothetical performance by applying buy and sell recommendations a week before the model would have actually made those suggestions, enabling the model to buy an ETF just before the price rose and sell just before it dropped. Because of the error, F-Squared’s calculations showed that the strategy had returned 135% during the period. In fact, the strategy’s hypothetical performance should have been 38%.

Back-testing is a nice way to understand and test ideas. But the very fact that back-test has been made easy by charting / statistical programs has meant that every one now just writes a few lines of code and runs a back-test on the whole data to see if the logic works. If yes, next step is in directly trying to implement the same without even worrying about the nuances of the program and whether the back-test results are close to the real results that will be possible.

Let me provide a few basic ways in which the back-test results may be way different from what can be expected in real trading scenario.

Cost of Slippage / Commission: How much of commission / slippage are you adding to the system for every trade that is carried out. If the system is trading on a lower time frame, its guaranteed that this number will be big regardless of which low cost broker you trade with.

There are basically two ways as to how people place their orders based on signals. The first way is to enter the trade as soon as the bar is completed. The second way is to enter the trade only if the bar high / low is broken (high in case of Long / Cover and low in case of Sell / Short).

The first method is straight forward and simple. As soon as the bar gets completed, you take a trade. But how much of a difference can be expected there? A point or two would always be there. Now while that does not seem to be a big number, think about how much of difference you shall see in reality if your system has even a average slippage of 1.5 to 2 points every trade and it trades say 300 trades per year.

The second method where one places a order above the high / low of the bar is even more liable for slippage due to the very fact that it being a stop order, the order trigger and execution price will be at least 3 – 5 points to ensure guaranteed fill.

Quite a few people I know have a filter of another few points just to ensure that they do not get a entry if the bar high / low is not conclusively crossed. That adds in a way to the misery of the system since every trade will be having a even higher slippage than what is generally thought of.

And last but not the least, slippage is high when markets are volatile. So, if your system seemed to have performed pretty good during the 2008 crisis, do add a note to yourself that liquidity in moving markets can be very poor and not as per expectations.

Cost of Rolls: Most guys who back-test use data from running futures contract since that is the most widely available data plus also enables one to test a large period in one go. But embedded into that data is the rollover cost.

Lets take a recent example. Assume you were long going into the December 2014 expiry. Since once the data series of December ends, the fresh data from January starts adding to the series. But the gap which one sees is not a real gap in the sense that if you had rolled over paying 100 Rupees (during the closing 5 minutes of Expiry day), this is seen as a profit in the back-test when in reality you never saw that 100 bucks 🙂

To overcome this issue, I believe a lot of guys use Spot data for their testing. But since we cannot trade Spot and will end up trading the futures, how big will be the difference between the two. Even on intra-day basis, we sometimes see enormous shifts in the premium. The other day, Nifty opened with a premium of around 58 points and slipped to 40 by the time the day ended. And yesterday, one saw it dip down to 20 before bouncing back to 30.

And finally, how much of data are you testing? This is a very big challenge for many since the larger the sample size of data one uses, the worse the system results seem to show. So, to avoid that should one use a smaller sample, one that is closer to the current market?

Well, if you want to fool yourselves, go ahead. But if you don’t, test for as long a period as possible. The greater the period, the more robust the strategy will be (if it hold up, that is). Falls we saw in 2008 / 2000 are the kind of things one should expect going forward as well. And if your system will get hanged up due to that kind of volatility, its just a matter of time before your system reduces your equity balance to Zero.

One way to measure system robustness is also in the number of back-test trades you see. The higher the number, the better it is. Remember reading somewhere that anything over 400 trades in a back-test signifies a good sample size.

And remember that even after accounting for everything that I have written about above, you will still need to beat the Buy & Hold returns by a handsome margin. Else, you are just wasting time on a venture that will get you sleepless nights without the benefit of a bounty at the end of the road.

Finally, all of the above will be useless if your system is based on some optimization. It shall pass any and every hind-sight test but shall fail the moment you start trading it in a real environment. I personally abhor optimization though even I fall prey to it (Trading one time frame than the other because historical records indicate that this is the better than the other). But if you understand the process of why the signal is generated and if you have enough out of sample test results, this is something that can be more or less overcome to a extent.