Efficiently Inefficient #4 : Backtest Noob musings.

There are some murmurings that I am emerging as some sort of back-test guru but that cannot be further from the truth. I am still very much of a noob and have a whole lot more to learn.

Today I will take about some aspects of backtesting that I will need to straighten out to improve some of my own investing strategies.

a) Understanding superior returns

The book explains that there are just two sources of superior returns. First, acceptance of liquidity risk. And second, superior information. Back-testing may refers to obtaining of superior information that compares strategies against each other although some of the counters flagged by a stock screener can lack liquidity. ( Like my frustration of being unable to buy more Global Testing stocks )

b) The problem of trading costs

Bloomberg's backtesting tool should allow us to factor in trading costs. In such a case, monthly rebalancing is costlier than annual rebalancing.

c) Data-mining as a serious form of bias when backtesting the markets

Kyith Ng of Investment Moats asked me privately what would backtesting results be if I changed the order of stock selection in my screens. So instead of choosing the highest dividend yielding stocks and then looking for those with the lowest gearing, I would look for REITs with the lowest gearing and then find the highest yielding stocks in the set. Both backtests will yield different results so one will clearly be superior to the other.

We will always be seduced into backtesting more sophisticated screens to get a higher return or lower risk but we are still operating on one historical dataset and may overfit the data.

d) Need alternative sets of data.

One day to preventing ourselves from data mining would be to test a different set of data with the screens which I have unfortunately not done in any of my talks so far.

We first find a strategy that works in Singapore, for example dividend stocks with sustainable free cash flows. We use a completely different set of data like the US or Japan and we observe to see if we can obtain outperformance in a different market.

This gives us better assurance that our investment ideas are not completely off the wall.