Backtesting: Interpreting The Past

Backtesting is a key component of effective trading-system development. It is accomplished by reconstructing, with historical data, trades that would have occurred in the past using rules defined by a given strategy. The result offers statistics that can be used to gauge the effectiveness of the strategy. Using this data, traders can optimize and improve their strategies, find any technical or theoretical flaws, and gain confidence in their strategy before applying it to the real markets. The underlying theory is that any strategy that worked well in the past is likely to work well in the future, and conversely, any strategy that performed poorly in the past is likely to perform poorly in the future. This article takes a look at what applications are used to backtest, what kind of data is obtained, and how to put it to use!

Tutorial: Basics of Technical Analysis

The Data and The Tools
Backtesting can provide plenty of valuable statistical feedback about a given system. Some universal backtesting statistics include:

• Net Profit or Loss - Net percentage gain or loss.
• Time frame - Past dates in which testing occurred.
• Universe - Stocks that were included in the backtest.
• Volatility measures - Maximum percentage upside and downside.
• Averages - Percentage average gain and average loss, average bars held.
• Exposure - Percentage of capital invested (or exposed to the market).
• Ratios - Wins-to-losses ratio.
• Annualized return - Percentage return over a year.
• Risk-adjusted return - Percentage return as a function of risk.

Typically, backtesting software will have two screens that are important. The first allows the trader to customize the settings for backtesting. These customizations include everything from time period to commission costs. Here is an example of such a screen in AmiBroker:


The second screen is the actual backtesting results report. This is where you can find all of the statistics mentioned above. Again, here is an example of this screen in AmiBroker:

In general, most trading software contains similar elements. Some high-end software programs also include additional functionality to perform automatic position sizing, optimization and other more-advanced features.

The 10 Commandments
There are many factors traders pay attention to when they are backtesting trading strategies. Here is a list of the 10 most important things to remember while backtesting:

Take into account the broad market trends in the time frame in which a given strategy was tested. For example, if a strategy was only backtested from 1999-2000, it may not fare well in a bear market. It is often a good idea to backtest over a long time frame that encompasses several different types of market conditions. Take into account the universe in which backtesting occurred. For example, if a broad market system is tested with a universe consisting of tech stocks, it may fail to do well in different sectors. As a general rule, if a strategy is targeted towards a specific genre of stock, limit the universe to that genre; but, in all other cases, maintain a large universe for testing purposes. Volatility measures are extremely important to consider in developing a trading system. This is especially true for leveraged accounts, which are subjected to margin calls if their equity drops below a certain point. Traders should seek to keep volatility low in order to reduce risk and enable easier transition in and out of a given stock. The average number of bars held is also very important to watch when developing a trading system. Although most backtesting software includes commission costs in the final calculations, that does not mean you should ignore this statistic. If possible, raising your average number of bars held can reduce commission costs, and improve your overall return. Exposure is a double-edged sword. Increased exposure can lead to higher profits or higher losses, while decreased exposure means lower profits or lower losses. However, in general, it is a good idea to keep exposure below 70% in order to reduce risk and enable easier transition in and out of a given stock. The average-gain/loss statistic, combined with the wins-to-losses ratio, can be useful for determining optimal position sizing and money management using techniques like the Kelly Criterion. (See Money Management Using the Kelly Criterion.) Traders can take larger positions and reduce commission costs by increasing their average gains and increasing their wins-to-losses ratio. Annualized return is important because it is used as a tool to benchmark a system\'s returns against other investment venues. It is important not only to look at the overall annualized return, but also to take into account the increased or decreased risk. This can be done by looking at the risk-adjusted return, which accounts for various risk factors. Before a trading system is adopted, it must outperform all other investment venues at equal or less risk. Backtesting customization is extremely important. Many backtesting applications have input for commission amounts, round (or fractional) lot sizes, tick sizes, margin requirements, interest rates, slippage assumptions, position-sizing rules, same-bar exit rules, (trailing) stop settings and much more. To get the most accurate backtesting results, it is important to tune these settings to mimic the broker that will be used when the system goes live. Backtesting can sometimes lead to something known as over-optimization. This is a condition where performance results are tuned so highly to the past that they are no longer as accurate in the future. It is generally a good idea to implement rules that apply to all stocks, or a select set of targeted stocks, and are not optimized to the extent that the rules are no longer understandable by the creator. Backtesting is not always the most accurate way to gauge the effectiveness of a given trading system. Sometimes strategies that performed well in the past fail to do well in the present. Past performance is not indicative of future results. Be sure to paper trade a system that has been successfully backtested before going live to be sure that the strategy still applies in practice.

Conclusion
Backtesting is one of the most important aspects of developing a trading system. If created and interpreted properly, it can help traders optimize and improve their strategies, find any technical or theoretical flaws, as well as gain confidence in their strategy before applying it to the real world markets.

Resources Tradecision (http://www.tradecision.com) - High-end Trading System Development AmiBroker (http://www.amibroker.com) - Budget Trading System Development.