Backtesting is the process of testing a trading strategy on relevant historical data to ensure its viability before the trader risks any actual capital. A trader can simulate the trading of a strategy over an appropriate period of time and analyze the results for the levels of profitability and risk.
A significant amount of the volume traded in today’s financial market is done by traders that use some sort of computer automation. This is especially true for trading strategies based on technical analysis. Backtesting is an integral part of developing an automated trading system.
When done correctly, backtesting can be an invaluable tool for making decisions on whether to utilize a trading strategy. The sample time period on which a backtest is performed on is critical. The duration of the sample time period should be long enough to include periods of varying market conditions including uptrends, downtrends and range-bound trading. Performing a test on only one type of market condition may yield unique results that may not function well in other market conditions, which may lead to false conclusions.
The sample size in the number of trades in the test results is also crucial. If the sample number of trades is too small, the test may not be statistically significant. A sample with too many trades over too long a period may produce optimized results in which an overwhelming number of winning trades coalesce around a specific market condition or trend that is favorable for the strategy. This may also cause a trader to draw misleading conclusions.
A backtest should reflect reality to the best extent possible. Trading costs that may otherwise be considered to be negligible by traders when analyzed individually may have a significant impact when the aggregate cost is calculated over the entire backtesting period. These costs include commissions, spreads and slippage, and they could determine the difference between whether a trading strategy is profitable or not. Most backtesting software packages include methods to account for these costs.
Perhaps the most important metric associated with backtesting is the strategy’s level of robustness. This is accomplished by comparing the results of an optimized back test in a specific sample time period (referred to as in-sample) with the results of a backtest with the same strategy and settings in a different sample time period (referred to as out-of-sample). If the results are similarly profitable, then the strategy can be deemed to be valid and robust, and it is ready to be implemented in real-time markets. If the strategy fails in out-of-sample comparisons, then the strategy needs further development, or it should be abandoned altogether.