What is 'Backtesting'

Backtesting assesses the viability of a trading strategy by discovering how it would play out with historical data.

Breaking Down 'Backtesting'

Backtesting allows a trader to simulate a trading strategy using historical data to generate results and analyze risk and profitability before risking any actual capital.

A well-conducted backtest that yields positive results assures traders that the strategy is fundamentally sound and is likely to yield profits when implemented in reality. A well-conducted backtest that yields suboptimal results will prompt traders to alter or reject the strategy.

Particularly complicated trading strategies, such as strategies implemented by automated trading systems, rely heavily on backtesting to prove their worth, as they are too arcane to evaluate otherwise.

The Ideal Backtesting Scenario

The ideal backtest chooses sample data from a relevant time period of a duration that reflects a variety of market conditions. In this way, one can better judge whether the results of the backtest represent a fluke or sound trading.

The historical data set must include a truly representative sample of stocks, including those of companies which eventually went bankrupt or were sold or liquidated. The alternative, including only data from historical stocks that are still around today, will produce artificially high returns in backtesting.

A backtest should consider all trading costs, however insignificant, as these can add up over the course of the backtesting period and drastically affect the appearance of a strategy’s profitability. Traders should ensure that their backtesting software accounts for these costs.

The Pitfalls of Backtesting

For backtesting to provide meaningful results, traders must develop their strategies and test them in good faith, avoiding bias as much as possible. That means the strategy should be developed without relying on the data used in backtesting. That’s harder than it seems. Traders generally build strategies based on historical data. They must be strict about testing with different data sets from those they train their models on. Otherwise, the backtest will produce glowing results that mean nothing.

Similarly, traders must also avoid data dredging, in which they test a wide range of hypothetical strategies against the same set of data with will also produce successes that fail in real-time markets, because there are many invalid strategies that would beat the market over a specific time period by chance.

One way to compensate for the tendency to data dredge or cherry pick is to use a strategy that succeeds in the relevant, or in-sample, time period and backtest it with data from a different, or out-of-sample, time period. If in-sample and out-of-sample backtests yield similar results, then they are likely generally valid.

RELATED TERMS
  1. Optimization

    In the context of technical analysis, it is the process of adjusting ...
  2. Representative Sample

    A representative sample is a subset of a statistical population ...
  3. Sampling Distribution

    A sampling distribution is a probability distribution of a statistic ...
  4. Mechanical Investing

    Buying and selling stocks according to a screen based on predetermined ...
  5. Sample Selection Bias

    Sample selection bias is a type of bias caused by using non-random ...
  6. Simple Random Sample

    A simple random sample is a subset of a statistical population ...
Related Articles
  1. Trading

    The Best Ways to Learn Technical Analysis

    Discover the best ways to learn technical analysis without risking thousands of dollars in the market.
  2. Trading

    Bad Luck With Trades? 5 Things You Might Be Doing Wrong

    If you're in a trading rut, ask yourself these five questions to help turn the corner.
  3. Trading

    Automated Trading Systems: The Pros and Cons

    Automated trading systems minimize emotions, allow for faster order entry, lead to greater consistency and resolve "pilot error."
  4. Trading

    Create Your Own Trading Strategies

    Do-it-yourself trading can be very rewarding – both psychologically and for your wallet.
  5. Trading

    How to Practice as a Day Trader With Paper Trading

    Day traders face intense competition in today's market. Paper trading practice is more important than ever for getting above-market risk-adjusted returns.
  6. Trading

    Top 10 Rules For Successful Trading

    Whether you're a novice or an expert, these 10 rules should be the backbone of your trading career.
  7. Investing

    TD Ameritrade's Backtesting Tool Lets Investors Relive, Retest Their Trades

    TD Ameritrade's backtesting tool lets traders implement lessons from the past.
RELATED FAQS
  1. What percentage of the population do you need in a representative sample?

    Learn about representative samples and how they are used in conjunction with other strategies to create useful data with ... Read Answer >>
  2. What's the difference between a representative sample and a random sample?

    Explore the differences between representative samples and random samples, and discover how they are often used in tandem ... Read Answer >>
  3. What are the disadvantages of using a simple random sample to approximate a larger ...

    Learn what a simple random sample is, how researchers use it as a statistical tool and the disadvantages it carries when ... Read Answer >>
  4. What's an example of stratified random sampling?

    Stratified random sampling divides a population into subgroups or strata, whereby the members in each of the stratum formed ... Read Answer >>
  5. What are the pros and cons of stratified random sampling?

    Stratified random sampling provides a more accurate sampling of a population, but can be disadvantageous when researchers ... Read Answer >>
  6. What assumptions are made when conducting a t-test?

    Learn what a t-test is and discover the five standard assumptions made regarding the validity of sampling and data used in ... Read Answer >>
Hot Definitions
  1. Yield Curve

    A yield curve is a line that plots the interest rates, at a set point in time, of bonds having equal credit quality, but ...
  2. Portfolio

    A portfolio is a grouping of financial assets such as stocks, bonds and cash equivalents, also their mutual, exchange-traded ...
  3. Gross Profit

    Gross profit is the profit a company makes after deducting the costs of making and selling its products, or the costs of ...
  4. Diversification

    Diversification is the strategy of investing in a variety of securities in order to lower the risk involved with putting ...
  5. Intrinsic Value

    Intrinsic value is the perceived or calculated value of a company, including tangible and intangible factors, and may differ ...
  6. Current Assets

    Current assets is a balance sheet item that represents the value of all assets that can reasonably expected to be converted ...
Trading Center