Overfitting

Definition of 'Overfitting'


A modeling error which occurs when a function is too closely fit to a limited set of data points. Overfitting the model generally takes the form of making an overly complex model to explain idiosyncrasies in the data under study. In reality, the data being studied often has some degree of error or random noise within it. Thus attempting to make the model conform too closely to slightly inaccurate data can infect the model with substantial errors and reduce its predictive power.

Investopedia explains 'Overfitting'


Financial professionals must always be aware of the dangers of overfitting a model based on limited data. For instance, a common problem is using computer algorithms to search extensive databases of historical market data in order to find patterns. Given enough study, it is often possible to develop elaborate theorems which appear to predict things such as returns in the stock market with close accuracy. However, when applied to data outside of the sample, such theorems may likely prove to be merely an overfitting of a model to what were in reality just chance occurrences. In all cases, it is important to test a model against data which is outside of the sample used to develop it.



comments powered by Disqus
Hot Definitions
  1. Cash and Carry Transaction

    A type of transaction in the futures market in which the cash or spot price of a commodity is below the futures contract price. Cash and carry transactions are considered arbitrage transactions.
  2. Amplitude

    The difference in price from the midpoint of a trough to the midpoint of a peak of a security. Amplitude is positive when calculating a bullish retracement (when calculating from trough to peak) and negative when calculating a bearish retracement (when calculating from peak to trough).
  3. Ascending Triangle

    A bullish chart pattern used in technical analysis that is easily recognizable by the distinct shape created by two trendlines. In an ascending triangle, one trendline is drawn horizontally at a level that has historically prevented the price from heading higher, while the second trendline connects a series of increasing troughs.
  4. National Best Bid and Offer - NBBO

    A term applying to the SEC requirement that brokers must guarantee customers the best available ask price when they buy securities and the best available bid price when they sell securities.
  5. Maintenance Margin

    The minimum amount of equity that must be maintained in a margin account. In the context of the NYSE and FINRA, after an investor has bought securities on margin, the minimum required level of margin is 25% of the total market value of the securities in the margin account.
  6. Leased Bank Guarantee

    A bank guarantee that is leased to a third party for a specific fee. The issuing bank will conduct due diligence on the creditworthiness of the customer looking to secure a bank guarantee, then lease a guarantee to that customer for a set amount of money and over a set period of time, typically less than two years.
Trading Center