Sample Size Neglect

DEFINITION of 'Sample Size Neglect'

Sample size neglect occurs when an individual infers too much from a small new sample of information. It is described by behavioral finance as a cognitive error, and can lead people to make poor judgments based on inadequate information. The underlying cause of sample size neglect is that much of the time people fail to understand that more variance is likely to occur in smaller samples. Knowing what a large enough sample size is can be an issue for people who do not have a full understanding of statistical methods.

BREAKING DOWN 'Sample Size Neglect'

When a sample size is too small, accurate and trustworthy conclusions cannot be drawn. An example is an experiment where people were presented with the following scenario:

A person draws from a sample of 5 balls, and finds that 4 are red and 1 is green
A person draws from a sample of 20 balls, and finds that 12 are red and 8 are green
Which sample provides better evidence that the balls are predominantly red?
Most people said that the first, smaller sample provides much stronger evidence because the ratio (4:1) was so much higher. But, in fact, the high ration is outweighed by the much smaller sample size. The sample of 20 actually provides much stronger evidence.

In another oft-referenced experiment people were presented with the following:

A certain town is served by two hospital - In the larger hospital an average of 45 babies are born each day, and in the smaller hospital around 15 babies are born. Roughly 50% of all babies are boys, however, the actual percentage fluctuates from day to day.During one year, each hospital recorded the days on which more than 60% of the babies happened to be boys. Which hospital recorded more such days?

Statistically the larger sample will show less total variability than a smaller sample. Therefore, the smaller hospital will observe more days on which the number of boys born deviates significantly from the expected average. However, 22% of respondents said the larger hospital would experience more such days, and 56% said that the results would be the same for both hospitals. Most people responded incorrectly, and half of those had the incorrect intuition that larger samples exhibit more variability.