What is 'Statistical Significance'

Statistical significance means that a result from testing or experimenting is not likely to occur randomly or by chance, but is instead likely to be attributable to a specific cause. Statistical significance can be strong or weak, and it is important to disciplines that rely heavily on analyzing data and research, such as finance, investing, medicine, physics and biology.

!--break--When analyzing a data set and doing the necessary tests to discern whether one or more variables have an effect on an outcome, statistical significance helps show that the results from the test are real and not just by luck or chance. Problems arise in tests of statistical significance because researchers are usually working with samples of larger populations. As a result, the samples must be representative of the population, so the data contained in the sample must not be biased in any way.

Statistical significance does not always indicate practical significance, meaning the results cannot be applied to real-world business situations. In addition, statistical significance can be misinterpreted when researchers do not use language carefully in reporting their results. Another problem that may arise with statistical significance is that past data, and the results from that data, whether statistically significant or not, may not reflect ongoing or future conditions. In investing, this may manifest itself in a pricing model breaking down during times of financial crisis as correlations change and variables do not interact as usual. Statistical significance can also help an investor discern whether one asset pricing model is better than another.

Calculating Statistical Significance

The calculation of statistical significance (significance testing) is subject to a certain degree of error. The researcher must define in advance the probability of a sampling error, which exists in any test that does not include the entire population. Sample size is an important component of statistical significance in that larger samples are less prone to flukes. Only random, representative samples should be used in significance testing. The level at which one can accept whether an event is statistically significant is known as the significance level. Researchers use a test statistic known as the p-value to discern whether the event falls below the significance level; if it does, the result is statistically significant. The p-value is a function of the means and standard deviations of the data samples.

The p-value indicates the probability under which a statistical result occurred by chance or by sampling error. In other words, the p-value indicates the risk that there is no actual difference or relationship. The p-value must fall under the significance level for the results to at least be considered statistically significant. The opposite of the significance level, calculated as 1 minus the significance level, is the confidence level. The confidence level indicates the degree of confidence that the statistical result did not occur by chance or by sampling error. The customary confidence level in many statistical tests is 95%, leading to a customary significance level or p-value of 5%.

BREAKING DOWN 'Statistical Significance'

RELATED TERMS
  1. Statistically Significant

    The likelihood that a result or relationship is caused by something ...
  2. Sample

    A subset containing the characteristics of a larger population. ...
  3. Sampling Distribution

    A probability distribution of a statistic obtained through a ...
  4. Sampling

    A process used in statistical analysis in which a predetermined ...
  5. Standard Error

    The standard deviation of the sampling distribution of a statistic. ...
  6. Statistics

    A type of mathematical analysis involving the use of quantified ...
Related Articles
  1. Investing

    Understanding Statistics

    Statistics provide the means to analyze data and then summarize it into a numerical form.
  2. Investing

    Explaining Standard Error

    Standard error is a statistical term that measures the accuracy with which a sample represents a population.
  3. Investing

    How Statistical Significance is Determined

    If something is statistically significant, it’s unlikely that it happened by chance.
  4. Investing

    How Does Sampling Work?

    Sampling is a term used in statistics that describes methods of selecting a pre-defined representative number of data from a larger data population.
  5. Investing

    What is a Representative Sample?

    In statistics, a representative sample accurately represents the make-up of various subgroups in an entire data pool.
  6. Investing

    What is Systematic Sampling?

    Systematic sampling is similar to random sampling, but it uses a pattern for the selection of the sample.
  7. Investing

    Understanding the Simple Random Sample

    A simple random sample is a subset of a statistical population in which each member of the subset has an equal probability of being chosen.
  8. Insights

    Can Investors Trust Official Statistics?

    The official statistics in some countries need to be taken with a grain of salt. Find out why you should be skeptical.
  9. Investing

    What is Descriptive Statistics?

    Descriptive statistics is the term applied to meaningful data analysis.
  10. Investing

    Hypothesis Testing in Finance: Concept & Examples

    When you're indecisive about an investment, the best way to keep a cool head might be test various hypotheses using the most relevant statistics.
RELATED FAQS
  1. What's the difference between a representative sample and a random sample?

    Explore the differences between representative samples and random samples, and discover how they are often used in tandem ... Read Answer >>
  2. What's the difference between a representative sample and a convenience sample?

    Learn the difference between convenience sampling and representative sampling and the advantages and disadvantages of each ... Read Answer >>
  3. What percentage of the population do you need in a representative sample?

    Learn about representative samples and how they are used in conjunction with other strategies to create useful data with ... Read Answer >>
  4. How can a representative sample lead to sampling bias?

    Learn how using representative samples alone is not enough to make sampling bias negligible and why elements such as randomization ... Read Answer >>
  5. How do researchers ensure that a simple random sample is an accurate representation ...

    Learn which methods researchers employ to ensure that a simple random sample best approximates the larger population being ... Read Answer >>
  6. What are the benefits of financial sampling?

    Learn more about how financial sampling is used to determine whether or not inaccurate or fraudulent information exists in ... Read Answer >>
Hot Definitions
  1. IRS Publication 970

    A document published by the Internal Revenue Service (IRS) that provides information on tax benefits available to students ...
  2. Federal Direct Loan Program

    A program that provides low-interest loans to postsecondary students and their parents. The William D. Ford Federal Direct ...
  3. Cash Flow

    The net amount of cash and cash-equivalents moving into and out of a business. Positive cash flow indicates that a company's ...
  4. PLUS Loan

    A low-cost student loan offered to parents of students currently enrolled in post-secondary education. With a PLUS Loan, ...
  5. Graduate Record Examination - GRE

    A standardized exam used to measure one's aptitude for abstract thinking in the areas of analytical writing, mathematics ...
  6. Graduate Management Admission Test - GMAT

    A standardized test intended to measure a test taker's aptitude in mathematics and the English language. The GMAT is most ...
Trading Center