Least Squares Method

What is the 'Least Squares Method'

The least squares method is a form of mathematical regression analysis that finds the line of best fit for a dataset, providing a visual demonstration of the relationship between the data points. Each point of data is representative of the relationship between a known independent variable and an unknown dependent variable.

BREAKING DOWN 'Least Squares Method'

The least squares method provides the overall rationale for the placement of the line of best fit among the data points being studied. The most common application of the least squares method, referred to as linear or ordinary, aims to create a straight line that minimizes the sum of the squares of the errors generated by the results of the associated equations, such as the squared residuals resulting from differences in the observed value and the value anticipated based on the model.

This method of regression analysis begins with a set of data points to be graphed. An analyst using the least squares method will be seeking a line of best fit that explains the potential relationship between an independent variable and a dependent variable. In regression analysis, dependent variables are designated on the vertical Y axis and independent variables are designated on the horizontal X axis. These designations will form the equation for the line of best fit, which is determined from the least squares method.

Example of Least Squares Method

For example, an analyst may want to test the relationship between a company’s stock returns and the index returns for which the stock is a component. In this example, the analyst seeks to test the dependence of the stock returns on the index returns. To do this, all of the returns are plotted on a chart. The index returns are then designated as the independent variable, and the stock returns are the dependent variable. The line of best fit provides the analyst, with coefficients explaining the level of dependence.

Line of Best Fit Equation

The line of best fit determined from the least squares method has an equation that tells the story of the relationship between the data points. Computer software models are used to determine the line of best fit equation, and these software models include a summary of outputs for analysis. The least squares method can be used for determining the line of best fit in any regression analysis. The coefficients and summary outputs explain the dependence of the variables being tested.

In contrast to a linear problem, a non-linear least squares problem has no closed solution and is generally solved by iteration. The earliest description of the least squares method was by Carl Friedrich Gauss in 1795.