Weighted least squares

Weighted least squares is a method of linear regression similar to the ordinary least squares method, except that points are weighted, which causes that some points have greater effect on the approximation than the others. The weighted least squares method is particularly important in function approximation and in estimation of model parameters from measured data.

Problem statement

Consider the problem of adjusting a model function to best fit a given data set. The chosen model function has a set of adjustable parameters a. The data set consist of n points

$(y_{i},{\mathbf {x}}_{i}),i=1,2,\dots ,n.$

The model function has the form

$y=f({\mathbf {x}};{\mathbf {a}}),$

where y is the dependent variable, x are the independent variables, and a are the adjustable parameters of the model. We wish to find the values of these parameters such that the model best fits the data according to a defined error criterion. The least squares method minimizes the weighted sum of squares of errors,

$S(a)=\sum _{i=1}^{n}w_{i}(y_{i}-f({\mathbf {x}}_{i};{\mathbf {a}}))^{2},$

with respect to the adjustable parameters of the model a.

When y_i are some quantities measured in some real physical system, and f is a model of this system, then we often set the weights w_i as inverse of the variances of the corresponding measurements (y_i),

$w_{i}=1/\sigma _{i}^{2}.$

In this way, measurements with lower statistical variance are given greater statistical weight.

See also

• Least squares
• Moving least squares
• Regression analysis
• Function approximation