Weighted least squares: Difference between revisions

From Citizendium
Jump to navigation Jump to search
imported>Igor Grešovnik
(added Problem statement)
imported>Igor Grešovnik
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Subpages}}
'''Weighted least squares''' is a method of linear regression similar to the ordinary [[least squares]] method, except that points are weighted, which causes that some points have greater effect on the approximation than the others. The weighted least squares method is particularly important in function approximation and in estimation of model parameters from measured data.
'''Weighted least squares''' is a method of linear regression similar to the ordinary [[least squares]] method, except that points are weighted, which causes that some points have greater effect on the approximation than the others. The weighted least squares method is particularly important in function approximation and in estimation of model parameters from measured data.


==Problem statement==
==Problem statement==
Consider the problem of adjusting a model function to best fit a data set. The chosen model function has a set of adjustable parameters '''a'''. The data set consist of ''n'' points  
Consider the problem of adjusting a model function to best fit a given data set. The chosen model function has a set of adjustable parameters '''a'''. The data set consist of ''n'' points  


:<math>(y_i,\bold{x}_i), i = 1, 2,\dots, n .</math>
:<math>(y_i,\bold{x}_i), i = 1, 2,\dots, n .</math>


The model function has the form  
The modelfunction has the form  


:<math>y=f(\bold{x};\bold{a}) ,</math>  
:<math>y=f(\bold{x};\bold{a}) ,</math>  


where ''y'' is the dependent variable, '''x''' are the independent variables, and '''a''' are the adjustable parameters of the model. We wish to find the values of these parameters such that the model best fits the data according to a defined error criterion. The least squares method minimizes the sum of squares of errors,
where ''y'' is the dependent variable, '''x''' are the independent variables, and '''a''' are the adjustable parameters of the model. We wish to find the values of these parameters such that the model best fits the data according to a defined error criterion. The least squares method [[Function minimization|minimizes]] the weighted sum of squares of errors,
:<math> S(\bold{a}) = \sum_{i=1}^n  w_i (y_i - f(\bold{x}_i;\bold{a}))^2 ,</math>


:<math> S(a) = \sum_{i=1}^n  w_i (y_i - f(\bold{x}_i;\bold{a}))^2 ,</math>
with respect to the adjustable parameters of the model '''a'''.


with respect to the adjustable parameters of the model '''a'''.
When ''y<sub>i</sub>'' are some quantities measured in some real physical system, and ''f'' is a model of this system, then we often set the weights ''w<sub>i</sub>'' as inverse of the [[Variance|variances]] of the corresponding measurements (''y<sub>i</sub>''),
 
:<math> w_i = 1/\sigma_i^2 .</math>


In this way, measurements with lower statistical variance are given greater statistical weight.


== See also ==
== See also ==
Line 22: Line 27:
*[[Regression analysis]]
*[[Regression analysis]]
*[[Function approximation]]
*[[Function approximation]]
*[[Optimization (mathematics)]]

Latest revision as of 19:20, 17 January 2008

This article is developing and not approved.
Main Article
Discussion
Related Articles  [?]
Bibliography  [?]
External Links  [?]
Citable Version  [?]
 
This editable Main Article is under development and subject to a disclaimer.

Weighted least squares is a method of linear regression similar to the ordinary least squares method, except that points are weighted, which causes that some points have greater effect on the approximation than the others. The weighted least squares method is particularly important in function approximation and in estimation of model parameters from measured data.

Problem statement

Consider the problem of adjusting a model function to best fit a given data set. The chosen model function has a set of adjustable parameters a. The data set consist of n points

The modelfunction has the form

where y is the dependent variable, x are the independent variables, and a are the adjustable parameters of the model. We wish to find the values of these parameters such that the model best fits the data according to a defined error criterion. The least squares method minimizes the weighted sum of squares of errors,

with respect to the adjustable parameters of the model a.

When yi are some quantities measured in some real physical system, and f is a model of this system, then we often set the weights wi as inverse of the variances of the corresponding measurements (yi),

In this way, measurements with lower statistical variance are given greater statistical weight.

See also