Biology, images, analysis, design...
Use/Abuse Principles How To Related
"It has long been an axiom of mine that the little things are infinitely the most important" (Sherlock Holmes)




Locally weighted regression

Locally weighted regression scatterplot smoothing (LOWESS) is used to model a relationship where no single functional form (such as a linear relationship) is adequate. One option for non-linear regression is to fit a polynomial model of the form Y = β0 + β1X + β2X2 + ...... These are superficially attractive, because higher order polynomials can be made to 'fit' almost any pattern. However, interpretation of such models is very difficult and their shape is sensitive to outliers. LOWESS regression is often a preferable approach.

The rationale of the approach is that the Y values of neighbouring X values are the best indicators of what the Y value should be at a given X value. Given this we just have to decide how many X values should be considered, and how the Y values that correspond to neighbouring X values should be weighted. We have already seen how this approach is applied for time series with running means. However, running means are only appropriate when X values are equally spaced ordinal values. LOWESS regression fits a line to a bivariate scatter of points in a series of iterations using the following procedure:

  1. One first decides how smooth the fitted relationship should be by deciding on the number of adjacent points (q) to be used in the estimation procedure; the greater the number, the smoother will be the fitted line.
  2. Each point to be used is then given a neighbourhood weight in relation to its distance from the focal point (xi).
  3. A simple linear regression is then fitted to each of the q values for a given focal point by weighted least squares, and an estimate i is computed for the focal point.
  4. The procedure is repeated until all n points have estimated (fitted) Y values.
  5. Ordinary residuals are then calculated from the difference between observed and fitted values, and robustness weights calculated.
  6. A further weighted least squares regression is then run using the product of the neighbourhood and robustness weights.
  7. The procedure is repeated until there is little or no change in the final fitted line.

Details on the precise weighting to apply can be found in Trexler & Travis(1993). Further information on the technique is given by Chambers (1983) and Cleveland (1979).