Least-angle regression

In statistics, least-angle regression (LARS) is a regression algorithm for high-dimensional data, developed by Bradley Efron, Trevor Hastie, Iain Johnstone and Robert Tibshirani. [cite journal
author = Efron, Bradley
coauthors = Hastie, Trevor; Johnstone, Iain and Tibshirani, Robert
title = Least Angle Regression
journal = Annals of Statistics
year = 2004
volume = 32
issue = 2
pages = "pp." 407–499
url = http://stat.stanford.edu/~imj/WEBLIST/2004/LarsAnnStat04.pdf]

Suppose we expect a response variable to be determined by a linear combination of a subset of potential covariates. Then the LARS algorithm provides a means of producing an estimate of which variables to include, as well as their coefficients.

Instead of giving a vector result, the LARS solution consists of a curve denoting the solution for each value of the L1 norm of the parameter vector. The algorithm is similar to forward stepwise regression, but instead of including variables at each step, the estimated parameters are increased in a direction equiangular to each one's correlations with the residual.

The advantages of the LARS method are:
# It is computationally just as fast as forward selection.
# It produces a full piecewise linear solution path, which is useful in cross-validation or similar attempts to tune the model.
# If two variables are almost equally correlated with the response, then their coefficients should increase at approximately the same rate. The algorithm thus behaves as intuition would expect, and also is more stable.
# It is easily modified to produce solutions for other estimators, like the Lasso.
# It is effective in contexts where "p" > "n".

References