Orthant-wise limited-memory quasi-Newton

Orthant-wise limited-memory quasi-Newton

Orthant-wise limited-memory quasi-Newton (OWL-QN) is a numerical optimization algorithm that belongs to the class of quasi-Newton methods, and is specifically designed to serve in the training/fitting algorithm of log-linear (MaxEnt) models with \ell_1-regularization. It minimizes functions of the form

f(\vec x) = g(\vec x) + C \|\vec x\|_1

where g is a differentiable convex loss function.

OWL-QN's design is based on limited-memory BFGS (L-BFGS) but extended to exploit the sparsity of \ell_1-regularized models.[1]

Implementations

References

  1. ^ a b Andrew, Galen; Gao, Jianfeng (2007). "Scalable Training of L1-Regularized Log-Linear Models". ICML. 

Wikimedia Foundation. 2010.

Игры ⚽ Нужно решить контрольную?

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”