- ALOPEX
ALOPEX (an acronym from ""ALgorithms Of Pattern EXtraction") is a correlation based machine learning algorithm first proposed by Tzanakou and Harth in 1974.
Principle
In
machine learning , the goal is to train a system to minimize a cost function or (referring to ALOPEX) a response function. Many training algorithms, such asbackpropagation , have an inherent susceptibility to getting "stuck" in local minima or maxima of the response function. ALOPEX uses a cross-correlation of differences and a stochastic process to overcome this in an attempt to reach the absolute minimum (or maximum) of the response function.Method
ALOPEX, in its simplest form is defined by an updating equation:
Delta W_{ij}(n) = gamma Delta W_{ij}(n-1) Delta R(n) + r_i(n)
Where:
*n geq 0 is the iteration or time-step.
*Delta W_{ij}(n) is the difference between the current and previous value of system variable W_{ij} at iteration n .
*Delta R(n) is the difference between the current and previous value of the response function R, at iteration n .
*gamma is the learning rate parameter gamma < 0 minimizes R, and gamma > 0 maximizes R )
*r_i(n) sim N(0,sigma ^2)Discussion
Essentially, ALOPEX changes each system variable W_{ij}(n) based on a product of: the previous change in the variable DeltaW_{ij}(n-1), the resulting change in the cost function DeltaR(n), and the learning rate parameter gamma. Further, to find the absolute minimum (or maximum), the stochastic process r_{ij}(n) (Gaussian or other) is added to stochastically "push" the algorithm out of any local minima.
References
*Harth, E., & Tzanakou, E. (1974) Alopex: A stochastic method for determining visual receptive fields. Vision Research, 14:1475-1482. [http://dx.doi.org/10.1016/0042-6989(74)90024-8 Abstract from ScienceDirect]
Wikimedia Foundation. 2010.