Rprop

Rprop, short for resilient backpropagation, is a learning heuristics for supervised learning in artificial neural networks. Similarly to the Manhattan update rule, Rprop takes into account only the sign of the partial derivative over all patterns (not the magnitude), and acts independently on each "weight". For each weight, if there was a sign change of the partial derivative of the total error function compared to the last iteration, the update value for that weight is multiplied by a factor η^-, where η^- <1. If the last iteration produced the same sign, the update value is multiplied by a factor of η⁺, where η⁺ >1. The update values are calculated for each weight in the above manner, and finally each weight is changed by its own update value, in the opposite direction of that weight's partial derivative , so as to minimise the total error function.

η⁺ is empirically set to 1.2 and η^- to 0.5.

Next to the Cascade correlation algorithm and the Levenberg-Marquardt algorithm, Rprop is one of the fastest weight update mechanisms.

It was created by Martin Riedmiller.

References

* [http://citeseer.ist.psu.edu/rd/2171473%2C711503%2C1%2C0.25%2CDownload/http://citeseer.ist.psu.edu/cache/papers/cs2/20/http:zSzzSzamy.informatik.uos.dezSzriedmillerzSzpublicationszSzrprop.details.pdf/riedmiller94rprop.pdf Rprop - Description and Implementation Details] Martin Riedmiller, 1994. Technical report.