Neural gas

Neural gas


Neural gas is an artificial neural network, inspired by the self-organizing map and introduced in 1991 by Thomas Martinetz and Klaus Schulten.[1] The neural gas is a simple algorithm for finding optimal data representations based on feature vectors. The algorithm was coined "neural gas" because of the dynamics of the feature vectors during the adaptation process, which distribute themselves like a gas within the data space. It is applied where data compression or vector quantization is an issue, for example speech recognition[2], image processing[3] or pattern recognition. As a robustly converging alternative to the k-means clustering it is also used for cluster analysis.[4]

Contents

Algorithm

Given a probability distribution P(x) of data vectors x and a finite number of feature vectors wi, i=1,...,N.

With each time step t a data vector randomly chosen from P is presented. Subsequently, the distance order of the feature vectors to the given data vector x is determined. i0 denotes the index of the closest feature vector, i1 the index of the second closest feature vector etc. and iN-1 the index of the feature vector most distant to x. Then each feature vector (k=0,...,N-1) is adapted according to

 w_{i_k}^{t+1} = w_{i_k}^{t} + \epsilon\cdot  e^{-k/\lambda}\cdot (x-w_{i_k}^{t})

with ε as the adaptation step size and λ as the so-called neighborhood range. ε and λ are reduced with increasing t. After sufficiently many adaptation steps the feature vectors cover the data space with minimum representation error.[5]

The adaptation step of the neural gas can be interpreted as gradient descent on a cost function. By adapting not only the closest feature vector but all of them with a step size decreasing with increasing distance order, compared to k-means clustering a much more robust convergence of the algorithm can be achieved. The neural gas model does not delete a node and also does not create new nodes.

Further reading

  • T. Martinetz, S. Berkovich, and K. Schulten. "Neural-gas" Network for Vector Quantization and its Application to Time-Series Prediction. IEEE-Transactions on Neural Networks, 4(4):558-569, 1993.
  • T. Martinetz and K. Schulten. Topology representing networks. Neural Networks, 7(3):507-522, 1994.

References

  1. ^ Thomas Martinetz and Klaus Schulten (1991). "A "neural gas" network learns topologies". Artificial Neural Networks. Elsevier. pp. 397-402. http://www.ks.uiuc.edu/Publications/Papers/PDF/MART91B/MART91B.pdf. 
  2. ^ F. Curatelli and O. Mayora-Iberra (2000). "Competitive learning methods for efficient Vector Quantizations in a speech recognition environment". In Osvaldo Cairó, L. Enrique Sucar, Francisco J. Cantú-Ortiz. MICAI 2000: Advances in artificial intelligence : Mexican International Conference on Artificial Intelligence, Acapulco, Mexico, April 2000 : proceedings. Springer. p. 109. ISBN 9783540673545. http://books.google.com/books?id=nJKLv5eheZoC&pg=PA109&dq=%22neural+gas%22+speech+recognition&hl=en&ei=ER4RTJqsF57hnQfuzbDrBw&sa=X&oi=book_result&ct=result&resnum=1&ved=0CC0Q6AEwAA#v=onepage&q=%22neural%20gas%22%20speech%20recognition&f=false. 
  3. ^ Angelopoulou, Anastassia and Psarrou, Alexandra and Garcia Rodriguez, Jose and Revett, Kenneth (2005). "Automatic landmarking of 2D medical shapes using the growing neural gas network". In Yanxi Liu, Tianzi Jiang, Changshui Zhang. Computer vision for biomedical image applications: first international workshop, CVBIA 2005, Beijing, China, October 21, 2005 : proceedings. Springer. p. 210. doi:10.1007/11569541_22. ISBN 9783540294115. http://books.google.com/books?id=xI0g7vqVkdoC&pg=PA210&dq=%22neural+gas%22+image+processing&hl=en&ei=pR8RTLynLOLpnQeOxsHpBw&sa=X&oi=book_result&ct=result&resnum=1&ved=0CC0Q6AEwAA#v=onepage&q=%22neural%20gas%22%20image%20processing&f=false. 
  4. ^ Fernando Canales and Max Chacon (2007). "Modification of the growing neural gas algorithm for cluster analysis". In Luis Rueda, Domingo Mery, Josef Kittler, International Association for Pattern Recognition. Progress in pattern recognition, image analysis and applications: 12th Iberoamerican Congress on Pattern Recognition, CIARP 2007, Viña del Mar-Valparaiso, Chile, November 13–16, 2007 ; proceedings. Springer. pp. 684-693. doi:10.1007/978-3-540-76725-1_71. ISBN 9783540767244. http://books.google.com/books?id=JMQk1HJmhv0C&pg=PA684&dq=%22neural+gas%22+cluster+analysis&cd=1#v=onepage&q=%22neural%20gas%22%20cluster%20analysis&f=false. 
  5. ^ http://www.neuroinformatik.ruhr-uni-bochum.de/ini/VDM/research/gsn/JavaPaper/img187.gif

External links


Wikimedia Foundation. 2010.

Игры ⚽ Поможем написать курсовую

Look at other dictionaries:

  • Neural Gas — ein Künstliches neuronales Netz, angelehnt an die Self Organizing Maps und vorgestellt 1991 von Thomas Martinetz und Klaus Schulten. Das Neural Gas ist ein einfacher Algorithmus zur möglichst fehlerfreien Datenkodierung mit Hilfe von… …   Deutsch Wikipedia

  • Growing neural gas — is a self organization neural network first proposed by Bernd Fritzke. Unlike the earlier Neural Gas, Growing Neural Gas (GNG) can add and delete nodes during algorithm execution. The growth mechanism is based on Growing Cell Structures and… …   Wikipedia

  • Growing Neural Gas — Das Growing Neural Gas (GNG) ist ein künstliches neuronales Netz, bei dem während des Adaptionsprozesses Neuronen eingefügt und gelöscht werden können. Bei anderen künstlichen neuronalen Netzen wie Self Organizing Maps (SOM) oder Neural Gas (NG)… …   Deutsch Wikipedia

  • Neuronales Gas — Neural Gas ein Künstliches neuronales Netz, angelehnt an die Self Organizing Maps und vorgestellt 1991 von Thomas Martinetz und Klaus Schulten. Das Neural Gas ist ein einfacher Algorithmus zur möglichst fehlerfreien Datenkodierung mit Hilfe von… …   Deutsch Wikipedia

  • Neuronen-Gas — Neural Gas ein Künstliches neuronales Netz, angelehnt an die Self Organizing Maps und vorgestellt 1991 von Thomas Martinetz und Klaus Schulten. Das Neural Gas ist ein einfacher Algorithmus zur möglichst fehlerfreien Datenkodierung mit Hilfe von… …   Deutsch Wikipedia

  • Artificial neural network — An artificial neural network (ANN), usually called neural network (NN), is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an… …   Wikipedia

  • Cellular neural network — Cellular neural networks (CNN) are a parallel computing paradigm similar to neural networks, with the difference that communication is allowed between neighbouring units only. Typical applications include image processing, analyzing 3D surfaces,… …   Wikipedia

  • Künstliche neuronale Netze — (kurz: KNN, engl. artificial neural network – ANN) sind Netze aus künstlichen Neuronen. Sie sind ein Zweig der künstlichen Intelligenz und prinzipieller Forschungsgegenstand der Neuroinformatik. Der Ursprung der künstlichen neuronalen Netze liegt …   Deutsch Wikipedia

  • Künstliche neuronale Netzwerke — Künstliche neuronale Netze (kurz: KNN, engl. artificial neural network – ANN) sind Netze aus künstlichen Neuronen. Sie sind ein Zweig der künstlichen Intelligenz und prinzipieller Forschungsgegenstand der Neuroinformatik. Der Ursprung der… …   Deutsch Wikipedia

  • GNG-SOM — Das Growing Neural Gas (GNG) ist ein künstliches neuronales Netz, bei dem während des Adaptionsprozesses Neuronen eingefügt und gelöscht werden können. Bei anderen künstlichen neuronalen Netzen wie Self Organizing Maps (SOM) oder Neural Gas (NG)… …   Deutsch Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”