Hierarchical Temporal Memory

Hierarchical Temporal Memory (HTM) is a machine learning model developed by Jeff Hawkins and Dileep George of Numenta, Inc. that models some of the structural and algorithmic properties of the neocortex using an approach somewhat similar to Bayesian networks. HTM model is based on the memory-prediction theory of brain function described by Jeff Hawkins in his book "On Intelligence". HTMs are claimed to be biomimetic models of cause inference in intelligence.

While criticized by the AI community as rehashing existing material (for example, in the December 2005 issue of the Artificial Intelligence journal), the model is quite novelFact|date=January 2008 in proposing functions for cortical layers. As such it is related to similar work by Tomaso Poggio and David Mumford amongst others.

imilarity to other models

Bayesian Networks

An HTM can be considered a form of Bayesian network where the network consists of a collection of nodes arranged in a tree-shaped hierarchy. Each node in the hierarchy self-discovers a set of causes in its input through a process of finding common spatial patterns and then finding common temporal patterns. Unlike many Bayesian networks, HTMs are self-training, have a well-defined parent/child relationship between each node, inherently handle time-varying data, and afford mechanisms for covert attention.

Neural Networks

Numenta's Director of Developer Services addressed how HTMs differ from neural networks.

First of all, HTM's are a type of neural network. But in saying that, you should know that there are many different types of neural networks (single layer feed-forward network, multi-layer network, recurrent, etc). 99% of these types of networks tend to emulate the neurons, yet don't have the overall infrastructure of the actual cortex.

Additionally, neural networks tend not to deal with temporal data very well, they ignore the hierarchy in the brain, and use a different set of learning algorithms than our implementation.

But, in a nutshell, HTMs are built according to biology.

Whereas neural networks ignore the structure and focus on the emulation of the neurons, HTMs tend to focus on the structure and ignores the emulation of the neurons.

Implementation

The HTM idea has been implemented in a research release of a software API called "Numenta Platform for Intelligent Computing" (NuPIC). Currently, the software is available as a free download and can be licensed either for general research, or for academic research.

The implementation is written in C++ and Python.Fact|date=January 2008

ee also

*On Intelligence
*Memory-prediction framework
*List of artificial intelligence projects
*Belief propagation
*Bionics

Related models

*Hierarchical hidden Markov model
*Bayesian networks
*Neural networks

References

*PDFlink| [http://www.numenta.com/Numenta_HTM_Concepts.pdf "Hierarchical Temporal Memory - Concepts, Theory, and Terminology"] |804 KiB by Jeff Hawkins and Dileep George, "Numenta Inc.", 2006-05-17
*"On Intelligence"; Jeff Hawkins, Sandra Blakeslee; Henry Holt, 2004; ISBN 0312712340
*Citation
title = How HTMs differ from Neural networks
last = Shoemaker
first = Phillip B.
url = http://onintelligence.org/forum/viewtopic.php?t=255
accessdate = 2007-10-17

External links

Official

* [http://www.numenta.com Numenta, Inc.]
* [http://www.onintelligence.org/forum OnIntelligence.org Forum] , an Internet forum for the discussion of relevant topics, especially relevant being the [http://www.onintelligence.org/forum/viewforum.php?f=3 Models and Simulation Topics] forum.
* [http://www.almaden.ibm.com/institute/resources/2006/Almaden%20Institute%20Jeff%20Hawkins.ppt Hierarchical Temporal Memory] (Microsoft PowerPoint presentation)
* [http://video.google.com/videoplay?docid=-2500845581503718756 Hierarchical Temporal Memory: Theory and Implementation] (Google Video)

Other

* [http://www.gartner.com/research/fellows/fellows_interview_jeff_hawkins_tom_austin.jsp The Gartner Fellows: Jeff Hawkins Interview] by Tom Austin, "Gartner", March 2, 2006
* [http://www.cioinsight.com/article2/0,1540,1955963,00.asp Emerging Tech: Jeff Hawkins reinvents artificial intelligence] by Debra D'Agostino and Edward H. Baker, "CIO Insight", May 1 2006
* [http://insight.zdnet.co.uk/hardware/emergingtech/0,39020439,39268542,00.htm "Putting your brain on a microchip"] by Stefanie Olsen, "CNET News.com", May 12 2006
* [http://www.wired.com/wired/archive/15.03/hawkins.html "The Thinking Machine"] by Evan Ratliff, Wired, March 2007
* [http://spectrum.ieee.org/apr07/4982 Think like a human] by Jeff Hawkins , IEEE Spectrum, April 2007
* [http://sourceforge.net/projects/neocortex Neocortex - Memory-Prediction Framework] — Open Source Implementation with GNU General Public License
* [http://www.pembrokeballet.com/10701-HTM_CAPTCHA.pdf Using Numenta’s hierarchical temporal memory to recognize CAPTCHAs] by Yensy James Hall and Ryan E. Poplin, December 12, 2007
* [http://pages.sbcglobal.net/louis.savain/AI/memory.htm Another type of Temporal Memory] by Louis Savain , November 13, 2002