Hierarchical Temporal Memory
Encyclopedia
Hierarchical temporal memory (HTM) is a machine learning
model developed by Jeff Hawkins
and Dileep George of Numenta, Inc.
that models some of the structural and algorithm
ic properties of the neocortex
. HTM is a biomimetic
model based on the memory-prediction
theory of brain function described by Jeff Hawkins
in his book On Intelligence
. HTM is a method for discovering and inferring the high-level causes of observed input patterns and sequences, thus building an increasingly complex model of the world.
Jeff Hawkins states that HTM does not present any new idea or theory, but combines existing ideas to mimic the neocortex with a simple design that provides a large range of capabilities. HTM combines and extends approaches used in Bayesian networks, spatial and temporal clustering algorithms, while using a tree-shaped hierarchy of nodes that is common in neural networks
.
Each HTM node has the same basic functionality. In learning and inference modes, sensory data comes into the bottom level nodes. In generation mode, the bottom level nodes output the generated pattern of a given category. The top level usually has a single node that stores the most general categories (concepts) which determine, or are determined by, smaller concepts in the lower levels which are more restricted in time and space. In inference mode, a node in each level interprets information coming in from its child nodes in the lower level as probabilities of the categories it has in memory.
Each HTM region learns by identifying and memorizing spatial patterns - combinations of input bits that often occur at the same time. It then identifies temporal sequences of spatial patterns that are likely to occur one after another.
During inference (recognition), the node calculates the set probabilities that a pattern belongs to each known coincidence. Then it calculates the probabilities that the input represents each temporal group. The set of probabilities assigned to the groups is called a node's "belief" about the input pattern. (In a simplified implementation, node's belief consists of only one winning group). This belief is the result of the inference that is passed to one or more "parent" nodes in the next higher level of the hierarchy.
"Unexpected" patterns to the node do not have a dominant probability of belonging to any one temporal group, but have nearly equal probabilities of belonging to several of the groups. If sequences of patterns are similar to the training sequences, then the assigned probabilities to the groups will not change as often as patterns are received. The output of the node will not change as much, and a resolution in time is lost.
In a more general scheme, the node's belief can be sent to the input of any node(s) in any level(s), but the connections between the nodes are still fixed. The higher-level node combines this output with the output from other child nodes thus forming its own input pattern.
Since resolution in space and time is lost in each node as described above, beliefs formed by higher-level nodes represent an even larger range of space and time. This is meant to reflect the organization of the physical world as it is perceived by human brain. Larger concepts (e.g. causes, actions and objects) are perceived to change more slowly and consist of smaller concepts that change more quickly. Jeff Hawkins postulates that brains evolved this type of hierarchy to match, predict, and affect the organization of the external world.
More details about the functioning of HTM can be found in Numenta's documentation.
distributed representations. It models cortical columns that tend to inhibit neighboring columns in the neocortex thus creating a sparse activation of columns. A region creates a sparse representation from its input, so that a fixed percentage of columns are active at any one time.
Each HTM region consists of a number of highly interconnected cortical column
s. A region is similar to layer III of the neocortex
. A cortical column is understood as a group of cells that have the same receptive field. Each column has a number of cells that able to remember several previous states. A cell can be in one of three states: active, inactive and predictive state.
Spatial pooling: The receptive field of each column is a fixed number of inputs that are randomly selected from a much larger number of node inputs. Based on the input pattern, some columns will receive more active input values. Spatial pooling selects a relatively constant number of the most active columns and inactivates (inhibits) other columns in the vicinity of the active ones. Similar input patterns tend to activate a stable set of columns. The amount of memory used by each region can be increased to learn more complex spatial patterns or decreased to learn simpler patterns.
Representing the input in the context of previous inputs: If one or more cells in the active column are in the predictive state (see below), they will be the only cells to become active in the current time step. If none of the cells in the active column are in the predictive state (during the initial time step or when the activation of this column was not expected), all cells are made active.
Predicting future inputs and temporal pooling: When a cell becomes active, it gradually forms connections to nearby cells that tend to be active during several previous time steps. Thus a cell learns to recognize a known sequence by checking whether the connected cells are active. If a large number of connected cells are active, this cell switches to the predictive state in anticipation of one of the few next inputs of the sequence. The output of a region includes columns in both active and predictive states. Thus columns are active over longer periods of time, which leads to greater temporal stability seen by the parent region.
Cortical learning algorithms are able to learn continuously from each new input pattern, therefore no separate inference mode is necessary. During inference, HTM tries to match the stream of inputs to fragments of previously learned sequences. This allows each HTM region to be constantly predicting the likely continuation of the recognized sequences. The index of the predicted sequence is the output of the region. Since predictions tend to change less frequently than the input patterns, this leads to increasing temporal stability of the output in higher hierarchy levels. Prediction also helps to fill in missing patterns in the sequence and to interpret ambiguous data by biasing the system to infer what it predicted.
Cortical learning algorithms are currently being implemented and are planned to be released in the middle of 2011.
is remotely similar to the highest HTM level. A single HTM node may represent a group of cortical column
s within a certain region.
Although it is primarily a functional model, several attempts have been made to relate the algorithms of the HTM with the structure of neuronal connections in the layers of neocortex. The neocortex is organized in vertical columns of 6 horizontal layers. The 6 layers of cells in the neocortex should not be confused with levels in an HTM hierarchy. In Jeff Hawkins' theory of the neocortex, the output from a number of layer 1 neurons is analogous to the output of a node. Some layer 4, 5, 6 neurons are analogous to the input to a node. Neurons in layers 2 and 3 are believed to be where the "concepts" of the neocortex reside.
, an HTM comprises a collection of nodes that are arranged in a tree-shaped hierarchy. Each node in the hierarchy discovers an array of causes in the input patterns and temporal sequences it receives. A Bayesian belief revision
algorithm is used to propagate feed-forward and feedback beliefs from child to parent nodes and vice versa. However, the analogy to Bayesian networks is limited, because HTMs can be self-trained (such that each node has an unambiguous family relationship), cope with time-sensitive data, and grant mechanisms for covert attention.
A theory of hierarchical cortical computation based on Bayesian belief propagation
was proposed earlier by Tai Sing Lee and David Mumford
. While HTM is mostly consistent with these ideas, it adds details about handling invariant representations in the visual cortex.
. The tree-shaped hierarchy commonly used in HTMs resembles the usual topology of traditional neural networks. However, unlike most types of neural networks, HTM does not attempt to simulate individual neurons or their connections, but rather models the large-scale hierarchy of cortical regions. HTM is an unconventional type of neural network which uses abstract neural modules that represent complete subsystems of the brain.
LAMINART and similar neural networks researched by Stephen Grossberg attempt to model both the infrastructure of the cortex and the behavior of neurons in a temporal framework to explain neurophysiological and psychophysical data. However, these networks are, at present, too complex for realistic application.
HTM is also related to work by Tomaso Poggio
, including an approach for modeling the ventral stream of the visual cortex known as HMAX. Similarities of HTM to various AI ideas are described in the December 2005 issue of the Artificial Intelligence journal.
Platform for Intelligent Computing" (NuPIC). Currently, the software is available as a free download and can be licensed for general or academic research as well as for developing commercial applications. NuPIC is written in C++
and Python
.
A number of HTM software development tools have been implemented using NuPIC:
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
model developed by Jeff Hawkins
Jeff Hawkins
Jeffrey Hawkins is the founder of Palm Computing and Handspring...
and Dileep George of Numenta, Inc.
Numenta
Numenta is a company founded March 24, 2005, by Palm founder Jeff Hawkins with his longtime business partner Donna Dubinsky and Stanford graduate student Dileep George. It is headquartered in Redwood City, California.-Origin:...
that models some of the structural and algorithm
Algorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
ic properties of the neocortex
Neocortex
The neocortex , also called the neopallium and isocortex , is a part of the brain of mammals. It is the outer layer of the cerebral hemispheres, and made up of six layers, labelled I to VI...
. HTM is a biomimetic
Bionics
Bionics is the application of biological methods and systems found in nature to the study and design of engineering systems and modern technology.The word bionic was coined by Jack E...
model based on the memory-prediction
Memory-prediction framework
The memory-prediction framework is a theory of brain function that was created by Jeff Hawkins and described in his 2004 book On Intelligence...
theory of brain function described by Jeff Hawkins
Jeff Hawkins
Jeffrey Hawkins is the founder of Palm Computing and Handspring...
in his book On Intelligence
On Intelligence
On Intelligence: How a New Understanding of the Brain will Lead to the Creation of Truly Intelligent Machines is a book by Palm Pilot-inventor Jeff Hawkins with New York Times science writer Sandra Blakeslee. The book explains Hawkins' memory-prediction framework theory of the brain and describes...
. HTM is a method for discovering and inferring the high-level causes of observed input patterns and sequences, thus building an increasingly complex model of the world.
Jeff Hawkins states that HTM does not present any new idea or theory, but combines existing ideas to mimic the neocortex with a simple design that provides a large range of capabilities. HTM combines and extends approaches used in Bayesian networks, spatial and temporal clustering algorithms, while using a tree-shaped hierarchy of nodes that is common in neural networks
Neural Networks
Neural Networks is the official journal of the three oldest societies dedicated to research in neural networks: International Neural Network Society, European Neural Network Society and Japanese Neural Network Society, published by Elsevier...
.
HTM structure and algorithms
A typical HTM network is a tree-shaped hierarchy of levels that are composed of smaller elements called nodes or columns. A single level in the hierarchy is also called a region. Higher hierarchy levels have fewer nodes, so the resolution in space is lost. Higher hierarchy levels reuse patterns learned at the lower levels by combining them to memorize more complex patterns.Each HTM node has the same basic functionality. In learning and inference modes, sensory data comes into the bottom level nodes. In generation mode, the bottom level nodes output the generated pattern of a given category. The top level usually has a single node that stores the most general categories (concepts) which determine, or are determined by, smaller concepts in the lower levels which are more restricted in time and space. In inference mode, a node in each level interprets information coming in from its child nodes in the lower level as probabilities of the categories it has in memory.
Each HTM region learns by identifying and memorizing spatial patterns - combinations of input bits that often occur at the same time. It then identifies temporal sequences of spatial patterns that are likely to occur one after another.
Zeta 1: first generation node algorithms
During training, a node receives a temporal sequence of spatial patterns as its input. The learning process consists of two stages:- Spatial pooling identifies frequently observed patterns and memorizes them as coincidences. Patterns that are significantly similar to each other are treated as the same coincidence. A large number of possible input patterns are reduced to a manageable number of known coincidences.
- Temporal pooling partitions coincidences that are likely to follow each other in the training sequence into temporal groups. Each group of patterns represents a "cause" of the input pattern (or "name" in On Intelligence).
During inference (recognition), the node calculates the set probabilities that a pattern belongs to each known coincidence. Then it calculates the probabilities that the input represents each temporal group. The set of probabilities assigned to the groups is called a node's "belief" about the input pattern. (In a simplified implementation, node's belief consists of only one winning group). This belief is the result of the inference that is passed to one or more "parent" nodes in the next higher level of the hierarchy.
"Unexpected" patterns to the node do not have a dominant probability of belonging to any one temporal group, but have nearly equal probabilities of belonging to several of the groups. If sequences of patterns are similar to the training sequences, then the assigned probabilities to the groups will not change as often as patterns are received. The output of the node will not change as much, and a resolution in time is lost.
In a more general scheme, the node's belief can be sent to the input of any node(s) in any level(s), but the connections between the nodes are still fixed. The higher-level node combines this output with the output from other child nodes thus forming its own input pattern.
Since resolution in space and time is lost in each node as described above, beliefs formed by higher-level nodes represent an even larger range of space and time. This is meant to reflect the organization of the physical world as it is perceived by human brain. Larger concepts (e.g. causes, actions and objects) are perceived to change more slowly and consist of smaller concepts that change more quickly. Jeff Hawkins postulates that brains evolved this type of hierarchy to match, predict, and affect the organization of the external world.
More details about the functioning of HTM can be found in Numenta's documentation.
Cortical learning algorithms
The new generation of HTM learning algorithms relies on fixed-sparsitySparse coding
The sparse code is a kind of neural code in which each item is encoded by the strong activation of a relatively small set of neurons. For each item to be encoded, this is a different subset of all available neurons....
distributed representations. It models cortical columns that tend to inhibit neighboring columns in the neocortex thus creating a sparse activation of columns. A region creates a sparse representation from its input, so that a fixed percentage of columns are active at any one time.
Each HTM region consists of a number of highly interconnected cortical column
Cortical column
A cortical column, also called hypercolumn or sometimes cortical module, is a group of neurons in the brain cortex which can be successively penetrated by a probe inserted perpendicularly to the cortical surface, and which have nearly identical receptive fields...
s. A region is similar to layer III of the neocortex
Neocortex
The neocortex , also called the neopallium and isocortex , is a part of the brain of mammals. It is the outer layer of the cerebral hemispheres, and made up of six layers, labelled I to VI...
. A cortical column is understood as a group of cells that have the same receptive field. Each column has a number of cells that able to remember several previous states. A cell can be in one of three states: active, inactive and predictive state.
Spatial pooling: The receptive field of each column is a fixed number of inputs that are randomly selected from a much larger number of node inputs. Based on the input pattern, some columns will receive more active input values. Spatial pooling selects a relatively constant number of the most active columns and inactivates (inhibits) other columns in the vicinity of the active ones. Similar input patterns tend to activate a stable set of columns. The amount of memory used by each region can be increased to learn more complex spatial patterns or decreased to learn simpler patterns.
Representing the input in the context of previous inputs: If one or more cells in the active column are in the predictive state (see below), they will be the only cells to become active in the current time step. If none of the cells in the active column are in the predictive state (during the initial time step or when the activation of this column was not expected), all cells are made active.
Predicting future inputs and temporal pooling: When a cell becomes active, it gradually forms connections to nearby cells that tend to be active during several previous time steps. Thus a cell learns to recognize a known sequence by checking whether the connected cells are active. If a large number of connected cells are active, this cell switches to the predictive state in anticipation of one of the few next inputs of the sequence. The output of a region includes columns in both active and predictive states. Thus columns are active over longer periods of time, which leads to greater temporal stability seen by the parent region.
Cortical learning algorithms are able to learn continuously from each new input pattern, therefore no separate inference mode is necessary. During inference, HTM tries to match the stream of inputs to fragments of previously learned sequences. This allows each HTM region to be constantly predicting the likely continuation of the recognized sequences. The index of the predicted sequence is the output of the region. Since predictions tend to change less frequently than the input patterns, this leads to increasing temporal stability of the output in higher hierarchy levels. Prediction also helps to fill in missing patterns in the sequence and to interpret ambiguous data by biasing the system to infer what it predicted.
Cortical learning algorithms are currently being implemented and are planned to be released in the middle of 2011.
Comparing HTM and neocortex
Comparing high-level structures and functionality of neocortex with HTM is most appropriate. HTM attempts to implement the functionality that is characteristic of a hierarchically related group of cortical regions in the neocortex. A region of the neocortex corresponds to one or more levels in the HTM hierarchy, while the hippocampusHippocampus
The hippocampus is a major component of the brains of humans and other vertebrates. It belongs to the limbic system and plays important roles in the consolidation of information from short-term memory to long-term memory and spatial navigation. Humans and other mammals have two hippocampi, one in...
is remotely similar to the highest HTM level. A single HTM node may represent a group of cortical column
Cortical column
A cortical column, also called hypercolumn or sometimes cortical module, is a group of neurons in the brain cortex which can be successively penetrated by a probe inserted perpendicularly to the cortical surface, and which have nearly identical receptive fields...
s within a certain region.
Although it is primarily a functional model, several attempts have been made to relate the algorithms of the HTM with the structure of neuronal connections in the layers of neocortex. The neocortex is organized in vertical columns of 6 horizontal layers. The 6 layers of cells in the neocortex should not be confused with levels in an HTM hierarchy. In Jeff Hawkins' theory of the neocortex, the output from a number of layer 1 neurons is analogous to the output of a node. Some layer 4, 5, 6 neurons are analogous to the input to a node. Neurons in layers 2 and 3 are believed to be where the "concepts" of the neocortex reside.
Bayesian networks
Likened to a Bayesian networkBayesian network
A Bayesian network, Bayes network, belief network or directed acyclic graphical model is a probabilistic graphical model that represents a set of random variables and their conditional dependencies via a directed acyclic graph . For example, a Bayesian network could represent the probabilistic...
, an HTM comprises a collection of nodes that are arranged in a tree-shaped hierarchy. Each node in the hierarchy discovers an array of causes in the input patterns and temporal sequences it receives. A Bayesian belief revision
Belief revision
Belief revision is the process of changing beliefs to take into account a new piece of information. The logical formalization of belief revision is researched in philosophy, in databases, and in artificial intelligence for the design of rational agents....
algorithm is used to propagate feed-forward and feedback beliefs from child to parent nodes and vice versa. However, the analogy to Bayesian networks is limited, because HTMs can be self-trained (such that each node has an unambiguous family relationship), cope with time-sensitive data, and grant mechanisms for covert attention.
A theory of hierarchical cortical computation based on Bayesian belief propagation
Belief propagation
Belief propagation is a message passing algorithm for performing inference on graphical models, such as Bayesian networks and Markov random fields. It calculates the marginal distribution for each unobserved node, conditional on any observed nodes...
was proposed earlier by Tai Sing Lee and David Mumford
David Mumford
David Bryant Mumford is an American mathematician known for distinguished work in algebraic geometry, and then for research into vision and pattern theory. He won the Fields Medal and was a MacArthur Fellow. In 2010 he was awarded the National Medal of Science...
. While HTM is mostly consistent with these ideas, it adds details about handling invariant representations in the visual cortex.
Neural networks
Like any system that models details of the neocortex, HTM can be viewed as an artificial neural networkArtificial neural network
An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
. The tree-shaped hierarchy commonly used in HTMs resembles the usual topology of traditional neural networks. However, unlike most types of neural networks, HTM does not attempt to simulate individual neurons or their connections, but rather models the large-scale hierarchy of cortical regions. HTM is an unconventional type of neural network which uses abstract neural modules that represent complete subsystems of the brain.
LAMINART and similar neural networks researched by Stephen Grossberg attempt to model both the infrastructure of the cortex and the behavior of neurons in a temporal framework to explain neurophysiological and psychophysical data. However, these networks are, at present, too complex for realistic application.
HTM is also related to work by Tomaso Poggio
Tomaso Poggio
Tomaso Armando Poggio, born in Genoa, Italy, is the Eugene McDermott Professor in the Department of Brain and Cognitive Sciences, an investigator at the McGovern Institute for Brain Research, a member of the MIT Computer Science and Artificial Intelligence Laboratory and the director of The Center...
, including an approach for modeling the ventral stream of the visual cortex known as HMAX. Similarities of HTM to various AI ideas are described in the December 2005 issue of the Artificial Intelligence journal.
NuPIC platform and development tools
The HTM model has been implemented in a research release of a software API called "NumentaNumenta
Numenta is a company founded March 24, 2005, by Palm founder Jeff Hawkins with his longtime business partner Donna Dubinsky and Stanford graduate student Dileep George. It is headquartered in Redwood City, California.-Origin:...
Platform for Intelligent Computing" (NuPIC). Currently, the software is available as a free download and can be licensed for general or academic research as well as for developing commercial applications. NuPIC is written in C++
C++
C++ is a statically typed, free-form, multi-paradigm, compiled, general-purpose programming language. It is regarded as an intermediate-level language, as it comprises a combination of both high-level and low-level language features. It was developed by Bjarne Stroustrup starting in 1979 at Bell...
and Python
Python (programming language)
Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability. Python claims to "[combine] remarkable power with very clear syntax", and its standard library is large and comprehensive...
.
A number of HTM software development tools have been implemented using NuPIC:
- Numenta Vision Toolkit - allows to create a customized image recognition system. It assists in collecting and preparing images for training, trains the HTM network and recognizes new images. Vision Toolkit can also optimize network training parameters by selecting one of the predefined network configurations that were found to work well with certain image types.
- Vitamin D Toolkit (discontinued) - provides a set of visual tools to inspect network configuration, find recognition problems and fine-tune network parameters.
- Numenta Prediction Toolkit (future) - is planned to include tools for simple development of general-purpose HTM networks.
Applications
The following commercial applications have been developed using NuPIC:- Vitamin D Video - a video surveillance application that uses HTM to detect people in video by differentiating them from other moving objects.
- EDSA power analytics system - an electrical power analytics, supervision and diagnostic system scheduled to be deployed in an oil field in the North Sea. It uses HTM to learn and distinguish between “routine” and “non-routine” events in an electrical power network. The system alerts an operator when a situation is not normal.
- Lockheed MartinLockheed MartinLockheed Martin is an American global aerospace, defense, security, and advanced technology company with worldwide interests. It was formed by the merger of Lockheed Corporation with Martin Marietta in March 1995. It is headquartered in Bethesda, Maryland, in the Washington Metropolitan Area....
has been using and modifying HTM technology for several applications such as integrating multiple types of sensory inputs and object recognition from geospatial imagery of an urban environment. - iResemble - an iPhone application implemented using the Vision Toolkit. It has a trained HTM network that classifies a submitted photo and outputs a belief of what type of person the photo resembles.
See also
- On IntelligenceOn IntelligenceOn Intelligence: How a New Understanding of the Brain will Lead to the Creation of Truly Intelligent Machines is a book by Palm Pilot-inventor Jeff Hawkins with New York Times science writer Sandra Blakeslee. The book explains Hawkins' memory-prediction framework theory of the brain and describes...
- Memory-prediction frameworkMemory-prediction frameworkThe memory-prediction framework is a theory of brain function that was created by Jeff Hawkins and described in his 2004 book On Intelligence...
- Belief revisionBelief revisionBelief revision is the process of changing beliefs to take into account a new piece of information. The logical formalization of belief revision is researched in philosophy, in databases, and in artificial intelligence for the design of rational agents....
- Belief propagationBelief propagationBelief propagation is a message passing algorithm for performing inference on graphical models, such as Bayesian networks and Markov random fields. It calculates the marginal distribution for each unobserved node, conditional on any observed nodes...
- BionicsBionicsBionics is the application of biological methods and systems found in nature to the study and design of engineering systems and modern technology.The word bionic was coined by Jack E...
- List of artificial intelligence projects
Related models
- Hierarchical hidden Markov modelHierarchical hidden Markov modelThe hierarchical hidden Markov model is a statistical model derived from the hidden Markov model . In an HHMM each state is considered to be a self-contained probabilistic model. More precisely each state of the HHMM is itself an HHMM....
- Bayesian networks
- Neural networksNeural NetworksNeural Networks is the official journal of the three oldest societies dedicated to research in neural networks: International Neural Network Society, European Neural Network Society and Japanese Neural Network Society, published by Elsevier...
Official
- Numenta, Inc.
- OnIntelligence.org Forum, an Internet forumInternet forumAn Internet forum, or message board, is an online discussion site where people can hold conversations in the form of posted messages. They differ from chat rooms in that messages are at least temporarily archived...
for the discussion of relevant topics, especially relevant being the Models and Simulation Topics forum. - Hierarchical Temporal Memory (Microsoft PowerPoint presentation)
- Hierarchical Temporal Memory: Theory and Implementation (Google Video)
Other
- The Gartner Fellows: Jeff Hawkins Interview by Tom Austin, GartnerGartnerGartner, Inc. is an information technology research and advisory firm headquartered in Stamford, Connecticut, United States. It was known as GartnerGroup until 2001....
, March 2, 2006 - Emerging Tech: Jeff Hawkins reinvents artificial intelligence by Debra D'Agostino and Edward H. Baker, CIO Insight, May 1, 2006
- "Putting your brain on a microchip" by Stefanie Olsen, CNET News.comCNETCNET is a tech media website that publishes news articles, blogs, and podcasts on technology and consumer electronics. Originally founded in 1994 by Halsey Minor and Shelby Bonnie, it was the flagship brand of CNET Networks and became a brand of CBS Interactive through CNET Networks' acquisition...
, May 12, 2006 - "The Thinking Machine" by Evan Ratliff, WiredWired (magazine)Wired is a full-color monthly American magazine and on-line periodical, published since January 1993, that reports on how new and developing technology affects culture, the economy, and politics...
, March 2007 - Think like a human by Jeff Hawkins , IEEE SpectrumIEEE SpectrumIEEE Spectrum is a magazine edited by the Institute of Electrical and Electronics Engineers. The IEEE's description of it is:IEEE Spectrum began publishing in January 1964 as a successor to Electrical Engineering...
, April 2007 - Neocortex - Memory-Prediction Framework — Open SourceOpen sourceThe term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
Implementation with GNU General Public LicenseGNU General Public LicenseThe GNU General Public License is the most widely used free software license, originally written by Richard Stallman for the GNU Project.... - Hierarchical Temporal Memory related Papers and Books