Leabra
Encyclopedia
Leabra stands for "Local, Error-driven and Associative, Biologically Realistic Algorithm". It is a model
of learning
which is a balance between Hebbian and error-driven learning
with other network
-derived characteristics. This model is used to mathematically predict outcomes based on inputs and previous learning influences. This model is heavily influenced by and contributes to neural network designs and models.
This algorithm is the default algorithm in Emergent (successor of PDP++) when making a new project, and is extensively used in various simulations.
Hebbian learning is performed using conditional principal components analysis (CPCA) algorithm with correction factor for sparse expected activity levels.
Error-driven learning
is performed using GeneRec
, which is a generalization of the Recirculation algorithm, and approximates Almeida-Pineda recurrent backpropagation
. The symmetric, midpoint version of GeneRec is used, which is equivalent to the contrastive Hebbian learning algorithm (CHL). See O'Reilly (1996; Neural Computation) for more details.
The activation function is a point-neuron approximation with both discrete spiking
and continuous rate-code output.
Layer or unit-group level inhibition can be computed directly using a k-winners-take-all (KWTA) function, producing sparse distributed representations.
The net input is computed as an average, not a sum, over connections, based on normalized, sigmoidaly transformed weight values, which are subject to scaling on a connection-group level to alter relative contributions. Automatic scaling is performed to compensate for differences in expected activity level in the different projections.
Documentation about this algorithm can be found in the book "Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain" published by MIT press. and in the Emergent Documentation
pieces of the algorithm described in more detail in the subsequent
sections fit together.
Computer simulation
A computer simulation, a computer model, or a computational model is a computer program, or network of computers, that attempts to simulate an abstract model of a particular system...
of learning
Learning
Learning is acquiring new or modifying existing knowledge, behaviors, skills, values, or preferences and may involve synthesizing different types of information. The ability to learn is possessed by humans, animals and some machines. Progress over time tends to follow learning curves.Human learning...
which is a balance between Hebbian and error-driven learning
Error-driven learning
Error-driven learning is a sub-area of machine learning concerned with how an agent ought to take actions in an environment so as to minimize some error feedback. It is a type of reinforcement learning....
with other network
Neural network
The term neural network was traditionally used to refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes...
-derived characteristics. This model is used to mathematically predict outcomes based on inputs and previous learning influences. This model is heavily influenced by and contributes to neural network designs and models.
This algorithm is the default algorithm in Emergent (successor of PDP++) when making a new project, and is extensively used in various simulations.
Hebbian learning is performed using conditional principal components analysis (CPCA) algorithm with correction factor for sparse expected activity levels.
Error-driven learning
Error-driven learning
Error-driven learning is a sub-area of machine learning concerned with how an agent ought to take actions in an environment so as to minimize some error feedback. It is a type of reinforcement learning....
is performed using GeneRec
GeneRec
GeneRec is a generalization of the Recirculation algorithm, and approximates Almeida-Pineda recurrent backpropagation. It is used as part of the Leabra algorithm for error-driven learning....
, which is a generalization of the Recirculation algorithm, and approximates Almeida-Pineda recurrent backpropagation
Almeida-Pineda recurrent backpropagation
Almeida–Pineda recurrent backpropagation is an extension to the backpropagation algorithm that is applicable to recurrent neural networks. It is a type of supervised learning....
. The symmetric, midpoint version of GeneRec is used, which is equivalent to the contrastive Hebbian learning algorithm (CHL). See O'Reilly (1996; Neural Computation) for more details.
The activation function is a point-neuron approximation with both discrete spiking
Spiking neural network
Spiking neural networks fall into the third generation of neural network models, increasing the level of realism in a neural simulation. In addition to neuronal and synaptic state, SNNs also incorporate the concept of time into their operating model...
and continuous rate-code output.
Layer or unit-group level inhibition can be computed directly using a k-winners-take-all (KWTA) function, producing sparse distributed representations.
The net input is computed as an average, not a sum, over connections, based on normalized, sigmoidaly transformed weight values, which are subject to scaling on a connection-group level to alter relative contributions. Automatic scaling is performed to compensate for differences in expected activity level in the different projections.
Documentation about this algorithm can be found in the book "Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain" published by MIT press. and in the Emergent Documentation
Overview of the Leabra Algorithm
The pseudocode for Leabra is given here, showing exactly how thepieces of the algorithm described in more detail in the subsequent
sections fit together.
Iterate over minus and plus phases of settling for each event.
o At start of settling, for all units:
- Initialize all state variables (activation, v_m, etc).
- Apply external patterns (clamp input in minus, input & output in
plus).
- Compute net input scaling terms (constants, computed
here so network can be dynamically altered).
- Optimization: compute net input once from all static activations
(e.g., hard-clamped external inputs).
o During each cycle of settling, for all non-clamped units:
- Compute excitatory netinput (g_e(t), aka eta_j or net)
-- sender-based optimization by ignoring inactives.
- Compute kWTA inhibition for each layer, based on g_i^Q:
* Sort units into two groups based on g_i^Q: top k and
remaining k+1 -> n.
* If basic, find k and k+1th highest
If avg-based, compute avg of 1 -> k & k+1 -> n.
* Set inhibitory conductance g_i from g^Q_k and g^Q_k+1
- Compute point-neuron activation combining excitatory input and
inhibition
o After settling, for all units, record final settling activations
as either minus or plus phase (y^-_j or y^+_j).
After both phases update the weights (based on linear current
weight values), for all connections:
o Compute error-drivenError-driven learningError-driven learning is a sub-area of machine learning concerned with how an agent ought to take actions in an environment so as to minimize some error feedback. It is a type of reinforcement learning....
weight changes with CHL with soft weight bounding
o Compute Hebbian weight changes with CPCA from plus-phase activations
o Compute net weight change as weighted sum of error-driven and Hebbian
o Increment the weights according to net weight change.
Special algorithms
- Temporal Differences and General Da (dopamine) Modulation. Temporal differences (TD)Temporal difference learningTemporal difference learning is a prediction method. It has been mostly used for solving the reinforcement learning problem. "TD learning is a combination of Monte Carlo ideas and dynamic programming ideas." TD resembles a Monte Carlo method because it learns by sampling the environment according...
is widely used as a modelComputer simulationA computer simulation, a computer model, or a computational model is a computer program, or network of computers, that attempts to simulate an abstract model of a particular system...
of midbrain dopaminergicDopaminergicDopaminergic means related to the neurotransmitter dopamine. For example, certain proteins such as the dopamine transporter , vesicular monoamine transporter 2 , and dopamine receptors can be classified as dopaminergic, and neurons which synthesize or contain dopamine and synapses with dopamine...
firing. - Primary value learned value (PVLV). PVLVPVLVThe primary value learned value model is a possible explanation for the reward-predictive firing properties of dopamine neurons. It simulates behavioral and neural data on Pavlovian conditioning and the midbrain dopaminergic neurons that fire in proportion to unexpected rewards. It is an...
simulates behavioral and neural data on Pavlovian conditioningClassical conditioningClassical conditioning is a form of conditioning that was first demonstrated by Ivan Pavlov...
and the midbrain dopaminergicDopaminergicDopaminergic means related to the neurotransmitter dopamine. For example, certain proteins such as the dopamine transporter , vesicular monoamine transporter 2 , and dopamine receptors can be classified as dopaminergic, and neurons which synthesize or contain dopamine and synapses with dopamine...
neurons that fire in proportion to unexpected rewards (an alternative to TDTemporal difference learningTemporal difference learning is a prediction method. It has been mostly used for solving the reinforcement learning problem. "TD learning is a combination of Monte Carlo ideas and dynamic programming ideas." TD resembles a Monte Carlo method because it learns by sampling the environment according...
). - Prefrontal Cortex Basal Ganglia Working Memory (PBWM). PBWM uses PVLVPVLVThe primary value learned value model is a possible explanation for the reward-predictive firing properties of dopamine neurons. It simulates behavioral and neural data on Pavlovian conditioning and the midbrain dopaminergic neurons that fire in proportion to unexpected rewards. It is an...
to train Prefrontal cortexPrefrontal cortexThe prefrontal cortex is the anterior part of the frontal lobes of the brain, lying in front of the motor and premotor areas.This brain region has been implicated in planning complex cognitive behaviors, personality expression, decision making and moderating correct social behavior...
working memoryWorking memoryWorking memory has been defined as the system which actively holds information in the mind to do verbal and nonverbal tasks such as reasoning and comprehension, and to make it available for further information processing...
updating system, based on the biology of the prefrontal cortex and basal gangliaBasal gangliaThe basal ganglia are a group of nuclei of varied origin in the brains of vertebrates that act as a cohesive functional unit. They are situated at the base of the forebrain and are strongly connected with the cerebral cortex, thalamus and other brain areas...
.
Links
- Emergent about Leabra
- PDP++ about Leabra
- O'Reilly, R.C. (1996). The Leabra Model of Neural Interactions and Learning in the Neocortex. Phd Thesis, Carnegie Mellon University, Pittsburgh, PA [ftp://grey.colorado.edu/pub/oreilly/thesis/oreilly_thesis.all.pdf PDF]