BELBIC
Encyclopedia
In recent years, the use of biologically inspired methods such as the evolutionary algorithm
have been increasingly employed to solve and analyze complex computational problems. BELBIC (Brain Emotional Learning Based Intelligent Controller) is one such controller
which adopts the network model developed by Moren and Balkenius to mimic those parts of the brain which are known to produce emotion (namely, the amygdala
, orbitofrontal cortex
, thalamus
and sensory input cortex).
and emotion
. However these phenomena can not be separated. Motivation is the drive that causes any system to do anything – without it, there is no reason to act. Emotions indicate how successful a course of actions have been and whether another set of actions should have been taken instead - they are a constant feedback to the learning system. Learning on the other hand, guarantees that motivation and emotional subsystems are able to adapt to constantly changing conditions.
Thus, in the study of biological organisms, emotions have arisen to prominence as an integral part of any biologically inspired system. But how does any living organism benefit from its emotions? It is crucial to answer this question as we attempt to increasingly employ biologically inspired methods in solving computational problems.
Every creature has innate abilities that accommodate its survival in the world. It can identify food, shelter, partners, and danger. But these “simple mappings between stimuli and reactions will not be enough to keep the organisms from encountering problems.” For example, if a given animal knows that its predator has qualities A, B and C, it will escape all creatures that have those qualities. And thus waste much of its energy and resources on non-existent danger.
We can not expect evolution to provide more advanced algorithms for assessing danger, because the predator is also evolving at the same speed. Thus, biological systems need to be equipped with the ability to learn. This learning and re-learning mechanism allows them to adapt to highly complex and advanced situations.
To learn effectively, every learning organism needs an evaluation of the current situation and also feedback on how beneficial the results of learning were. On the most part, these evaluation mechanisms are built-in. And so we encounter a new problem: whereas creatures take appropriate measures in real time based on their evaluations, these built-in evaluation procedures are developed in evolutionary time. But all creatures need to learn of new evaluation techniques in their lifetime just as they learn the proper reactions.
This is where the ability to condition emotional reactions comes into play. Biological organisms associate innate emotional stimuli with other stimuli they encounter in the world and thus give them an emotional significance when needed. These evaluations can be monitored to operate at very specific times, specific places or when accompanied by other specific stimuli.
There is another reason why these observations are so significant and that is the creation of artificial systems. These systems do not evolve over time but are designed with certain abilities from the start. Thus, their adaptability must be built-in.
A model
is a simplified description of a phenomenon. It brings to life some aspects of this phenomenon while overlooking others. What aspects are kept in the model and what are overlooked greatly depends on the topic of study. Thus, the nature of a model depends on the purpose the investigator plans to carry out. A computational model is one which can be mathematically analyzed, tested and simulated using computer systems.
To construct a computational model of emotional learning in the brain requires a thorough analysis of the amygdala
and the orbitofrontal cortex
and the interaction between them:
In mammals, emotional responses are processed in a part of the brain called the limbic system
which lies in the cerebral cortex
. The main components of the limbic system are the amygdala
, orbitofrontal cortex
, thalamus
and the sensory cortex.
The amygdala is an almond shaped area which is placed such that it can communicate with all other cortices within the limbic system. The primary affective conditioning of the system occurs within the amygdala. That is, the association between a stimulus and its emotional consequence takes place in this region.
It has been suggested that learning takes place in two fundamental steps. First, a particular stimulus is correlated with an emotional response. This stimulus can be an endless number of phenomena from observing a face, to detecting a scent, hearing a noise, etc. Second, this emotional consequence shapes an association between the stimulus and the response. This analysis is quite influential in part because it was one of the first to suggest that emotions play a key part in learning. In more recent studies, it has been shown that the association between a stimulus and its emotional consequence take place in the amygdala. “In this region, highly analyzed stimulus representations in the cortex are associated with an emotional value. Therefore, emotions are properties of stimuli”.
The task of the amygdala is thus to assign a primary emotional value to each stimulus that has been paired with a primary reinforcer - the reinforcer is the reward and punishment that the mammal receives. This task is aided by the orbitofrontal complex. “In terms of learning theory, the amygdala appears to handle the presentation of primary reinforcement, while the orbitofrontal cortex is involved in the detection of omission of reinforcement.”
The first thing we notice in the computational model developed by Moren and Balkenius is that quite a number of interacting learning systems exist in the brain that deal with emotional learning. The computational model is presented below where:
This image shows that the sensory input enters through the thalamus TH. In biological systems, the thalamus takes on the task of initiating the process of a response to stimuli. It does so by passing the signal to the amygdala and the sensory cortex.
This signal is then analyzed in the cortical area – CX. In biological systems, the sensory cortex operates by distributing the incoming signals appropriately between the amygdala and the orbitofrontal cortex. This sensory representation in CX is then sent to the amygdala A, through the pathway V.
This is the main pathway for learning in this model. Reward and punishment enter the amygdala to strengthen the connection between the amygdala and the pathway. At a later stage if a similar representation is activated in the cortex, E becomes activated and produces an emotional response.
O, the orbitofrontal cortex, operates based on the difference between the perceived (i.e. expected) reward/punishment and the actual received reward/punishment. This perceived reward/punishment is the one that has been developed in the brain over time using learning mechanisms and it reaches the orbitofrontal cortex via the sensory cortex and the amygdala. The received reward/punishment on the other hand, comes courtesy of the outside world and is the actual reward/punishment that the specie has just obtained. If these two are identical, the output is the same as always through E. If not, the orbitofronal cortex inhibits and restrains emotional response to make way for further learning. So the path W is only activated in such conditions.
models. One reason is that these linear models are developed using straightforward methods from process test data.
However, if the process is highly complex and nonlinear, subject to frequent disturbances, a nonlinear model will be required. Biologically motivated intelligent controllers have been increasingly employed in these situations. Amongst them, fuzzy logic
, neural networks
and genetic algorithms are some of the most widely employed tools in control applications with highly complex, nonlinear settings.
BELBIC is one such nonlinear controller
– a neuromorphic
controller based on the computational learning model shown above to produce the control action. This model is employed much like an algorithm in these control engineering applications. In these new approaches, intelligence is not given to the system from the outside but is actually acquired by the system itself.
This simple model has been employed as a feedback
controller
to be applied to control design problems. One logic behind this use in control engineering is a belief held by many experts in the field that there has been too much focus on fully rational deliberative approaches, whereas in many real world circumstances, we are only provided with a bounded rationality. Factors like computational complexity, multiplicity of objectives and prevalence of uncertainty lead to a desire to obtain more ad-hoc, rule-of-thumb approaches. Emotional decision making is highly capable of addressing these issues because it is neither fully cognitive nor fully behavioral.
Evolutionary algorithm
In artificial intelligence, an evolutionary algorithm is a subset of evolutionary computation, a generic population-based metaheuristic optimization algorithm. An EA uses some mechanisms inspired by biological evolution: reproduction, mutation, recombination, and selection...
have been increasingly employed to solve and analyze complex computational problems. BELBIC (Brain Emotional Learning Based Intelligent Controller) is one such controller
Controller (control theory)
In control theory, a controller is a device which monitors and affects the operational conditions of a given dynamical system. The operational conditions are typically referred to as output variables of the system which can be affected by adjusting certain input variables...
which adopts the network model developed by Moren and Balkenius to mimic those parts of the brain which are known to produce emotion (namely, the amygdala
Amygdala
The ' are almond-shaped groups of nuclei located deep within the medial temporal lobes of the brain in complex vertebrates, including humans. Shown in research to perform a primary role in the processing and memory of emotional reactions, the amygdalae are considered part of the limbic system.-...
, orbitofrontal cortex
Orbitofrontal cortex
The orbitofrontal cortex is a prefrontal cortex region in the frontal lobes in the brain which is involved in the cognitive processing of decision-making...
, thalamus
Thalamus
The thalamus is a midline paired symmetrical structure within the brains of vertebrates, including humans. It is situated between the cerebral cortex and midbrain, both in terms of location and neurological connections...
and sensory input cortex).
Emotions and Learning
Traditionally, the study of learning in biological systems was conducted at the expense of overlooking its lesser known counterparts: motivationMotivation
Motivation is the driving force by which humans achieve their goals. Motivation is said to be intrinsic or extrinsic. The term is generally used for humans but it can also be used to describe the causes for animal behavior as well. This article refers to human motivation...
and emotion
Emotion
Emotion is a complex psychophysiological experience of an individual's state of mind as interacting with biochemical and environmental influences. In humans, emotion fundamentally involves "physiological arousal, expressive behaviors, and conscious experience." Emotion is associated with mood,...
. However these phenomena can not be separated. Motivation is the drive that causes any system to do anything – without it, there is no reason to act. Emotions indicate how successful a course of actions have been and whether another set of actions should have been taken instead - they are a constant feedback to the learning system. Learning on the other hand, guarantees that motivation and emotional subsystems are able to adapt to constantly changing conditions.
Thus, in the study of biological organisms, emotions have arisen to prominence as an integral part of any biologically inspired system. But how does any living organism benefit from its emotions? It is crucial to answer this question as we attempt to increasingly employ biologically inspired methods in solving computational problems.
Every creature has innate abilities that accommodate its survival in the world. It can identify food, shelter, partners, and danger. But these “simple mappings between stimuli and reactions will not be enough to keep the organisms from encountering problems.” For example, if a given animal knows that its predator has qualities A, B and C, it will escape all creatures that have those qualities. And thus waste much of its energy and resources on non-existent danger.
We can not expect evolution to provide more advanced algorithms for assessing danger, because the predator is also evolving at the same speed. Thus, biological systems need to be equipped with the ability to learn. This learning and re-learning mechanism allows them to adapt to highly complex and advanced situations.
This is where the ability to condition emotional reactions comes into play. Biological organisms associate innate emotional stimuli with other stimuli they encounter in the world and thus give them an emotional significance when needed. These evaluations can be monitored to operate at very specific times, specific places or when accompanied by other specific stimuli.
There is another reason why these observations are so significant and that is the creation of artificial systems. These systems do not evolve over time but are designed with certain abilities from the start. Thus, their adaptability must be built-in.
A Computational Model of Emotional Conditioning
Scientific modelling
Scientific modelling is the process of generating abstract, conceptual, graphical and/or mathematical models. Science offers a growing collection of methods, techniques and theory about all kinds of specialized scientific modelling...
is a simplified description of a phenomenon. It brings to life some aspects of this phenomenon while overlooking others. What aspects are kept in the model and what are overlooked greatly depends on the topic of study. Thus, the nature of a model depends on the purpose the investigator plans to carry out. A computational model is one which can be mathematically analyzed, tested and simulated using computer systems.
To construct a computational model of emotional learning in the brain requires a thorough analysis of the amygdala
Amygdala
The ' are almond-shaped groups of nuclei located deep within the medial temporal lobes of the brain in complex vertebrates, including humans. Shown in research to perform a primary role in the processing and memory of emotional reactions, the amygdalae are considered part of the limbic system.-...
and the orbitofrontal cortex
Orbitofrontal cortex
The orbitofrontal cortex is a prefrontal cortex region in the frontal lobes in the brain which is involved in the cognitive processing of decision-making...
and the interaction between them:
In mammals, emotional responses are processed in a part of the brain called the limbic system
Limbic system
The limbic system is a set of brain structures including the hippocampus, amygdala, anterior thalamic nuclei, septum, limbic cortex and fornix, which seemingly support a variety of functions including emotion, behavior, long term memory, and olfaction. The term "limbic" comes from the Latin...
which lies in the cerebral cortex
Cerebral cortex
The cerebral cortex is a sheet of neural tissue that is outermost to the cerebrum of the mammalian brain. It plays a key role in memory, attention, perceptual awareness, thought, language, and consciousness. It is constituted of up to six horizontal layers, each of which has a different...
. The main components of the limbic system are the amygdala
Amygdala
The ' are almond-shaped groups of nuclei located deep within the medial temporal lobes of the brain in complex vertebrates, including humans. Shown in research to perform a primary role in the processing and memory of emotional reactions, the amygdalae are considered part of the limbic system.-...
, orbitofrontal cortex
Orbitofrontal cortex
The orbitofrontal cortex is a prefrontal cortex region in the frontal lobes in the brain which is involved in the cognitive processing of decision-making...
, thalamus
Thalamus
The thalamus is a midline paired symmetrical structure within the brains of vertebrates, including humans. It is situated between the cerebral cortex and midbrain, both in terms of location and neurological connections...
and the sensory cortex.
The amygdala is an almond shaped area which is placed such that it can communicate with all other cortices within the limbic system. The primary affective conditioning of the system occurs within the amygdala. That is, the association between a stimulus and its emotional consequence takes place in this region.
It has been suggested that learning takes place in two fundamental steps. First, a particular stimulus is correlated with an emotional response. This stimulus can be an endless number of phenomena from observing a face, to detecting a scent, hearing a noise, etc. Second, this emotional consequence shapes an association between the stimulus and the response. This analysis is quite influential in part because it was one of the first to suggest that emotions play a key part in learning. In more recent studies, it has been shown that the association between a stimulus and its emotional consequence take place in the amygdala. “In this region, highly analyzed stimulus representations in the cortex are associated with an emotional value. Therefore, emotions are properties of stimuli”.
The task of the amygdala is thus to assign a primary emotional value to each stimulus that has been paired with a primary reinforcer - the reinforcer is the reward and punishment that the mammal receives. This task is aided by the orbitofrontal complex. “In terms of learning theory, the amygdala appears to handle the presentation of primary reinforcement, while the orbitofrontal cortex is involved in the detection of omission of reinforcement.”
The first thing we notice in the computational model developed by Moren and Balkenius is that quite a number of interacting learning systems exist in the brain that deal with emotional learning. The computational model is presented below where:
- Th : Thalamus
- CX : Sensory Cortex
- A : Input structures in the amygdala
- E : Output structures in the amygdala
- O : Orbitofrontal Cortex
- Rew/Pun : External signals identifying the presentation of reward and punishment
- CR/UR : conditioned response/unconditioned response
- V : Associative strength from cortical representation to the amygdala that is changed by learning
- W : Inhibitory connection from orbitofrontal cortex to the amygdala that is changed during learning
This image shows that the sensory input enters through the thalamus TH. In biological systems, the thalamus takes on the task of initiating the process of a response to stimuli. It does so by passing the signal to the amygdala and the sensory cortex.
This signal is then analyzed in the cortical area – CX. In biological systems, the sensory cortex operates by distributing the incoming signals appropriately between the amygdala and the orbitofrontal cortex. This sensory representation in CX is then sent to the amygdala A, through the pathway V.
This is the main pathway for learning in this model. Reward and punishment enter the amygdala to strengthen the connection between the amygdala and the pathway. At a later stage if a similar representation is activated in the cortex, E becomes activated and produces an emotional response.
O, the orbitofrontal cortex, operates based on the difference between the perceived (i.e. expected) reward/punishment and the actual received reward/punishment. This perceived reward/punishment is the one that has been developed in the brain over time using learning mechanisms and it reaches the orbitofrontal cortex via the sensory cortex and the amygdala. The received reward/punishment on the other hand, comes courtesy of the outside world and is the actual reward/punishment that the specie has just obtained. If these two are identical, the output is the same as always through E. If not, the orbitofronal cortex inhibits and restrains emotional response to make way for further learning. So the path W is only activated in such conditions.
The Controller
In most industrial processes that contain complex nonlinearities, control algorithms are used to create linearizedLinearization
In mathematics and its applications, linearization refers to finding the linear approximation to a function at a given point. In the study of dynamical systems, linearization is a method for assessing the local stability of an equilibrium point of a system of nonlinear differential equations or...
models. One reason is that these linear models are developed using straightforward methods from process test data.
However, if the process is highly complex and nonlinear, subject to frequent disturbances, a nonlinear model will be required. Biologically motivated intelligent controllers have been increasingly employed in these situations. Amongst them, fuzzy logic
Fuzzy logic
Fuzzy logic is a form of many-valued logic; it deals with reasoning that is approximate rather than fixed and exact. In contrast with traditional logic theory, where binary sets have two-valued logic: true or false, fuzzy logic variables may have a truth value that ranges in degree between 0 and 1...
, neural networks
Neural Networks
Neural Networks is the official journal of the three oldest societies dedicated to research in neural networks: International Neural Network Society, European Neural Network Society and Japanese Neural Network Society, published by Elsevier...
and genetic algorithms are some of the most widely employed tools in control applications with highly complex, nonlinear settings.
BELBIC is one such nonlinear controller
Controller (control theory)
In control theory, a controller is a device which monitors and affects the operational conditions of a given dynamical system. The operational conditions are typically referred to as output variables of the system which can be affected by adjusting certain input variables...
– a neuromorphic
Neuromorphic
Neuromorphic engineering or neuromorphic computing is a concept developed by Carver Mead, in the late 1980s, describing the use of very-large-scale integration systems containing electronic analog circuits to mimic neuro-biological architectures present in the nervous system...
controller based on the computational learning model shown above to produce the control action. This model is employed much like an algorithm in these control engineering applications. In these new approaches, intelligence is not given to the system from the outside but is actually acquired by the system itself.
This simple model has been employed as a feedback
Feedback
Feedback describes the situation when output from an event or phenomenon in the past will influence an occurrence or occurrences of the same Feedback describes the situation when output from (or information about the result of) an event or phenomenon in the past will influence an occurrence or...
controller
Controller (control theory)
In control theory, a controller is a device which monitors and affects the operational conditions of a given dynamical system. The operational conditions are typically referred to as output variables of the system which can be affected by adjusting certain input variables...
to be applied to control design problems. One logic behind this use in control engineering is a belief held by many experts in the field that there has been too much focus on fully rational deliberative approaches, whereas in many real world circumstances, we are only provided with a bounded rationality. Factors like computational complexity, multiplicity of objectives and prevalence of uncertainty lead to a desire to obtain more ad-hoc, rule-of-thumb approaches. Emotional decision making is highly capable of addressing these issues because it is neither fully cognitive nor fully behavioral.
Applications
To date, BELBIC has been tested in the following applications:- HVACHVACHVAC refers to technology of indoor or automotive environmental comfort. HVAC system design is a major subdiscipline of mechanical engineering, based on the principles of thermodynamics, fluid mechanics, and heat transfer...
Systems (heating, ventilating and air conditioning): these are some of the most challenging plants in control systems which consume 50% of the total world energy consumption.
- Nonlinear Systems
- Cell-to-Cell Mapping Algorithm
- Electrically Heated Micro Heat Exchanger: this device has been developed to accelerate fluid and heat exchange in reduced systems.
- The Motion Control of Three Wheeled Robots: three wheeled robots are commonly used in RoboCupRoboCupRoboCup is an international robotics competition founded in 1997. The aim is to develop autonomous soccer robots with the intention of promoting research and education in the field of artificial intelligence...
soccer because they are omnidirectional with minimum wheels.
- RoboCupRoboCupRoboCup is an international robotics competition founded in 1997. The aim is to develop autonomous soccer robots with the intention of promoting research and education in the field of artificial intelligence...
Rescue Simulation: a large, multi-agent system is one of the most challenging environments to control and coordinate because there needs to be a precise coordination between agents.
- Control of Intelligent Washing Machines: intelligent control of home applicances has gained considerable attention by scientists and the industry in recent years. In the case of washing machines, intelligent control could mean both easier use and energy and water conservation.
- Auto Landing System
- Speed Regulation of DC motors
- Active Queue Management
- Aerospace launch vehicle control
- Impossibles AIBO 4-legged Robocup competition
- Predicting Geomagnetic Activity Index
- Speed Control of Switched Reluctance Motor
- Intelligent control of Micro Heat Exchanger
- Model Free Control of Overhead Travelling Crane
- Model Free Control of Perturbed Overhead Crane
- Autopilot Control Design for a 2-DOF Helicopter Model
- Path Tracking for a Car
See also
- Soft computingSoft computingSoft computing is a term applied to a field within computer science which is characterized by the use of inexact solutions to computationally-hard tasks such as the solution of NP-complete problems, for which an exact solution cannot be derived in polynomial time.-Introduction:Soft Computing became...
- Fuzzy LogicFuzzy logicFuzzy logic is a form of many-valued logic; it deals with reasoning that is approximate rather than fixed and exact. In contrast with traditional logic theory, where binary sets have two-valued logic: true or false, fuzzy logic variables may have a truth value that ranges in degree between 0 and 1...
- Evolutionary AlgorithmEvolutionary algorithmIn artificial intelligence, an evolutionary algorithm is a subset of evolutionary computation, a generic population-based metaheuristic optimization algorithm. An EA uses some mechanisms inspired by biological evolution: reproduction, mutation, recombination, and selection...
- Neural NetworkNeural networkThe term neural network was traditionally used to refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes...
- Genetic AlgorithmGenetic algorithmA genetic algorithm is a search heuristic that mimics the process of natural evolution. This heuristic is routinely used to generate useful solutions to optimization and search problems...
- Caro LucasCaro LucasCaro Lucas was a renowned Iranian- Armenian scientist. His many areas of contribution to Iranian scientific society include biological computing, computational intelligence, uncertain systems, intelligent control, fuzzy systems, neural networks, multiagent systems, swarm intelligence, data mining,...
External links
- A Practical Tutorial on Genetic Algorithm Programming a Genetic Algorithm step by step.
- Fuzzy logic - article at Stanford Encyclopedia of PhilosophyStanford Encyclopedia of PhilosophyThe Stanford Encyclopedia of Philosophy is a freely-accessible online encyclopedia of philosophy maintained by Stanford University. Each entry is written and maintained by an expert in the field, including professors from over 65 academic institutions worldwide...
- International Society for Genetic and Evolutionary Computation
- IEEE Computational Intelligence Society (IEEE CIS)
- A collection of non-linear models and demo applets (in Monash University's Virtual Lab)
- Nonlinear Dynamics I: Chaos at MIT's OpenCourseWare
- PSO-BELBIC scheme for two-coupled distillation column process