Evolutionary Acquisition of Neural Topologies
Encyclopedia
Evolutionary Acquisition of Neural Topologies (EANT/EANT2) is an evolutionary
reinforcement learning
method that evolves both the topology and weights of artificial neural network
s. It is closely related to the works of Angeline et al. and Stanley and Miikkulainen. Like the work of Angeline et al., the method uses a type of parametric mutation that comes from evolution strategies
and evolutionary programming
(now using the most advanced form of the evolution strategies CMA-ES
in EANT2), in which adaptive step sizes are used for optimizing the weights of the neural networks. Similar to the work of Stanley (NEAT), the method starts with minimal structures, which are complexified along the evolution path.
.
It introduces a genetic encoding called Common Genetic Encoding (CGE) that handles both direct and indirect encoding of neural networks with in the same theoretical framework. The encoding has important properties that makes it suitable for evolving neural networks:
These properties have been formally proven in.
For evolving the structure and weights of neural networks, an evolutionary process is used, where the exploration of structures is executed at a larger timescale (structural exploration), and the exploitation of existing structures is done at a smaller timescale (structural exploitation). In the structural exploration phase, new neural structures are developed by gradually adding new structures to an initially minimal network that is used as a starting point. In the structural exploitation phase, the weights of the currently available structures are optimized using an evolution strategy
.
Keepaway benchmark. In all the tests, EANT was found to perform very well. Moreover, a newer version of EANT, called EANT2, was tested on a visual servoing task and found to outperform NEAT and the traditional iterative Gauss–Newton method. Further experiments include results on a classification problem
Evolutionary computation
In computer science, evolutionary computation is a subfield of artificial intelligence that involves combinatorial optimization problems....
reinforcement learning
Reinforcement learning
Inspired by behaviorist psychology, reinforcement learning is an area of machine learning in computer science, concerned with how an agent ought to take actions in an environment so as to maximize some notion of cumulative reward...
method that evolves both the topology and weights of artificial neural network
Artificial neural network
An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
s. It is closely related to the works of Angeline et al. and Stanley and Miikkulainen. Like the work of Angeline et al., the method uses a type of parametric mutation that comes from evolution strategies
Evolution strategy
In computer science, evolution strategy is an optimization technique based on ideas of adaptation and evolution. It belongs to the general class of evolutionary computation or artificial evolution methodologies.-History:...
and evolutionary programming
Evolutionary programming
Evolutionary programming is one of the four major evolutionary algorithm paradigms. It is similar to genetic programming, but the structure of the program to be optimized is fixed, while its numerical parameters are allowed to evolve....
(now using the most advanced form of the evolution strategies CMA-ES
CMA-ES
CMA-ES stands for Covariance Matrix Adaptation Evolution Strategy. Evolution strategies are stochastic, derivative-free methods for numerical optimization of non-linear or non-convex continuous optimization problems. They belong to the class of evolutionary algorithms and evolutionary computation...
in EANT2), in which adaptive step sizes are used for optimizing the weights of the neural networks. Similar to the work of Stanley (NEAT), the method starts with minimal structures, which are complexified along the evolution path.
Contribution of EANT to neuroevolution
Despite sharing these two properties, the method has the following important features which distinguish it from previous works in neuroevolutionNeuroevolution
Neuroevolution, or neuro-evolution, is a form of machine learning that uses evolutionary algorithms to train artificial neural networks. It is useful for applications such as games and robot motor control, where it is easy to measure a network's performance at a task but difficult or impossible to...
.
It introduces a genetic encoding called Common Genetic Encoding (CGE) that handles both direct and indirect encoding of neural networks with in the same theoretical framework. The encoding has important properties that makes it suitable for evolving neural networks:
- It is complete in that it is able to represent all types of valid phenotype networks.
- It is closed, i.e. every valid genotype represents a valid phenotype. (Similarly, the encoding is closed under genetic operators such as structural mutation and crossover.)
These properties have been formally proven in.
For evolving the structure and weights of neural networks, an evolutionary process is used, where the exploration of structures is executed at a larger timescale (structural exploration), and the exploitation of existing structures is done at a smaller timescale (structural exploitation). In the structural exploration phase, new neural structures are developed by gradually adding new structures to an initially minimal network that is used as a starting point. In the structural exploitation phase, the weights of the currently available structures are optimized using an evolution strategy
Evolution strategy
In computer science, evolution strategy is an optimization technique based on ideas of adaptation and evolution. It belongs to the general class of evolutionary computation or artificial evolution methodologies.-History:...
.
Performance
EANT has been tested on some benchmark problems such as the double-pole balancing problem, and the RoboCupRoboCup
RoboCup is an international robotics competition founded in 1997. The aim is to develop autonomous soccer robots with the intention of promoting research and education in the field of artificial intelligence...
Keepaway benchmark. In all the tests, EANT was found to perform very well. Moreover, a newer version of EANT, called EANT2, was tested on a visual servoing task and found to outperform NEAT and the traditional iterative Gauss–Newton method. Further experiments include results on a classification problem