Evolutionary Acquisition of Neural Topologies

Evolutionary Acquisition of Neural Topologies (EANT/EANT2) is an evolutionary reinforcement learning method that evolves both the topology and weights of neural networks. It is closely related to the works of Angeline et al. [Peter J Angeline, Gregory M Saunders, and Jordan B Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5:54–65, 1994. [http://demo.cs.brandeis.edu/papers/ieeenn.pdf] ] and Stanley and Miikkulainen [NeuroEvolution of Augmented Topologies (NEAT) by Stanley and Miikkulainen, 2005 http://nn.cs.utexas.edu/downloads/papers/stanley.ieeetec05.pdf] . Like the work of Angeline et al., the method uses a type of parametric mutation that comes from evolution strategies and evolutionary programming (now using the most advanced form of the evolution strategies CMA-ES in EANT2), in which adaptive step sizes are used for optimizing the weights of the neural networks. Similar to the work of Stanley, the method starts with minimal structures, which are complexified along the evolution path.

Contribution of EANT to Neuroevolution

Despite sharing these two properties, the method has the following important features which distinguish it from previous works in neuroevolution:

It introduces a genetic encoding called Common Genetic Encoding (CGE) that handles both direct and indirect encoding of neural networks with in the same theoretical framework. The encoding has important properties that makes it suitable for evolving neural networks: (1) It is "complete" in that it is able to represent all types of valid phenotype networks. (2) It is "closed", i.e. every valid genotype represents a valid phenotype. Similarly, the encoding is "closed under genetic operators" such as structural mutation and crossover. These properties have been formally proven in [Yohannes Kassahun, Mark Edgington, Jan Hendrik Metzen, Gerald Sommer and Frank Kirchner. Common Genetic Encoding for Both Direct and Indirect Encodings of Networks. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), London, UK, 1029-1036, 2007. [http://portal.acm.org/citation.cfm?id=1277162] ] .

For evolving the structure and weights of neural networks, an evolutionary process is used, where the exploration of structures is executed at a larger timescale (structural exploration), and the exploitation of existing structures is done at a smaller timescale (structural exploitation). In the structural exploration phase, new neural structures are developed by gradually adding new structures to an initially minimal network that is used as a starting point. In the structural exploitation phase, the weights of the currently available structures are optimized using an Evolution Strategy.

Performance

EANT has been tested on some benchmark problems such as the double-pole balancing problem [Yohannes Kassahun and Gerald Sommer. Efficient reinforcement learning through evolutionary acquisition of neural topologies. In Proceedings of the 13th European Symposium on Artificial Neural Networks (ESANN 2005), pages 259–266, Bruges, Belgium, April 2005. [http://www.ks.informatik.uni-kiel.de/~yk/ESANN2005EANT.pdf] ] , and the RoboCup Keepaway benchmark [ Jan Hendrik Metzen, Mark Edgington, Yohannes Kassahun and Frank Kirchner. Performance Evaluation of EANT in the RoboCup Keepaway Benchmark. In Proceedings of the Sixth International Conference on Machine Learning and Applications (ICMLA 2007), pages 342-347, USA, 2007 [http://portal.acm.org/citation.cfm?id=1336953.1337145&coll=GUIDE&dl=GUIDE] ] . In all the tests, EANT was found to perform very well. Moreover, a newer version of EANT, called EANT2, was tested on a visual servoing task and found to outperform NEAT and the traditional iterative Gauss-Newton method [Nils T Siebel and Gerald Sommer. Evolutionary reinforcement learning of artificial neural networks. International Journal of Hybrid Intelligent Systems 4(3): 171-183, October 2007. [http://www.ks.informatik.uni-kiel.de/~vision/doc/Publications/nts/SiebelSommer-IJHIS2007.pdf] ] .

References