Channel Configuration for Neural Architecture: Insights from the Search Space

Citation

Thomson SL, Ochoa G, Veerapen N & Michalak K (2023) Channel Configuration for Neural Architecture: Insights from the Search Space. In: TBC. The Genetic and Evolutionary Computation Conference (GECCO) 2023, Lisbon, Portugal, 15.07.2023-19.07.2023. New York: ACM. https://doi.org/10.1145/nnnnnnn.nnnnnnn

Abstract
We consider search spaces associated with neural network channel configuration. Architectures and their accuracy are visualised using low-dimensional Euclidean embedding (LDEE). Optimisation dynamics are captured using local optima networks (LONs). LONs are a compression of a fitness landscape: the nodes are local optima and the edges are search transitions between them. Several neural architecture search algorithms are tested on the search space and we discover that iterated local search (ILS) is a competitive algorithm for neural channel configuration. We additionally implement a landscape-aware ILS which performs well. Observations from the search and landscape space analyses bring visual clarity and insight to the science of neural network channel design: the results indicate that a high number of channels, kept constant throughout the network, is beneficial.

Keywords
Fitness Landscapes; Neural Architecture Search; Local Optima Net- works (LONs)

Notes
Output Status: Forthcoming

People

Professor Gabriela Ochoa

Professor, Computing Science