Learning Spatial Relations with a Standard Convolutional Neural Network

Citation

Swingler K & Bath M (2020) Learning Spatial Relations with a Standard Convolutional Neural Network. In: Merelo JJ, Garibaldi J, Wagner C, Bäck T, Madani K & Warwick K (eds.) Proceedings of the 12th International Joint Conference on Computational Intelligence - Volume 1: NCTA. 12th International Conference on Neural Computation Theory and Applications, Budapest, Hungary, 02.11.2020-04.11.2020. Setubal, Portugal: SCITEPRESS - Science and Technology Publications, pp. 464-470. https://doi.org/10.5220/0010170204640470

Abstract
This paper shows how a standard convolutional neural network (CNN) without recurrent connections is able to learn general spatial relationships between different objects in an image. A dataset was constructed by placing objects from the Fashion-MNIST dataset onto a larger canvas in various relational locations (for example, trousers left of a shirt, both above a bag). CNNs were trained to name the objects and their spatial relationship. Models were trained to perform two different types of task. The first was to name the objects and their relationships and the second was to answer relational questions such as ``Where is the shoe in relation to the bag?". The models performed at above 80\% accuracy on test data. The models were also capable of generalising to spatial combinations that had been intentionally excluded from the training data.

Keywords
Convolutional Neural Networks; Spatial Reasoning; Computer Vision

People

Professor Kevin Swingler

Professor, Computing Science