Conference Proceeding

Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features



Abel A, Gao C, Smith L, Watt R & Hussain A (2018) Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features. In: 2018 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE Symposium Series on Computational Intelligence, SSCI 2018, Bangalore, India, 18.11.2018-21.11.2018. Piscataway, NJ, USA: IEEE, pp. 1033-1040.

The extraction of relevant lip features is of continuing interest in the speech domain. Using end-to-end feature extraction can produce good results, but at the cost of the results being difficult for humans to comprehend and relate to. We present a new, lightweight feature extraction approach, motivated by glimpse based psychological research into racial barcodes. This allows for 3D geometric features to be produced using Gabor based image patches. This new approach can successfully extract lip features with a minimum of processing, with parameters that can be quickly adapted and used for detailed analysis, and with preliminary results showing successful feature extraction from a range of different speakers. These features can be generated online without the need for trained models, and are also robust and can recover from errors, making them suitable for real world speech analysis.

Feature extraction; Lips; Mouth; Psychology; Adaptation models; Shape; Training

FundersEPSRC Engineering and Physical Sciences Research Council
Publication date31/12/2018
Publication date online31/01/2019
Place of publicationPiscataway, NJ, USA
ConferenceIEEE Symposium Series on Computational Intelligence, SSCI 2018
Conference locationBangalore, India

People (2)


Professor Leslie Smith

Professor Leslie Smith

Emeritus Professor, Computing Science

Professor Roger Watt

Professor Roger Watt

Emeritus Professor, Psychology