A Haptic Interface for Guiding People with Visual Impairment using Three Dimensional Computer Vision

Citation

Swingler K & Grigson C (2022) A Haptic Interface for Guiding People with Visual Impairment using Three Dimensional Computer Vision. In: Back T, van Stein B, Wagner C, Garibaldi J, Lam HK, Cottrell M, Doctor F, Filipe J, Warwick K & Kaprzyk J (eds.) Proceedings of the 14th International Joint Conference on Computational Intelligence (IJCCI 2022) - NCTA. 14th International Conference on Neural Computation Theory and Applications, Valletta, Malta, 24.10.2022-26.10.2022. Setubal, Portugal: SCITEPRESS - Science and Technology Publications, pp. 315-322. https://doi.org/10.5220/0011307800003332

Abstract
Computer vision technology has the potential to provide life changing assistance to blind or visually impaired (BVI) people. This paper presents a technique for locating objects in three dimensions and guiding a person’s hand to the object. Computer vision algorithms are used to locate both objects of interest and the user’s hand. Their relative locations are used to calculate the movement required to take the hand closer to the object. The required direction is signaled to the user via a haptic wrist band, which consists of four haptic motors worn at the four compass points on the wrist. Guidance works both in two and three dimensions, making use of both colour and depth map inputs from a camera. User testing found that people were able to follow the haptic instructions and move their hand to locations on vertical or horizontal surfaces. This work is part of the Artificial Intelligence Sight Loss Assistant (AISLA) project.

Keywords
3D Computer Vision; Object Detection; Assistive Technology; Visual Impairment; Haptic Feedback

People

Professor Kevin Swingler

Professor, Computing Science