What the eye doesn't see: Using infrared to improve face recognition of individuals with highly pigmented skin

Abstract

Face recognition technology has become commonplace in security and access control applications. However, their performance leaves a lot to be desired when working with highly pigmented skin tones. One reason for this is the training bias introduced by under-representation in existing datasets. The other is inherent to pigmentation -- darker skins absorb more light and therefore could reflect less discernible detail in the visible spectrum. We show how this can be enhanced by incorporating the infrared spectrum, which electronic sensors can perceive. We augment existing datasets with images of highly pigmented individuals, captured using the visible, infrared and full spectra We fine-tune state-of-the-art face recognition systems and compare the performance of these three spectra. We also assess the impact of narrow and wide cropping, different facial orientations, and sunlight and shaded conditions. We find a marked improvement in the accuracy and in the AUC values of the ROC curves when including the infrared spectrum, with performance increasing from 97.5% to 99% for highly pigmented faces. Including different facial orientations and narrow cropping also improves the performance, and can therefore be deemed as recommended best practices for future research.