Dense 3D Facial Landmark Identification by Using Depth Maps

Abstract

2D facial landmark identification is a well-established research area, with Dlib and MediaPipe models achieving high success rates. 3D facial landmark identification on point clouds, however, is a less prominent research area due to the complexity of topology variation. Previous automated 3D facial landmark identification approaches include resource-intensive AI training, with the capability to recognize only ~12-68 topologically distinct features (tip of the nose/eyes). In this paper, we present an approach that identifies 468 3D landmarks. To our knowledge, our algorithm’s results are the densest to date and the first to identify landmarks on topologically smooth regions (cheeks, forehead). Our approach utilizes 2D depth maps as an intermediate to circumvent the complexity of 3D data while still relying solely on spatial data. Our approach projects a 3D point cloud onto an optimized 2D depth map, utilizes existing robust 2D models to identify landmarks, and maps them back to the original 3D point cloud. After testing on the LYHM dataset, our pipeline is shown to yield high accuracy with error rates of 1.03 ± 0.08% (∝=0.01), showing the potential of 2D representations in processing 3D data.