Dense Identification of 3D Facial Landmarks by Utilizing 2D as Intermediate

Abstract

2D facial landmark identification is a well-established research area, with Dlib and MediaPipe models achieving high success rates. 3D facial landmarks identification on point clouds, however, is a less prominent research area due to the complexity of topology variation. Previous automated 3D facial landmark identification approaches include resource-intensive AI training, with the capability to recognize only ~12-68 typologically distinct features (tip of the nose/eyes). In this paper, we present an approach that identifies 468 3D landmarks. To our knowledge, our algorithm’s results are the densest to date and the first to identify landmarks on topologically smooth regions (cheeks, forehead). Our approach utilizes 2D depth maps as an intermediate to circumvent the complexity of 3D data while still relying solely on spatial data. Our approach projects a 3D point cloud onto an optimized 2D depth map, utilizes existing robust 2D models, and maps landmarks back to the original 3D point cloud. After testing on the LYHM dataset, our pipeline is shown to yield high accuracy with error rates of 1.03 ± 0.08% (∝=0.01), showing the potential of 2D representations in processing 3D data.