Faithful but Not Plausible? A Comparative Evaluation of Explainable AI Methods for Bridge Deck Crack Detection

Abstract

Objectives: Automated crack detection using deep learning is increasingly proposed for bridge deck inspection, but transportation agencies hesitate to adopt models whose decisions cannot be examined. Explainable AI (XAI) saliency methods are offered as a remedy, yet are rarely evaluated for both faithfulness to the model and plausibility to a human inspector. This study quantifies both, and their relationship, for crack detection.

Methods: Four post-hoc explanation methods (Grad-CAM⁺⁺, Eigen-CAM, Score-CAM, and SHAP) were compared across three lightweight classifiers (ResNet-18, MobileNetV3-Small, and ViT-Tiny) fine-tuned on the SDNET2018 bridge deck subset. Faithfulness was measured with deletion and insertion curves, localization against pixel-level DeepCrack masks, and plausibility through blinded expert ratings by a professional engineer.

Findings: All models exceeded 90 percent accuracy. Gradient- and activation-based explanations degraded sharply on the vision transformer (ViT), producing negative faithfulness gaps, while perturbation-based Score-CAM partially recovered. Most consequentially, SHAP was among the most faithful methods on convolutional models by machine metrics yet was rated least plausible by the expert, because its scattered pixel-level attributions did not align with the crack. Faithfulness and plausibility were only moderately correlated overall.

Novelty: This study is among the first infrastructure-inspection studies to jointly evaluate explanation faithfulness, localization, and expert plausibility, and to document a direct dissociation between machine-measured faithfulness and human-judged plausibility for crack detection.

Practical Applications: Agencies should not select XAI methods for inspection on machine faithfulness metrics alone; a lightweight expert-rating step during validation surfaces explanations that are technically faithful but practically unusable, and explanation method and model architecture must be chosen together.