Interpretability-Driven Defense Against Physical Adversarial Attacks in Facial Recognition Systems

This study aims to develop and evaluate a defense mechanism, the Self-Explaining Gate (SEG), to protect facial recognition (FR) systems against physical adversarial attacks (PAAs) by combining identity recognition, feature attribution, and anatomical landmark detection. SEG uses a three-stream architecture comprising identity feature extraction, real-time Grad-CAM feature attribution, and Haar-cascade-based anatomical landmark detection, integrated through a novel Anatomical Consistency Score (ACS) that measures the overlap between model attention and biological facial landmarks. The approach was evaluated on the CASIA-FASD and SiW benchmark datasets and a custom adversarial dataset, using detection accuracy, false acceptance/rejection rates, and inference latency as evaluation metrics. Our SEG achieved 94% attack detection accuracy, rejecting 96.1% of adversarial images while accepting 90% of genuine images, with inference latency under 160 milliseconds, outperforming adversarial training, feature squeezing, and SSR-FCN baseline defenses. Performance degrades on low-resolution images and under extreme occlusion, and further validation on video-based and larger real-world datasets is required. SEG can be integrated into real-time facial recognition deployments, such as border control, banking, and security systems, to provide interpretable and anatomically grounded rejection of adversarial inputs. By making automated identity verification more transparent and resistant to spoofing, SEG has the potential to increase public trust in facial recognition systems used in security-sensitive contexts. This study is the first to combine intrinsic, feedforward explainability with anatomical landmark validation as a real-time defense mechanism, introducing the Anatomical Consistency Score and a zero-background weighting scheme.