Do explanations expose bias? How saliency maps affect judgements of biased face-recognition models

Justyn Rodrigues, Krista A. Ehinger, Oliver Obst, X. Rosalind Wang

Research output: Chapter in Book / Conference PaperChapterpeer-review

2 Downloads (Pure)

Abstract

Saliency-map explanations are intended to make computer-vision models more transparent, but it is unclear whether they help people recognise biased behaviour. We conducted a controlled on-line study with 40 participants who compared Layer-wise Relevance Propagation maps from convolutional face-recognition models. A fair model was trained on a balanced synthetic dataset; two biased models were trained on data in which either light- or dark-skinned faces appeared only in frontal pose. Each participant completed 32 comparison trials. When the fair model was paired with the dark-skinned-pose-biased model, selections were near chance (52.8% favouring the fair model, binomial p =.36). When the fair model was paired with the light-skinned-pose-biased model, participants chose the biased model significantly more often (58.1%, p =.005). Confidence ratings varied with condition and did not systematically track model fairness. These results indicate that pixel-level attribution alone does not reliably expose training bias and can, in some settings, mislead non-expert users.

Original languageEnglish
Title of host publicationECAI 2025: 28th European Conference on Artificial Intelligence, 25-30 October 2025, Bologna, Italy, Including 14th Conference on Prestigious Applications of Intelligent Systems (PAIS 2025), Proceedings
EditorsInês Lynce, Nello Murano, Mauro Vallati, Serena Villata, Federico Chesani, Michela Milano, Andrea Omicini, Mehdi Dastani
Place of PublicationNetherlands
PublisherIOS Press
Pages1229-1236
Number of pages8
ISBN (Electronic)9781643686318
DOIs
Publication statusPublished - 2025
EventEuropean Conference on Artificial Intelligence - Bologna, Italy
Duration: 25 Oct 202530 Oct 2025
Conference number: 28th

Publication series

NameFrontiers in Artificial Intelligence and Applications
Volume413
ISSN (Print)0922-6389
ISSN (Electronic)1879-8314

Conference

ConferenceEuropean Conference on Artificial Intelligence
Abbreviated titleECAI
Country/TerritoryItaly
CityBologna
Period25/10/2530/10/25

Fingerprint

Dive into the research topics of 'Do explanations expose bias? How saliency maps affect judgements of biased face-recognition models'. Together they form a unique fingerprint.

Cite this