Detailed Information

Grounding Counterfactual Explanation of Image Classifiers to Textual Concept Space

Cited 0 time in Web of Science Cited 0 time in Scopus

Kim, Siwon; Oh, Jinoh; Lee, Sung Jin; Yu, Seung Hak; Do, Jae Young; Taghavi, Tara

Issue Date
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.10942-10950
Concept-based explanation aims to provide concise and human-understandable explanations of an image classifier. However, existing concept-based explanation methods typically require a significant amount of manually collected concept-annotated images. This is costly and runs the risk of human biases being involved in the explanation. In this paper, we propose Counterfactual explanation with text-driven concepts (CounTEX), where the concepts are defined only from text by leveraging a pre-trained multimodal joint embedding space without additional concept-annotated datasets. A conceptual counterfactual explanation is generated with text-driven concepts. To utilize the text-driven concepts defined in the joint embedding space to interpret target classifier outcome, we present a novel projection scheme for mapping the two spaces with a simple yet effective implementation. We show that CounTEX generates faithful explanations that provide a semantic understanding of model decision rationale robust to human bias.
Files in This Item:
There are no files associated with this item.
Appears in Collections:

Related Researcher

  • College of Engineering
  • Department of Electrical and Computer Engineering
Research Area AI 애플리케이션을 위한 알고리즘-시스템 공동 설계, AI-powered Big Data Management, Generative AI, Large Language Model, ML, 고성능 대규모 AI 데이터 분석 및 처리, 모달 AI


Item View & Download Count

  • mendeley

Items in S-Space are protected by copyright, with all rights reserved, unless otherwise indicated.