Home / Current Issue / Paper 1718150
Mechanistic Interpretability of Vision-Language Models: Tracing Multimodal Concept Binding in CLIP and SigLIP
Subject area: Science,Engineering and Technology · Area of research: Vision-Language Models
DOI: https://doi.org/10.64388/IREV9I7-1718150
Abstract
Vision-Language Models like CLIP and SigLIP align images and text in a shared embedding space, but the internal mechanisms by which they bind visual concepts to language remain opaque. We apply mechanistic interpretability methods to localize and characterize circuits responsible for cross-modal concept binding in CLIP-ViT-B/32 and SigLIP-SO400M. Using causal tracing, sparse autoencoders, and activation patching, we identify three functional subsystems: early visual feature extraction, cross-attention-mediated grounding, and late-stage concept fusion. We find that 12–18% of MLP neurons and 7% of attention heads exhibit polysemantic, modality-invariant concept selectivity for objects, colors, and relations. Causal intervention on these circuits produces predictable changes in image-text alignment scores, confirming causal roles. Our analysis reveals that concept binding relies on a small set of highly interpretable circuits rather than distributed representations. These findings provide a foundation for targeted model editing, bias mitigation, and robust multimodal alignment.
Keywords
mechanistic interpretability, vision-language models, CLIP, concept binding, sparse autoencoders, causal tracing
How to cite this paper
@article{1718150,
author = {Dr. Madumere Smart Onyemaechi, Frank Uchehara O.},
title = {Mechanistic Interpretability of Vision-Language Models: Tracing Multimodal Concept Binding in CLIP and SigLIP},
journal = {Iconic Research And Engineering Journals},
year = {2026},
volume = {9},
number = {7},
pages = {3019-3021},
issn = {2456-8880},
url = {https://www.irejournals.com/formatedpaper/1718150.pdf},
abstract = {Vision-Language Models like CLIP and SigLIP align images and text in a shared embedding space, but the internal mechanisms by which they bind visual concepts to language remain opaque. We apply mechanistic interpretability methods to localize and characterize circuits responsible for cross-modal concept binding in CLIP-ViT-B/32 and SigLIP-SO400M. Using causal tracing, sparse autoencoders, and activation patching, we identify three functional subsystems: early visual feature extraction, cross-attention-mediated grounding, and late-stage concept fusion. We find that 12–18% of MLP neurons and 7% of attention heads exhibit polysemantic, modality-invariant concept selectivity for objects, colors, and relations. Causal intervention on these circuits produces predictable changes in image-text alignment scores, confirming causal roles. Our analysis reveals that concept binding relies on a small set of highly interpretable circuits rather than distributed representations. These findings provide a foundation for targeted model editing, bias mitigation, and robust multimodal alignment.},
keywords = {mechanistic interpretability, vision-language models, CLIP, concept binding, sparse autoencoders, causal tracing},
month = {January},
doi = {https://doi.org/10.64388/IREV9I7-1718150}
}