Luke Malveaux
Computer science
Hometown: Tempe, AZ, US
Graduation date: Fall 2027
FURI | Fall 2026, Needs Review
Where Vision-Language Models Look: Position-Resolved Analysis of Attention Heads on Documents
Mechanistic interpretability asks which parts of a model do what. Recent interpretability work on vision-language models (VLMs) has largely focused on attention heads, the components that decide where the model looks, and typically scores each head at a single position on natural-image grounding tasks. The research team argues that these two practices jointly constrain what such analyses can establish. Attention scored at a single position reveals where a head attends but not its function there, and natural-image grounding tasks rarely separate different functions in space. This research drops both, recording every head of a VLM at every token of the question and answer on multi-page documents, whose structured layouts and varied question types separate those functions in space, and tests the observed patterns with causal interventions. The resulting account of how VLM heads locate and use visual evidence is intended to give head-level interpretability a position-resolved foundation and to identify where attention steering and supervision should act in document understanding.
Mentor: Vivek Gupta
It’s hip to be square.
Students presenting projects at the Fulton Forge Student Research Expo are encouraged to download this personal QR code and include it within your poster. This allows expo attendees to explore more about your project and about you in the future.
Right click the image to save it to your computer.