Amount awarded: $9,800

Project Narrative: This project will help build conceptual foundations for explainable AI by asking whether and how concepts from neuroscience (e.g., circuit, representation) can help explain AI models. To do this, we will collaborate with experts in each field to build a consensus glossary of core concepts in neuroscience and AI, and we will compare the fields’ glossaries via expert cross-commentary. The result will be a critical analysis of the conceptual toolkits available to study natural and artificial intelligence, and the relationship between them.

Project Summary: Mechanistic Interpretability (MI) aims to open the black box of AI systems by uncovering the internal mechanisms that generate their behavior. It is described by practitioners as a “pre-paradigmatic field” (Lindsey et al. 2025), still in search of the right concepts and constructs. Currently, the field sources many of its foundational concepts from neuroscience, characterizing AI systems (especially LLMs) as, e.g., processing information through circuits that perform computations over representations, and studying all of the above with decoding methods adapted from neuroscience (e.g., Ameisen et al. 2025; Li et al. 2023; Lindsey et al. 2025; Templeton et al. 2024). This conceptual toolkit has been fruitful in neuroscience, and it is not unreasonable to think it will serve MI as well. But the concepts in question have served neuroscience by helping it achieve particular goals, in a particular context, under particular constraints and assumptions. There is no guarantee they will also suit MI and its particular goals, context, constraints, and assumptions. It is therefore necessary to understand how these concepts serve their home discipline, what features they bring with them to MI, and how, in light of this, they might be tailored to fit MI’s purposes.

Our project will therefore study the meaning of the concepts that travel from neuroscience to MI, aiming to bring neuroscientists and MI researchers into a productive dialogue about the concepts they use to characterize natural and artificial intelligence. To achieve this, we will build a comparative consensus glossary, recruiting a sample of scientist-participants in MI and neuroscience to (1) define key concepts and their scientific role and significance, (2) vote on the definitions, and (3) iterate until the votes pass a threshold, allowing us to identify a shared, core meaning for the relevant concepts. This follows an existing methodology for consensus glossaries, but we will extend it in two ways. First, our goal is not consensus but comparison. So we will build two consensus glossaries for the same concepts: one with neuroscientists and one with MI researchers. This will be followed by a step (4) in which scientists give structured commentary on the other field’s definitions, aiming to surface differences between the two fields. Second, we are not just interested in definitions, but in the role concepts play in research. So our glossaries and commentaries will be built following the lessons of pragmatist philosophy of science, and indexing concepts directly to their scientific role and the research programs they serve (cf. Cao 2022; Chirimuuta 2024; Egan 2025; Feest 2025; Haueis and Margulies 2026; Mace 2025; Richmond 2025a, 2025b).

In addition to first-order gains for MI, philosophy of neuroscience, and philosophy of AI, a major contribution of this project will be the construction of an open-source pipeline for conducting (comparative) consensus glossaries — a useful addition to the case study and experimental methods philosophers typically use to study scientific concepts.