A Survey on Actionable Interpretability in Large Language Models

1 Introduction

Large Language Models (LLMs) have become a foundational paradigm in modern artificial intelligence, demonstrating remarkable performance across diverse domains and tasks. Their rapid progress has enabled increasingly capable systems that process language, images, and structured information within a unified modeling framework, leading to widespread deployment in real-world applications. However, the increasing capability and deployment of LLMs have amplified a fundamental challenge: the growing epistemic gap between observable model behavior and the underlying computational mechanisms that produce it. LLMs are large-scale, data-driven nonlinear systems whose internal representations and computations remain only partially understood. In particular, the relationship between their observable behaviors and the underlying mechanisms that encode information, perform reasoning, and generate outputs remains opaque. Such opacity is not merely a theoretical concern-it limits our ability to verify correctness, diagnose failure modes, or enforce safety constraints, thereby presents a central challenge for trustworthy deployment. These limitations are particularly consequential in high-stakes settings, where reliability, accountability, fairness, and safety are essential requirements [DoshiVelez and Kim, 2017].

Before the advent of LLMs, interpretability research primarily focused on traditional smaller-scale machine learning models, with the goal of understanding how input features influence model predictions. Representative approaches include post-hoc explanation methods such as LIME and Shapley-value-based methods, as well as inherently interpretable models such as linear models and decision trees [Dwivedi et al., 2023]. Although interpretability continues to serve both as a scientific lens for understanding model mechanisms and as a tool for explanation-oriented applications, the rise of LLMs has substantially expanded the scope, complexity, and practical significance of interpretability.

Early LLM interpretability research primarily focused on descriptive explanations of model behavior and improve

Application Objectives Interpretability Targets InterpretabilityMethods Actionable Interventions

1 Mitigating Hallucinations

2 Knowledge Editing

3 Fairness and Bias Mitigation

4 Improving Safety

5 Enhancing Capabilities

6 Improving Efficiency

1 Tokens

2 Representations

3 Layers

4 Attention Heads

5 Neurons

6 Circuits

1 Attribution-based Methods

2 Attention-based Methods

3 Sparse Autoencoder

4 Probing

5 Logit Lens

6 Component Analysis

1 Steering

2 Editing

3 Suppression

4 Enhancement

5 Pruning

Table 1: A unified taxonomy of actionable LLM interpretability. The framework can be viewed through four dimensions: application objectives, interpretability targets, interpretability methods, and actionable interventions. The categories shown are representative rather than exhaustive, highlighting the most commonly studied directions in current research.

transparency. However, descriptive explanation alone provides limited support for modifying model behavior. Practical objectives such as improving reliability, editing knowledge, and enforcing safety constraints require the ability to identify and intervene on the mechanisms that give rise to model behavior. In this context, interpretability must move beyond describing model behavior toward enabling principled intervention. Explanations should clarify not only how models behave, but also which mechanisms can be monitored, modified, or controlled. Without this actionable role, interpretability risks remaining passive observation with limited practical utility. [Miller, 2019] emphasizes the utility of interpretability for specific stakeholders and tasks, referring to methods that provide explanations with tangible benefits-for example, supporting human decision-making, building trust, or facilitating model improvement. Building on the utility of interpretability, recent work [Zhang et al., 2026] emphasizes "actionable" as moving beyond insight toward enabling direct use, informing concrete steps for further model improvement, behavioral control or downstream applications of explanations. Under this view, the value of interpretability lies not only in what it explains, but also in the actions it enables.

This survey provides a systematic review of LLM interpretability methods through the lens of actionability. We organize existing methods into attributional and mechanistic paradigms, while also discussing emerging actionable interpretability techniques for VLMs. We further examine how actionable interpretability supports downstream objectives, including hallucination mitigation, knowledge editing, fairness, safety, capability enhancement and efficiency inference. Figure 1 contrasts prior descriptive interpretability, which primarily explains model behavior, with actionable interpretability, which seeks to identify intervention targets and enable behavior change. Our objective is not merely to catalog existing techniques, but to develop a conceptual framework for actionable LLM interpretability: one that moves beyond descriptive explanations toward understanding and control.

2 Definitions and Taxonomy

The rise of LLMs has shifted interpretability research from explaining small, task-specific models to analyzing openended generative models trained on broad and heterogeneous corpora. LLMs encode implicit knowledge, implement context-dependent computations and exhibit complex ability, posing challenges beyond the assumptions of traditional interpretability. Prior work has shown that interpretability is inherently context-dependent, with the validity and utility of explanations varying across stakeholders, tasks, and operational settings [Miller, 2019]. In the LLM era, this dependency becomes even more pronounced, necessitating principled frameworks that situate interpretability claims within clearly defined user roles and objectives. Accordingly, this survey adopts an operational view of LLM interpretability as the study of how internal representations and computations give rise to model behavior. Under this view, interpretability aims to make the mechanisms underlying model generations understandable, analyzable, and ultimately actionable for diagnosis, intervention, control and improvement.

Actionable LLM Interpretability. In this survey, we define actionable LLM interpretability as interpretability that enables targeted interventions with verifiable effects on model behavior. Such interpretability goes beyond providing descriptive or post hoc accounts of model behavior; it must identifie functionally relevant representations, features, neurons, heads, layers, or circuits that can be manipulated to achieve a specified downstream objective. Although no universally accepted quantitative metric for actionability exists, a reasonable qualitative criterion is to ask: Who is the intended user of this explainability method and what concrete notion of utility is increased by using it? In practice, this entails both (i) accurately characterizing the aspects of the interpretability most relevant to the user's goals, and (ii) enabling targeted interventions that address identified issues, ranging from dataand input-level modulation to representation-level interventions and parameter-level editing. Importantly, the operational meaning of "actionable" is inherently role- and contextdependent: end-users may value clarity and transparency in the reasoning process, while researchers may prioritize mechanistic insights into internal computation that guide architectural design or training process.

Taxonomic Dimensions of Actionable LLM Interpretability. LLM interpretability methods can be systematically categorized along several orthogonal dimensions that together structure the landscape of LLM interpretability.

1. Scope: This dimension distinguishes between local and global interpretability. Local interpretability explains model behavior for a particular generation, while global interpretability provides an understanding of general model behavior across diverse inputs.
2. Stage: Intrinsic interpretability incorporates transparency into the model design, allowing the model's in-

ternal generation process to be examined directly. Posthoc interpretability, in contrast, explains the behavior of already trained black-box models using auxiliary explanatory methods.

1. Approach Paradigm: Interpretability methods can be broadly categorized into two major paradigms. Attribution-based methods estimate the contribution of input features to model outputs. Mechanistic interpretability seeks to reverse-engineer the computations underlying model behavior by identifying functionally relevant representations and components.
2. Task-Specific Functionality. Selecting suitable interpretability approaches requires considering the specific task context because different LLM capabilities engage different computational processes. For instance, reasoning and memory retrieval involve different computational processes and thus require tailored interpretability techniques [Jin et al., 2025a].
3. Application Objectives. Interpretability becomes practically valuable when it supports concrete interventions in model behaviorr. Its value lies not merely in explanatory plausibility, but in enabling downstream actions such as hallucination mitigation, knowledge editing, debiasing, and safety alignment.

Based on the taxonomy discussed above, we summarize the landscape of actionable LLM interpretability in Table 1. The table presents a unified taxonomy that organizes existing research along four complementary dimensions: application objectives, interpretability targets, interpretability methods, and actionable interventions. We include key relatedworks for Transformer interpretability because many remain applicable to LLMs when appropriately adapted. Nonetheless, extending these methods to LLMs requires careful consideration of the fundamental differences between large-scale language models and earlier, smaller Transformer models.

3 Explainability Paradigms for LLMs

3.1 Feature Attribution Interpretability for LLMs

Feature attribution methods seek to quantify the contribution of individual input tokens to an LLM's output. Given an input token sequence x = {x1, x2, . . . , xn}, and an LLM, an attribution method assigns each token xi an importance score S(xi) that estimates its contribution to a target output. The attribution map provides a local explanation of which parts of the input strongly influence the model's output for this instance x. Representative families include perturbation-based and local-surrogate methods, attention-based methods, and gradient- and backpropagation-based methods.

Perturbation and Local Surrogate. Perturbation-based methods estimate feature importance by measuring how model outputs change when parts of the input are masked, removed, or otherwise modified. SHAP [Lundberg and Lee, 2017] and LIME [Ribeiro et al., 2016] are representative examples: the former estimates Shapley-value contributions, while the latter fits an interpretable local surrogate to approximate the model's behavior using simpler, interpretable functions. Recent work extends these ideas to LLMs by perturbing internal components like attention weights [Deiseroth et al., 2023], scaling attribution to long-context generation through efficient Shapley-value approximations [Enouen et al., 2024; Liu et al., 2025], and modeling higher-order feature interactions using sparse representations [Kang et al., 2025a]. [Sudhi et al., 2024] extends perturbation-based explanation to retrieval-augmented generation (RAG) by modifying retrieved contexts and measuring the resulting changes in model outputs, enabling attribution of model responses to specific retrieved evidence. A key advantage of perturbationbased approaches is their model-agnostic nature and direct causal interpretation: a feature is deemed important if modifying it changes the prediction. However, perturbation and surrogate-based methods often incur substantial computational overhead, rely on simplifying assumptions about feature interactions, and provide only indirect approximations of model behavior. Moreover, attribution estimates can be sensitive to the choice of perturbation scheme, while modern models may remain highly confident under heavily perturbed or semantically degraded inputs, raising concerns about the reliability of perturbation-based explanations [Feng et al., 2018].

Attention-based. Attention-basedmethods leverage Transformers attention patterns as a source of token-level attribution. The most direct approach treats raw attention weights as importance scores. However, high attention weights do not necessarily correspond to high predictive importance, making raw attention an unreliable explanation signal. A variety of improved methods have been proposed - for example, Attention Rollout aggregates attention across layers, while Attention Flow models information propagation from output tokens back to input tokens using a maximum-flow formulation [Abnar and Zuidema, 2020]. [Chefer et al., 2021a; Chefer et al., 2021b] augment attention weights with gradient signals to derive class-specific relevance estimates and leverage positive gradient-attention interactions to improve faithfulness. Nevertheless, attention weights do not necessarily identify the causal factors driving model outputs [Wiegreffe and Pinter, 2019; Jain and Wallace, 2019]. Recent studies on attention sinks provide additional evidence for this limitation, showing that LLMs may assign disproportionately high attention to semantically uninformative tokens [Kang et al., 2025b; Gu et al., 2025].

Gradient-Based and backpropagation-based. GradientBased and backpropagation-based methods estimate feature importance by tracing how prediction-relevant signals propagate through the network. Early approaches such as Input × Gradient (I×G) [Simonyan et al., 2013] estimate token importance by multiplying input representations with their gradients, but can be noisy and unstable due to gradient saturation or shattering effects in deep architectures. SmoothGrad [Smilkov et al., 2017] stabilizes attributions by averaging gradients across perturbations, while Integrated Gradients (IG) [Sundararajan et al., 2017] mitigates gradient saturation by integrating gradients along an interpolation path. To better accommodate language models, [Enguehard, 2023] propose Sequential Integrated Gradients (SIG), which attributes importance by interpolating one token at a time while holding all others fixed, avoiding unrealistic mixed-token interpolation paths. However, gradient integration methods remain computationally expensive for large Transformer models and provide limited insight into how intermediate representations contribute to the final prediction. A related line of work uses Layer-wise Relevance Propagation (LRP) [Bach et al., 2015; Ding et al., 2017] backpropagates relevance scores through layers. More recent variants such as AttnLRP [Achtibat et al., 2024] enable relevance propagation across both input tokens and internal representations. Despite their strong theoretical grounding and fine-grained attributions, backpropagationbased methods remain constrained by computational cost and the difficulty of relating relevance scores to underlying computational mechanisms.

3.2 Mechanistic Interpretability for LLMs

Mechanistic interpretability aims to identify the internal computational mechanisms that explain model behavior [Rai et al., 2024]. Its objects of study range from residual-stream directions and sparse features to neurons, attention heads, and multi-component circuits. In contrast to attribution-based methods, which quantify input-output relationships, mechanistic approaches aim to uncover how information is encoded, transformed, and routed through the network. Recent advances span several directions: (1) SAEs that extract interpretable feature bases, (2) probing methods that localize task-relevant information across internal representations, (3) logit-lens-based methods that track the emergence of outputrelevant signals across layers, and (4) mechanism-discovery methods that identify causal computational structures at different levels of granularity.

Sparse Autoencoder. Sparse Autoencoders (SAEs) are widely used to decompose internal features into sparse representations, which can often be interpreted as humanunderstandable concepts [Dong et al., 2025]. Given a vector x, for example, neuron activation, a SAE encodes it into a sparse latent representation and reconstructs it through a decoder, typically written as SAE(x) = Dec(Act(Enc(x))), where Enc and Dec are learned linear maps and Act enforces sparsity. The model is trained to minimize reconstruction error while encouraging only a small number of latent features to be active for each input. After training, individual neurons are interpreted via inputs that strongly activate them. The corresponding decoder vectors serve as candidate directions in representation space, yielding a sparse feature (or concept) dictionary. SAEs have been used to identify features associated with semantic concepts, factual knowledge, and alignment-relevant behaviors, and have also been extended to multimodal representations [Huben et al., 2024; Lou et al., 2025; He et al., 2025].

Probing. Probing is a representation analysis technique that treats an LLM as a frozen feature extractor and trains an auxiliary lightweight classifier, typically a linear probe, to determine whether a target property can be decoded from internal representations. If the probe achieves high predictive performance, the corresponding property is considered to be encoded in the representation. Probes can be applied to residual-stream states, attention-head outputs, feed-forward network (FFN) activations, and other intermediate representations. By evaluating decodability across layers and components, probing enables the localization of properties such as factual knowledge, linguistic information, truthfulness signals, and sentiment [Morris et al., 2023; Yu et al., 2025]. For example, [Jiang et al., 2025c] show that first-token logit distributions contain signals predictive of confidence, truthfulness and safety, enabling probes to detect unanswerable questions, deceptive prompts, and jailbreak attempts before generation. While probing is simple, efficient, and widely applicable, it requires supervised datasets with property-specific labels, making probe construction a major limitation.

Logit Lens. The Logit Lens is a post hoc vocabularyprojection method that interprets intermediate representations by projecting them into the vocabulary space using the unembedding matrix [Belrose et al., 2023]. Applied layer by layer, it tracks how output-relevant token probabilities change during forward computation, providing a direct view of the semantic content accessible from hidden states. Beyond standard layer-wise projections, recent extensions apply Logit Lens to neuron-level activations in FFN layers, revealing specialized neurons that encode domain-specific knowledge and contribute to model generalization [Huo et al., 2024]. The Attention Lens [Jiang et al., 2025c] visualizes how visual evidence is reflected in generation trajectories and attention patterns, finding that hallucinated tokens often show weaker visual support in critical layers. Overall, the Logit Lens provides a training-free and intuitive tool for tracing how outputrelevant information becomes accessible across layers. Its diagnostic value lies in revealing the vocabulary-level content linearly readable from intermediate states. However, such projections remain inherently approximate, as vocabulary projections reflect what is linearly decodable through the final unembedding layer rather than what is causally utilized by the model during computation.

Component-Level Analysis at Different Granularities. Mechanistic analysis investigates model behavior at multiple levels of granularity, ranging from layers and attention heads to neurons and circuits. At the layer level, prior studies characterize how computational roles are distributed across model depth. For example, [Jin et al., 2025b] introduce Concept Depth to characterize where different types of information become linearly decodable across model layers. Through layer-wise probing on factual, emotional, and reasoning tasks, they show that simpler concepts are often accessible from shallow representations, whereas more complex concepts typically require deeper layers. At the head level, analyses identify functionally specialized attention heads, such as truthfulness-related heads whose activations encode truthfulness-related signals [Li et al., 2023], retrieval heads that access long-range contextual information and streaming heads that primarily attend to recent tokens and attention sinks [Xiao et al., 2025]. Beyond layer-level and headlevel explanations, interpretability research also investigate model behavior at the granularity of individual neurons and sparse circuits [Gurnee et al., 2023]. A line of work seeks to identify small subsets of neurons, or causal computational pathways, that support specific capabilities, including factual knowledge storage [Dai et al., 2022], language processing [Tang et al., 2024] and multimodal reasoning [Yang et al., 2026]. Beyond functional localization, recent studies suggest that even neurons activated by seemingly specialized tasks can participate in broader computational processes, revealing coupling between distinct capabilities and their influence on more general model behavior [Gurnee et al., 2024; Chen et al., 2025b].

4 VLMs Interpretability: Unique Challenges and Solutions

VLMs pose additional interpretability challenges because model behavior depends on how visual information is integrated with language representations throughout the model. This complicates the attribution of outputs to visual and textual evidence, as well as the analysis of multimodal information flow within the model. Moreover, VLMs are deployed in modality-rich tasks such as text-rich image perception, video understanding, and 3D spatial reasoning that require fundamentally different abilities, motivating task-specific analysis that can capture the unique computational processes.

4.1 Token-level Analysis

Token-level analysis examines how discrete visual and textual tokens encode information, interact across modalities, and influence model outputs. In particular, visual tokens are a central object of analysis because they provide the interface through which image content enters the language model and becomes available for grounding, reasoning, and generation.

Visual Token Attribution. Visual token attribution aims to characterize the contribution of individual visual tokens to model outputs and how their influence changes across layers. [Zhang et al., 2025a] investigate the importance of visual tokens through an information-flow perspective. By combining attention scores with Grad-CAM, they find that image-text interactions are concentrated in shallow decoder layers, where visual information becomes progressively concentrated among a small subset of tokens, revealing substantial redundancy among visual tokens in these layers. [Li et al., 2025b] identify contextual interference as a limitation of token-level explanations in VLMs, where visual activations induced by preceding tokens can obscure the evidence associated with later tokens. They propose an estimated-causalinference-based method that separates token-specific visual cues from contextual effects.

Information Inside Visual Tokens. Another line of work examines what information visual tokens encode. [Fan et al., 2026] show that visual tokens in VLMs exhibit pronounced semantic sparsity: visual tokens naturally separate into sink, dead, and alive groups, with only alive tokens carrying imagespecific semantics. They further find that grounded object semantics are concentrated in a very small subset of visual tokens. [Papadimitriou et al., 2025] use SAEs to decompose VLM embedding spaces into sparse concepts and study which concepts are predominantly visual, which are predominantly textual, and which support cross-modal semantic alignment. While many concepts are activated predominantly by either images or text, the authors show that numerous concept directions are nearly orthogonal to modality-separating subspaces and instead encode shared semantic structure.

4.2 InnerWorkings of VLMs

Recent works investigate how visual information is transformed inside VLMs. Using logit-lens analysis and attention knockout, [Neo et al., 2025; Zhang et al., 2025b] suggest a stage-wise processing hierarchy: lower layers capture scenelevel information, middle and late layers integrate questionrelevant visual evidence, and later layers support answer formulation and syntactic refinement. Similarly, [Huang et al., 2025b; Kaduri et al., 2025] examine how textual instructions modulate visual information processing in VLMs. Their analyses suggest a staged process in which visual information is first pre-processed and filtered according to the instruction, then transferred into language representations and retrieved during generation. [Li et al., 2026] uses logit-lens to trace the layer-wise evolution of visual-encoder representations and reveal that early layers primarily encode local attributes such as color and texture, while deeper layers progressively compose these low-level cues into object-level semantic representations. [Wu et al., 2025] provide evidence for the semantic hub hypothesis, showing that semantically equivalent inputs across languages, modalities, and data types are mapped to a shared representational space. This shared space appears to be scaffolded by the model's dominant pretraining language, helping explain both cross-domain generalization and why logit-lens analyses can recover interpretable semantic structure from intermediate representations.

5 FromExplanation to Intervention: Actionable Explainability for LLMs and VLMs

Actionable explainability treats explanations not as descriptive outputs, but as tools for guiding downstream interventions on model behavior. In this view, an explanation is actionable when it identifies a concrete targe whose manipulation can produce a measurable change in model behavior. Recent work [Orgad et al., 2026] argues that actionability should be judged by two coupled properties: concreteness, meaning that the explanation identifies a specific action, and validation, meaning that the action is empirically shown to improve an objective. In practice, actionable LLM interpretability typically follows a three-stage workflow: localizing a relevant mechanism, verifying its causal or functional role, and leveraging the verified signal for model improvement. The following sections review representative applications of this paradigm in LLMs and VLMs.

5.1 Mitigating Hallucinations

Hallucinations refer to generated content that is unsupported by contextual evidence or inconsistent with facts [Huang et al., 2025a]. Such evidence may comes from the prompt, retrieved context, or factual knowledge expected from the model. VLM hallucinations also involve failures of visual grounding, where generated content is insufficiently supported by image evidence. Actionable interpretability addresses hallucinations by identifying internal mechanisms associated with hallucinations and using these mechanisms to guide hallucination detection or mitigation.

Interpretability studies suggest that hallucinations are associated with internal mechanisms governing how models use evidence and factual knowledge. [Sun et al., 2025] disentangle the roles of retrieved context and parametric knowledge, showing that hallucinations arise when copying heads fail to preserve retrieved evidence while knowledge FFNs overinject parametric knowledge into the residual stream. This imbalance provides both diagnostic signals and intervention targets for hallucination mitigation. A related line of work studies whether models internally recognize the entities they are asked about. Using SAEs, [Ferrando et al., 2025] identify directions that distinguish known from unknown entities. These directions are causally linked to hallucination and refusal behavior. This suggests that hallucinations are partly associated with internal representations of entity familiarity. In VLMs, hallucinations arise when visual evidence is weakly incorporated into the model's internal generation process. Via logit lens, [Jiang et al., 2025a] find that hallucinated tokens exhibit lower logit confidence, reflecting weaker internal association with visual evidence. At the attention level, [Jiang et al., 2025c] show that hallucinated objects receive weaker or less consistent cross-head attention in intermediate layers than grounded objects At the representation level, [Phukan et al., 2025] find lower similarity between hallucinated answers and image-region embeddings, indicating weaker alignment between hallucinated answers and visual evidence. These grounding signals also motivate targeted interventions. [Jiang et al., 2025a] mitigate hallucinations by projecting the hallucinated image embeddings onto the subspace orthogonal to the corresponding hallucinated object's textual embeddings, thereby reducing unsupported object semantics in the visual representation. [Jiang et al., 2025c] improve grounding by injecting an additional attention term computed from the average attention across all heads in intermediate layers, encouraging the model to prioritize image regions that consistently attract attention across heads.

5.2 Knowledge Editing

Knowledge editing encompasses several objectives, including knowledge updating and knowledge unlearning, which aim to modify targeted knowledge while preserving unrelated model behavior. Mechanistic interpretability provides a natural foundation for this objective by localizing the internal representations that encode and retrieve knowledge. Most knowledge editing methods therefore follow a locatethen-edit paradigm, where knowledge-bearing components are first localized and then selectively modified.

At the neuron level, [Dai et al., 2022] introduce the concept of knowledge neurons, identifying a small set of FFN neurons whose activations are causally associated with the expression of specific factual knowledge. By manipulating these neurons and their corresponding FFN parameters, they demonstrate preliminary capabilities for knowledge editing without full model fine-tuning. [Li et al., 2024] further refine mechanistic localization for knowledge editing by disentangling the roles of self-attention and feed-forward networks. Their analysis suggests that FFNs primarily store factual associations, whereas attention layers encode more general knowledgeretrieval patterns. Guided by this insight, they restrict edits to FFN weights, leading to more precise and reliable knowledge updates. [Guo et al., 2025] show that effective knowledge editing depends on localizing the mechanisms responsible for factual lookup. By restricting edits to fact-lookup MLPs, they achieve more robust unlearning across prompt variations. Similar mechanistic editing paradigms have been extended to VLMs. [Basu et al., 2024] introduce a multimodal causal tracing framework to identify the layers responsible for storing and transferring factual information in VLMs. Their analyses reveal that factual associations are concentrated in early causal MLP layers, and editing these layers enables effective correction of erroneous knowledge and insertion of new factual associations.

5.3 Fairness and Bias Mitigation

Interpretability-informed debiasing seeks to identify internal mechanisms responsible for biased behavior and intervene on them directly. Existing studies suggest that bias can be localized to identifiable neurons, latent representations, and sparse semantic features, enabling targeted interventions that mitigate biased behavior while preserving general capabilities. [Liu et al., 2024] introduce social bias neurons, whose activations are causally associated with demographic disparities in model outputs. Using gradient-based attribution, they localize these neurons and mitigate biased behavior by suppressing their activations during inference. In contrast to prior neuron-suppression and neuron-editing approaches, [Pan et al., 2026] identify know-bias neurons associated with bias awareness and selectively amplify their activations during inference, mitigating social bias through knowledge activation rather than behavior suppression. [Neplenbroek et al., 2025] show that demographic biases can also emerge in latent user representations. Using linear probes, they reveal that stereotypical conversational cues induce demographic representations that may even override explicitly stated user identities. They further steer these representations toward the userspecified identity, mitigating stereotype-driven implicit personalization. [Yalavarthi et al., 2026] leverage SAEs to decompose VLM embeddings into interpretable feature directions and identify those associated with spurious attributes. By removing the corresponding directions through orthogonal projection, they mitigate spurious correlations, demonstrating that SAE features can serve as actionable targets for fairness and robustness interventions.

5.4 Improving Safety

As LLMs are deployed in increasingly consequential settings, unsafe outputs raises substantial concerns, while the opacity internal mechanisms further complicates safety auditing and risk mitigation. Actionable interpretability addresses this challenge by identifying safety-relevant components and representations that can be monitored or intervened [Lee et al., 2025]. A recurring finding across both LLMs and VLMs is that unsafe behavior can be traced to a small subset of internal mechanisms. At the component level, safety-critical heads, neurons, and layers have been shown to play disproportionate roles in refusal, toxicity mitigation, and harmfulquery detection, enabling selective editing, targeted tuning, or activation control [Zhou et al., 2025; Zhao et al., 2025; Li et al., 2025a]. At the representation level, safety-relevant behavior such as refusal behavior can be modulated through low-dimensional latent representations [Arditi et al., 2024], while toxicity-related SAE features [Goyal et al., 2025] provide sparse targets for steering harmful generations. In VLMs, visual inputs introduce additional failure modes by allowing harmful signals to propagate through multimodal representations. Multimodal jailbreaks have been associated with distinctive hidden-state trajectories [Jiang et al., 2025b], harmful activation directions that can be suppressed during inference [Wang et al., 2025a], and sparse visual-textual tokens whose pruning and restoration mitigate unsafe generations [Chen et al., 2025a]. Furthermore, [Xu et al., 2025] show that text-side safety mechanisms do not reliably transfer to visual inputs, motivating cross-modal alignment as a mechanism-aware safety intervention.

5.5 Enhancing Capability

Interpretability can improve performance by turning explanatory findings into actionable intervention targets. Recent studies leverage mechanistic insights into attention, representations, neurons and heads to guide inference-time interventions and targeted fine-tuning. A representative example is post-hoc attention steering. [Zhang et al., 2024] propose PASTA, which identifies influential a small set of functionally influential attention heads and reweights them toward useremphasized spans at inference time, enabling fine-grained control without additional training. This interpretabilitydriven intervention translates head-level mechanistic insights into substantial improvements in instruction following and knowledge integration. [Chen et al., 2025c] analyze attention trajectories over image tokens and show that successful spatial reasoning hinges on attention aligning with true object locations. Building on this diagnosis, they propose to reshape attention based on confidence score: sharpening attention when confidence is high and broadening it when low. Beyond attention, residual-stream representations can serve as actionable targets for intervention. [Højer et al., 2025] identify latent directions associated with successful reasoning by contrasting internal activations from correct and incorrect reasoning trajectories. The resulting control vectors are injected into the residual stream at inference time, steering the model toward reasoning-favorable representational states and improving performance across multiple reasoning benchmarks. Actionable interpretability can inform fine-tuning. By identifying a sparse set of keystone neurons that are consistently important across multiple capability dimensions, [Ji et al., 2026] demonstrate that updating only parameters associated with these neurons can achieve comparable or even better performance to full-parameter fine-tuning while better preserving all capabilities.

5.6 Improving Efficiency

Interpretability reveals how computational roles are distributed across model components, exposing functional specialization and computational redundancy at different levels such as tokens, heads and layers that can be leveraged for efficient inference. At the head level, [Xiao et al., 2025] identify a functional division between retrieval heads, which access long-range contextual information, and streaming heads, which primarily attend to recent tokens and attention sinks. Exploiting this decomposition, they retain full KV caches for retrieval heads and use compressed caches for streaming heads, reducing memory and latency while preserving long-context performance. At the layer level, [Chen et al., 2025d] show that consecutive layers contribute unequally to representation transformation, with some layer blocks inducing only minor changes in hidden representations. Thus, they replace low-impact layer blocks with lightweight modules, achieving compression with limited performance degradation. At the channel level, [Le et al., 2025] use a small subset of informative hidden states to probe future representations and infer batch-specific channel importance. These activation-derived signals are then used to guide adaptive structured pruning during inference. Similar principles extend to VLMs. [Wang et al., 2025b] reveal that fewer than 5% of attention heads are consistently responsible for visual concept understanding. Exploiting this head-level sparsity, they allocate KV-cache budgets asymmetrically across attention heads. Through visual information-flow analysis, [Yin et al., 2025] show that shallow layers inject visual cues into instruction tokens, whereas deeper layers consolidate unimodal visual semantics through image-image interactions. Guided by this insight, they prune image tokens at selected layers to reduce computation while preserving performance.

6 Evaluation: Datasets and Benchmarks

Developing reliable benchmarks for actionable LLM interpretability remains challenging due to the lack of consensus on evaluation standards, especially as model capabilities continue to evolve and expand beyond purely text-based tasks. Nonetheless, recent efforts have begun to address this gap by moving toward evaluations grounded in intervention effectiveness and downstream utility.

Different from descriptive LLM interpretability benchmarks that primarily assess whether an explanation is faithful, stable, or human-understandable, actionable LLM interpretability introduces a stronger objective: after using the explanation, can the evaluator produce a verified improvement in model behavior without unacceptable collateral damage [Orgad et al., 2026]? This shift necessitates new evaluation protocols that measure not only explanatory quality but also the effectiveness of the actions derived from those explanations. In this survey, we argue that actionable interpretability should be evaluated along five complementary dimensions. First, localization validity assesses whether a method identifies the feature, token, head, layer, or circuit that mediates the behavior. Second, causal efficacy measures whether intervening on the identified target changes the intended behavior. Third, specificity evaluates whether the intervention selectively changes the targeted behavior while preserving non-targeted model behavior. Fourth, robustness and scalability examine whether intervention effects persist across prompts, distribution shifts, and model scales. Fifth, utility asks whether the explanation supports a better downstream action.

Existing benchmarks cover these criteria unevenly. InterpBench [Gupta et al., 2024] constructs semi-synthetic Transformers with known ground-truth circuits using Strict Interchange Intervention Training (SIIT), enabling controlled evaluation of circuit discovery methods. By constraining non-circuit nodes from influencing model outputs, SIIT provides realistic testbeds while preserving circuit-level ground truth. MIB [Mueller et al., 2025] benchmarks mechanistic interpretability methods along circuit localization and causalvariable localization. It evaluates whether methods recover concise causal pathways and whether internal features align with task-relevant causal variables. SAEBench [Adam et al., 2025] evaluates SAEs beyond reconstruction quality, covering interpretability, feature disentanglement, concept detection, and practical applications such as unlearning, thereby providing a more comprehensive assessment of SAE utility. AxBench [Zhengxuan et al., 2025] evaluates representationbased steering methods against prompting and finetuning baselines, finding that prompting and finetuning consistently outperform SAE-based interventions, while even simple steering vectors achieve stronger control than SAEs. [Guo et al., 2025] use knowledge editing and unlearning as functional testbeds for evaluating localization quality. Their results show that localization methods should be assessed not only by whether the identified components influence model outputs, but also by whether interventions on those components produce robust changes with limited unintended effects.

Looking forward, progress in interpretability benchmarking will depend on the development of rigorous evaluation standards. Future benchmarks should clearly distinguish among explanatory objectives and intervention utility-and measure their relationships rather than treating interpretability as a single property. For VLMs, a key challenge is establishing whether explanations are grounded in the evidence actually used by the model.

7 Challenges and Opportunities

7.1 Challenges

Several challenges limit the effectiveness and broader adoption of actionable interpretability in modern foundation models. The first challenge is how to rigorously measure whether an LLM interpretability method is effectively actionable. There is no widely accepted standard for what constitutes a "good" explanation. Human evaluations suffer from subjectivity and low scalability, while automatic metrics-such as fidelity or stability-often fail to align with human understanding [Braun and Forell, 2025]. This lack of consensus undermines reproducibility and meaningful cross-method comparison. Developing reliable benchmarks remains challenging due to the absence of standardized evaluation criteria, particularly as model scale and modality complexity continue to increase. Existing benchmarks [Gupta et al., 2024; Adam et al., 2025] remain fragmented and task-specific, underscoring the need for unified evaluation frameworks that jointly assess faithfulness, robustness, and practical utility across textual and multimodal settings. The second challenge arises from the ambiguity and multifactorial nature of LLM outputs. Outputs are often shaped by entangled influences-including input phrasing, training distributions, prompt context, and internal hidden states. Attribution-based interpretability tends to conflate correlation with causation, making it hard to determine whether a model truly "used" a feature or merely "noticed" it. The localization of decisive mechanisms an open problem.

A third challenge concerns intervention reliability. Many interpretability-guided interventions show effectiveness under specific settings, yet their robustness across prompts, domains, modalities, or model scales remains poorly understood. Recent studies suggest that internal representations can shift substantially across contexts and languages through specialized latent-space transition mechanisms [Tezuka and Inoue, 2025], while activation-space interventions may transfer across model families through shared representation mappings [Oozeer et al., 2025]. Nevertheless, the broader stability and generalizability of mechanistic interventions remain open questions. A related challenge is the scalability of actionable interventions. Many existing methods rely on computationally expensive procedures such as perturbation-based methods and SAEs. While effective in certain settings, these approaches may introduce significant cost overhead in LLMs and VLMs. Developing lightweight and scalable intervention mechanisms therefore remains an important direction for actionable interpretability research.

Finally, modern LLMs operate as highly non-linear complex systems. Their internal dynamics, driven by deep transformer stacks and attention mechanisms, render inference process opaque and resistant to mechanistic explanation. Unlike small-scale models, tracing causal chains within these architectures is prohibitively complex. Recent work [Sharkey et al., 2025] on mechanistic interpretability further underscores the difficulty of mapping high-dimensional representations back to human-intelligible concepts. Even when technical explanations (e.g., saliency maps or token attributions) are available, end users like policymakers or domain experts often struggle to translate them into actionable insight. This gap between technical explanations and human understanding calls for more user-centric designs, such as contrastive explanations or interactive interpretability systems.

7.2 Opportunities

Despite these challenges, actionable interpretability presents significant opportunities for improving LLM deployment. It is central to building trust in high-stakes settings by enabling the detection of hallucinations, bias, and unsafe behaviors. It also serves as a powerful diagnostic tool and guides targeted interventions across the entire deployment pipeline including data curation, prompting strategies, and architecture adjustments. By generating intelligible and verifiable explanations, interpretability strengthens human-AI collaboration, facilitating oversight and reducing misuse. Moreover, actionable LLM interpretability have catalyzed methodological innovation, inspiring self-explaining models, LLM-as-explainer frameworks, and interactive explanation systems. Thus, future research may further enable scalable and real-time interpretability frameworks that support continuous monitoring and adaptive intervention in LLMs.

Actionable interpretability also enables LLMs to surface mechanistic patterns in biology, chemistry, and medicine, as early efforts have shown that interpretable models can reveal meaningful structure, thereby transforming models from passive predictors into tools for hypothesis generation and controlled scientific reasoning. Moreover, emerging approaches [Kim et al., 2025] treat LLMs as cooperative agents that provide multi-turn, concept-aware explanations to support human oversight. The concept of "agentic interpretability" advocates for LLMs to function not merely as providers of static explanations, but as interactive partners that adapt their explanatory behavior to support timely and effective human intervention.

8 Conclusion

This survey reviewed recent advances in LLM interpretability through the lens of actionability, organizing existing methods across attributional and mechanistic paradigms, extending this taxonomy to emerging approaches for VLMs, and examining how interpretable signals enable targeted interventions for downstream applications. Despite notable progress, the opacity and growing complexity of modern LLMs continue to limit trustworthy understanding and control. Future work should develop reliable evaluation frameworks and practical interventions that remain scalable across prompts, domains, modalities, and model scales, while enabling human-AI collaboration that is more transparent, controllable, and aligned with human needs. Advancing actionable interpretability will be essential for building models that are not only more effectively improved and controlled, but also more closely aligned with human needs.

Acknowledgments

This work was supported by the Army Research Office (ARO) under cooperative agreement W911NF-24-2-0133.