Mitigating Tool Overuse for LLMs via Active Knowledge Boundary Probing
Mitigating Tool Overuse for LLMs via Active Knowledge Boundary Probing
Zhaoyu Yang, Wenjun Ke, Yuanyao Li, Yuchen Liu, Junqi Xu, Peng Wang, Hengyuan Xu
Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence
Main Track. Pages 6018-6026.
https://doi.org/10.24963/ijcai.2026/670
Tool-augmented methods aim to enhance the reasoning capabilities of large language models (LLMs) by invoking external tools, which can be broadly categorized into training-free and training-based methods. Training-free methods can directly instruct LLMs to invoke external tools, but they exhibit limited generalization in dynamic environments. In contrast, training-based methods use a teacher model to generate tool-use trajectories and fine-tune a student model to distill the trajectories that can generalize across tasks. However, they still face two major challenges: (1) Knowledge boundary mismatch: teacher trajectories may contain steps that the student cannot execute without tools due to their mismatch of knowledge boundaries. (2) Tool overuse propagation: the student inherits the misaligned tool-use strategy from teacher model. To address these challenges, we propose RADAR, an automated framework for knowledge boundary discovery and tool overuse mitigation. First, RADAR performs semantic anchoring to select representative seeds. Second, simulated annealing probing is employed to actively explore the student model’s tool-use boundary and acquire sparse, verified student-aware labels. Third, it globally propagates these labels and rewrites conflicts, yielding deployment-aligned decisions with fewer unnecessary tool calls. We conduct experiments on multiple benchmarks, compared to SOTA method, RADAR reduces average tool-use by 52.7% while improving accuracy by 3.2% across three models.
Keywords:
Natural Language Processing: Interpretability and analysis of models for NLP
Natural Language Processing: Resources and evaluation
Natural Language Processing: Tools
