Learning Multi-Indicator Weights for Data Selection: A Joint Task-Model Adaptation Framework with Efficient Proxies

1 Introduction

Instruction tuning adjusts large language models (LLMs) to match human intent, and the choice of training data significantly influences model performance. Recent studies indicate that larger datasets may not necessarily lead to better performance, as noisy data can incur high computational costs and introduce hallucinations [Chen and Mueller, 2024; Si et al., 2025]. Instead, the effectiveness of instruction finetuning mainly depends on the quality and diversity of the samples, rather than the size of the dataset [Zhou et al., 2023; Chen et al., 2024].

Previous work generally evaluates instruction tuning data from three dimensions: quality, which reflects the correctness and utility of responses [Liu et al., 2024; Chen et al., 2024]; complexity, which captures the logical depth or difficulty of queries [Li et al., 2024b]; and diversity, which ensures broad coverage across various domains [Ge et al., 2024; Lu et al., 2024]. Consequently, most selection pipelines aggregate these indicators using static weighting schemes or predefined manual rules [Cao et al., 2024]. However, two challenges persist in these approaches. First, static weighting schemes encounter the problem of poor generalizability [Diddee and Ippolito, 2025] and do not account for the varying requirements of different downstream tasks [Cao et al., 2024] and different LLMs. The optimal weighting among indicators is universal across different tasks; for instance, mathematical reasoning tasks demand high logical complexity data, whereas general-purpose alignment may prioritize linguistic diversity. Second, while some data-driven approaches optimize the selection by identifying influential samples [Xia et al., 2024; Tang et al., 2025], they often incur high computational costs. These methods typically rely on "selectfinetune-evaluate" loops or high-dimensional gradient calculations to score the data. To reduce training costs, some recent approaches employ in-context learning (ICL) as a proxy for fine-tuning performance [Wang et al., 2024]. However, they remain constrained by their sample-level granularity, suffering from high inference time costs due to the traversal of the dataset.

In this paper, we propose a joint Task-Model Adaptation framework with efficient Proxies (TMAP) for instruction data selection. To deal with the first challenge, unlike approaches that rely on static heuristic rules, TMAP treats data selection as an optimization problem by dynamically adjusting the weighting of various indicators, to match the requirements of both the downstream task and the base model. To deal with the second challenge, TMAP introduces an efficient performance proxy that utilizes ICL signals on a tiny validation sets, bypassing the prohibitive cost of fine-tuning an LLM. By leveraging these high-fidelity proxies, TMAP can rapidly evaluate weight configurations and update them without training the model. This approach significantly reduces execution time while ensuring the selected data remains effective for target tasks. Our main contributions are summarized as follows:

• We propose a joint task-model adaptation framework that tackle the problem of static heuristics by using learned weights over multiple selection indicators to tailor data subset to specific task requirements and model capabilities.
• We introduce efficient proxies of select-finetuneevaluate loops via performing ICL on tiny validation sets. By utilizing validation results as proxies, we enable efficient weight-level optimization without the need for full-scale finetuning and full-validation inference.
• Experiments across various model families and benchmarks demonstrate that our method achieves performance comparable to full-dataset tuning, reaching considerable results on reasoning tasks such as GSM8K while utilizing only 30% of the training data.

2 RelatedWork

Instruction data selection with different indicators. Current indicators for instruction tuning can be broadly categorized into three types: heuristic-based, LLM-based, and ICLbased. Heuristic-based approaches assess data quality using predefined heuristic metrics. A simple yet effective metric is selecting data with the longest output length [Zhao et al., 2024]. [Li et al., 2024b; Li et al., 2024a] propose an instruction following difficulty (IFD) metric that leverages GPT-2 to compare the perplexity of model responses with and without instructions. LLM-based methods employ LLMs as scorers to identify high-quality data samples. For instance, ALPAGASUS [Chen et al., 2024] and DEITA [Liu et al., 2024] use template prompts to allow LLMs to rate data for both quality and complexity.

Some LLM-based selection use data-driven optimization to identify influential data best aligned with model requirements. LESS [Xia et al., 2024] utilizes gradients from LLM to select data, while iterative methods like Middo [Tang et al., 2025] continually update the training LLM with selected high complexity and quality data in each iteration, aiming to select data that is aligned with the current model's needs. Additionally, LASER [Mirza et al., 2025] trains and employs data-specialized scorers to assess data quality.

ICL-based methods assess the quality of each candidate data point by measuring its usefulness as an ICL example for a given evaluation set. Nuggets [Li et al., 2024c] computes a quality score by comparing the correctness gap between a zero-shot setting and a one-shot setting that uses the candidate data as the prompt. [Zhang et al., 2025] propose an in-context approximation (ICA) approach, which estimates the loss of candidate examples by conditioning on a carefully selected in-context held-out set, and then re-weights gradient updates accordingly. Data Whisperer [Wang et al., 2025a] randomly samples data from the candidate pool as demonstrations and queries, then computes the weighted evaluation score for each query based on the attention mechanism. The scores quantify the usefulness of the candidate data.

These data selection methods, however, often overlook the specific needs of diverse downstream applications. Furthermore, ICL-based methods incur high computational costs for new data, and the reliability of using pre-defined or randomly sampled evaluation sets is limited. Iterative approaches typically rely on "select-finetune-evaluate" loops that are computationally expensive and difficult to scale.

Benchmark compression for efficient evaluation. The increasing computational costs required for evaluating LLMs have spurred significant research into efficient benchmarking methods. Many studies have identified redundancy in benchmark items [Ye et al., 2023; Perlitz et al., 2024]. Consequently, methods now employ carefully selected small evaluation subsets to predict full benchmark performance. For instance, [Vivek et al., 2024] proposes the Anchor Points method and identifies critical data points to assess models with far fewer examples [Polo et al., 2024].introduces TinyBenchmarks, demonstrating that by training an Item Response Theory (IRT) model [Cai et al., 2016] on just 319 curated items can reliably estimate full performance on MMLU within 2% error [Wang et al., 2025b]. proposes the EssenceBench framework, which formulates benchmark compression as an optimization problem and addresses it through an iterative genetic algorithm. Our work differs by focusing on learning task-model-adaptive weights over a rich set of indicators, while sharing only the goal of reducing the cost of efficient evaluation.

3 Methodology

3.1 ProblemFormulation

We formulate the problem of instruction data selection as follows. Given a candidate pool D = {d1, d2, . . . , dN} containing N samples, for each sample di, we pre-calculate K heuristic-based and LLM-based indicators {si,j}Kj=1 . These indicators are then min-max normalized to form a feature vector si ∈ [0, 1]K . Our objective is to determine an optimal weight configuration w = [w1, w2, . . . , wK]⊤, wi ∈ [0, 1] to calculate a weighted score Si(w) for each sample:

Based on these scores, a training subset Dsub(w) is formed by selecting the top-M samples where M < N. Our goal is to find an optimal w that maximizes the target task performance of the target LLM after instruction tuning.

Efficient Proxies. Ideally, data-driven optimization for the selection problem is formulated as a select-finetune-evaluate loop. The optimization objective in this paradigm is typically denoted as follows:

where SFT(·) denotes the supervised fine-tuning of a LLM Mon the selected subset, and Eval(·, Vfull) represents evaluating on the full downstream validation set. However, this process is computationally prohibitive due to the repeated LLM training and large-scale evaluation inference. To address these challenges, we redefine the selection process using two efficient proxies: ICL and tinybenchmark. Instead of performing full-scale fine-tuning, we utilize few-shot ICL to measure the utility of Dsub(w). Instead of the full validation set Vfull, we evaluate performance on a tiny validation set Vtiny ⊂ Vfull. We define a reward function R(w) that serves as a proxy for the fine-tuning outcome:

By shifting the optimization target to argmaxwR(w), the framework finds approximately optimal configurations at the indicator level while reducing computational costs and execution time. The remaining problem is how to guarantee that the performance approximation via efficient proxies is reasonable. We introduce the details in Section 3.4

3.2 Overview of TMAP

As show in Figure 1, we treat data selection as a weight optimization problem. The TAMP pipeline consists of: (1) sampling a tiny validation set via K-means on IRT embeddings; (2) scoring the candidate data pool across an extensible set of indicators; (3) calculating weighted scores and selecting the highest-scoring data to form an ICL demonstration set; (4) iteratively optimizing indicator weights using Bayesian Optimization, guided by an ICL-based reward signal on the tiny validation set; and (5) selecting the final subset based on the optimized weights once optimization finished.

3.3 Multi-Dimensional Indicators

To capture multiple features of data quality, we score each candidate sample di with K dimensions to form a normalized feature vector si ∈ [0, 1]K . Following by prior work, we implement four key indicator groups: (1) Semantic Quality: LLM-based scores alpagasus score and deita quality score, which capture the faithfulness and helpfulness of responses [Chen et al., 2024; Liu et al., 2024]. (2) Complexity and Difficulty: ifd and deita complexity score, which measure how challenging an instruction-response pair is for the model [Li et al., 2024a; Liu et al., 2024]. (3) Linguistic Heuristics: output length [Zhao et al., 2024] metric, which aligns with general-purpose filtering. (4) Domain-specific metrics: For the GSM8K dataset, we use reason steps [Jin et al., 2024a] to approximate the length of the underlying chain-ofthought. Note that TMAP is easy to adapt to different domains (e.g., dropping reasoning metrics for dialogue tasks).

3.4 Efficient Proxies

Proxy for Finetuning. To avoid the high computation cost of repeated SFT in the optimization cycle, we leverage ICL as an efficient proxy. Recent studies confirm that a sample's utility as a few-shot demonstration strongly correlates with its utility as a training instance [Yin et al., 2024; Zhao et al., 2025]. Note that, unlike prior work, we do not replicate exact SFT performance. Instead, our goal is to preserve the relative ranking of data quality. As shown in Appendix A 1 , varying the number of ICL demonstrations consistently maintain the same performance trends as observed in SFT training. This demonstrates that ICL can serve as a reliable proxy of SFT in evaluating the influence of different data on the target model. To reduce the impact of ICL prompt order sensitivity on the reward signal R(w), we partition the selected subset Dsub into m distinct groups of n-shot exemplars. We then evaluate the base model using these groups as prompts, deriving the reward from the aggregated accuracy.

1 Due to space constraints, the appendix is available in our extended arXiv version: https://arxiv.org/abs/2605.09665.

Proxy for Full-validation Evaluating. To avoid timeconsuming full-set inference, we construct a compact tinybenchmark Vtiny of S representative samples. Our target is to approximate the full-set performance via weighted evaluation on this subset:

YvM ∈ {0, 1} is the correctness of sample v and wv is the weight of sample v. The underlying principle is that an LLM can accurately answer a broad range of questions if and only if it correctly answers these representative samples. To select representative samples, we employ IRT to generate embedding vectors. In the context of TMAP, we adopt the Multidimensional Two-Parameter Logistic (M-2PL) model from IRT. The M-2PL model assumes that the probability of the LLMMcorrectly solve the validation sample v is:

where θM represents latent LLM abilities and (αv, βv) represent item difficulty and discrimination. We concatenate the point estimates parameters to form embedding vectors Ev = (ˆα, ˆ β) for sample v. We apply K-means clustering to identify centroids and select the item v∗ closest to form Vtiny :

To further refine precision of tinybenchmark, following [Polo et al., 2024], we combine the observed results and IRT prediction:

(6) where λ is a learnable parameters, δ1 = ˆλ |Vtiny| , δ2 = 1-ˆλ |Vfull \Vtiny | and ˆλ = |Vtiny|/|Vfull| ∈ [0, 1]. Experiment in Appendix B proves the reliability of the proxy for full-validation evaluating.

3.5 Task-Model AdaptiveWeight Optimization

We formulate data selection as a black-box optimization problem: w∗ = argmaxwR(w), where R(w) represents the proxy reward signal. To efficiently solve this nondifferentiable objective, we employ Bayesian Optimization (BO) [Moˇckus, 1974].

As outlined in Algorithm 1, the process iteratively updates weights w. In each iteration t, we compute weighted scores Si = ∑Kj=1 wt,j · si,j for all candidates, select the top-M instances, and partition them into m groups. These groups are evaluated via ICL on Vtiny to obtain a robust reward signal Rt, which guides the Bayesian optimizer to propose the next configuration wt+1.

Algorithm 1 Task-Adaptive Weight Optimization

• 1: Input: Candidate pool D, Indicators {sj}Kj=1 , Target task tiny validation set Vtiny, Selection size M
• 2: Initialize: Weights w0, Bayesian Optimizer B
• 3: for t = 1 to T do
• 4: Si ← ∑Kj=1 wt,j · si,j for each di ∈ D
• 5: Dsub ←Top-M(D, Si)
• 6: {Dsubl }ml=1 ← Partition(Dsub)
• 7: Rt ← EvalICL({Dsubl }ml=1 , Vtiny) {Average performance over m groups}
• 8: wt+1 ←Update(wt,Rt)
• 9: end for
• 10: Return w∗

3.6 Final Filtering and Robustness Refinement

To capture the multifaceted nature of data quality, we score each candidate sample di across K dimensions to form a normalized feature vector si ∈ [0, 1]K . Drawing from prior work, we implement four key indicator groups:

• Semantic Quality: LLM-based scores, specifically alpagasus score [Chen et al., 2024] and deita quality score [Liu et al., 2024], which capture the faithfulness and helpfulness of responses.
• Complexity and Difficulty: Metrics like ifd and deita complexity score, which measure how challenging an instruction-response pair is for the

model [Li et al., 2024a; Liu et al., 2024].

• Linguistic Heuristics: Simple statistics such as output length [Zhao et al., 2024], consistent with general-purpose filtering metrics.
• Domain-Specific Metrics: For reasoning tasks like GSM8K, we include reason steps [Jin et al., 2024a] to approximate the length of the underlying chain-ofthought.

Note that TMAP is designed to be extensible, allowing for easy adaptation to different domains (e.g., excluding reasoning metrics for dialogue tasks).

4 Experiments

4.1 Experimental Setup

Datasets. We focus on knowledge and reasoning benchmarks that are widely used for evaluating instructiontuned LLMs. Specifically, we used the following datasets. GSM8K, a dataset of grade-school math word problems requiring multi-step reasoning [Cobbe et al., 2021]. ARC-C, a challenging multiple-choice science QA subset with strong distractors [Clark et al., 2018]. MUSR, a dataset for evaluating LLMs on multistep soft reasoning tasks specified in a natural language narrative [Sprague et al., 2023]. GPQA, a multiple-choice dataset of hard questions created by experts in biology, physics, and chemistry. For the synthetic-data experiments, we additionally construct DeepSeek-distilled variants of GSM8K and ARC-C by augmenting each example with model-generated chain-of-thought solutions, following the setup summarized in Section 2.

4.2 Baselines

We compared TMAP with several widely used data selection techniques: (i) Random, which randomly samples subsets from the dataset. (ii) IFD, which uses GPT-2-computed IFD scores to select high-quality samples. (iii) Longest, which selects data with longest output length. (iv) Alpagasus, which prompts a LLM with a hand-craft template to score and select data. (v) DEITA, which selects data with highest quality, complexity, and diversity. (vi) Data Whisperer, which leverages ICL and attention scores to select data.

4.3 Implementation Details

We use four backbone models of different scales and families: Mistral-v0.2-7B-Instruct [Jiang et al., 2023], Qwen2.57B-Instruct, Llama3-8B-Instruct and Llama2-13B-Chat. For GSM8K and ARC-C, we train the model using 30% samples from their corresponding training set, aligning with [Wang et al., 2025a]. For GPQA and MUSR, we use 10% samples from DS2-50K dataset for training following the settings of prior work [Pang et al., 2025]. We report the Exact Match (EM) accuracy on the test set for GSM8K and the other three tasks are reported with their multi-choice accuracy. All experiments were performed on 8 NVIDIA A100 GPUs. We utilized LoRA [Hu et al., 2022] for model fine-tuning. During the Bayesian optimization process, we set the ICL evaluation parameters to m = 5 groups and n = 3 exemplars per group. We employ 5 random initialization steps followed by 200 optimization iterations. Further details can be found in supporting materials.

4.4 Main Results: Effectiveness of TMAP

We evaluate the effectiveness of TMAP through multiple settings, including cross-model validation (Exp 1), synthetic data distillation (Exp 2), and multi-model generalization (Exp 3).

Performance Across Models and Tasks

As shown in Table 1, TMAP consistently outperforms heuristic baselines across all models. Notably, on Qwen2.5-7B with GSM8K, it achieves comparable performance to baselines while using only 30% of training data. These results demonstrate that learning task-specific weights for multidimensional heuristics is more effective than relying on a single fixed metric or uniform weighting.

Validation on Synthetic Data

In Exp 2, we evaluate TMAP on distilled datasets where reasoning paths are augmented using DeepSeek [Liu et al., 2025]. We expand our evaluation to include both the mathematical reasoning benchmark GSM8K and the science reasoning benchmark ARC-C. For this setting, we select 30% of the synthetic data pool for fine-tuning Llama2-13B and Mistral-v0.2-7B. As shown in Table 2, TMAP consistently outperforms other data selection baselines. For Llama2-13B on GSM8K, TMAP achieves a score of 52.77, surpassing DEITA (51.78) and Alpagasus (49.28). More notably, on the ARC-C benchmark using Mistral-v0.2-7B, our method reaches 77.05, which is virtually equivalent to the performance of fine-tuning on the full synthetic dataset (77.22), despite using only 30% of the training samples. This result highlights TMAP's capability to filter high-quality reasoning traces from DeepSeek-generated data, effectively identifying the "gold" within synthetic pools while discarding redundant or lower-quality generations.

Multi-Model Generalization

In Exp 3, we replace the single-model reward with a multimodel consensus signal obtained by averaging the ICL-based rewards fromMistral-v0.2-7B, Qwen2.5-7B, and Llama3-8B. As shown in Table 3, this consensus reward leads to slightly lower peak scores on each individual model compared to the single-model optimal configuration. However, the consensus weights remain competitive with strong baselines and, more importantly, capture a more robust data mixture that transfers well across architectures. This highlights the crossmodel generalizability of TMAP: even when no single model is explicitly favored during optimization, the learned weights deliver stable gains across Mistral, Qwen2.5, and Llama3.

4.5 Ablation Study: Impact of Diversity

In Exp 4, we investigate the impact of incorporating diversitybased filtering (following the DEITA approach [Liu et al., 2024]) after determining the optimal weights via TMAP. Contrary to the prevailing intuition that diversity is always beneficial, our results in Table 4 reveal a consistent performance decline across all tested models when explicit diversity constraints are enforced. Specifically, the accuracy for Mistral drops by 4.17%, while Llama3 and Qwen2.5 see reductions of 1.90% and 0.38%, respectively. This confirms our hypothesis that for reasoning-dense tasks, the priority should be logic-complexity rather than semantic surface variation. In such reasoning-dense settings, explicit diversity filtering may prune semantically similar yet logically distinct hard examples that are crucial for strengthening step-by-step mathematical reasoning.

4.6 Ablation Study: Optimal Granularity m

To investigate the impact of partitioning granularity on reward signal stability, we evaluate our method across various m ∈ {1, 2, 4, 8, 16} on the MUSR datasets using Qwen2.57B.

Analysis of Granularity m. As illustrated in Table 5, the choice of granularity m significantly influences reasoning performance, exhibiting an "inverted U-shape" trend. We observe that performance improves as m increases from 1 to 8, peaking at 46.00%, before declining at m = 16. This suggests that extreme granularities are suboptimal: overly coarse granularity (e.g., m = 1) fails to capture sufficient variance for robust ranking, while excessively fine granularity (e.g., m = 16) introduces noise that masks underlying weight patterns, thereby hindering precise optimization. Considering the trade-off between performance and computational efficiency, we adopt m = 5 in our main experiments. Notably, the superior performance observed at m = 8 implies that our reported results do not represent the absolute ceiling, indicating that TMAP possesses an even higher performance upper bound.

4.7 Efficiency Analysis

To assess the computational scalability of TMAP, we benchmark its overhead against three baselines: (1) Full-set Evaluation, utilizing the complete validation set for ICL reward calculation; (2) Data Whisperer, which computes scores by traversing the entire candidate dataset; (3) Standard SFT, representing the wall-clock time for a single fine-tuning epoch. To simulate resource-constrained environments, all benchmarks are conducted on two NVIDIA A100 GPUs using the Llama3-8B backbone.

As presented in Table 6, TMAP (m = 5) requires approximately 3 minutes per optimization iteration. A defining advantage of TMAP is its scalability regarding candidate pool size. While Data Whisperer exhibits linear complexity, where runtime surges from 11 hours to over 72 hours as the pool expands from GSM8K (7.5k) to DS2-50K (50k), TMAP maintains a nearly constant optimization cost (≈ 3.0 to 3.5 minutes per iteration) across both settings.This efficiency stems from the decoupling of the evaluation proxy from the size of the training set. TMAP's computational cost is determined solely by the inference complexity of the target task (specifically, generation length and the size of Vtiny). By evaluating weight configurations via a compact proxy rather than scoring individual samples, TMAP achieves high-fidelity selection with significantly reduced overhead, facilitating scalability to massive instruction tuning datasets.

5 Weight Interpretation and Analysis

A core contribution of TMAP is discovering autonomous, model-specific weighting schemes. We analyze the converged weights for Qwen2.5, Llama3, and Mistral on GSM8K to understand their distinct definitions of "highquality" data.

5.1 Model-Specific Heuristic Preferences

Table 7 and Figure 2 show that optimal weight configurations vary significantly, confirming that data quality is relative to the model's architecture rather than a static property.

Qwen2.5: Complexity andLength. Qwen2.5 assigns high weights to IFD (0.700), Reasoning Steps (0.679), and Output Length (0.674), while minimizing Alpagasus Score (0.050). As a strong reasoner, Qwen benefits most from structural complexity and hard data, relying less on external semantic judgments.

Llama3: Reliance on Strong Judges. Llama3 heavily weights Alpagasus Score (0.965) while ignoring DEITA Complexity (0.029). This indicates a priority for correctness and clear instruction-following over raw difficulty. TMAP steers selection toward authoritative samples to avoid introducing noise during alignment.

Mistral: High Information Density. Mistral prioritizes Perplexity (0.991) and DEITA Score (0.787) but ignores Reasoning Steps (0.101). Given the high-quality candidate pool, this preference for high perplexity suggests Mistral benefits from information density or surprisal, requiring data that diverges from its pre-training priors rather than predictable patterns.

5.2 The Necessity of Task-AdaptiveWeighting

The fact that DEITA Complexity is prioritized by Mistral (0.675) but largely ignored by Llama3 (0.029) highlights a critical trade-off: a sample that is "appropriately difficult" for one model might be redundant or incorrectly scored for another. By bypassing manual rule-setting, TMAP identifies these hidden alignment points between data characteristics and model capabilities.

6 Conclusion

In this paper, we introduced TMAP, a joint task-model adaptation framework designed for instruction tuning data selection, which overcomes the limitations of static heuristic weighting and the high computation costs of existing data selection methods. By combining Bayesian Optimization with two efficient proxies In-Context Learning (ICL) and tinyBenchmark, TMAP iteratively searches optimal, multidimensional indicator weights tailored to specific target tasks and model. Experiments across various model families and benchmarks demonstrate that TMAP significantly outperforms static baselines with high computational efficiency. These results validate the necessity of adaptive, task-specific data selection strategies in instruction tuning. Future work will focus on extending TMAP to other instruction-tuning categories, such as writing or coding, as well as exploring more efficient proxy signals for continual model improvement.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (62306344), Guangdong Basic and Applied Basic Research Foundation (2024A1515010253, 2026A1515011800), and the Open Research Fund of the State Key Laboratory of Blockchain and Data Security, Zhejiang University.

Contribution Statement

Jingze Song and Zihao Chen contributed equally to this work as co-first authors. They jointly developed the methodology, implemented the codebase, and conducted the experiments. Wenqing Chen supervised the research, provided methodological guidance, and served as the corresponding author. Zibin Zheng contributed to project administration and research support.