A Unifying Perspective on Model Reuse: From Small to Large Pre-Trained Models

1 Introduction

Recent years have witnessed the rapid advancement of machine learning, achieving competitive performance across various real-world applications. A typical machine learning pipeline consists of two core steps: collecting data and training a model [Mitchell, 1997], as illustrated in Figure 1 (top). This data-centric paradigm heavily relies on the quantity and quality of training data to ensure strong generalization. However, in resource-constrained settings, acquiring high-quality data at scale poses significant challenges, limiting the effectiveness of traditional machine learning approaches.

To this end, researchers are exploring how to reduce the cost by reusing existing knowledge. In this way, reusing PreTrained Models (PTMs) rather than training models from scratch-has emerged as a promising solution for knowledge transfer, especially with the growing availability of PTMs across diverse domains [Wolf, 2019]. This paradigm leverages a vast array of PTMs that encapsulate valuable inductive biases, making them highly beneficial for various downstream tasks. With well-designed reuse strategies, knowledge from these PTMs can be efficiently extracted and extended beyond their original scope, facilitating numerous applications [Zhou, 2016; Zhou and Tan, 2024; Tan et al., 2025; Lei et al., 2024].

The idea of reusing PTMs dates back to the 1990s when researchers explored the use of pre-trained neural networks as feature extractors for tasks such as waveform and handwritten character recognition [Guo and Gelfand, 1992]. As PTMs have grown more powerful-especially those trained on large-scale datasets-model reuse has emerged as a resourceefficient and performance-competitive alternative to training models from scratch. The concept of model reuse was formally introduced in [Zhou, 2016], highlighting the reusable property of machine learning models. Early implementations of model reuse include vanilla fine-tuning [Yosinski et al., 2014] and biased regularization [Kuzborskij and Orabona,

2017]. However, to address task-specific challenges such as distribution shifts and sample scarcity [Zheng et al., 2023; Zhuang et al., 2020], more advanced model reuse techniques have been developed across various areas [Kundu et al., 2020; Hinton et al., 2015; Chen et al., 2022].

With the rapid expansion of machine learning, an increasing number of PTMs are being developed and organized into public PTM repositories [Wolf, 2019; Tan et al., 2024], enabling the next evolution of model reuse based on a model zoo. The diversity of PTMs within a model zoo-ranging from small specialized PTMs tailored for domains such as medicine, finance, and education [Luo et al., 2024], to large foundation models with strong zero-shot capabilities [Bommasani et al., 2021]-introduces new challenges and opportunities. As shown in Figure 1 (bottom), a more advanced model reuse paradigm with a model zoo involves several key steps [Zhou, 2016; Zhou and Tan, 2024], e.g., (1) selecting appropriate PTMs from the model zoo, (2) leveraging the selected PTMs to help the learning of the target task, and (3) managing and improving the model zoo for easier selection and enhanced overall performance.

Despite the success of model reuse, two key challenges remain in comprehensively exploring its core principles. First, as model reuse becomes a natural choice in various applications, a holistic and task-agnostic perspective is needed to unify fragmented research efforts across different fields. Establishing a big-picture view of model reuse can bridge the gaps between specialized subfields and promote broader applicability. Second, with the emergence of large PTMs such as foundation models [Bommasani et al., 2021], reusing such models involves both shared principles and diverse implementations. Integrating model reuse across both small and large PTMs can help connect diverse methodologies and inspire the design of novel model reuse strategies.

To tackle these existing challenges, this survey provides a unifying perspective on model reuse by categorizing research efforts based on two fundamental steps: model selection and adaptation. We introduce a novel taxonomy to organize existing approaches. Model selection methods are categorized based on the selection mechanism, namely, semantic/rulebased, metric-based, and learning-based strategies. Model adaptation methods are classified according to the role of PTMs in the learning process, including leveraging PTMs for data preparation, model training, and inference.

Additionally, we distinguish between small and large PTMs, recognizing that while there is no strict threshold, we define small PTMs as relatively compact, specialized models, whereas large PTMs refer to general-purpose models trained on diverse data with strong zero-shot generalization across domains. By analyzing both their commonalities and differences, we highlight the advantages, challenges, and broader impact of model reuse across the machine learning pipeline.

The main contributions of this survey include:

• A unifying framework for model reuse, categorizing research based on PTM selection and adaptation strategies.
• A comprehensive review covering both small and large PTMs, highlighting their commonalities and differences.
• We summarize insights into emerging trends and challenges in the future of model reuse.

2 Preliminaries

2.1 Reusing PTMs from aModel Zoo

We illustrate the model reuse process using classification as an example, though the methodology can be extended to other tasks with different types of PTMs. In standard machine learning, a C-class classification task consists of a training set D = {(xi, yi)}Ni=1 with N examples, where each instance xi ∈ Rd and its corresponding label yi ∈ [C] = {1, . . . , C}. The goal is to construct a classifier fθ : Rd → RC with parameters θ that maps an input instance to C-dimensional confidence scores. Typically, f represents a neural network, and θ corresponds to its weights, which are learned through empirical risk minimization.

Reusing a Single PTM. Instead of training fθ from scratch, a well-trained model gΘ is often available and can be leveraged to facilitate the training of fθ. Here, gΘ is pre-trained on a dataset D′ = {(x′j , y′j )}N ′ j=1 with instances x′j ∈ Rd′ and labels y′j ∈ [C ′ ]. To reuse the expert knowledge in gΘ, an adaptation strategy is applied fθ = Adapt(fθ0 | D, gΘ), where θ0 is the initial state of the model before adaptation. Since the PTM gΘ may differ from fθ in various aspects-such as model architecture (f ̸= g), data distribution (Pr(D) ̸= Pr(D′ )), feature dimension (d ̸= d′ ), or label space (C ̸= C ′ )-the adaptation function Adapt(·) must address these heterogeneities. Reusing Multiple PTMs from a Model Zoo. With the proliferation of PTMs across different domains, model zoos, denoted as M = {g1Θ, . . . , gM Θ }, have emerged as valuable resources. These PTMs encode diverse inductive biases, making them highly beneficial for a wide range of target tasks. To effectively utilize these rich model repositories, [Zhou, 2016; Zhou and Tan, 2024] summarizes and highlights the importance of a "select-then-adapt" workflow. For simplicity, consider selecting a single PTM, and scenarios selecting multiple PTMs could be easily extended. In the model selection stage, a selection mechanism ranks the PTMs in M based on their relevance to the target task D and outputs the index of the most appropriate PTM:

fθ in Eq. 1 indicates the selection process also depends on the required architecture f over the target task. The workflow steps into the model adaptation stage once a PTM is selected, which is applied as follows:

Model selection ensures the most relevant PTM is identified, while adaptation efficiently bridges the gap between the PTM and the target task. The adaptation could be applied to the selected model itself (i.e., adapt gm Θ ) or to the target model fθ where the selected gm Θ provides auxiliary expert knowledge.

2.2 Small and Large PTMs

Model zoos often contain PTMs of varying types, including those pre-trained on diverse domains such as finance, education, and scientific research. These PTMs also differ in functionality, covering tasks such as classification, regression, object detection, image/text recognition, text generation, and reinforcement learning.

The emergence of large PTMs has significantly impacted the model reuse workflow, as computational costs have become a crucial factor, and there are diverse strategies for extracting expert knowledge from these models. While reusing both small and large PTMs shares fundamental principles, the differences in scale introduce distinct challenges and opportunities. This survey connects the reuse of both types of PTMs, highlighting their commonalities and differences to provide deeper insights into model reuse methodologies.

Since PTM size is relative, there is no strict boundary between small and large PTMs. Generally, we define small PTMs as models based on classical architectures and pre-trained for specialized tasks, such as ResNet for image classification [He et al., 2016]. In contrast, large PTMs typically contain billions of parameters, are general-purpose, and possess zero-shot capabilities across various tasks. For example, a large language model (LLM) can handle text generation, classification, and reasoning without explicit task-specific fine-tuning, adapting dynamically to new queries [Zhao et al., 2023].

2.3 Advantages and Goals

Model reuse facilitates knowledge transfer by leveraging pretrained models to improve generalization and reduce dependence on large labeled datasets. It also enables knowledge aggregation, where combining multiple PTMs enhances robustness, accuracy, and adaptability. By building on existing models, reuse accelerates training, lowers computational and data costs, and mitigates catastrophic forgetting in dynamic or continually evolving environments. Overall, model reuse improves scalability and efficiency while extending the applicability of machine learning across diverse tasks and domains.

3 Taxonomy

Model reuse consists of two core steps: model selection and model adaptation. Accordingly, we systematically categorize existing approaches into these two components, as illustrated in Figure 2. The objective of model selection is to identify relevant PTMs from the model zoo, which hinges on defining an appropriate retrieval metric. Based on how this matching is determined, we classify model selection methods into three categories: (i) semantic/rule-based methods, (ii) metric-based methods, and (iii) learning-based methods. These approaches differ in how they quantify model-task relevance. Once a PTM is selected, model adaptation ensures its effective integration into the target task. We categorize adaptation methods based on their application stage: (i) PTMs for data preparation, (ii) PTMs for model training, and (iii) PTMs for model inference. Since model reuse is a broad topic within machine learning, some related areas extend beyond the selection-adaptation paradigm. Therefore, we discuss model collaboration, compression, and representation learning, which contribute to the broader landscape of model reuse.

3.1 Model Selection

The target of model selection is to choose one or multiple related models from the model zoo that are suitable for the related dataset. This is a typical retrieval process, where the core problem is to rank all models in the model zoo correctly. Hence, we need to design a metric that measures the matching degree between models and the downstream tasks. For this perspective, we divide current approaches into three categories, each highlighting a different perspective to calculate the matching degree, i.e., semantic/rule-based methods, metric-based methods, and learning-based methods. Specifically, semantic/rule-based methods are designed to utilize the semantic information (e.g., natural language descriptions) or manually designed rules to find related models. Metric-based methods rely on defining the matching degree between the model's output and the corresponding dataset, while learningbased methods directly learn the similarity between data and models. This topic is also regarded as measuring the transferability score of a PTM [Xue et al., 2024].

Semantic/rule-based methods

For small models. Before the prosperity of large models, typical works rely on semantic information to choose proper models from the model zoo. This semantic information is commonly represented by natural languages, expressing the capability of models and the upstream training data. For example, [Zhou and Tan, 2024] proposes a typical model selection paradigm, where models in the model zoo are accompanied by descriptions (known as "semantic specification"). The semantic specification contains descriptive information related to the model, including the target task (e.g., classification or regression), the kind of machine learning applications (e.g., natural language processing or image recognition), the model structure (e.g., SVM, CNN, or Decision Tree), etc. Hence, we can form a key containing all this related information as s = {s1, s2, · · · , sk}, where si represents a dimension to describe the model. Hence, facing a new task, the users are only required to describe their requirements as a new query and search within the model zoo. Similar ideas are also explored in Taskonomy [Zamir et al., 2018], where neural networks are adopted to find transfer learning dependencies among a wide array of tasks, leading to a structured approach that reduces the need for labeled data in training new tasks.

For large Models. Semantic specifications are manually described by the model owners, costing extra effort and domain knowledge. To this end, the prosperity of large models helps get rid of the manual description. HuggingGPT [Shen et al., 2024] utilizes GPT to manage millions of models on HuggingFace [Wolf, 2019]. Similarly, natural language is utilized as the interface to align all models. Facing the user's new requirement, it plans out the necessary sub-tasks, and selects proper models from HuggingFace. Similar ideas have also been explored in automatically selecting and combining computer vision models [Gupta and Kembhavi, 2023].

Pros & Cons: We visualize the pipeline of semantic/rulebased methods in Figure 3. These methods provide a naive idea to manage models in the model zoo since natural language is a common tool for description. They can easily adapt to various domains and tasks by simply tagging models with appropriate semantic labels. This flexibility makes them suitable for dynamic environments where new types of tasks frequently emerge. However, there are also some drawbacks. The effectiveness of these methods heavily relies on the accuracy and comprehensiveness of the semantic specifications provided. Inaccurate or vague descriptions can lead to poor model selection, affecting overall performance. Besides, manual semantic tagging becomes impractical as the number of models grows. This is particularly challenging in environments where new models are constantly added [Wolf, 2019].

Metric-based methods

For small models. Apart from manually designing the specification, there are also works aiming to define the metric to represent the fitness between models and datasets. Correspondingly, metric-based methods determine whether a PTM is suitable for the target task by evaluating various proxy fitness metrics. There are various ways to design the metric, e.g., using feature statistics [P´andy et al., 2022], joint distribution of the source and target data [Nguyen et al., 2020], mutual information-based transferability [Tran et al., 2019], and gradients with few updates [Song et al., 2019].

Specifically, we can utilize the pre-trained model to encode the downstream task's data, and evaluate the separability of per-class instances considering inter-class and intra-class information [P´andy et al., 2022]. Other works seek to jointly model the upstream and downstream data. LEEP [Nguyen et al., 2020] applies the PTM to the downstream dataset to estimate the label distribution for each input. Then, it assesses how well the predictions from the model align with the actual distribution of ground truth labels. The metric is based on the alignment of the model's predictions with the target task's requirements. NCE [Tran et al., 2019] uses the conditional entropy between label sequences of source and target tasks as a metric. This measure reflects the information required to predict labels of one task based on the knowledge of another, thereby providing an estimate of task transferability and hardness. The aforementioned works only require the simple forward pass of the related dataset. However, if the computational resources are sufficient and can support backpropagation, we can also utilize the matching degree of attribution maps to define the metric [Song et al., 2019]. This line of work is very similar to transferability estimation, and we refer readers to [Xue et al., 2024] for more details.

For large models. In the era of large models, the cost of evaluating transferability becomes a critical factor due to the high computational expense involved in even simple operations like the forward pass. To address these challenges, recent developments focus on reducing the computational demands of metric-based evaluations. To this end, recent work focuses on designing surrogate tasks to approximate the behaviors of larger models without needing to operate them directly, thus speeding up the evaluation process. For example, LOVM [Zohar et al., 2023] defines the vision-language model selection paradigm based solely on textual descriptions of tasks, without needing access to specific datasets. SWAB [Yi et al., 2024] enhances the selection performance by bridging the modality gap between the textual descriptions and the visual model.

Pros & Cons. Metric-based methods provide a quantitative measure of suitability, making comparisons between models straightforward and reducing subjective bias in model selection. Besides, they adapt well to different types of data and tasks by adjusting the metrics to reflect the specific needs of the target task. However, the effectiveness of metric-based methods heavily relies on the appropriateness and robustness of the chosen metric. Poorly designed metrics can lead to suboptimal model selection. Some metrics, especially those involving backpropagation or complex statistical analyses, can be computationally intensive, limiting their use in resourceconstrained environments.

Learning-based methods

For small models. Apart from manually describing the models' behaviors, there are also works on automatically learning the specifications. This line of work aims to project models, as well as datasets, into the same embedding space, where the retrieval process can be easily done by distance calculation in the unified space. A representative work, i.e., Task2Vec [Achille et al., 2019] involves passing data through a probe network to generate a task-specific embedding that captures the complexity and characteristics of the task. Recent papers in this area also involve learning the specifications of PTMs and tasks from data. By training an encoder on known tasks and PTMs, these methods enable the generalization of the ability to represent both datasets and PTMs to unseen tasks and PTMs [Ding et al., 2022]. Model Spider [Zhang et al., 2023] learns model representations through vast historical performance.

For large models. Learning-based methods enable LLM selection with adaptive characteristics [Guha et al., 2024; Muqeeth et al., 2024]. A representative work is mixture of experts (MoE) [Jacobs et al., 1991], where multiple experts are constructed for diverse tasks. The training stage will adaptively assign the weight of each expert for the aggregated prediction to identify the best fit for each piece of reused tasks. Recent advancements in LLM selection have focused on learning effective routers for multiple pretrained LLMs [Pfeiffer et al., 2020; Zhang et al., 2023; Ong et al., 2025; Wu et al., 2024; Jitkrittum et al., 2025; Frick et al., 2025], e.g., Zooter [Lu et al., 2024] employs a reward model to score query-output pairs for routing decisionmaking, while [Mohammadshahi et al., 2024] uses self-play in reinforcement learning to create query-response-score triplets. Model-SAT [Zhang et al., 2025] further utilizes an extra foundation model as the model selector.

Pros & Cons. Learning-based methods automate the specification learning process, enabling scalability by adapting to new models and tasks in an end-to-end manner. However, their effectiveness heavily depends on the quality and diversity of the training data. Inadequate or biased data can lead to poor performance. Besides, the generalization ability of these works is also highly dependent on the scale of training tasks. Finally, the process of training and deploying these systems, especially for large models, can be resource-intensive, requiring substantial computational power and time.

3.2 Adapting PTMs

After selecting models from the model zoo, we then discuss how the selected model can help obtain better performance in the target task. In the following section, we will explore how PTMs can be effectively integrated into different learning phases, namely, target data preparation, target model training, and target model inference.

Adapt PTMs during target data preparation

For small models. PTMs serve as experts capable of generating enriched data for the target task, enabling the target model to achieve better generalization ability. For example, generative PTMs model the distribution of pretext tasks, and utilizing them to generate related datasets can help the training process of resource-constrained scenarios [Smith et al., 2023]:

where Dm Θ is generated by the selected model gm Θ . The enriched data can also emerge in other forms, e.g., the dataset statistics can help calibrate the distribution of downstream tasks [Wu et al., 2019]. The output of PTM can also be applied to tackle black-box source-free adaptation [Liang et al., 2022] via knowledge adaptation.

For large models. The advancement of large language models offers a new insight into the zero-shot ability, which can be further utilized for data preparation. Some recent methods also treat the pre-trained LLMs as a knowledge base and retrieve the common knowledge in the text form [Menon and Vondrick, 2023; Pratt et al., 2023]. In this case, the word-level labels are further refined into the semantic level (e.g., "hen"→"two legs", "red, brown, or white feathers", "a small body", "a small head", etc.) to further facilitate recognition.

Pros & Cons. With the help of PTMs, we can improve the diversity and quality of datasets, which is crucial for training robust models. The enriched data help models learn more generalized features, potentially improving performance on unseen data. However, there are also some drawbacks. The effectiveness of the data preparation heavily relies on the quality of the PTMs used. If the PTMs are biased or trained on nonrepresentative data, this can adversely affect the quality of the prepared data. Besides, there is a risk that models may overfit the characteristics of data generated or modified by PTMs, especially if the diversity of the synthetic data is limited or too closely mirrors the training scenarios of the PTMs.

Adapt PTMs during target model training

For small models. With the selected pre-trained model, an intuitive way to apply it to downstream tasks is fine-tuning. PTMs, when related to the target task, offer a strong foundation upon which the target model can be built:

In Eq. 4, the selected PTM is directly applied as the initialization of the target model. Fine-tuning these PTMs often leads to significant improvements in the target model's performance [Yosinski et al., 2014].

Since PTMs have been extensively trained on the pretext task, directly fine-tuning will ruin the existing knowledge contained in the PTM. Hence, there are some works with regularization to regularize PTM adaptation. Hypothesis transfer serves as a regularization technique that encourages the weights of the target model to resemble those of the PTM.

where R(fθ, gm Θ ) denotes the regularization term on the similarity between (fθ, gm Θ ). This is achieved by defining similarity measures in weight space, effectively guiding the target model to be close to the PTM. Various norm-based methods, such as the Frobenius Norm, ℓ2-norm [Li et al., 2018], and spectral norm [Miyato et al., 2018] are employed to quantify the similarity between the weights. When dealing with weight spaces of diverse shapes, additional mapping and attention layers [Ye et al., 2021] are applied to facilitate alignment.

Another line of work considers utilizing PTMs as the external supervision signal. Knowledge distillation was thought to be proposed by [Hinton et al., 2015], while ten years earlier a similar idea has already been published by [Zhou and Jiang, 2004]. These approaches match the PTM and the target model in the prediction space, i.e., the output predictions of both models should be similar:

where Sim(·, ·) measures the similarity between the model outputs. While KL-divergence and JS divergence are commonly used for this purpose, they are constrained to comparing models within the same label space. When the PTM and target model possess non-overlapping class sets, alternative techniques that consider statistics of hidden layers [Romero et al., 2015], as well as the relationships between instances [Park et al., 2019], prove to be powerful tools for achieving alignment. For large models. Fine-tuning, hypothesis transfer, and knowledge distillation have been proven effective in adapting PTMs for small models. Correspondingly, similar ideas are also applied for large models, e.g., distilling large LLMs into smaller scale [Gu et al., 2024], reprogramming the model with extra mapping layers [Chen, 2024], applying lightweight tuning techniques for LLM [Hu et al., 2022]. Additionally, when facing diverse modalities, reusing multiple large models of different modalities for unified recognition becomes popular in the field. For example, LLaVA [Liu et al., 2023] and BLIP [Li et al., 2023] build multi-modal LLMs by learning the projection between frozen visual and textual features. Pros & Cons. We visualize the steps to reuse PTMs during model training in Figure 4. By leveraging PTMs, the training time is significantly reduced, as the complex foundational learning has already been accomplished. Furthermore, PTMs often lead to improved model performance, especially when fine-tuned for specific tasks, because they have been pre-trained on extensive and diverse datasets. However, potential risks include overfitting the PTM's data characteristics, especially if the target task differs significantly from the tasks the PTMwas originally trained on. Finally, the success of the adapted model heavily depends on the quality of the PTM. If the PTM has biases or was trained on non-representative data, these issues can propagate to the adapted model.

Adapt PTMs during model inference

For small models. The advantages of PTMs extend to the model inference stage of the target model. One straightforward approach involves constructing an embedding space using the PTM, resulting in more discriminative features. This enriched space facilitates the creation of linear classifiers and nearest class mean classifier [Mensink et al., 2013], simplifying the process of obtaining the target model. Some recent methods also learn to concatenate middle layers' features [Evci et al., 2022] or learn to weight features from different PTMs [Tu et al., 2023] to enhance the generalization of the features.

Furthermore, model merging methods have been developed to fuse PTMs together. This technique has the potential to flatten the optimization space and increase the likelihood of converging towards an optimal solution [Matena and Raffel, 2022]. Various methods have emerged in this domain, including those emphasizing partial channels during the model merging process [Wortsman et al., 2022]. Some other methods address the model heterogeneity. These approaches involve learning a mapping matrix to rectify the weights of different layers [Singh and Jaggi, 2020], accommodating the diverse characteristics of the models being merged.

For large models. The idea of model merging is also proven effective in LLM inference to enhance its general capability [Yang et al., 2024; Kim et al., 2024; Dang et al., 2025]. Besides, Retrieval-Augmented Generation (RAG) [Lewis et al., 2020] utilizes the external database for retrieval. Facing a new input, it fetches relevant documents from the database and produces responses based on the retrieved information. When an external database is unavailable, methods also explore the LLMs' reasoning ability to self-guide the inference [Brown et al., 2020] by reasoning "step by step".

Pros & Cons. Leveraging PTMs during inference can reduce the need for expensive model retraining, thus saving computational resources and time. However, there are also some drawbacks. While PTMs can be powerful, integrating them effectively with the target model can be challenging, especially when concatenating or combining features from different layers. This may require careful fine-tuning and architecture adjustments. Besides, although PTMs can simplify some aspects of inference, the added complexity of using intermediate layer features or constructing chain of thoughts may introduce additional computation, potentially increasing inference time compared to simpler models.

3.3 Other Topics

Apart from the above topics in model reuse, there are also related fields focusing on other aspects of model reuse. These topics are not fully independent of model selection and adaptation, which partially share the same goal.

Model Assembly. Rather than treating each PTM as a whole, model reuse can operate at a finer granularity by assembling models from components such as network blocks [Yang et al., 2022; Pfeiffer et al., 2024; Guo et al., 2023] or parameterefficient tuning modules [Hu et al., 2024]. This approach enables fine-grained model selection targets, where reusable parts are identified and recombined to build models tailored to specific tasks. Some methods also modularize networks or extract key functional units to improve interpretability and flexibility [Wang et al., 2022; Shi et al., 2024].

Model Representation Learning [Sch¨urholt et al., 2022; Unterthiner et al., 2020] vectorizes PTMs to encode their characteristics, forming hyper-representations that allow model selection without direct access to the original model parameters. These representations can be learned based on intrinsic permutation-invariant properties of a PTM and can even be decoded back into model weights [Sch¨urholt et al., 2024]. Learned model representations serve multiple purposes: they act as model specifications within a model zoo, assist in efficient retrieval, and serve as auxiliary modalities during the adaptation process [Zhou et al., 2024].

Model Compression. Given the computational demands of large PTMs, model compression aims to reduce model size while preserving functionality. Techniques such as pruning [Cheng et al., 2024] and quantization [Zhou et al., 2018] are used to transform large PTMs into compact versions that maintain performance while reducing inference costs.

Model Repair and Editing. Model repair corrects erroneous behaviors in a PTM without retraining from scratch [Luo et al., 2023], often through targeted weight modifications, counterfactual learning, or fine-grained intervention. Model editing [Mitchell et al., 2022; Fang et al., 2024], on the other hand, involves modifying a model's knowledge base or decision boundaries to reflect new information or correct biases. Unlike standard fine-tuning, model editing aims to localize changes while maintaining overall model consistency.

Managing LLMs. Managing model repositories such as "LLM Market" requires dynamic specification of model traits, rapid adaptation to new data, and efficient fusion and configuration of multiple models [Shen et al., 2024; Prabhu et al., 2024; Zhou et al., 2025; Zhang et al., 2024]. Methods have been developed to learn representations of model characteristics and add semantic specifications to guide model selection. Efficient evaluation frameworks assess language understanding, reasoning, and task proficiency of LLMs, facilitating benchmarking and optimization for realworld deployment [Polo et al., 2024; Zhong et al., 2025; Saranathan et al., 2024].

Model Relationship Discovery. PTM relationship discovery aims to trace the relationship between models, providing helpful information in model zoo management. Neural Lineage [Yu and Wang, 2024] predicts which parent model a child model has been fine-tuned from, and [Horwitz et al., 2025] formulates this task in an unsupervised manner.

4 Future Directions

Coupling of Selection and Adaptation. The generalization ability of an adapted model depends not only on the selected PTM but also on the adaptation strategy employed. Thus, PTM selection and adaptation are inherently coupled [Arango et al., 2024]. While some methods, such as transferability estimation, attempt to improve selection, they often fail to account for the mismatches between fine-tuning and advanced adaptation techniques. Future research should focus on joint optimization of selection and adaptation to enhance end-to-end performance.

Diversity and Strength of the Model Zoo. The success of model reuse depends on the quality, diversity, and generalization of PTMs in the model zoo. A well-curated collection improves the chance of selecting suitable models-even at finegrained levels such as instances or tokens. Therefore, improving the curation and management of model zoos is essential for maximizing the impact and utility of model reuse [Casta˜no et al., 2023; Dong et al., 2023].

Universal Adaptation Methods. Adaptation methods must bridge heterogeneity between the target task and the selected PTM while remaining universally applicable across different models and tasks. However, most adaptation techniques are task-specific and require extensive hyper-parameter tuning, making them difficult to generalize. Developing standardized, flexible, and efficient adaptation strategies is essential for improving model reuse in diverse settings.

Model Collaboration. Instead of selecting a single PTM, aggregating multiple PTMs to collaborate in parallel or sequentially can enhance model robustness and accuracy [Feng et al., 2025]. In this way, complex target tasks can often be decomposed into multiple sub-tasks, each handled by a different PTMwith distinct expertise. There are several ways to achieve this goal. Firstly, models can collaborate sequentiallysmall models can tackle easy tasks, while large models can tackle what small models cannot handle. Furthermore, some tasks can be decomposed into sub-tasks, where different PTMs handle each step sequentially. Optimizing the order and dependencies of PTMs is crucial, considering task complexity, computational cost, and adaptation requirements [Chen and Varoquaux, 2024]. Secondly, multiple models can collaborate in a parallel manner, where multiple PTMs contribute jointly to task performance through techniques such as weighted averaging or model merging [Wortsman et al., 2022; Stoica et al., 2023]. However, effectively combining PTMs with varying strengths remains a challenge, requiring proper ensemble strategies [Fu et al., 2025]. Lastly, modules of different PTMs can also collaborate together to enhance prediction robustness [Wang et al., 2024; Li et al., 2025].

Model Market and Dock System Construction. As model reuse continues to gain traction, the seamless deployment of reusable models requires a robust, containerized system. Systems like Learnware [Zhou, 2016; Zhou and Tan, 2024] and Beimingwu [Tan et al., 2024] have paved the way for efficient model reuse, offering quick and easy application. However, beyond containerization, several critical topics in the context of the model market remain underexplored. For instance, managing the model zoo to facilitate smoother model reuse, as well as determining appropriate pricing strategies for models, are areas that warrant further research and development.

Privacy-Preserving and Robust PTM Representations. The effectiveness of PTM selection and adaptation often relies on access to large, labeled datasets, which may not always be feasible due to privacy concerns or data scarcity. Future work should explore privacy-preserving PTM representations and methods that require minimal task-specific data [Lei et al., 2024]. In particular, large PTMs may produce nearly identical outputs on few-shot samples, making it difficult to distinguish their capabilities. Developing task-aware probing mechanisms that can effectively differentiate PTMs under lowdata conditions is an important research direction.

Multi-Objective Model Reuse. In the real world, apart from vanilla measuring the performance on target tasks, there are other important targets, e.g., adapting efficiency [Houlsby et al., 2019] that require fast model reuse and carbon emission [Lacoste et al., 2019] that recurring low computation cost. These multiple objects may also evolve in the learning process, posing more challenging targets to model reuse.

5 Conclusions

Model reuse has become a common paradigm in modern machine learning, enabling the efficient utilization of PTMs to enhance performance while reducing computational and data requirements. This survey provides a unifying perspective on model reuse, categorizing methods based on PTM selection and adaptation, and exploring its applications across both small specialized PTMs and large foundation models. By outlining a comprehensive taxonomy, discussing emerging trends, and identifying open research questions, this survey aims to provide valuable insights to drive future research and foster the adoption of model-centric AI development.

Acknowledgments

This work is supported by the National Key R&D Program of China (2024YFE0202800), NSFC (62376118), Fundamental Research Funds for the Central Universities (14380021), and the Collaborative Innovation Center of Novel Software Technology and Industrialization.