Beyond Homophily: Spectrum-Based Graph Pre-Training and Cluster-Augmented Prompt Tuning

1 Introduction

Graph learning has become a powerful paradigm for various machine learning tasks, especially node classification [Zang et al., 2025]. Graph neural networks (GNNs) [Jiang et al., 2019; Velickovic et al., 2017; Ju et al., 2025] are central to this paradigm due to their ability to aggregate information from local neighborhoods. However, with limited labeled data, GNNs are prone to overfitting and often generalize poorly, motivating graph pre-training methods that learn transferable representations from large-scale unlabeled graphs via self-supervised objectives, such as DGI [Veliˇckovi´c et al., 2019] and GraphCL [You et al., 2020]. Building on pre-trained models, graph prompt tuning has emerged as an efficient adaptation strategy that introduces lightweight, task-specific prompts while keeping the backbone fixed [Sun et al., 2022; Liu et al., 2023]. Recent unified frameworks, including HGPrompt [Yu et al., 2024a], SAMGPT [Yu et al., 2025a], and HS-GPPT [Luo et al., 2025], further combine pre-training with prompt tuning to enhance adaptability across diverse graph structures.

∗Corresponding authors: Lingfei Ren, Xin Yang.

Despite these advances, existing graph pre-training and prompt tuning methods face two critical challenges. (1) Graph spectral bias. Graph frequency distributions are closely correlated with homophily: homophilic graphs are dominated by low-frequency components, whereas heterophilic graphs contain richer high-frequency information [Ren et al., 2024; Zheng et al., 2022]. As shown in Fig. 1a and Fig. 1b, datasets with lower homophily exhibit higher proportions of high-frequency energy. However, most pretraining methods focus on a single spectral band, limiting generalization across graphs with varying homophily levels [Chen et al., 2025]. Moreover, without supervision, spectral GNNs struggle to learn optimal filter shapes, leading to suboptimal downstream adaptation. (2) Graph prompt tuning misalignment. Existing graph prompt methods are predominantly designed in the spatial domain and do not explicitly model or control the spectral properties of node representations [Yu et al., 2025b]. Since heterophily varies across graphs and even across nodes within the same graph (Fig. 1c), standard prompt methods exhibit node- and dataset-sensitive performance (Fig. 1d). When combined with frequencyaware pre-trained models, spatial prompts often fail to modulate critical spectral bands and may even disrupt learned frequency information, particularly on heterophilic graphs. Spectrally isolated bias or prompt tuning is inadequate, as current methods lack joint optimization of spectral learning and adaptation, limiting homophily-agnostic generalization.

To address these challenges, we propose Spectrum-based Graph Pre-training and Cluster Augmented Prompt Tuning (SCA-GPPT). During pre-training, we first employ a smoothness-aware feature alignment module to map heterogeneous node features from different domains into a unified semantic space. Complementary low and high-frequency filters are then learned via parameterized Chebyshev polynomials, together with band-aware contrastive learning, to capture diverse spectral patterns in the pretraining stage. For downstream adaptation, we freeze the pre-trained backbone and introduce a spectral prompting mechanism that directly adjusts Chebyshev filter coefficients, enabling fine-grained frequency adaptation without modifying backbone parameters. In addition, cluster-augmented prompts further exploit global structural consistency, yielding representations that are both discriminative and globally coherent. By jointly modeling feature alignment, spectral diversity learning, and prompt alignment, SCA-GPPT achieves robust generalization across graphs with varying homophily levels.

In summary, our contributions are threefold: (i) We pioneer a systematic spectral analysis of graph prompt tuning, revealing spectral bias and spatial prompt misalignment that necessitate spectral pre-training and prompt tuning across varying homophily. (ii) We propose SCA-GPPT, a spectrum-aware framework integrating spectral pre-training and cluster-augmented prompt tuning. It achieves finegrained spectral alignment through learnable parameterized Chebyshev polynomials and adaptive clustering prompts, improving cross-graph transfer in both transductive and inductive scenarios. (iii) Extensive experiments on synthetic and real-world graphs with varying homophily demonstrate that SCA-GPPT consistently outperforms existing baselines in both transductive and inductive settings.

2 Preliminaries and Theoretical Analysis

2.1 Preliminaries

Homophily. Let G = (V, E,X) be a graph equipped with a node label vector yi ∈ Y. The homophily level quantifies the extent to which adjacent nodes share the same label. It is defined as the proportion of edges with the same label:

where |E| denotes the total number of edges and I{·} is the indicator function that returns 1 when its argument is true and 0 otherwise.

Spectral Graph Filters. A spectral graph filter is a realvalued function g(·) applied to the spectrum of L [Bruna et al., 2014]. Given a node feature matrix L ∈ RN×d , the filtered representation Z ∈ RN×d is obtained by

where g(Λ) = diag(g(λ1), . . . , g(λN)). Different choices of g emphasise different spectral bands and thus capture different patterns of neighbour aggregation: low-pass filters promote similarity among connected nodes [Kipf and Welling, 2017], while high-pass filters highlight discrepancies between nodes and their neighbors [Bo et al., 2021].

Spectral Energy. Let G = (V,X, E) be an undirected graph with |V| = n and Laplacian matrix L = D - A. For a nodewise scalar signal x ∈ Rn , we define its spectral energy as:

when using the normalized Laplacian Lsym = I - D-1/2AD-1/2 , one may analogously define F (norm) G (x) := 12 x⊤Lsymx.

High-FrequencyEnergy. Recall that the eigenvalues of the normalized Laplacian satisfy λi ∈ [0, 2], Then we define the high-frequency energy fraction as

where Bh := {i|λi ≥ λ0} is high spectral bands. For node classification, we instantiate x as the label indicator signals {yc}Cc=1 :

2.2 Theoretical Analysis

Lemma 1 (Spectral Limitation of Spatial Prompts). Considering a general prompt as a perturbation δ to the feature matrix: Xp = X + δ, where δ is learnable and lightweight. In the spectral domain, the prompted output becomes: Zp = Ug (Λ) U (X+ δ) = g (L)X + g (L) δ, the prompted highfrequency energy satisfies:

where ε → 0 as the high-pass attenuation of g increases. This implies that spatial prompts cannot effectively recover high-frequency information if it is absent from the pre-trained representations.

Lemma 1 highlights that traditional space-domain prompt fine-tuning loses high-frequency energy in graphs and is only suitable for homophilic graphs. Figure 1d also confirms that nodes with lower homophily exhibit poorer fine-tuning performance.

Theorem 1 (Transfer Ability to Spectral Discrepancy). Assume the optimal downstream decision function depends on filtered representations g⋆ (L)X and the learned filter is g. For Lipschitz-continuous filters g on [0, 2], the representation mismatch between source and target satisfies

for some constants C1, C2 depending on ∥g∥∞ and the Lipschitz constant of g.

Theorem 1 highlights that cross-graph transfer ability is fundamentally limited by spectral discrepancy. Therefore, a pretraining objective that only emphasizes pure low-pass is brittle when ∣∣∣S(s) high - S(t) high ∣∣∣ is large.

3 Methodology

Preliminary analysis confirms that spatial prompt tuning causes loss of high-frequency energy and is only applicable to homophilic graphs. To address this issue from both spectral representation learning and prompt-based adaptation perspectives, we propose SCA-GPPT, as shown in Figure 2.

3.1 Smoothness-based Feature Alignment (SFA)

Node features often vary across domains due to diverse textual descriptions, metadata, or domain-specific attributes (e.g., personal profiles in pre-training versus transaction attributes in fine-tuning). To integrate such heterogeneous features, we propose an alignment module that maps them into a unified representation space through two stages: feature projection and semantic alignment.

Feature Projection. Motivated by GCOPE [Zhao et al., 2024], we align the dimensions of features from different domains via a shared linear transformation. Specifically, for the feature matrix X(i) ∈ Rni×di from domain i, we apply the projection operation:

where du denotes the predefined projection dimension, while Proj (·) represents the feature projection operation, typically implemented as singular value decomposition (SVD) [Golub and Reinsch, 1971] or principal component analysis (PCA) [Pearson, 1901].

Semantic Alignment. Despite achieving dimensional alignment, semantic inconsistencies may still arise between the pre-training and fine-tuning stages [Liu et al., 2024]. To further optimize this issue, we introduce a smoothness-based semantic alignment that draws from the perspective of graph signal processing to evaluate the contribution of each feature dimension. Formally, given a graph G = (V, E,X), we define the smoothness for the k-th feature dimension as:

where a lower sk value indicates substantial variation between connected nodes, reflecting strong heterophily similar to high-frequency components.

3.2 Spectrum-based Pre-training

Theorem 1 demonstrates that a graph's transfer ability correlates with spectral energy differences across graphs. To achieve generalization capabilities across diverse spectral distributions, spectral information must be extracted. To this end, we designed a spectrum-based graph pre-training mechanism that ensures balanced integration of both lowfrequency and high-frequency components in graphs, thereby enhancing the model's ability to discern subtle differences in spectral patterns.

Adaptive Chebyshev Encoding (ACE). Traditional heterophilic graph learning incurs parameters that scale linearly with the polynomial degree K and may overemphasize specific frequency bands [He et al., 2022]. Recent studies [Chen et al., 2024; Wan et al., 2024] build on cosine-parameterized Chebyshev polynomials to propose PolyGCL, a graph contrastive learning paradigm that effectively handles both homophilic and heterophilic graphs without relying on complex data augmentation used in conventional GCL [You et al., 2020]. Consequently, we adopt PolyGCL as our backbone model. Specifically, given Chebyshev polynomials with interpolation ∑Kk=0 wkTk ( ˆL)X, where ˆL is normalized Laplace matrix, and the enhanced Chebyshev polynomials with Chebyshev interpolation wk is:

where xj = cos ( j+1/2 K+1 π) , j = 0, · · · ,K denoted the Chebyshev node for TK+1, and Tk (x) is Chebyshev polynomials recursively defined as Tk (x) = 2xTk-1 (x) - Tk-2 (x), with T0 (x) = 1 and T1 (x) = x. As for the learnable parameter γj , we propose to use a learnable sliding cosine-parametric-based formulation to build our low-pass and high-pass filters based on parameters γ lj and γ h j as follows:

For initialisation, we set βh j and βlj to 0 and 2, respectively, whereas βlb and βh b are set to 2. These four parameters are all trainable during the learning process. To sum up, substituting γi with γ l and γ h in Eq.10, respectively, the graph filter can be expressed as:

where fθ (X) denotes the use of MLP on the raw feature matrix X.

Local Homophily. Specifically, we first utilise the smoothing properties of low-pass filters to pre-train them using traditional link prediction methods. For node vi in the graph, its positive set N ′ i consists of nodes vj that are first-order neighbours of vi, while its negative set V \ vi comprises randomly sampled nodes that are not first-order neighbours. The cross-low objective is formulated to maximise the similarity between representations of such pairs:

where zl i and zlp emphasise low-frequency components, capturing the local similarity and homophily within the spatial domain, s(·, ·) denotes a similarity function (e.g., cosine similarity). This objective encourages the model to align the embeddings of structurally proximate nodes, thereby fostering robustness in both homophilic and heterophilic graphs.

Global Heterophily. The high-pass filter reveals the diversity within local neighbourhoods and the global long-range similarity among semantic neighbours. We introduce semantically neighbours as positive sample pairs, determined based on feature-level similarity. Specifically, for each node vi, we identify its k most similar neighbours based on the aligned features, denoted as N ′′ i = kNN(vi, k). This enables the derivation of the all-pass component of the objective function, as shown in Eq.14:

where zh i and zh p emphasizes high-frequency components, capturing long-range dependencies and dissimilarities within the spatial domain.

Optimization and Balancing. The overall loss combines these objectives with a balancing coefficient:

where α is a tunable hyperparameter that modulates the emphasis between local and global contributions. This formulation promotes inference efficiency, making it particularly suitable for large-scale graphs and real-time applications. Extensive experiments demonstrate that this approach yields superior adaptability and performance in diverse graph learning scenarios.

3.3 Cluster-Augmented Prompt Tuning

In the downstream stage, we use a parameter-efficient prompt tuning with adapters on the Chebyshev order K. The backbone, including K, remains frozen to preserve pre-trained knowledge. For a graph G = (V, E,X), the frozen backbone produces spectral representations Zl and Zh capturing homophilic and heterophilic patterns.

To adapt the model without updating frozen parameters, we design lightweight prompts as adapters to K. These adapters modulate the Chebyshev filter weights {wk}Kk=0 associated with the frozen K. The prompt function PK produces adaptive adjustments:

where Θprompt are trainable parameters. The promptenhanced representation is:

The adapter PK is implemented as a bottleneck structure:

where ϕ(·) encodes weight information, Wdown ∈ Rd×d′ , Wup ∈ Rd′×d with d′ ≪ d, and σ(·) is an activation function. For training stability, Wup is initialized to zero, ensuring minimal initial perturbation to the pre-trained filters.

After prompt modulation, the low-frequency and highfrequency representations are fused to form the final node embedding:

where β ∈ [0, 1] controls the relative contribution of homophilic and heterophilic information. This design enables flexible adaptation to different graph structural regimes without modifying the backbone architecture.

While supervised classification loss provides signals for labeled nodes, it may not sufficiently regularize the global embedding space, especially in low-label or semi-supervised settings, so we introduce a cluster-augmented objective that enforces structural consistency among prompt-enhanced representations. Let Z = {zi}|V| i=1 denote the final node embeddings, and {µk}Kk=1 be a set of learnable cluster centroids. We compute the soft assignment of node vi to cluster k using a Student-t kernel:

where τ is a temperature parameter controlling the smoothness of the assignment distribution.

Following the Deep Embedded Clustering (DEC) paradigm [Xie et al., 2016], we construct a sharpened target distribution to enhance cluster discrimination:

and optimize the clustering objective by minimizing the Kullback-Leibler divergence between the target distribution and the soft assignments:

It is worth noting that the clustering objective does not introduce additional supervision, but instead serves as a structural regularizer that complements the classification loss [Caron et al., 2018; Bo et al., 2020]. While the classification objective enforces label-level separability, the clustering constraint encourages compactness and consistency in the embedding space, thereby improving training stability and generalization.

The overall downstream optimization objective is defined as:

where Lcls denotes the supervised classification loss and λ is a balancing coefficient. During downstream training, only the prompt modules, clustering module, and classification head are updated, while the pre-trained spectral backbone remains frozen. This design enables efficient and stable adaptation with minimal trainable parameters.

Theorem 2 (Stability Gain from Clustering Regularization). Assume the backbone is frozen and only prompt parameters are learned. If the clustering loss enforces a margin ∥µk - µk′∥ ≥ m for k ̸= k′ , then the prompt solution has improved uniform stability, leading to a tighter generalization bound:

where S is the labeled set and Θprompt are prompt parameters.

Theorem 2 formalizes that clustering regularization reduces the effective hypothesis space explored by prompt tuning under few-shot supervision, thus improving generalization and robustness.

4 Experiments

4.1 Experimental Setup

Datasets. We conduct experiments on eight real-world datasets with various homophily levels. Among them, Cora, Citeseer, PubMed [Sen et al., 2008], Amazon product network Photo [Shchur et al., 2018] are homophilic graphs, Wisconsin, Cornell [Pei et al., 2020], Chameleon, Squirrel [Rozemberczki et al., 2021]are considered as heterophilic graphs.

Baselines. The baseline methods fall into three categories: (i) Traditional GNN models: GCN [Kipf and Welling, 2017], GAT [Velickovic et al., 2017], H2GCN [Zhu et al., 2020], and FAGCN [Bo et al., 2021]; (ii) Graph "Pre-training + Fine-tuning" models: DGI [Veliˇckovi´c et al., 2019], GraphCL [You et al., 2020], and PolyGCL [Chen et al., 2024]; (iii) Graph "Pre-training + Prompt-tuning" models: GPPT [Sun et al., 2022], GPrompt [Liu et al., 2023], GPFplus [Yu et al., 2024b], All-in-One [Zhao et al., 2024], ProNoG [Yu et al., 2025b], and HS-GPPT [Luo et al., 2025]. Among these methods, H2GCN, FAGCN, and PolyGCL are designed to handle heterophily. For GPPT, GPrompt, GPFplus, and All-in-One, we use GraphSAGE [Hamilton et al., 2017] as the backbone. For HS-GPPT, we follow the official implementation released by the authors and keep its original backbone and training settings.

4.2 Performance Comparison

We focus on two scenarios: transductive and inductive settings. We define our task as 1-shot node classification. Results are averaged over 100 runs with different random seeds

, using ACC as the metric Results of transductive learning are in Table 1, inductive learning in Table 2.

Transductive Performance. From the results, we observe that SCA-GPPT performs best on most datasets and remains highly competitive on PubMed across both homophilic and heterophilic graphs. On homophilic datasets such as Cora and Citeseer, SCA-GPPT achieves accuracies of 61.63% and 53.61%, respectively, outperforming the second-best methods by a substantial margin. On heterophilic datasets including Cornell, Chameleon, and Squirrel, our method remains robust (41.61%, 31.30%, and 23.95%), demonstrating its ability to leverage both structural and feature cues across diverse graph types. Overall, our approach achieves strong and competitive performance across datasets, further demonstrating its effectiveness and robustness in few-shot scenarios.

Inductive Performance. We evaluate inductive learning performance across graphs with varying levels of homophily, considering both in-domain settings where pre-training and downstream tasks share the same domain and cross-domain settings. Experimental results demonstrate that our model achieves strong inductive learning performance, even when discrepancies between pre-trained and downstream graphs pose challenges for knowledge transfer. This effectiveness stems from its ability to capture rich spectral representations during pre-training. It then adapts the spectral distribution of downstream graphs to the pre-trained knowledge through cluster-augmented prompt tuning, facilitating more effective knowledge transfer.

Limitation of SCA-GPPT. Under the transductive setting, our method achieves slightly lower accuracy than the bestperforming baseline (by 0.27%) on PubMed. This slight gap may be attributed to the 1-shot setting and the subtle class feature variations in PubMed, which restrict the effective use of structural and feature information.

4.3 Ablation Study

(a) Cora→Cora

• (c) Wisconsin→Cornell

(d) Citeseer→Squirrel

Figure 3: Ablation study on 1-shot transductive and inductive setting.(a)(b)present the ablation results under the transductive setting, while (c)(d) present the ablation results under the inductive setting in in-domain settings and cross-domain setting. The red border indicates the best-performing model in each dataset.

We conduct a systematic ablation study under the 1-shot inductive node classification setting to evaluate the contributions of pre-training, prompt tuning, and spectral modeling in Figures 3. (1) w/o SFA removes smoothness-based feature alignment. (2) w/o PreTrain replaces the adaptive Chebyshev encoding with a vanilla GCN. (3) w/o Cluster removes the cluster assignments and relies only on the classifier. (4) w/o HighFreq removes high-frequency filter. (5) w/o Prompt disables prompt tuning entirely.

The ablation study shows that all components contribute to the final performance. The clustering head (w/o Cluster) has a more noticeable impact on homophilic graphs (Figure 3a), while removing prompt tuning (w/o Prompt) also causes a moderate decline in performance. Among them, removing spectrum-aware pre-training (w/o PreTrain) leads to the largest performance drop, particularly in the transductive setting (Figure 3c). Removing smoothness-based feature alignment (w/o SFA) or high-frequency modeling (w/o HighFreq) results in consistent performance degradation across both transductive and inductive settings.Overall, the complete model consistently outperforms all ablated variants under both settings.

4.4 Spectral Analysis of Prompt tuning

As shown in Fig. 4, including the Wisconsin→Cornell (Fig. 4a) and Citeseer→Squirrel (Fig. 4b) inductive settings, the spectral analysis illustrates how the spectral prompt tuning modulates graph signals in the 1-shot transfer learning scenario. The eigenvalue spectrum λ is partitioned into lowfrequency (λ < 0.3), mid-frequency (0.3 ≤ λ < 1.0), and high-frequency (λ ≥ 1.0) bands for band-wise spectral energy analysis. Prompt tuning performs targeted spectral shaping by enhancing task-relevant frequencies while attenuating bands associated with noise or domain discrepancies. In the Wisconsin→Cornell transfer (Fig. 4a), spectral energy is primarily suppressed in the low- and mid-frequency bands as well as most of the high-frequency band, with limited enhancement in localized high-frequency regions. In contrast, the Citeseer→Squirrel transfer (Fig. 4b) exhibits broad highfrequency suppression and partial mid-frequency attenuation, accompanied by mild enhancement in the low-frequency region. It is evident that our proposed spetral prompt finetuning method can automatically adjust the spectral parameters of the pre-trained filter based on the spectral differences between the source and target graphs, thereby better adapting to the spectral distribution of the downstream target graph.

4.5 Experimental Results and Analysis

(a) FAGCN on Cora

(b) PolyGCL on Cora

(c) HS-GPPT on Cora

(d) Ours on Cora

Figure 5: Comparative 1-shot learning scatter plots on the Cora dataset. (a) FAGCN; (b) PolyGCL; (c) HS-GPPT; (d) Ours.

As shown in Fig. 5, we present the scatter plot of the 1-shot transductive learning on Cora dataset. We have the following observations: On both homophilic and heterophilic graphs, our method effectively captures structural consistency across different graph domains and topological patterns by forming compact and well-separated clusters through the clustering head. Moreover, our approach maintains clear class separability through the classification head, indicating that the proposed spectral filtering and prompt-tuning mechanisms can flexibly adapt to graphs with varying degrees of homophily and enable robust and stable knowledge transfer under challenging few-shot settings.

5 Conclusion

Our study examines graph prompt alignment from a spectral perspective, revealing that spectral bias and prompt misalignment constitute the primary obstacles to cross-homophily generalization. We propose SCA-GPPT, unifying spectrumbased pre-training and cluster-augmented prompting for fewshot node classification. Experimental results indicate that in transductive and inductive settings, SCA-GPPT consistently outperforms existing baselines.

Broader impact. The framework provides a principled approach to integrating spectral information and prompts in graph learning, offering new insights into few-shot adaptation and cross-domain transfer, with potential applications in social networks, biological networks, and other graphstructured domains.

Limitations and future work. This study focuses on eight real-world datasets and few-shot settings. Future work includes exploring other few-shot ratios, diverse graph types, real-world scenarios with spectral misalignment, and alternative spectral representations or prompt designs to enhance adaptation and robustness under extremely weak supervision conditions.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 62506308) and Chengdu Science and Technology Program (No.2025-YF12-00030-RC).

ContributionStatement

Tingting Li and Xiangkun Wang contributed equally to this work.