Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graph

1 Introduction

Text-attributed graph (TAG) [Yan et al., 2023] is a prevalent type of graph-structured data, where each node is associated with a text description. For instance, in a citation network, the papers (i.e., nodes) are linked by the citation relations (i.e., edges), and the abstract of each paper serves as the text description. Few-shot and zero-shot node classification on TAGs (FZNC-TAG) predict the categories of the nodes using a few or even no labeled data since labeled data are expensive to obtain [Liu et al., 2021; Liu et al., 2022]. The two tasks have many applications in areas such as recommender system [Gao et al., 2022], social network analysis [Yu et al., 2020], and anomaly detection [Noble and Cook, 2003].

∗Corresponding authors.

Existing methods for FZNC-TAG typically follow a twostep process: first learn node and text embeddings on the TAGs and then use prompting to produce classification results [Wen and Fang, 2023; Huang et al., 2023]. They mainly differ in embedding learning and can be classified into three categories. ❶ Self-supervised graph learning methods employ graph augmentation techniques, such as node adding and feature masking, to generate more generalized node embeddings [You et al., 2020; Deng and Hooi, 2021]. They exploit the graph topology but ignores the information in the text semantics. ❷ The two-stage learning methods encode the text using large language models (LLMs) and add the text embedding as additional node features [Tang et al., 2024; Chen et al., 2024]. Then, graph neural network (GNN) methods, e.g., TextGCN [Yao et al., 2019] and GraphSAGE [Hamilton et al., 2017], are used to learn the node embeddings. The limitation of these approaches is that the LLMs are not updated during GNN training [Yan et al., 2023]. ❸ The state-of-theart end-to-end learning method, G2P2 [Wen and Fang, 2023], jointly trains the GNN and language model via contrastive learning [He et al., 2020]. As shown in Figure 1(a), G2P2 contrasts each node-text pair to ensure that the GNN and language model are aligned in the embedding space.

Existing work utilizes various graph-based augmentation to train the node and text embeddings, however, neglecting the semantics information in the texts. As shown in Figure 1(a), G2P2 merely aligns the raw node-text pairs in the TAGs. Furthermore, we observe that the classification accuracy of G2P2 is low. For instance, on the Fitness dataset [Yan et al., 2023], G2P2 only achieves an accuracy of 68.24% and 45.99% for few- and zero-shot classification, respectively. The significantly lower accuracy in zero-shot classification also suggests that the problems are caused by the lack of text semantics and that G2P2's sole reliance on the graph-based augmentation is inadequate for training high-quality models. Thus, we ask the following research question:

How to exploit more text semantics to enhance fewand zero-shot classification on TAGs?

To answer the question, we present TSA, where embedding learning can be improved by enforcing similarity relations among embeddings. Inspired by this idea, we design two augmentation techniques, i.e., positive semantics matching and negative semantics contrast, as shown in Figure 1(b,c). These techniques create additional node-text pairs that have similar or dissimilar embeddings to facilitate model learning.

Positive semantics matching. In Figure 1(b), we provide multiple positive text embeddings for each node embedding. This is achieved by searching the texts that have similar embeddings to the text of the considered graph node. We also encourage the node embedding to be similar to those text embeddings. This augmentation provides more supervision to GNN training and enforces the prior that nodes may belong to the same categories if their texts have similar semantics.

Negative semantics contrast. In Figure 1(c), we pair each text with a semantically opposite negative text, which is constructed by adding a learnable negative prompt to the original text. We then encourage the original text embeddings and its corresponding node to be dissimilar to the negative text. This augmentation provides additional text semantics to make the classification robust. For instance, to classify a paper as being related to artificial intelligence, it should not only be similar to the description "a paper is published at IJCAI" but also be dissimilar to "a paper is published at The Lancet".

We conduct extensive experiments to evaluate TSA, using 5 datasets and comparing with 13 state-of-the-art baselines. The results show that TSA consistently achieves higher accuracy than all baselines for both few-shot and zero-shot setting. In particular, TSA improves the accuracy and F1 scores of few-shot classification by 4.6% and 6.9% on average, and zero-shot classification by 8.8% and 9.3%, respectively.

In summary, we make the following contributions:

• We observe that prior methods only contrast node-text pairs, thus failing to effectively capture text semantics. To solve this issue, we propose TSA to enhance the semantic understanding in both model pre-training and inference.
• We incorporate two novel augmentation techniques into TSA: positive semantics matching and negative semantics contrast. These techniques create additional node-text pairs by mining more text semantics from diverse perspectives.
• We conduct extensive experiments to evaluate TSA and compare with state-of-the-art baselines, demonstrating that

TSA enjoys high model accuracy and training efficiency.

2 Preliminaries

Text-attributed graph. We denote a text-attributed graph (TAG) as G = (V, E,X), in which V, E, and X are the node set, edge set, and text set, respectively. Take citation network as an example for TAG. Each node vi ∈ V is a paper, interconnected by the edges e ∈ E that signify citation relations. Let xi ∈ X denote the text (i.e., paper abstract) of the i-th node. Each node has a label to indicate the topic of the paper. Since the graph nodes and papers have a strict one-to-one correspondence, node vi and text xi share an identical label.

Few- and zero-shot learning. For few-shot classification, the test dataset encompasses a support set S and a query set Q. S comprises C classes of nodes, with K labeled nodes drawn from each class. These nodes can be used to train or fine-tune the classifier, which is then utilized to classify the nodes in Q. Zero-shot node classification is essentially a special case of few-shot classification with K = 0. There are no labeled nodes for both training and testing, and classification depends solely on the class names.

Contrastive loss. Recent researches [Wen and Fang, 2023; Zhao et al., 2024] use the contrastive loss to jointly train the graph and text encoders. Specifically, they employ GCN [Kipf and Welling, 2016] as the graph encoder ϕ to encode each node vi into a node embedding ni, and adopt Transformer [Vaswani et al., 2017] as the text encoder ψ to map each text xi to a text embedding ti. That is,

Then, they use InfoNCE [He et al., 2020] loss LCL to maximize the similarity between each node ni and its corresponding text ti, while simultaneously minimizing the similarity between node ni and other mismatched texts tj . As shown in part (1) of Figure 2, LCL is calculated as follows:

where B is a data batch, sim(, ) is the cosine similarity, and τ is a learnable temperature.

3 The TSA Framework

In this section, we present a novel pre-training and inference framework, named TSA. We start with a overview and follow up with the detailed descriptions of its components.

3.1 Overview

The overall architecture of our framework is illustrated in Figure 2. The model for few-shot consists of a graph encoder and a text encoder, and an extra negative text encoder is included for zero-shot pre-training. We introduce them as follows.

• Graph encoder ϕ. We adopt a graph neural network as the encoder to generate the node embedding n.
• Text encoder ψ. We choose Transformer [Vaswani et al., 2017] as the text encoder, and it produces a text embedding t for each text description.

• Negative text encoder ψneg . This maintains the same architecture as the text encoder, with the difference that we train it independently with a negative prompt to generate the negative text presentation tneg .

To effectively train the above encoders, we design two novel loss functions: positive semantics matching loss and negative semantics contrast loss, which can assist the pretraining model in mining more semantic information. Next, we propose a strategy in Section 3.3, probability-average, to enhance zero-shot node classification.

3.2 Text Semantics Augmentation

In this section, we introduce two novel augmentations: negative semantics contrast and positive semantics matching, to exploit text semantics to enhance model training on TAGs.

Positive semantics matching loss. We provide more positive text embeddings for each node embedding. G2P2 [Wen and Fang, 2023] defaults to only one text embedding similar to each node embedding, however there may be multiple similar texts to the target node in TAGs [He et al., 2020; Chuang et al., 2020]. Therefore, we search for multiple text embeddings that are similar to the text embedding of the target node and subsequently encourage the target node embedding to align with these similar text embeddings ˆt. The positive semantics matching loss is denoted as follow:

where K is the number of similar text embeddings.

However, the above method has two serious drawbacks: first, the complexity of brute-force search for similar text embeddings among all text embeddings is unacceptable; second, storing all text embeddings in GPU memory may lead to outof-memory. To address these issues, we create a text bank with a capacity of 32K to model the whole text embedding space. As illustrated in the Figure 2(2), whenever a new batch of data arrives, the earliest text embedding is discarded if the capacity of the text bank exceeds a predetermined threshold.

Otherwise, it is stored in the bank. Subsequently, we identify the most K similar text embeddings for target node through similarity calculations. In this way, our text bank is both timeefficient and space-efficient.

Negative prompt. After co-training the graph and text encoder using Equations (2) and (3), the model now possesses the base capability to distinguish node-text pairs. However, understanding the negative semantics within the input text description poses a challenge for the model. For example, we represent a text description such as "a paper is published at IJCAI" and its negation "a paper is published at The Lancet". In the embedding space, these two descriptions are likely to be very similar, as their raw texts differ by only few words. To address this issue, we employ negative prompts to generate multiple negative texts that are semantically opposed to the original text descriptions. These negative texts are then used to train a negative text encoder independently. This process helps the negative text encoder learn parameters that are contrary to those of the text encoder.

Our initial idea is to manually construct a series of negative prompts. Specifically, we manually alter the text descriptions by incorporating negation terms such as "no", "not", "without", etc., thus creating a negative prompt corpus that are semantically opposite to the original ones, denoted as Xneg . Then, we input the negative text xneg i into negative text encoder ψneg to generate negative text embedding tneg i .

However, manual modification of the raw text is timeconsuming and labor-intensive. To solve this problem, inspired by CoOp [Zhou et al., 2022], we propose a learnable negative prompt and add it to the front of raw text. The underlying logic is to represent negative semantics by constantly optimizing the learnable prompt, thereby mirroring the handcrafted negative prompts Xneg . Specifically, we concatenate the raw text with M learnable vectors to generate the negative prompt h, allowing Xneg to be replaced by h. Then we input h into the negative text encoder ψneg , denoted as follows:

where the negative text encoder is a Transformer with the same architecture as the text encoder. We further demonstrate from the perspective of information entropy that negative text can replace hand-crafted negative prompt.

Theorem 1. Given the learnable negative prompt h and hand-crafted negative prompt Xneg , the information entropy of the learnable negative prompt H(h) is related to the lower bound of the information entropy of the hand-crafted negative prompt LowerBound(H(Xneg )) as follow:

This indicates that learnable negative prompts exhibit higher information entropy than the lower bound of hand-crafted negative prompts. Therefore, learnable negative prompts can effectively capture the negative semantics present in handcrafted negative prompts. Proof see in Appendix A of [Wang et al., 2025b].

Negative semantics contrast loss. There is still an unsolved problem: how do we train a negative text encoder? In other words, how do we ensure that the semantics of the negative text embeddings contradict the original text embeddings. To address this problem, we introduce a novel loss functions, termed negative semantics contrast loss, which comprises margin loss and semantics-opposite loss.

The margin loss anticipates the greatest possible similarity between positive pairs, and conversely, it expects dissimilarity in the case of negative pairs. As shown in Figure 2(3), given a target node vi, the corresponding negative text description tneg i is deemed a negative text, while any other noncorresponding text tneg j̸=i are considered positive texts. Subsequently, we employ margin loss to assess the degree of matching between the target nodes, positive texts, and negative texts. Specifically, margin loss ensures that the similarity between the target node and the positive text is at least a margin m higher than the similarity with the negative text. Here, we use margin loss instead of InfoNCE loss because margin loss remains constant when the gap between positive and negative samples exceeds m. This is favorable because positive and negative samples are already easily distinguishable. The margin loss LML is denoted as follows:

As shown in Figure 2(4), semantics-opposite loss seeks to maximize the mean square error between positive and negative text embeddings. As text xi and negative text tneg i are semantically opposite, their corresponding embeddings should be as far apart as possible in the text embedding space. We compute the semantics-opposite loss LSO as follow:

where ∥∥2 is the L2 norm. Thus, the negative semantics contrast loss is equal to the sum of margin loss and semanticsopposite loss, denoted as LNSC = LML +LSO. It enforces both the node and text embeddings are dissimilar to the corresponding negative text embedding.

Objective. In summary, we denote the total loss of TSA as:

where α is the hyperparameter with respect to loss activation. In few-shot pre-training, we do not activate the loss LNSC (i.e., α = 0) because the prompts in few-shot are inherently learnable, incorporating negative prompts would introduce more noise and lead to sub-optimal performance. In contrast, zero-shot classification lacks labeled data during the pre-training. We analyze the ablation experiments on the loss function in detail in Section 4.3.

Complexity Analysis. TSA incorporates both a GNN and a Transformer. The GNN takes O(LNd2 ) time for aggregating the neighboring nodes, where L is the network depth, N is the number of nodes and d is the number of dimensions. The Transformer's time complexity is O(sd2 + s2d), where s the maximum length of the input sequence. O(sd2 ) time is used for mapping vectors at each position to query, key and value vectors, and O(s2d) time is utilized for the computation of the attention score. Consequently, the overall time complexity of our method is O(LNd2 + sd2 + s2d).

3.3 Prompt Tuning and Inference

Based on the pre-trained model, we tune the model parameters to adapt to the few/zero-shot tasks, which enables classification with a few even no labeled samples while concurrently freezing the pre-trained model's parameters. Prompt tuning is a prompt optimization technique in multimodal models for improving accuracy in few/zero-shot tasks. It uses learnable context vectors [v] to replace the static prompt words. For example, "a photo of a [class]" is replaced by [v1][v2][v3][class]. Next, we introduce the foundational paradigm of few- and zero-shot node classification.

Zero-shot classification. In the zero-shot setting, we operate without any labeled samples and rely solely on class name description. To perform C-way node classification, we construct a series of class descriptions {Dc}Cc=1 , via discrete prompts, such as "a paper of [class]". Then, we input the description text into the pre-trained text encoder to generate the class embedding gc = ψ(Dc). We predict the category of a node vi by computing the similarity between the node embedding ni with the class embedding gc. The insight behind this is that we align the pre-training and prompting objectives (i.e., to determine whether nodes and texts are similar). Thus, we do not have to tune the parameters of the pre-trained model. The similarity probability between the target node and the candidate class description is calculated as follows:

Few-shot classification. In the few-shot setting, we conduct a C-way K-shot classification task. Unlike discrete prompts (i.e., "a paper of ...") in the zero-shot setting, we have C×K labeled samples to train learnable prompts. Specifically, we construct a continuous prompt gc by adding M learnable vectors to the front of the class description Dc. Formally, we denote gc = ψ([e1, e2, ..., eM,Dc]). Then, we use Equation (9) to predict the node category, and update the continuous prompts by minimizing the discrepancy between the predicted and ground-truth labels via cross-entropy loss. It is worth noting that because C ×K is a small value, the parameters required to fine-tune the prompts are considerably less than those needed for the pre-trained model.

Probability-average. As shown in Figure 3, we propose probability-average to predict node category. Specifically, we first compute pi by Equation (9). We use the negative text encoder to generate the negative class embedding. Then, we compute negative probability pneg i by contrasting these negative class embeddings with the target node embedding using Equation (9). pi denotes the probability that a node belongs to each category and vice versa, pneg i represents the probability that a node does not belong to each category. Finally, we utilize (pi + 1 - pneg i )/2 to predict the node label. Unlike using a single-encoder model, the negative text encoder provides additional predictive auxiliary for the positive text encoder. Therefore, we use probability-average to balance the output probabilities of the positive and negative text encoders and thus enhancing classification accuracy. Formally, the probability-average strategy can be denoted as follows:

Note that the probability-average stragety is only applicable to zero-shot classification, as it requires the negative prompts and negative text encoder to calculate pneg i . In contrast, few-shot classification directly uses pi to predict labels.

4 Experimental Evaluation

In this section, we conduct extensive experiments to evaluate TSA and answer the following research questions.

• RQ1: How does TSA compare with state-of-the-art methods for few- and zero-shot classification on TAGs?
• RQ2: Do our augmentation techniques improve accuracy?
• RQ3: How efficient is TSA in training and inference?

4.1 Experiment Settings

Datasets. Following related researches [Yan et al., 2023], we use 5 datasets for experiments. Cora [McCallum et al., 2000] is a citation network, where papers are linked by citation relations and abstract serves as the text. Art, Industrial, M.I., and Fitness are derived from Amazon product categories [Yan et CVPR KDD ICML 0.16 0.64 0.08 (a) Probability-product 𝑝(1 - 𝑝!"# ) ✅ al., 2023], namely, arts, crafts and sewing for Art; industrial and scientific for Industrial; musical instruments for M.I.; and sports-fitness for Fitness, respectively. For the four datasets, an edge is added to construct the graph if a user visits two products successively, and the text is the product description. The five datasets cover different scales (from thousands to millions of nodes) and number of classes (from tens to thousands). These datasets are stored in OceanBase [Yang et al., 2022] and the dataset statistics is provided in Appendix B of [Wang et al., 2025b].

Baselines. We compare TSA with 13 baselines from 5 categories, briefly describe as follow.

• Supervised GNNs: GCN [Kipf and Welling, 2016], SAGEsup [Hamilton et al., 2017], TextGCN [Yao et al., 2019]. They are trained in a supervised or semi-supervised manner for the node classification tasks.
• Self-supervised GNNs: GraphCL [You et al., 2020], InfoGCL [Xu et al., 2021], PGCL [Lin et al., 2022]. They are first pre-trained via contrastive learning and then finetuned for the classification tasks.
• Graph prompt methods: GPPT [Sun et al., 2022], GFP [Fang et al., 2024], GraphPromt [Liu et al., 2023]. They reduce the divergence between the pre-training and inference by designing the training objectives and prompts.
• Language models: BERT [Devlin et al., 2018], LLMGNN [Chen et al., 2023], GraphTranslator [Zhang et al., 2024]. Bert is first pre-trained and then fine-tuned for text classification. LLM-GNN and GraphTranslator translate graph into language and predict the labels by LLMs.
• Co-trained model: G2P2 [Wen and Fang, 2023]. It employs the contrastive loss to train the GNN and language model jointly such that they produce similar node embedding and text embedding for each node-text pair.

Following G2P2, we use classification accuracy and F1 score to measure performance. We report the average value and standard deviation across 5 runs. Note that we only select language models and G2P2 as the baselines for zero-shot classification, since the other baselines require at least one labeled sample per class for either training or inference.

Environment. During pre-training, we use Adam as the optimizer with a learning rate of 2e-5 for 2 epochs, and the batch size is 64. The number of similar text representations and the capacity of the text bank are set to 1 and 32K, respectively. The length of the learnable negative prompt is 16. Margin m is set to 1. For few-shot pre-training, we do not activate the negative semantics contrast loss, so α is set to 0. Instead, α is set to 0.5 during zero-shot pre-training. Our experiments are conducted on a server with Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz and 8 NVIDIA RTX A6000 48G GPUs.

Task configurations. For few-shot classification, we use a 5way 5-shot setup, i.e., 5 classes are taken from all classes, and then 5 nodes are sampled from these classes to construct the training set. The validation set is generated in the same way as the training set, and all remaining data is used as the test set. For zero-shot classification, we use 5-way classification, which samples classes but does not provide labeled nodes.

4.2 Main Results (RQ1)

Few-shot node classification. Table 1 reports the accuracy of TSA and the baselines for few-shot node classification. We can see that TSA consistently outperforms all baselines across the datasets, with an average improvement of 4.6% and 6.9% for classification accuracy and F1 score, respectively. Moreover, the improvements of TSA over the baselines are over than 5% in 6 out of the 10 cases. The results show that TSA can mine text semantics effectively to enhance model pre-training and thus improve accuracy. More explanations are in Appendix C.1 of [Wang et al., 2025b].

Zero-shot node classification. Table 2 reports the accuracy of TSA and the baselines for zero-shot node classification. We only include the language models and G2P2 because the other methods require at least one labeled sample for inference. The results show that TSA consistently outperform all baselines by a large margin. Compared with the bestperforming baseline G2P2, the average improvements of TSA in classification accuracy and F1 score are 8.8% and 9.3%, respectively. All methods have lower accuracy for zero-shot classification than few-shot classification because zero-shot classification does not provided labeled samples, and thus the task is more challenging. However, the improvements of TSA are larger for zero-shot classification because it introduces more text semantics for learning.

Robustness to task configuration. We conduct the fewerway and fewer-shots classification on M.I. dataset. The experimental results are provided in Appendix C.2 of [Wang et al., 2025b]. The results show that TSA outperforms G2P2 across different configurations of ways and shots. We observe that to achieve the same accuracy, TSA requires fewer labeled samples (i.e., shots) than G2P2. Moreover, TSA surpasses the G2P2 across the different ways for zero-shot classification.

4.3 Micro Experiments

Effect of the augmentations (RQ2). Table 3 presents an ablation study by gradually enabling different augmentations techniques in TSA. We can see that all augmentations are effective in improving accuracy, as adding each of them outperforms the baseline. The best-performing combination for few-shot classification deactivates negative semantics contrast (i.e., LNSC) while zero-shot classification actives LNSC. This is because few-shot classification uses labeled samples to learn the prompt, and the negative prompt learned by LNSC may interfere with prompt tuning. In contrast, zeroshot classification lacks labeled data for prompt tuning, and the LNSC can provide more text semantics.

Efficiency (RQ3). To examine the efficiency of TSA, we compare with G2P2 for pre-training time and prompting time at inference time. We experiment on Industrial and Art, the two largest datasets, as the running time is shorter on the smaller datasets. Figure 4 shows that TSA and G2P2 have similar pre-training time and prompting time. This is because they both jointly train the GNN and language model, and computing the loss terms has a small cost compared with computing the two models. Few-shot classification has longer prompting time than zero-shot classification because it needs to tune the prompt using the labeled samples.

Hyperparameter sensitivity. We conduct the hyperparameter experiments with respect to the number of similar texts and the capacity of text bank. Figure 5 examines the effect of the two parameters on the Industrial and M.I. datasets. We observe that accuracy first increases but then decreases with the number of similar texts. This is because while more similar texts can provide more text semantics, an excessive number of these signals may introduce noise by including texts that are not truly similar to the target node. Hence, the optimal accuracy are obtained at an intermediate value to balance between semantic supervisions and noises. When increasing the capacity of the text bank, accuracy first increases but then stabilizes. This is because using a larger text bank allows a node to identify texts that are more similar but the similarity will become sufficiently highly when the bank is large enough.

5 RelatedWork

Graph Pre-training and Prompting. GNNs [Kipf and Welling, 2016; Veliˇckovi´c et al., 2017] use message passing to aggregate features from neighboring nodes to compute graph node embedding. However, early GNN models, such as GCN [Kipf and Welling, 2016], and GAT [Veliˇckovi´c et al., 2017], are supervised and require many labeled nodes for training. To mine supervision signals from unlabeled data, graph self-supervised learning is proposed to train using welldesigned pretext tasks. For instance, DGI [Veliˇckovi´c et al., 2018] learns node embeddings by maximizing mutual information between the global and local node embeddings. GPTGNN [Hu et al., 2020] utilizes a self-supervised graph generation task to combine the graph structural and semantic information.

Graph self-supervised learning [Wang et al., 2024a; Wang et al., 2024b; Liang et al., 2025; Wang et al., 2025a; Zhang et al., 2025] methods still require many labeled to finetune specific tasks (e.g., node classification). To further reduce the reliance on labeled instances, graph prompt learning is proposed for few-shot node classification. For example, GPPT predicts the node label by deciding whether an edge exists between the target node and candidate labels. GraphPrompt [Liu et al., 2023] learns embeddings for subgraphs rather than nodes to unify graph-level and node-level tasks. These approaches consider only the graph and thus have limited accuracy for TAGs with text descriptions. To account for the text, TextGCN [Yao et al., 2019] generates text embeddings using pre-trained language models and adds these embeddings as node features for GNN training. G2P2 [Wen and Fang, 2023] jointly trains the language model and GNN with the contrastive strategy and uses prompting for few-shot and zero-shot node classification. However, TSA targets TAGs and considers the graph and text modalities jointly by mining more text semantics while graph pre-training methods consider only the graph.

Pre-trained Language Models (PLMs). PLMs [Devlin et al., 2018; Yang et al., 2022; Yang et al., 2023; Han et al., 2024] enhance the ability to understand natural language by pre-training on large-scale text corpus. The well-known BERT [Devlin et al., 2018], for instance, is pre-trained with two tasks, i.e., masked token reconstruction and next token prediction, to capture contextual information. RoBERTa [Liu et al., 2019] improves BERT by eliminating the next token prediction task, increasing the batch size and data volume during pre-training, and using a dynamic masking strategy. While PLMs achieve great success for text oriented tasks, they cannot capture the topology information for TAGs.

6 Conclusion

In this paper, we study few-shot and zero-shot node classification on text-attributed graphs. We observe that the prior methods is limited to graph-based augmentation techniques, thus we propose TSA as a novel pre-training and inference framework. TSA incorporates two key augmentation techniques, i.e., positive semantics matching and negative semantics contrast, to exploit more text semantics. Extensive experiments show that TSA outperforms existing methods by a large margin. We believe our methodology, i.e., generating node-text pairs that have similar/dissimilar embeddings, is general and can be extended beyond our augmentation techniques.

Acknowledgments

This work was sponsored by National Natural Science Foundation of China (62472327) and the Fundamental Research Funds for the Central Universities (2042025kf0040). This work was supported by Ant Group through CCF-Ant Research Fund (CCF-AFSG RF20240104) and Sichuan Clinical Research Center for Imaging Medicine (YXYX2402).