RegionMatch: Pixel-Region Collaboration for Semi-Supervised Semantic Segmentation in Remote Sensing Images

1 Introduction

Semantic segmentation in remote sensing images (RSIs) is a fundamental task in remote sensing data interpretation, aiming to classify each pixel into distinct categories. It is widely applied in land cover monitoring, urban planning, and disaster assessment. Currently, deep learning-based models have dominated this field. However, these models require labor-intensive pixel-level annotations to achieve satisfactory performance. To address this challenge, semisupervised semantic segmentation (S4) [Wang et al., 2022a; Ding et al., 2023] uses a small labeled dataset combined with a large amount of unlabeled data, reducing the dependence on annotation data.

The key to S4 lies in the effective utilization of unlabeled data. Currently, the combination of pseudo-labeling and consistency regularization techniques has become the dominant approach [Sohn et al., 2020; Yang et al., 2023a]. These methods select reliable pixels predicted from one augmented version of unlabeled data and use them as pseudo-labels to train another. Most of the improved methods [Wang et al., 2022b; Qiao et al., 2023; Wang et al., 2024] still utilize unlabeled data only from an individual pixel perspective. Recent studies revisit the potential of semi-supervised learning from the perspective of semantic segmentation tasks, focusing on more effective strategies for exploiting unlabeled data. For instance, RankMatch [Mai et al., 2024] designs consistency regularization methods from the perspective of inter-pixel correlations, CorrMatch [Sun et al., 2024a] utilizes pairwise similarity maps between pixels to propagate labels, and MPMC [Howlader et al., 2025] mitigates pseudo-labeling noise by utilizing multi-label classification predictions for image patches. However, these approaches are limited to pixellevel information when leveraging unlabeled data. This limitation is particularly pronounced in RSIs, where models are more vulnerable to noise due to complex backgrounds and diverse landscapes.

Semantic segmentation tasks inherently depend on understanding object-level relationships, where a pixel's label is often determined by the object region to which it belongs. RSIs cover wide ground areas, with ground objects exhibiting strong coherence, particularly in areas such as residential zones, forests, and agricultural lands. We observed that semantic segmentation models exhibit strong object region recognition ability in RSIs. As illustrated in Fig. 1, classification accuracy for object regions significantly outperforms pixel-level accuracy, indicating that region-level information is more stable and reliable than pixel-level's. Furthermore, the predicted regions exhibit high consistency, as evidenced by the dominance of a single category within these regions. Current S4 methods fail to incorporate valuable object-level information, limiting their performance in RSIs.

In this paper, we analyze the bottleneck of existing methods based on pixel-level information and propose a novel approach, RegionMatch, to leverage unlabeled data from a fresh object-level perspective for semi-supervised semantic segmentation of RSIs. Specifically, we introduce an object region classifier that trains the model to identify object regions composed of pixels, explicitly injecting object-level contextual information into the S4 pipeline. We then design the Pixel-Region Synergy Pseudo-Labeling strategy (PRSPL) to integrate both pixel-level and object-level information, generating high-quality pseudo-labels for model training and promoting knowledge collaboration across the two perspectives. To further unleash the potential of unlabeled data, we propose the Region-Structure Aware Correlation Consistency (RSCC). RSCC explicitly models object-level relationships by establishing multi-level correlations both within and between pixels and regions, providing stronger supervision signals for unlabeled data that are more in line with the nature of semantic segmentation. RSCC consists of two key components: Cross-Image Region-Region Correlation Consistency (CIR²R) and Pixel-Region Correlation Consistency (PRCC). CIR²R establishes inter-region correlation across images to capture object-level consistency, while PRCC strengthens intra-region pixel consistency, thus improving the model's understanding of object-level relationships.

The contributions of this paper can be summarized as follows:

1. We propose RegionMatch, a novel S4 method for RSIs, which utilizes unlabeled data from a fresh object-level perspective that is more tailored to the nature of semantic segmentation.
2. We designed the Pixel-Region Synergy PseudoLabeling strategy, which encourages knowledge cooperation between pixel and object-region perspectives to generate high-quality pseudo-labels for model training.
3. We propose Region-Structure Aware Correlation Consistency which models object-level relationships to provide more effective supervisory signals for unlabeled data, thus inspiring the potential of unlabeled data further.

2 RelatedWork

2.1 Semi-Supervised Semantic Segmentation

The primary challenge in S4 is effectively leveraging unlabeled data to enhance model generalization. Key strategies include adversarial method [Ke et al., 2020; Zhang et al., 2020], consistency regularization [Chen et al., 2021; Liu et al., 2022], pseudo-labeling [Yang et al., 2022; Teh et al., 2022], and contrastive learning [Zhou et al., 2021; Wang et al., 2023b; Wang et al., 2023a]. Among these, the combination of pseudo-labeling and consistency regularization [Sohn et al., 2020; Yang et al., 2023a; Chen et al., 2021] has emerged as the dominant approach. These methods generate pseudo-labels from a segmentation model's predictions on weakly augmented unlabeled data, which are then used to train strongly augmented versions. While previous approaches [Wang et al., 2022b; Qiao et al., 2023; Sun et al., 2024b; Na et al., 2024; Yang et al., 2025; Wang et al., 2024] utilize unlabeled data from the perspective of individual pixels, recent approaches construct supervised signals for unlabeled data from the perspective of inter-pixel relationships and multi-label classification of image patches. For example, RankMatch [Mai et al., 2024] models pixel correlation using representative proxies to optimize rank-aware consistency, while CorrMatch [Sun et al., 2024a] employs pairwise pixel similarity for label propagation. Another approach [Howlader et al., 2025] introduces a multi-label classifier with multi-scale patches to reduce pseudo-label noise. However, these methods utilize unlabeled data only from a pixel-level perspective, ignoring the fact that the semantic segmentation task is more dependent on the understanding of object-level relationships, thus limiting their performance.

Relatively few studies have focused on semi-supervised semantic segmentation for RSIs. Current research primarily concentrates on developing more effective data perturbation strategies [Lu et al., 2023; Lv and Zhang, 2024; Bai et al., 2024; Liu et al., 2024; Luo et al., 2024] to improve the effect of consistency regularization, and constructing representation learning methods for unlabeled data [Xin et al., 2024; Luo et al., 2024; Yang et al., 2023b]. These methods rely solely on individual pixel information when utilizing unlabeled data, overlooking the strong region consistency inherent in RSIs, which limits the performance of S4.

2.2 Context for Semantic Segmentation

Pixel classification is heavily influenced by its context, typically defined by spatially related locations, such as the surrounding neighborhood. Many semantic segmentation models enhance feature representations by incorporating contextual information. Early works [Zhao et al., 2017; Zhao et al., 2018; Chen et al., 2017] used convolution and pooling kernels of varying sizes to capture multi-scale context. However, these fixed-region aggregation methods are non-specific, and information collected from semantically irrelevant regions may introduce noise. Subsequent research has focused on more effective aggregation of contextual information from semantically relevant regions. For instance, [Ding et al., 2019] restricts context regions by learning semantic masks that match object shape and scale. [Yu et al., 2020] addresses intra- and inter-class context dependencies through affinity loss supervision. [Shen et al., 2020] enhances pixel representation by modeling contextual interactions between regions. [Hu et al., 2021a] proposed region-aware contrast learning to improve pixel features. Unlike these fully-supervised approaches, this paper aims to explore object-level information from unlabeled data as supervisory signals to maximize its utilization.

3 Method

S4 aims to train a semantic segmentation model using a small labeled dataset and a large unlabeled dataset. We propose RegionMatch, leveraging unlabeled data from a novel objectlevel perspective.

3.1 Preliminaries

RegionMatch is built upon the classic pseudo-labeling and weak-to-strong consistency regularization framework [Sohn et al., 2020]. Given a labeled set Dl = {(xli , yl i )}N l i=1 and an unlabeled set Du = {xui }N u i=1 , where N u ≫ N l . Pixel-wise cross-entropy loss Lp is applied to both labeled and unlabeled data:

For labeled data, the loss Llp is calculated as follows:

where lce represents the standard cross-entropy loss function, pl = fp(g(xl )) is the predicted probability distribution of the pixel classifier fp(·), and g(·) is the feature extractor of the segmentation model. For unlabeled images, weak augmentation α(·) and strong augmentation A(·) are applied. The prediction from the weakly augmented version is used to supervise the strongly augmented version. The unlabeled loss Lup is computed as follows:

where pu = fp(g(A(xu ))) represents the predicted probability distribution of the pixel classifier, qu = fp(g(α(xu ))), ˆ yu = argmax(qu ), and τ is a fixed confidence threshold.

3.2 Overview

Fig. 2 shows the pipeline of RegionMatch for unlabeled data. We generate region pseudo label using the model's predictions on a weakly augmented version of the image, dividing the image into multiple semantically consistent regions consisting of adjacent pixels that represent complete objects or parts of objects. An object region classifier is then introduced to classify these regions, explicitly incorporating object-level contextual information into the pipeline. We then propose the Pixel-Region Synergy Pseudo-Labeling strategy (PRSPL), which merges information from both pixel- and region-level perspectives to generate high-quality pseudo-labels for training. Additionally, we propose Region-Structure Aware Correlation Consistency (RSCC) to model object-level relationships in images to build supervisory signals for unlabeled data. RSCC consists of Cross-Image Region-Region Correlation Consistency (CIR²R) and Pixel-Region Correlation Consistency (PRCC). The overall loss function of RegionMatch consists of the pixel classification loss Lp, the region classification loss Lr, the CIR²R loss LCIR, and the PRCC loss LPR:

where λ1 and λ2 are the corresponding weight coefficients.

3.3 Object Region Classification

In the method described in Section 3.1, the supervised signal for unlabeled data relies only on pixel-level information, which makes pixel classification susceptible to noise in RSIs with complex and variable backgrounds. RSIs typically have extensive ground coverage and strong ground object coherence. We find that the model provides more stable and reliable pseudo-labels at the object level compared to the pixel level. Therefore, we introduce explicit supervision for objectregion classification to inject object-level contextual information into the S4 pipeline, improving the model's understanding of object-level relationships of the image.

For each image, both pixel-level and region-level labels are provided. The pixel label denotes the semantic category of each pixel, while the region label specifies the object region to which the pixel belongs. Region labels rl ∈ {1, . . . ,Nl}H×W for labeled image xl and ru ∈ {1, . . . ,Nu}H×W for unlabeled image xu are both obtained using a connected component algorithm. Labeled image uses the pixel-level ground truth yl, and unlabeled image uses the pixel-level pseudo-label ˆ yu. For further details, see the appendix. We introduce a region classifier fr to classify the regions in the image. For each region i, the feature extractor outputs feature maps F ∈ RH×W×d , which are aggregated using mask average pooling within the region and passed through a projection layer h(·) to obtain the corresponding region feature Rli . The region classification loss for the labeled image xl is defined as:

where yr i denotes the ground truth class label of region i.

To minimize interference from noisy regions in the unlabeled data, we apply a dual thresholding mechanism with τlow and τhigh to compute the region classification loss for unlabeled data. Specifically, we calculate the average confidence score ¯ cui of each region i as the quality assessment of the region. Noisy regions are filtered out using the lower threshold:

For regions with τlow ≤ ¯ cui < τhigh, we generate soft labels ˜ qu i by averaging the pseudo-label probability within the region. For regions with ¯ cui > τhigh, hard labels are used: ˆ yr i = argmax(˜ qu i ). The region classification loss for the unlabeled image xu is computed as:

The total region classification loss is defined as:

where λ = 0.5 is a balancing factor that controls the weight of the unlabeled loss relative to the labeled loss.

3.4 Pixel-Region Synergy Pseudo-Labeling

The pixel classifier captures features at the individual pixel level, while the region classifier catches class attributes at the object level. A natural idea is to promote cooperation between the two perspectives to improve the model's understanding of unlabeled data. To this end, we propose the PixelRegion Synergy Pseudo-Labeling strategy (PRSPL) strategy, which combines the knowledge of both perspectives to generate high-quality pseudo-labels for training models. If the two classifiers reach consensus on the category prediction of a pixel, we consider the pseudo-label of this pixel to be reliable. We therefore select pixels with consistent predictions from both perspectives to train the model, which in turn promotes knowledge cooperation between the two perspectives.

For the image xu, we use the region classifier fr to predict pixel-wise probability distributions qru under the region-level view. For the pixel-level view, the probability distribution is denoted as qpu . We calculate the entropy for both the regionlevel and pixel-level views, defined as Hru and Hp u , respectively:

where C is the number of classes. To avoid interference from noisy regions, we compute reliable pseudo labels in the low entropy regions of both views. In low-entropy regions, the predictions from both classifiers are considered more confident. The low-entropy region i for the region-level perspective I r and the pixel-level perspective I p are defined as:

The threshold values for entropy are calculated as:

Here, Quantile(·, α) represents the calculation of the α-th quantile. We use a dynamic quantile αt calculated as:

where t is the current epoch number during training.

In the intersection of the low-entropy regions from both views, if a pixel has consistent classification predictions from both the region-level and pixel-level views, we consider the pixel's pseudo-label to be reliable. Therefore, the pixel classification loss for unlabeled data in our method can be expressed as:

where:

(17)

Here, ⊙ denotes element-wise multiplication.

3.5 Region-Structure Aware Correlation Consistency

In semantic segmentation tasks, pixel class labels typically depend on the object regions to which they belong, making it essential to understand object-level relationships. When constructing supervisory signals for unlabeled data, these objectlevel relationships should be fully considered. To this end, we propose Region-Structure Aware Correlation Consistency (RSCC), which provides strong supervisory signals that align with the nature of semantic segmentation tasks by explicitly modeling object-level relationships. RSCC achieves this through two key components: Cross-Image Region-Region Correlation Consistency (CIR²R) and Pixel-Region Correlation Consistency (PRCC). By incorporating object-level relationships into the S4 pipeline, RSCC enhances the model's ability to understand complex scenes in RSIs.

Cross-Image Region-Region Correlation Consistency

CIR²R aims to compute feature correlations across image regions, explicitly model object-level semantic relationships and capture patterns shared between ground objects. By establishing object-level semantic consistency across augmented images, this approach captures richer semantic descriptions and strengthens the supervisory signal of unlabeled data. Specifically, for a batch of unlabeled images {xu }nu=1 , we apply weak augmentation to obtain the augmented image set {xwu }nu=1 . The segmentation model extracts region features {Rw i }Nb i=1 for the weakly augmented images, where Rw i represents the feature of the i-th region. We calculate the relationship matrix Sw ∈ RNb×Nb for all regions in the weakly augmented images. Each element of Sw is computed as:

where ∥ · ∥ denotes the L2 norm, and Sw ij represents the semantic similarity between region i and region j in the weakly augmented images. Similarly, we apply strong augmentation to the same batch of unlabeled images, obtaining the augmented image set {xsu }ni=1 . The segmentation model extracts region features {Rs i }Ns i=1 , and the relationship matrix Ss ∈ RNb×Nb is calculated. Finally, the loss function of this part is computed as the MSE between the similarity matrices of the weakly and strongly augmented images over all batches:

where B denotes the total number of batches in the unlabeled dataset, Sw b and Ss b are the similarity matrices of weakly and strongly augmented images in batch b, respectively, and ∥ · ∥22 represents the squared L2 norm (i.e., MSE).

Pixel-Region Correlation Consistency

Unlike CIR²R, which focuses on the correlations between image regions, PRCC emphasizes the semantic consistency among pixels within a region. This enables the model to capture the complete shape of ground objects and more superior object-level features, thus improving the understanding of object-level semantic relationships. To enhance pixel consistency within a region, a straightforward approach is to minimize the feature distance between any two pixels in the region. However, this approach requires computing pairwise distances between all pixels, leading to substantial computational overhead, especially for large areas. To address this, we use regional features as a proxy for pixel features, indirectly promoting pixel consistency by aligning regional and pixel features. This strategy significantly reduces the computational burden and improves the efficiency of model training.

For each region i and pixel j within that region, the similarity between pixel feature Fj and the region feature Ri is computed as:

where ∥ · ∥2 is the Euclidean norm. The goal is to maximize the similarity between each pixel and its corresponding region feature. The loss function is defined as:

4 Experiments

4.1 Experiment Setup

Dataset. We evaluated the performance of the proposed RegionMatch method using three widely used remote sensing (RS) semantic segmentation datasets: Vaihingen [Rottensteiner et al., 2012], Potsdam [Rottensteiner et al., 2012], and LoveDA [Wang et al., 2021]. The Vaihingen dataset consists of 33 images, with sizes ranging from 1996×1995 pixels to 3816×2550 pixels. For our experiments, 16 images are used for training, and 17 images are used for testing. The Potsdam dataset contains 38 images, each with a size of 6000×6000 pixels. Of these, 24 images are used for training and 14 for testing. The validation categories for both the Vaihingen and Potsdam datasets include impervious surfaces, buildings, low vegetation, trees, and cars. The LoveDA dataset comprises 1024×1024 images from urban and rural areas, with categories including background, building, road, water, barren land, forest, and agriculture. The dataset contains 2522 training images and 1669 testing images. For all datasets, we crop the images into 512×512 pixel sections.

Evaluation Metric. The segmentation performance is evaluated using the mean Intersection over Union (mIoU) on the union of the sets.

Implementation Details. All experiments were conducted on an NVIDIA 4090 GPU. The DeeplabV3+ [Chen et al., 2018] segmentation model with a ResNet50 [He et al., 2016] backbone was employed. Stochastic Gradient Descent (SGD) was used as the optimizer. The base learning rate was set to 0.002, weight decay was set to 0.0001, and momentum was set to 0.9. The total number of training epochs was set to 80. Data augmentation strategies follow the FixMatch [Sohn et al., 2020]. τlow and τhigh are set to 0.85 and 0.95, respectively. λ1 and λ2 are set to 2 and 1, respectively. α0 is set to 20%.

4.2 Comparison with State-of-the-art Methods

Results on Vaihingen. Table 1 presents the experimental results of different semi-supervised semantic segmentation methods on the Vaihingen dataset. Under the labeled data protocols of 1/8, 1/4, and 1/2, our method achieves mIoU scores of 69.25%, 71.20%, and 72.68%, respectively, significantly outperforming existing semi-supervised methods. Compared to supervised learning using only labeled data, our method improves by 7.38%, 5.75%, and 4.06% under the 1/8, 1/4, and 1/2 labeled data settings. These results demonstrate that our approach effectively leverages the advantages of semi-supervised learning, significantly enhancing the model's generalization ability.

Results on Potsdam. The results on the Potsdam dataset are shown in Table 2, where we compare our method with state-of-the-art semi-supervised semantic segmentation approaches. Under the 1/8, 1/4, and 1/2 labeled data protocols, our method achieves mIoU scores of 80.24%, 80.68%, and 81.17%, respectively. These results significantly outperform methods that rely on pixel-level relationships [Sun et al., 2024a] or contextual information from image patches [Howlader et al., 2025], indicating that our approach more effectively unleashes the potential of unlabeled data from an object-level perspective. Furthermore, compared to supervised learning, our method improves mIoU by 5.46%, 3.97%, and 3.02% under the 1/8, 1/4, and 1/2 labeled data protocols, respectively, providing further evidence of its efficacy.

Results on Loveda. Table 3 presents a comparison of various methods on the Loveda dataset. Our approach consistently outperforms existing state-of-the-art methods across different annotated data protocols. Specifically, under the 1/16 labeled data protocol, our method achieves 52.55% mIoU, representing 5.23% improvement over supervised learning. Under the 1/8 protocol, our method attains 52.96% mIoU, surpassing supervised learning by 4.45%. At the 1/4 and 1/2 protocols, our method continues to lead, achieving 53.37% and 53.96%mIoU, respectively, exceeding the supervised approach by 3.33% and 3.71%. These results highlight our method's superior ability to leverage unlabeled data.

4.3 Ablations Studies

Effectiveness of Components. Table 4 illustrates the impact of various components on performance using the Vaihingen dataset. As shown, the proposed components yield significant improvements across all protocols. By collaboratively generating pseudo-labels from both pixel-level and region-level perspectives during model training, PRSPL facilitates knowledge sharing and mitigates the negative effects of erroneous pseudo-labels. Fig. 4 shows the influence of PRSPL on pseudo-label accuracy under the 1/8 labeled data protocol. The results confirm that PRSPL significantly improves pseudo-label accuracy, validating its effectiveness. Fig. 3 illustrates the T-SNE feature visualization on the Vaihingen dataset under the 1/8 protocol for different components. The results indicate that incorporating PRSPL allows the model to capture region-level contextual information, which is beneficial for pixel classification and enhances class separability. RSCC further strengthens the model's understanding of object-level semantic relationships by establishing multi-level relationships both within and between pixels and regions, aiding the model in learning more compact intra-class feature representations. Finally, the combination of PRSPL and RSCC enables the model to achieve both improved inter-class separation and more compact intra-class representations.

Impact of Region Classification. Table 5 demonstrates the impact of object region classification on the Vaihingen dataset. It can be observed that incorporating unlabeled data yields greater benefits for object region classification compared to labeled data. Due to the absence of labels, unlabeled data is more challenging to classify, as it relies on useful contextual information. By introducing the region classification task, contextual information of object regions can be supplemented, which aids in pixel classification and subsequently improves model performance.

Threshold Strategy in PRSPL. In PRSPL, we adopt a dynamic entropy threshold strategy. Table 6 presents the impact of dynamic thresholds on the Vaihingen dataset. It can be observed that the dynamic threshold plays a crucial role in model performance. Since deep learning models first memorize clean labels and then gradually memorize noisy labels, a lower threshold can be set initially to provide more training data. A higher threshold is applied in later stages to prevent the model from fitting noisy data.

Labeling Strategy in Region Classification. We employed a combined labeling strategy of both hard and soft labels for region classification. Table 7 illustrates the impact of this strategy on the Vaihingen dataset. When using the τlow threshold, neither the hard labels nor the soft labels alone achieved optimal performance. The best performance was attained when both soft and hard labels were used in combination.

5 Conclusion

This paper presents a novel semi-supervised semantic segmentation method, RegionMatch, effectively leveraging unlabeled data from a fresh object-level perspective. We propose the Pixel-Region Synergy Pseudo-Labeling strategy to explicitly inject contextual information in the S4 pipeline and encourage knowledge cooperation from pixel and region perspectives. In addition, our proposed Region-Structure Aware Correlation Consistency (RSCC) models object-level relationships by establishing the multi-level correlation between regions and pixels, providing stronger supervision signals for unlabeled data that fit the semantic segmentation task. Extensive experiments on multiple authoritative remote sensing datasets demonstrate that our method effectively improves the potential of unlabeled data.