A Local-Rotation-Driven Global Consistency Framework with Dual-View Decoding for Semi-Supervised Medical Image Segmentation

1 Introduction

Numerous clinical imaging workflows fundamentally rely on image segmentation. This capability enables computer-aided diagnosis, treatment planning, radiotherapy contouring, and lesion evaluation [Han et al., 2024]. While deep learning has delivered substantial improvements, most high-performance segmenters still require fully supervised training paradigms, necessitating extensive pixel-wise annotations [Wang et al., 2021]. Manual labeling creates a significant bottleneck: specialists must typically trace volumetric data slice-by-slice. Moreover, the resultant masks frequently exhibit variability across different observers and imaging protocol. By capitalizing on unlabeled data to lessen this manual burden, semisupervised approaches in medical segmentation thus represent a pragmatic pathway forward.

∗Corresponding author.

The fundamental challenge in semi-supervised medical image segmentation lies in reconciling local precision with global anatomical plausibility. Fine-grained boundaries and textures depend largely on local evidence, yet clinically meaningful masks require coherent global context. With scarce annotations, models inevitably drift toward noisy local patterns or retreat into overly rigid global priors [Guan et al., 2024; Zhang et al., 2024]. Recent research tackles this balance from several architectural fronts. CNN-based approaches remain competitive through systematic extensions and meticulous re-benchmarking [Isensee et al., 2023; Isensee et al., 2024]. Hierarchical Transformers simultaneously strengthen long-range dependencies via structured attention mechanisms [Hatamizadeh et al., 2022; He et al., 2023]. Foundation and prompt-driven segmenters have further improved cross-domain generalization [Chen et al.,

2023; Zhu et al., 2024], while emerging state-space hybrids offer efficient alternatives for global modeling [Liu et al., 2024; Ruan et al., 2024].

Nevertheless, limited supervision leaves the model's internal reasoning sparsely constrained. Local features prove susceptible to artifactual cues, while learned global relations risk violating anatomical priors-particularly when local and global mechanisms remain tightly coupled [Guan et al., 2024]. This challenge motivates architectures that explicitly regularize both shape encoding and relational coherence [Zhang et al., 2024].

We therefore propose LR-GCF, a Local-Rotation-Driven Global Consistency Framework for semi-supervised medical segmentation. LR-GCF directly couples local patch-wise rotations with global structural constraints: from each image we derive two locally-rotated variants, encode them via a shared CNN backbone, and decode through parallel Swin Transformers. Consequently, learning under strong geometric perturbations becomes directly anchored by anatomical coherence.

To improve training with sparse annotations, we introduce complementary regularization strategies. First, an auxiliary Rotation Head operating on deep encoder features supervises patch-wise orientation, cultivating locally distinctive representations that resist rotational variations. This anchors local evidence to global anatomical priors despite aggressive perturbations. Second, we propose a rotation-synchronized attention consistency objective that constrains self-attention relationships across both views, fostering harmonized regionto-region reasoning beyond simple output agreement and bolstering anatomical soundness.

Additionally, independently learned multi-scale positional embeddings invite geometric drift-a difficulty amplified when inputs experience divergent local rotations. Our Positional Encoding Duplicating (PED) scheme counters this by deriving all positional encodings from a shared lowresolution anchor through scale-wise replication. This approach furnishes a consistent spatial reference throughout the decoding stack, calms attention fluctuations, and refines interview correspondence. In aggregate, LR-GCF harmonizes locally rotation-driven feature extraction with globally relational constraints, yielding resilient segmentation from limited supervision.

The main contributions are summarized as follows:

• We introduce LR-GCF, a dual-stream framework that integrates local rotational perturbations with global structural regularization for semi-supervised segmentation of medical images.
• We propose a localized rotation prediction module that cultivates orientation-robust, morphology-sensitive features, ensuring they preserve consistency with global anatomical priors under aggressive augmentations.
• We consolidate global relational regularization via rotation-synchronized attention consistency across selfattention matrices, coupled with PED-based propagation of multi-scale positional embeddings from a unified lowresolution anchor, jointly stabilizing Transformer decoding for all views and scales.
• Extensive experiments on multiple datasets demonstrate that LR-GCF consistently achieves state-of-the-art performance, with strong gains in both boundary accuracy and structural coherence under scarce-label settings.

2 RelatedWork

2.1 Semi-SupervisedMedical Image Segmentation

Medical semi-supervised segmentation (SSMIS) confronts dense prediction tasks where sparse annotations coexist with abundant unlabeled scans, leveraging internally-generated predictions and auxiliary constraints as surrogate supervision [Jiao et al., 2024]. Contemporary pipelines predominantly harness dual mechanisms-pseudo-label generation and consistency regularization-which operate synergistically to stabilize optimization when labeled data remains severely limited [Tarvainen and Valpola, 2018; Sohn et al., 2020; Luo et al., 2023].

For pseudo-label generation, temporal ensembling through mentor-learner architectures represents the dominant paradigm, wherein a gradually evolving teacher network synthesizes refined supervisory signals for unannotated volumes. Reliability enhancement typically incorporates uncertainty quantification or confidence-based filtering to suppress noisy pseudo-labels [Yu et al., 2019; Sohn et al., 2020]. Concurrently, mutual supervision across paired decoders mitigates self-reinforcement errors [Chen et al., 2021b; Luo et al., 2023], while region-mixing augmentations facilitate knowledge transfer from labeled to unlabeled instances [Olsson et al., 2020]. Invariance-driven consistency constraints complement pseudo-labeling by enforcing stability against perturbations, with multi-scale implementation demonstrating particular efficacy for dense prediction tasks [Luo et al., 2022]. Nevertheless, predominant reliance on output-space constraints biases optimization toward local texture patterns rather than long-range anatomical dependencies, yielding predictions that lack structural coherence under limited supervision [Jiao et al., 2024].

2.2 Vision Transformers for Medical Image Segmentation

Self-attention mechanisms in transformer architectures capture distant dependencies while aggregating global context explicitly, making them increasingly attractive for medical segmentation tasks involving ambiguous boundaries or extensive anatomical regions [Shamshad et al., 2023]. Initial implementations frequently embedded transformer encoders within U-shaped architectures such as TransUNet and UNETR to strengthen contextual inference [Chen et al., 2021a; Hatamizadeh et al., 2022]. For high-resolution imagery and volumetric data, window-based hierarchical attention mechanisms have emerged as viable solutions, spawning Swinderived segmenters and subsequent refinements [Cao et al., 2021; He et al., 2023].

Complementing pure transformer models, extensive investigation targets CNN-Transformer amalgamation, merging convolutional spatial priors with transformer's global receptive field. Standard configurations integrate convolutional feature extractors with transformer contextualization modules, subsequently fusing multi-scale information via skip pathways or stage-wise interactions [Zhang et al., 2021; Xie et al., 2021]. Although these hybrids boost performance, their local-global integration remains implicit. With sparse annotations, regional features frequently overwhelm optimization, producing texture-biased or disconnected predictions, while global reasoning depends on intermediate transformer representations (attention correlations and positional embeddings) prone to divergence across stages or augmented views.

Collectively, transformer-based and hybrid segmenters deliver powerful global modeling capabilities, yet preserving alignment between fine-grained boundary evidence and holistic anatomical consistency proves difficult under annotation scarcity. This challenge drives the need for direct regularization of intermediate transformer signals (attention patterns and positional correspondence) alongside shape-sensitive local representation learning, moving beyond sole reliance on output-level supervision.

3 Method

Given a labeled dataset DL = {(xli , yl i )}Ni=1 and an unlabeled dataset DU = {xuj }Mj=1 with M ≫ N, our framework aims to exploit strong local perturbations while preserving global anatomical consistency for semi-supervised segmentation. As illustrated in Fig. 2, for each input image x (either labeled or unlabeled), we construct two patchwise jigsaw-rotated views {x(1) , x(2) } via local patch rotation. Both views are fed into a shared CNN encoder to extract multi-scale features, which are then decoded by two parallel Swin Transformer decoders to produce segmentation predictions for each view. For labeled data, we apply standard supervised segmentation loss on both decoder outputs; for unlabeled data, we perform cross-pseudo supervision between decoders across the two views. To explicitly couple local and global learning, we attach a patch-wise Rotation Head on deep encoder features to predict each patch rotation state, and we further enforce rotation-aligned attention consistency on self-attention relation matrices to regularize global relational reasoning. In addition, we introduce Positional Encoding Duplicating (PED) to derive multi-scale positional embeddings from a shared low-resolution anchor, providing a unified geometric reference across decoder stages and stabilizing cross-view alignment.

3.1 Dual-Decoder Transformer Segmentation Framework

Segmentation Supervision

In each mini-batch, we generate two patch-wise rotated views for all samples. For a labeled pair (xl , yl ) ∈ DL, we apply the same patch-wise rotation operator to the image and mask, obtaining two rotated pairs (xl,(1) , yl,(1) ) and (xl,(2) , yl,(2) ) with v ∈ {1, 2}. Pixel-wise supervision is computed only on the labeled subset.

Given a labeled view xl,(v) , the dual-decoder network produces segmentation logits Sl,(v) d ∈ RK×H×W for decoder d ∈ {1, 2}. We supervise each prediction using a composite loss combining cross-entropy and Dice:

The supervised loss is averaged over the two decoders and the two rotated views:

Cross-Pseudo Supervision

For unlabeled samples, we adopt a symmetric cross-pseudo supervision (CPS) strategy to exploit complementary predictions of the two decoders under two independently rotated views. Given an unlabeled image xu , we construct two patchwise rotated views {xu,(1) , xu,(2) }, and the dual-decoder network produces four segmentation logits Su,(v) d ∈ RK×H×W , where v ∈ {1, 2} indexes the view and d ∈ {1, 2} indexes the decoder.

Since each view is generated by local patch rotations, we map the logits back to a canonical orientation using the recorded rotation index map Ku,(v) :

We then obtain hard pseudo-labels by

We perform CPS in a symmetric cross-view and crossdecoder manner, yielding four equivalent supervision terms:

where stopgrad(·) blocks gradients through the pseudo-label branch. In implementation, Lseg uses the same Dice+CE form as the supervised case, with pseudo-labels treated as hard targets.

Finally, the CPS loss on unlabeled data is computed as the average of the four terms:

Patch-wise Rotation Prediction Loss

The rotation head is trained with direct supervision from the patch-wise rotations that we apply when constructing the two views; this loss is independent of cross-pseudo supervision. For each sample, we generate two rotated views x(1) and x(2) on a P × P grid, and record the corresponding rotation label maps K(1) ,K(2) ∈ {0, 1, 2, 3}P×P , where label 0 indicates no rotation and labels {1, 2, 3} correspond to

{90◦ , 180◦ , 270◦ }. Given view v ∈ {1, 2}, the rotation head predicts patch-level logits R(v) ∈ R4×P×P .

We supervise rotation prediction using patch-wise crossentropy:

and average the loss over the two views:

Attention Consistency Regularization

We regularize the transformer behavior by enforcing attention consistency between the two patch-wise rotated views X(1) and X(2) . This constraint is applied independently to each decoder d ∈ {1, 2} and to all collected W-MSA / SW-MSA blocks.

For a given decoder d and one Swin attention block, let A(v) d ∈ RT×T be the head-averaged attention scores for view v ∈ {1, 2}, where T is the token number. We normalize attention into a probability distribution with temperature-scaled softmax:

where τ is the temperature. For each query token t, the vector Π(v) d (t, :) ∈ RT represents how token t distributes its attention over all key tokens in the same block.

We measure the cross-view discrepancy for each query token by

We then select a subset of query tokens with the largest discrepancies:

where ρ is a fixed ratio (e.g., ρ = 0.1). The attention consistency loss for this block is defined as the mean squared error between the attention distributions of the selected query tokens:

We compute Lblk attn for all W-MSA and SW-MSA blocks across stages and average them to obtain the per-decoder loss L(d),attn. Finally, the overall attention regularization is the mean of the two decoders:

Position Encoding Duplicating

To avoid geometric drift caused by learning independent positional embeddings at different decoder scales, we propose Position Encoding Duplicating. We learn a single absolute positional embedding on the coarsest token grid (e.g., 8 × 8 with 768 channels), and generate all higher-resolution encodings (e.g., 16 × 16/32 × 32/64 × 64 with 384/192/96 channels) from this shared anchor. Concretely, the anchor embedding is reshaped to the 2D grid, its channel dimension is reduced by group-wise averaging to match the target scale, and it is then expanded to the target spatial resolution using nearestneighbor duplication. The resulting positional encoding is added to the corresponding scale tokens before Swin attention blocks. By reusing the same anchor across all scales and both rotated views, PED provides a unified coordinate reference throughout decoding and stabilizes cross-view alignment under strong patch-wise rotations.

3.2 Overall Loss andWeight Scheduling

The overall training objective jointly optimizes the supervised segmentation loss (Sec. 3.1.1), the unlabeled cross-pseudo supervision loss (Sec. 3.1.2), the patch-wise rotation prediction loss (Sec. 3.1.3), and the attention consistency regularization (Sec. 3.1.4). The total loss for a mini-batch is defined as

In our implementation, we set λrot = 0.5 and λattn = 3. The weight of the unlabeled loss, λu, is scheduled with a ramp-up strategy: it increases linearly from 0 to 1 as training proceeds, so that the model first learns from reliable labeled supervision and then gradually incorporates the unlabeled objective.

4 Experiments

4.1 Dataset

ACDC dataset. The ACDC dataset is a four-class cardiac MRI segmentation benchmark, including background, right ventricle, left ventricle, and myocardium. It contains scans from 100 patients. Following the commonly-used split protocol, the dataset is divided into 70, 10, and 20 patients for training, validation, and testing, respectively.

ISIC 2018 dataset. The ISIC 2018 dataset is a widely used benchmark for dermoscopic skin lesion segmentation, containing 3,694 RGB images. These images cover various types of skin lesions, with a focus on melanoma and non-melanoma lesions, and include pixel-level segmentation masks. The dataset is split into 2,594 training images, 100 validation images, and 1,000 test images. The original image sizes range from 540×722 to 4499×6748 pixels.

BraTS 2019 dataset. The BraTS 2019 dataset, released for the Brain Tumor Segmentation challenge, consists of preoperative multimodal MRI scans from 335 glioma patients, including T1, contrast-enhanced T1 (T1Gd), T2, and T2FLAIRmodalities. It includes 259 high-grade glioma (HGG) cases and 76 low-grade glioma (LGG) cases. Following [Xu et al., 2023], we adopt the T2-FLAIR modality and focus on whole tumor segmentation. The dataset is divided into 250

training samples, 25 validation samples, and 60 testing samples.

4.2 EvaluationMetrics

We evaluate segmentation performance using four widely adopted metrics: Dice score (%), Jaccard score (%), 95% Hausdorff distance (95HD), and average surface distance (ASD). For 2D datasets, these metrics are computed in pixel space, while for volumetric datasets they are computed in voxel space. Dice and Jaccard quantify the overlap between the predicted segmentation and the reference annotation, reflecting global region-level agreement. In contrast, 95HD and ASD emphasize boundary accuracy by measuring the discrepancy between the predicted and ground-truth contours or surfaces, where 95HD reports a robust maximum distance excluding extreme outliers and ASD computes the mean surface distance. Together, these metrics provide a comprehensive assessment of segmentation quality, capturing both overall overlap consistency and fine-grained boundary precision.

4.3 Implementation Details

We conduct all experiments on a single NVIDIA RTX 3060 GPU with fixed random seeds for reproducibility.

ACDC dataset. All slices are resized to 256 × 256 and the single-channel images are replicated to three channels to match the network input format. We adopt standard data augmentation to enlarge the effective training set, including random rotation, random flipping, and color jittering. In addition, for the proposed patch-wise rotation augmentation, each 256 × 256 slice is partitioned into an 8 × 8 grid of nonoverlapping patches, and local patch rotations are applied on this grid during training.

ISIC 2018 dataset. All dermoscopic images are resized to 256 × 256 to ensure consistency in data and standardized inputs across the dataset. The images are kept in their original RGB format to align with the network's input requirements. To enhance the training dataset, we apply typical data augmentation techniques, including random rotations, random horizontal/vertical flipping, and color jittering, which adjusts brightness, contrast, saturation, and hue. Additionally, for the patch-wise rotation augmentation, each 256 × 256 image is divided into an 8 × 8 grid of non-overlapping patches, and local rotations are applied to these patches during training.

BraTS 2019 dataset. The BraTS 2019 dataset includes preoperative MRI scans of glioma patients with modalities such as T1, T1Gd, T2, and T2-FLAIR. We use the T2-FLAIR images for tumor segmentation, resampled to an isotropic resolution of 1 × 1 × 1 mm³. Each image has a size of 155 × 240 × 240 pixels. During preprocessing, 2D slices of size 224×224 are extracted from the 3D volumes. The slices are then converted to single-channel images. For patch-wise rotation augmentation, each 224 × 224 slice is divided into a 7 × 7 grid of non-overlapping patches, with local rotations applied during training and rotation labels used for supervision.

4.4 Comparison with State-of-the-Art Methods

ACDC dataset. We compare LR-GCF on the ACDC dataset with a series of competitive semi-supervised baselines, including UA-MT [Yu et al., 2019], SASSNet [Li et al., 2020], DTC [Luo et al., 2023], and SS-Net [Wu et al., 2022b], as well as recent strong augmentation/self-training based methods such as BCP [Bai et al., 2023] and other competitors listed in Table 1. Following the commonly used protocol, we conduct semi-supervised experiments under two labeled ratios, i.e., 10% and 20%, while treating the remaining scans as unlabeled. For fair comparison, the results of competing methods are taken from their reported numbers under the same experimental settings when available. As shown in Table 1, LR-GCF achieves the highest Dice and Jaccard scores under both labeled settings, while maintaining competitive boundary-distance results in terms of 95HD and ASD. These results indicate that LR-GCF can produce accurate and anatomically plausible segmentations under limited supervision.

ISIC 2018 dataset. We further evaluate LR-GCF on the ISIC 2018 skin lesion segmentation benchmark, comparing it with representative semi-supervised methods listed in Table 2. Following the standard split, we use 10% and 20%

labeled images and treat the remaining training images as unlabeled. As shown in Table 2, LR-GCF achieves the highest Dice score under both settings and obtains the best overall results under the 20% labeled setting. Under the 10% setting, it also remains competitive on Jaccard and boundary metrics. These results indicate that LR-GCF generalizes well to dermoscopic image segmentation and effectively leverages unlabeled data under limited supervision.

BraTS 2019 dataset. As shown in Table 3, LR-GCF achieves the highest Dice and Jaccard scores on BraTS 2019 under both 10% and 20% labeled settings, demonstrating strong region-level segmentation performance in the scarcelabel regime. Compared with prior semi-supervised baselines, LR-GCF produces more complete and consistent tumor regions, while maintaining competitive boundary distances in terms of 95HD and ASD. The gains become more evident with 20% labeled data, suggesting that LR-GCF can effectively exploit unlabeled scans while benefiting from additional supervision.

4.5 Ablation Study

Table 4 presents the ablation study of LR-GCF's components on the ACDC dataset under 10% labeled supervision. The baseline model shows the weakest performance, while introducing PatchRot improves the results by augmenting the data with patch-wise rotations. Adding the RotHead further improves Dice, Jaccard, 95HD, and ASD, demonstrating the benefit of auxiliary rotation prediction for enhancing highlevel representation learning. The introduction of AttnCons brings additional gains, especially in Dice, Jaccard, and ASD, by regularizing attention consistency between the two decoders. Finally, the full LR-GCF model, which combines all components, achieves the best Dice, Jaccard, and 95HD scores, while maintaining competitive ASD. These results demonstrate the effectiveness of jointly combining patchwise rotations, rotation prediction, attention consistency, and positional encoding for improving segmentation performance under limited supervision.

Table 5 presents the ablation study on the token selection ratio used in attention consistency regularization on the ACDC dataset with 10% labeled data. Here, "top-k" denotes the percentage of query tokens with the largest attention discrepancy selected for the MSE-based consistency loss. Due to the lower bound k = max(1, ⌊ρT ⌋), ρ = 0 corresponds to selecting only the single most discrepant token. As shown, selecting the top 10% discrepant tokens achieves the best Dice, Jaccard, and 95HD scores, while the 50% setting obtains the lowest ASD. Considering the overall balance across metrics, we adopt the top 10% setting in LR-GCF. Selecting too few tokens or using a much larger proportion leads to less effective regularization and slightly degraded overall performance.

5 Visualization

Fig. 3 presents a qualitative comparison on the ACDC dataset against several competitive baselines, such as MT, CPS, DTC, and BCP. LR-GCF consistently yields masks that exhibit superior morphological fidelity and boundary precision. This advantage is particularly evident in challenging slices where other methods produce over-segmented regions or disconnected structures due to low contrast. By contrast, our model preserves the inherent anatomical topology. Fig. 4 further illustrates the impact of different modules on the ISIC dataset. The baseline is prone to fragmented predictions and spurious blobs around weak boundaries. While PatchRot encourages more stable local cues, the addition of RotHead and AttnCons progressively refines the results, suppressing noise and enhancing structural integrity. In aggregate, LR-GCF produces the most coherent segmentations that closely align with the ground truth.

6 Conclusion

This work presents LR-GCF, a robust semi-supervised framework for medical image segmentation that integrates local patch-wise rotations with global consistency regularization. By combining rotation prediction, rotation-aligned attention consistency, and the proposed PED scheme, LR-GCF improves both local feature learning and global anatomical coherence. Extensive experiments on ACDC, ISIC 2018, and BraTS 2019 demonstrate its superiority over strong baselines, especially under limited annotation settings.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 62302378 in part by the Innovation and Entrepreneurship Talent Project of Qin Chuang Yuan under Grant QCYRCXM2023-115 and Talent Scientific Fund of Lanzhou University.