GeoSFLoRA: Geometry-Conditioned Spectral Flow Low-Rank Adaptation for 2D-to-3D Transfer in Medical Image Segmentation

1 Introduction

The advent of large scale Vision Foundation Models (VFMs), such as DINOv3 [Sim´eoni et al., 2025] and the SAM series [Mazurowski et al., 2023; Li et al., 2025], has fundamentally reshaped visual representation learning. Pretrained on massive collections of 2D natural images, these models acquire universal semantic priors that exhibit strong transferability across a wide range of downstream tasks [Awais et al., 2025; Jiao et al., 2025]. Despite this success, extending such models to three-dimensional (3D) medical image segmentation remains challenging. The core difficulty lies in the structural divergence between domains. Natural images are dominated by surface appearance, texture, and color distributions, whereas 3D medical volumes (for example, CT or MRI) are characterized by inter-slice continuity, cross-sectional correlations, and highly anisotropic geometric structures [Litjens et al., 2017; Wu et al., 2025a; Li et al., 2024].

This dimensional mismatch induces a pronounced representation shift that straightforward adaptation strategies struggle to reconcile. Naive approaches, including slice-byslice 2D inference or the insertion of generic 3D convolutional or inflated adapters, often disrupt volumetric coherence and lead to inter-slice inconsistencies, manifesting as partial-volume artifacts [Wang et al., 2024; Xu et al., 2023; Hung et al., 2024]. Even recent domain-specific adaptations, such as Medical SAM Adapter [Wu et al., 2025b] or 3D SAM Adapter [Gong et al., 2023], face difficulties in fully resolving the semantic gap without incurring significant computational costs or compromising the pretrained manifold structure [VanBerlo et al., 2024].

To alleviate the cost of full retraining [Wang et al., 2025], Parameter-Efficient Fine-Tuning (PEFT) has emerged as a compelling alternative. By introducing low-rank updates to frozen backbones, methods such as LoRA [Hu et al., 2022], DoRA [Liu et al., 2024], GeLoRA [Ed-dib et al., 2024], and their spectral variants attempt to adapt pretrained priors with minimal parameter overhead. However, a critical limitation arises when applying these paradigms to volumetric data. Existing PEFT methods, including recent spectral approaches [Zhang and Pilanci, 2024], typically rely on global or static spectral reweighting, implicitly assuming that feature importance is spatially uniform. This assumption breaks down in 3D anatomy, where local anisotropy varies significantly across voxels. As a result, standard PEFT approaches fail to dynamically redistribute representational energy according to local geometry. Consequently, 2D texture semantics are preserved, while 3D structural awareness remains insufficient. These observations suggest that effective 2D-to3D transfer requires more than parameter adaptation alone. The model must dynamically redistribute spectral energy according to local volumetric geometry.

To address this challenge, we propose GeometryConditioned Spectral Flow Low-Rank Adaptation (GeoSFLoRA), a parameter-efficient framework that injects volumetric priors directly into the spectral subspace of a frozen 2D encoder. GeoSFLoRA consists of two tightly coupled components. First, a Geometry-Conditioned Encoder constructs patch-level descriptors from raw volumetric data to capture local anisotropy and boundary sharpness. Second, a Spectral Flow Low-Rank Adapter leverages these descriptors as control signals to predict token-specific singular-value residuals. In practice, for each adapted linear projection, we compute a truncated singular value decomposition once and keep the left and right singular bases fixed. Geometry-conditioned low-rank residuals are learned and added directly to the singular values, with the truncation rank controlling the adaptation capacity. This design enables position-specific energy redistribution within the pretrained spectral subspace, selectively enhancing or suppressing spectral components in accordance with local anatomical geometry while preserving the integrity of the learned semantic basis.

We empirically evaluated GeoSFLoRA on widely used benchmarks, including BraTS20, MSD-Prostate, and MSDLung. We report mean performance and standard deviation across cross-validation folds and compare segmentation accuracy together with parameter counts and inference cost. Experimental results demonstrate that GeoSFLoRA consistently outperforms representative PEFT baselines and achieves performance comparable to full fine-tuning, while requiring only a fraction of the trainable parameters. These results indicate that geometry-conditioned spectral adaptation offers an effective and efficient pathway for migrating 2D foundation models to volumetric medical imaging tasks.

Our main contributions are summarized as follows:

• We propose GeoSFLoRA, a parameter-efficient framework that explicitly injects local volumetric geometric information into the spectral subspace of frozen 2D pretrained encoders.
• We design a Geometry-Conditioned Encoder that constructs stable patch-level descriptors using normalized gradient statistics and structure-tensor-inspired eigenspectrum analysis, capturing local anisotropy and directional sharpness to provide modality-agnostic geometric priors.
• We introduce a Spectral Flow Low-Rank Adapter that performs token-specific singular-value modulation based on geometric cues, enabling principled 2D-to3D transfer while preserving the pretrained orthogonal bases of the encoder.

2 RelatedWork

2.1 Volumetric Medical Image Segmentation

Early volumetric methods established that 3D convolutional encoder-decoder architectures, such as V-Net [Milletari et al., 2016] and U-Net [Ronneberger et al., 2015], effectively capture local details. System-level frameworks like nnU-Net [Isensee et al., 2021] further optimized these priors. Recent Transformer-based backbones, including TransUNet [Chen et al., 2021] and Swin-UNETR [Hatamizadeh et al., 2022], have pushed the boundaries of long-range context modeling, albeit at the cost of high computational and memory overhead. To mitigate these costs, 2.5D methods employ cross-slice attention [Hung et al., 2024] or bidirectional propagation [Xu et al., 2023]. While these approaches alleviate memory constraints, they often struggle to preserve global 3D geometric consistency, particularly under highly anisotropic sampling. More recently, adapting 2D foundation models such as SAM to volumetric medical data has emerged as a promising direction (e.g., 3DSAM-Adapter [Gong et al., 2023], Medical SAM Adapter [Wu et al., 2025b]). These methods leverage strong 2D semantic priors but typically rely on spatial or channel-wise prompting mechanisms that do not explicitly model volumetric geometry. In contrast, GeoSFLoRA addresses the 2D-to-3D gap at the level of representation geometry by enabling location-specific adaptation driven by local volumetric structure, ensuring anatomical coherence without the burden of full 3D convolutions.

2.2 Parameter-Efficient Fine-Tuning

Parameter-Efficient Fine-Tuning (PEFT) methods adapt large pretrained models by injecting a small number of trainable parameters while freezing the backbone. Representative approaches include LoRA [Hu et al., 2022], DoRA [Liu et al., 2024], and AdaLoRA [Zhang et al., 2023], which introduce low-rank updates to linear projections to reduce optimization cost. More recent work explores adaptation in the spectral domain. Spectral adapters manipulate singular values or truncated SVD representations of pretrained weights, offering a principled way to reshape pretrained subspaces without altering their orthonormal semantic bases [Zhang and Pilanci, 2024]. Geometry-aware variants such as GeLoRA [Ed-dib et al., 2024] further attempt to allocate adaptation capacity based on representation geometry. However, most existing PEFT methods apply global or layer-level adaptations that are shared across all tokens, implicitly assuming spatial homogeneity. This assumption is ill-suited to volumetric medical images, where anatomical geometry and anisotropy vary substantially across space. GeoSFLoRA departs from prior PEFT designs by enabling token-specific spectral modulation conditioned on local volumetric geometry. This allows representational energy to be redistributed within the pretrained subspace in a spatially adaptive and modality-agnostic manner, effectively tailoring the 2D foundation model to the local 3D structural context.

3 Methodology

3.1 Overview

GeoSFLoRA adapts 2D pretrained Vision Transformers (ViTs) to 3D medical segmentation with minimal additional parameters. The overall architecture consists of a frozen 2D-pretrained ViT encoder, GeoSFLoRA adapters inserted into all linear layers, and a lightweight 3D UNet-style decoder. We selected the powerful DINOv3 [Sim´eoni et al., 2025] as the pretrained backbone. To process volumetric data, the 2D patch embedding is converted into a 3D patch embedding by inflating the 2D convolutional kernels along the depth dimension. The inflated weights are initialized from the original 2D pretrained parameters, preserving semantic priors while enabling 3D tokenization. All transformer blocks, including self-attention layers and MLP layers, remain unchanged. This design allows direct loading of 2D pretrained weights. GeoSFLoRA introduces two modules: (1) a Geometry-Conditioned Encoder (GCE) that extracts perpatch descriptors of local anisotropy and directional sharpness from volumetric gradients; (2) a Spectral Flow LowRank Adapter (SFLoRA) that modulates pretrained singular values within each linear layer based on these descriptors. A lightweight WeightedFusionAdapter is further employed to fuse the geometry-conditioned residuals with backbone token representations through learnable scalar gating before decoder aggregation. During the forward pass, the backbone outputs are corrected by token-wise geometry-aware spectral adjustments, while all original weights remain frozen. This design preserves pretrained semantics and enables geometrysensitive 3D adaptation with minimal additional parameters.

3.2 Geometry-Conditioned Encoder (GCE)

While 2D-pretrained ViT encoders capture rich semantic priors, they lack explicit knowledge of volumetric geometry. Effective 3D segmentation requires distinguishing linelike, surface-like, and isotropic structures. To this end, the Geometry-Conditioned Encoder (GCE) extracts compact patch-level descriptors encoding local anisotropy, directional sharpness, and intensity statistics. We prioritize compact, interpretable descriptors in GCE because they are robust across modalities and sampling rates, inexpensive to compute, and provide explicit geometric signals that directly guide tokenlevel spectral modulation.

Given a volumetric input x ∈ RB×C×D×H×W , we partition it into non-overlapping patches of size (pd, ph, pw), producing Np patches with V = pdphpw voxels each. Each patch is flattened into a vector form xp ∈ RV . To capture local directional variations, we compute normalized centraldifference gradients along depth, height, and width:

where ϵ = 10-6 ensures numerical stability. Normalizing the gradients removes scale variations, emphasizing geometric patterns rather than absolute intensity magnitudes.

To distill local geometric context, we construct the patchwise Structure Tensor using the gradient vectors. Let J = [ ˜Gd, ˜Gh, ˜Gw] ∈ RV ×3 denote the gradient matrix for a patch. The structural information is encoded in the covariance matrix S = J⊤J ∈ R3×3 . We perform an eigenspectrum analysis on S, extracting the eigenvalues λ0 ≥ λ1 ≥ λ2. These eigenvalues characterize the dominant spatial modes independent of intensity shift. We then define two scale-invariant geometric descriptors:

The motivation for employing these specific ratios is twofold. First, they provide a continuous mapping from raw voxels to classical anatomical primitives: a high r1 identifies linelike structures (e.g., vascular segments); a high r2 and low r1 captures planar structures (e.g., organ membranes); while low values for both indicate isotropic regions (e.g., homogeneous parenchyma). Second, the logarithmic transformation ensures numerical stability across heterogeneous imaging modalities. Finally, we concatenate these geometric priors with the intensity statistics (mean µ and standard deviation σstd of the patch intensity xp) to form the 4D descriptor:

Sample-wise normalization produces the geometry prior:

To align with the ViT token sequence, the mean descriptor across all patches is assigned to the CLS token, ensuring a one-to-one correspondence between ξ and the full token sequence used in SFLoRA.

3.3 Spectral FlowAdaptation

The geometry descriptors produced by the GCE provide token-wise volumetric context absent in 2D-pretrained ViTs. To effectively incorporate these descriptors without disrupting pretrained semantics or incurring the computational overhead of full fine-tuning, we introduce Spectral Flow LoRA (SFLoRA). SFLoRA adapts each linear layer by modulating its frozen spectral components rather than updating parameters in the ambient weight space. This design injects 3D geometric sensitivity directly into the pretrained singular directions while ensuring that feature representations remain anchored to the robust 2D semantic manifold.

For a frozen linear layer W0 ∈ Rdout×din and a token vector x(i) ∈ Rdin from the input sequence X ∈ RN×din , the adapted output is formulated as:

where ∆sub represents the geometry-conditioned subspace correction computed per token using the geometry prior ξ from GCE.

Truncated SVD Decomposition

Prior to adaptation, we factorize each frozen linear layer via truncated SVD:

where Uin ∈ Rdin×r , Uout ∈ Rdout×r , and σ0 ∈ Rr denote the frozen input basis, output basis, and singular values, respectively. The truncation rank r acts as a bottleneck for adaptation capacity. Freezing Uin and Uout prevents arbitrary rotation of the semantic space, preserving the orthogonal feature axes learned from large-scale 2D pretraining. Consequently, adaptation is constrained to reweighting and mixing these existing axes, which empirically minimizes semantic drift.

Geometry-Conditioned Singular-Value Modulation

While σ0 defines the global importance of semantic directions, the optimal spectral gain varies locally depending on anatomical geometry. Regions with different geometric anisotropy often require different spectral emphases. For example, boundary-dominant structures benefit from highfrequency directional components, whereas homogeneous regions rely more on low-frequency semantic priors. SFLoRA predicts token-wise bounded residuals for the singular values:

The MLP fσ maps the joint embedding of the static spectrum σ0 and dynamic geometry ξ (i) to a residual vector. This achieves diagonal spectral modulation, selectively amplifying or suppressing specific semantic bases based on local structure. The usage of tanh with a global scaler α ensures the spectral updates remain bounded and perturbative, preventing the adapter from overriding the pretrained knowledge base.

Conditional Residual Mixing in Subspace

Diagonal modulation scales existing bases but cannot model interactions between them. To capture complex geometric adaptations, we introduce a dense residual mixing matrix:

Here, fR predicts a small, geometry-dependent mixing term. This allows for fine-grained cross-channel communication within the spectral subspace. We include residual mixing to capture off-diagonal interactions that scalar reweighting cannot express. By predicting residuals within the low-rank subspace r ≪ min(din, dout), we enable directional specificity with minimal parameter overhead.

Efficient Spectral Flow Computation

The diagonal modulation and residual mixing jointly define the token-specific spectral residual kernel:

To maintain high computational and memory efficiency, we avoid explicitly instantiating the full spectral kernel tensor S ∈ RN×r×r . Instead, we implement a streamlined pipeline to derive the subspace correction for each token in a batchwise manner. First, the input tokens are projected into the low-rank spectral subspace via U ⊤ in . Second, the geometryconditioned spectral adjustments are applied by multiplying the kernel S(i) (ξ) with the projected vectors to selectively modulate the spectral energy based on local anatomical priors. Finally, these modulated latent representations are backprojected into the original feature space via Uout to yield the final subspace correction. This sequential process is mathematically summarized as:

This implementation strategy effectively reduces the computational complexity from O(Ndr2 ) to O(Ndr+Nr2 ), where d = max(din, dout). Since the truncation rank r is significantly smaller than the embedding dimension d (i.e., r ≪ d), GeoSFLoRA maintains a memory footprint and inference latency comparable to standard LoRA while providing significantly higher representational flexibility, particularly for high-resolution volumetric medical data.

3.4 Loss Function

All adapter parameters and the 3D decoder are optimized endto-end while the ViT backbone remains frozen. The segmentation objective is a balanced combination of Dice and focal losses:

with λDice = λFocal = 0.5 by default.

4 Experiments

4.1 Datasets and Evaluation Protocol

To comprehensively validate the cross-domain generalization and geometry-conditioned adaptation of GeoSFLoRA, we evaluate our model on three diverse 3D medical segmentation benchmarks: BraTS20, which contains 369 multimodal MRI scans (T1, T1ce, T2, FLAIR) for enhancing tumor (ET), tumor core (TC), and whole tumor (WT) segmentation; MSD-Prostate, consisting of 48 dual-modal MRI volumes (T2, ADC) resampled to 1mm3 isotropic spacing for peripheral zone (PZ) and transition zone (TZ) segmentation; MSD-Lung, featuring 64 thoracic CT scans for lung tumor segmentation. A unified preprocessing pipeline is applied across all datasets, including intensity clipping to the 0.5th-99.5th percentiles, z-score normalization, and random cropping to target input dimensions: [4, 128, 128, 128] for BraTS20, [2, 20, 320, 256] for Prostate, and [1, 80, 192, 160] for Lung. To mitigate overfitting, we employ standard data augmentations including random flipping across three axes (p = 0.25), random rotations (±15◦ ), and intensity scaling (0.8-1.2×). All datasets use patient-level 5-fold crossvalidation, and Dice and HD95 are reported as mean ± standard deviation across folds.

4.2 Implementation Details

We implemented the proposed SFLoRA in PyTorch and conducted training on 4 NVIDIA RTX 4090 GPUs (24GB each). The Adam optimizer was adopted with an initial learning rate of 1×10-4 , complemented by a cosine annealing scheduling strategy [Loshchilov and Hutter, 2016] and a weight decay of 1 × 10-4 . The ViT encoder operates with a patch size of 4 × 16 × 16. The model was trained for 1000 epochs with a batch size of 8, and automatic mixed precision (AMP) [Lam et al., 2013] was employed to reduce memory consumption and accelerate training.

4.3 Comparison with State-of-the-Art Methods

We evaluate GeoSFLoRA against four categories: (1) 3D architectures (nnU-Net [Isensee et al., 2021], UNETR++ [Shaker et al., 2024], SegMamba-V2 [Xing et al., 2025]); (2) a 2D slice-by-slice variant of DinoU-Net [Gao et al., 2025] with a DINOv3-Large backbone (r = 32); (3) foundation adapters (MedSAM2 [Ma et al., 2025]); and (4) PEFT baselines (LoRA [Hu et al., 2022], DoRA [Liu et al., 2024], Spectral Adapters [Zhang and Pilanci, 2024]). As shown in Table 1, GeoSFLoRA achieves the best HD95 among PEFT baselines on MSD-Lung (14.01 mm) and the best overall HD95 on Prostate TZ (9.44 mm). On BraTS20ET, GeoSFLoRA improves the Dice score by ∼2.0% and reduces the HD95 error by over 6 mm compared to LoRA. These results validate that GeoSFLoRA efficiently bridges the 2D-to-3D representational gap while ensuring structural integrity across diverse volumetric modalities. Visual comparisons in Fig. 2 mirror the quantitative gains. GeoSFLoRA produces sharper boundaries and fewer artifacts in small prostate zones and complex lung tumor edges.

4.4 Component-wise Contribution of GeoSFLoRA

We evaluate component contributions on three benchmarks using frozen DINOv3-large [Sim´eoni et al., 2025]. Starting from baseline spectral modulation (SVD+Flow, r = 32), we progressively integrate the conditional residual matrix (Rcond) and GCE geometric priors, comparing against standard LoRA [Hu et al., 2022] (Fig. 3).

Although LoRA offers competitive benchmark solutions, its uniform weight update mechanism struggles to adapt to 3D medical images, resulting in suboptimal boundary accuracy. In contrast, the vanilla SVD+Flow lacks cross-dimensional awareness, but adding Rcond enables geometry-conditioned subspace mixing, improving Dice by 1.27% on MSD-Lung. The full GeoSFLoRA yields the most significant leap; by using GCE descriptors to guide local 3D adaptation based on structural sharpness, it effectively bridges the 2D-to-3D representational gap.

4.5 Ablation on the Truncated SVDRank r

We investigate the effect of the truncated SVD rank r on GeoSFLoRA's representational capacity and segmentation performance, as summarized in Table 2. Experiments are conducted on BraTS20, MSD-Prostate, and MSD-Lung datasets using a frozen DINOv3-Large encoder. As r increases from 4 to 32, the Dice coefficient consistently improves across all datasets, while the HD95 decreases, indicating more precise boundary delineation. Mechanistically, the SVD rank r defines the degrees of freedom available for spectral flow modulation. Lower ranks constrain the adaptation of pretrained singular vectors to token-specific geometric priors ξ, limiting the model's ability to capture complex local structures. Increasing r expands the spectral subspace, allowing richer geometry-aware modulation and enhanced crossdimensional interactions, thereby bridging the 2D-to-3D domain gap more effectively.

4.6 Visualizing Geometric Guidance

Fig. 4 and Fig. 5 illustrate how geometric descriptors guide the 2D ViT toward 3D structural awareness. While Frozen ViT and LoRA exhibit diffuse or coarse attention, incorporating descriptors (r1, r2) progressively localizes attention maps onto target boundaries, while the summary plot in Fig. 5 shows consistent improvements in average Dice and HD95 across benchmarks. Overall, the attention evolution in Fig. 4 is consistent with the quantitative trends shown in Fig. 5, demonstrating that geometric descriptors effectively redirect pretrained 2D semantic directions toward volumetric 3D anatomy.

4.7 Layer Insertion Strategy

Table 3 evaluates the placement of GeoSFLoRA adapters within the ViT encoder. Frozen weights alone yield the lowest performance, while the joint MLP+QKV configuration consistently achieves superior results.

4.8 Sensitivity of Modulation Strength α

We evaluate the modulation strength α ∈ {0.4, 0.6, 0.8, 1.0} across three benchmarks (Fig. 6). Results show that α = 0.8 consistently yields the highest and most stable Dice scores, providing an optimal balance between geometric adaptation and training stability.

Lower values (α = 0.4) lead to under-adaptation and slow convergence, while excessive modulation (α = 1.0) causes performance oscillations, particularly on BraTS20, by over-perturbing pretrained singular values. Consequently, we adopt α = 0.8 as the default setting to ensure robust 3D adaptation without destabilizing the frozen backbone.

4.9 Impact of Backbone Capacity

Across DINOv3-Small, Base, and Large variants, validation performance improves with model capacity in our setting; therefore, DINOv3-Large is used as the default backbone in the main experiments.

4.10 Efficiency Analysis

Table 4 shows that GeoSFLoRA matches or slightly reduces training time compared with LoRA while incurring only marginal inference overhead, indicating a favorable balance between cost and accuracy.

5 Conclusion

GeoSFLoRA bridges the 2D-to-3D domain gap via geometry-conditioned spectral modulation in frozen ViTs. By leveraging voxel-wise anisotropy and structural sharpness, it improves volumetric consistency with minimal overhead and outperforms standard PEFT methods across three benchmarks. Future work will explore modality-aware geometric conditioning for varied MRI and CT profiles.

Ethical Statement

There are no ethical issues.

Acknowledgements

This work was supported in part by the Tianshan Talent Training Program 2023TSYCLJ0023. This work was supported by the university-level computing platform of Xinjiang University's Computing and Data Center.

Contribution Statement

Qin Hao and Bonian Chen contributed equally to this work. Shengwei Tian and Long Yu are the corresponding authors. Qin Hao and Bonian Chen developed the method, conducted the experiments, and drafted the manuscript. Shengwei Tian and Long Yu supervised the study, revised the manuscript, and provided project support.