A Medical Image Classification Network Based on Multi-View Consistent Momentum Contrastive Learning

1 Introduction

In natural image processing and medical image analysis, color sensitivity is a common and challenging issue. For instance, in tasks such as image classification [Krizhevsky et al., 2012], object detection [Girshick et al., 2014], and medical image analysis [Shen et al., 2017], variations in color can significantly impact model performance, especially under conditions of lighting changes, color distortions, or the presence of noise. Additionally, in medical image analysis, color variations are often less critical compared to texture and shape, with the structure and boundaries of lesions serving as key diagnostic indicators [Litjens et al., 2017]. Moreover, medical imaging data may originate from different devices, leading to color distortions and inter-device discrepancies, which makes color robustness particularly important.

∗Corresponding Author.

To address this, researchers have proposed various methods to enhance model robustness against color variations, such as color histograms [Rabie et al., 2024], color jittering [Cubuk et al., 2019], and Color-Aware Convolutions in deep learning [Yao et al., 2025]. However, these methods often result in the loss of some image details or introduce noise due to inaccurate color representation, while also increasing computational overhead.

Another common issue is edge blur, where the details of object boundaries in images become unclear or distorted, making the visual quality of the image appear blurry and less sharp [Simonyan, 2014]. Edge information is crucial for accurately understanding and processing images. This is particularly true in tasks like image classification, object detection, and medical image analysis [Tianhao et al., 2024], where edge information typically contains essential details about object shapes, structures, and spatial relationships. Consequently, edge blur can lead to errors in the model's understanding of the image, thereby affecting task accuracy. Addressing edge blur is therefore critical for enhancing model performance. Current solutions include deep deblurring networks (DeepDeblur) [Kupyn et al., 2018], CycleGAN [Zhu et al., 2017], and super-resolution reconstruction [Dong et al., 2014]. Although these methods are effective, they generally require substantial computational resources and labeled data, and there is limited research on their application in selfsupervised learning.

To tackle the issues of color sensitivity and edge blur in images, this paper proposes a contrastive model with an adaptive contrastive loss under three-view consistency constraints, incorporating the Sobel operator. This model demonstrates significant advantages within a self-supervised learning framework. Firstly, the Sobel operator extracts edge information from images and integrates it into the deep network, enhancing the representation of image edges, improving the model's sensitivity to detailed features, and increasing color robustness. Compared to existing methods, our approach not only addresses color robustness but also mitigates edge blur. Secondly, we introduce a three-view consistency momentum contrastive loss function, which enhances the consistency of representations across different views through consistency constraints, thereby improving the model's robustness to images from various angles. Momentum contrast effectively utilizes non-synchronous updated data through its unique dynamic dictionary and encoder update strategy, enhancing the learning capacity on large-scale datasets. Considering the common challenges in contrastive learning, such as loss oscillation or slow convergence, we propose a Dynamic Contrastive Loss. This loss function dynamically adjusts with the training cycles, effectively reducing oscillations during training and accelerating model convergence. The contributions of this paper are summarized as follows:

• .We propose a three-view consistency momentum contrastive loss function that addresses color sensitivity, enhancing the model's robustness and generalization capabilities.
• We integrate the Sobel operator into the ResNet main model framework to resolve edge blur issues.
• We construct a three-view framework structure and employ momentum contrast to update view features, enriching feature extraction while enhancing stability.
• We develop a Dynamic Contrastive Loss for the training process, accelerating convergence speed and mitigating model oscillations.

2 RelatedWorks

Edge blur and color sensitivity are key challenges in image processing and medical image analysis, significantly affecting model performance. Recent advances in edge extraction, color robustness, and loss function optimization have greatly improved model accuracy and stability. Techniques like dynamic loss, consistency loss, and momentum contrast have further enhanced training efficiency and generalization. This review highlights research on these challenges and methods.

In recent years, significant advancements have been made in the fields of image processing and medical imaging to effectively address the challenges of edge blur. These improvements have been achieved by enhancing edge extraction capabilities and ensuring clustering consistency, thus improving task performance. For example, CDANet [Yang et al., 2024] adopts a dual-branch architecture that focuses on both regional and edge features, effectively enhancing edge extraction in building semantic segmentation. In the field of cancer imaging, a deep learning-based edge detection algorithm for cancer images [Li et al., 2020] has successfully achieved high-precision edge detection through three-dimensional reconstruction and fine-grained feature segmentation. To further improve the accuracy of edge information, researchers have used tumor edge data as pseudo-labels for the finegrained BI-RADS classification of breast ultrasound images [Xu et al., 2024]. These studies demonstrate the critical role of edge enhancement and self-supervised learning strategies in addressing edge blur issues, particularly in enhancing the precision and robustness of image processing.

To address the challenges of color sensitivity, various innovative solutions have also been proposed in the fields of color processing and normalization in recent years. For example, the n-color balancing method [Akazawa et al., 2021] was introduced to correct all colors. In the realm of color naming computation, the ColorMLP approach [Yan et al., 2022] combines the RGB color model with Graph Attention Networks

(GATs), learning universal color mappings and employing Partial Color Jittering (PCJ) data augmentation. In the analysis of histopathological images, the Color-Adaptive Generative Adversarial Network (CAGAN) [Cong et al., 2022] was proposed for stain normalization. These methods exhibit strong application potential when dealing with complex data and diverse tasks, particularly in the areas of color correction and normalization, effectively mitigating challenges related to color sensitivity.

Dynamic loss functions are widely used in deep learning to adaptively adjust loss values during training, accelerating convergence and reducing oscillations. Early in training, larger learning rates and smaller loss weights expedite learning, while gradually lowering the learning rate later stabilizes training and avoids local minima or overfitting. For instance, Dynamic Loss Threshold (DLT) [Yang et al., 2023] discards potentially incorrect labels by comparing sample losses with dynamic thresholds, greatly improving performance on noisy-label datasets. FarSeg++ [Zheng et al., 2020] progressively focuses on hard samples while reducing easy-sample gradients, balancing foreground and background segmentation. Dynamic loss also enhances backdoor attacks in image compression [Yu et al., 2023b] by adaptively balancing loss terms, illustrating its broad applicability in handling noisy labels, balancing sample difficulty, and bolstering robustness.

In recent years, numerous scholars have explored consistency methods[Jiang et al., 2023], leading to the emergence of many innovative approaches. For instance, a deformable registration method based on paired cyclic consistent neural representations [Van Harten et al., 2023] has improved accuracy and provided reliable uncertainty measurements. In semisupervised medical image segmentation, ASE-Net [Lei et al., 2022] utilizes dynamic convolution and consistency training to better align labeled and unlabeled data, thus enhancing prediction quality and reducing overfitting. Additionally, the Fuzzy Consensus Mean Teacher (AC-MT) model [Xu et al., 2023] integrates fuzzy target selection to strengthen consistency learning in information-rich areas, improving segmentation outcomes. Path consistency [Lu et al., 2024] enhances object matching in self-supervised learning through multiple observation paths. Together, these methods highlight the pivotal role of consistency learning in boosting model robustness and performance.

Momentum Contrast (MoCo) [He et al., 2020] has shown considerable promise across various deep learning tasks, particularly in enhancing model stability, accelerating convergence, and improving generalization. The HMMC framework [Shen et al., 2023] further strengthens representation generalization in text-video retrieval by integrating hierarchical matching with momentum contrast and increasing negative samples. Similarly, the USER [Zhang et al., 2024] approach leverages unified semantic augmentation with momentum contrast to boost image-text retrieval. In medical imaging, momentum contrast learning combined with prototype networks and few-shot learning significantly elevates diagnostic accuracy for COVID-19 [Chen et al., 2021a], while the MoMA method [Le Vuong and Kwak, 2024] employs knowledge distillation to enhance histopathological analysis. The CLEAN algorithm[Yu et al., 2023a] utilizes a contrastive learning framework to assign enzyme commission (EC) numbers to enzymes.

3 Method

We first present the overall framework of the model, followed by a detailed introduction of the main innovative components within the model, including the Sobel operator, the threeview consistency momentum contrast loss function, and the Dynamic Contrastive Loss. The specific components of the model are illustrated in Figure 1.

3.1 Sobel Operator

The Sobel operator is an edge detection method that can extract the edge information of an input image while ignoring color features, thereby preventing the model from overrelying on color. This approach helps the model focus more on the structural information within the image rather than simple color features.

The Sobel operator calculates the horizontal and vertical gradients of an image using two convolution kernels, one in the horizontal direction (Gx) and the other in the vertical direction (Gy). The output of the Sobel layer is a combination of the horizontal and vertical gradients. Equations (1) and (2) respectively compute the magnitude and direction of the gradients.

Direction: Represents the orientation of the edge and is calculated as the arctangent of the gradient.

In DeepCluster, Sobel filters were employed to preprocess the input in order to prevent the model from merely relying on color information for clustering, instead encouraging the use of more meaningful features such as edges and shapes. Building on this approach, we incorporate Sobel layers into the ResNet architecture to enhance its focus on structural information within pathology images. This integration helps alleviate color sensitivity issues in pathology image classification, thereby improving the model's generalization capability. Additionally, ResNetSobel supports the use of deep stem layers, which are analogous to the deep stem layers in the standard ResNet. Depending on the model configuration, the input may first pass through the stem layer or directly utilize the initial convolution, batch normalization, and ReLU activation.

3.2 Three-viewConsistencyMomentum Contrastive Loss

Consistency loss functions are widely applied in semisupervised and self-supervised learning. Inspired by the infoNCE consistency loss function, we extend it to a three-view framework by employing three different data augmentations (such as cropping, rotation, and color jittering) for contrastive learning of samples. The goal is to encourage the model to maintain consistent prediction results for the same input data under different views or data transformations. This is achieved by minimizing the differences in the model's outputs for identical samples under varying conditions, thereby enhancing the model's robustness and generalization capabilities. We refer to this as the three-view consistency contrastive loss function, as detailed in Equations (3-7).

In which t is a temperature hyperparameter used to control the scale of similarity.

We construct three distinct viewsf(v1), f(v2), f(v3) and perform cross-contrastive comparisons among them. Specifically, Lv1, v2 measures the similarity between view1 and view2, while Lv2, v3 and Lv3, v1 are computed in a similar manner. The contrastive loss encourages these similarities to be as close as possible, thereby achieving feature alignment by minimizing the distances between them. Finally, the three loss components are weighted and summed to obtain the final three-view consistency contrastive loss function.

We introduce a momentum contrast update mechanism to update the second views (f(v1), f(v2), f(v3)), utilizing momentum updates to smooth the updating process and prevent drastic fluctuations in model parameters, as shown in Equation (7).

Specifically, i = 1, 2, 3, the default value β = 0.99.

In self-supervised learning, a momentum encoder is a technique used to enhance training stability and performance by maintaining smooth updates of the model's historical parameters. A higher momentum coefficient indicates slower updates, resulting in greater retention of historical information. The default value is set to β = 0.99, which means that the encoder parameters are updated very minimally, allowing historical information to dominate. Momentum updates enhance stability during the optimization process by leveraging information from previous updates to adjust the current parameters. This enables the parameters to be "accelerated" along the gradient direction to some extent, thereby facilitating faster model convergence. The three-view consistency contrastive loss function, integrated with momentum updates, can be expressed as Equation (8):

Summing these terms results in the three-view consistency momentum contrast loss function, as shown in Equation (9).

The hyperparameter α = 0.1. As shown in Equation (11), the variation of λi(t) is determined by the loss. When the loss value is high, indicating a large error concerning the true value, we dynamically adjust it to reduce the loss.

Similarly, after incorporating it into the three-view consistency contrastive loss function, as shown in Equation (12):

In the subsequent experimental section, we will separately employ the two different loss functions, Ld total and Ld total mc, to conduct experimental validations.

4 Experiments

4.1 Datasets

To effectively validate the generalization and robustness of our model, we conducted comprehensive evaluations using three publicly available pathology image datasets: the multiscale BreaKHis dataset [Spanhol et al., 2015] and the largescale pathology dataset NCT-CRC-HE-100K [Kather et al., 2018]. Additionally, we utilized the natural image dataset Food-101 [Bossard et al., 2014] to assess the model's performance across different domains and tasks. By employing these diverse datasets, we are able to thoroughly evaluate the model's adaptability and stability, ensuring its wide applicability in real-world applications.

4.2 Parameter setting

To ensure a more accurate and unbiased evaluation of the model's performance, we standardized several parameter settings, as shown in Table 1.

3.3 Dynamic Contrastive Loss

To accelerate model convergence and mitigate training oscillations, we have developed a Dynamic Contrastive Loss. This loss function dynamically adjusts the loss between each pair of views based on the current training stage or model performance. This approach enables the model to adaptively adjust the weights of different view pairs during the training process, thereby allowing the training to flexibly and efficiently focus on the most important view information.

Dynamic contrastive loss refers to the dynamic adjustment of the loss function's weights or other parameters based on the training progress or current feedback from the model. We adjust the loss function weights corresponding to each view pair according to their current training performance. With the incorporation of dynamic loss, our total loss function is expressed in Equation (10):

Specifically, λ1(t), λ2(t), λ3(t) is a dynamic weight function that varies over time (training steps).

If the loss value of a particular view pair is large, it indicates that the model's learning progress for this view is slow, and it may be necessary to dynamically decrease its loss weight. The specific adjustment method is detailed in Equation (11).

We used data augmentation techniques including RandomResizedCrop, ColorJitter, and RandomFlip. RandomResizedCrop first performs random cropping on the image and then resizes it to the target dimensions of 224×224. This method effectively increases data diversity, thereby enhancing the robustness of the model. ColorJitter randomly adjusts the image's brightness, contrast, hue, and saturation, generating images with varied visual effects. This helps the model adapt to different lighting and color conditions, thereby improving its generalizability. RandomFlip applies horizontal or vertical flips to images with a certain probability, assisting the model in handling scenarios where flipping does not affect classification outcomes, thereby further enhancing its robustness. Through these data augmentation methods, the model is better able to adapt to complex environments, improving performance and stability during the training process.

4.3 Comparative Experiments

Firstly, we conducted comparative experiments on the BreaKHis pathology image dataset at various magnification scales. The objective was to evaluate the model's performance across different magnification levels of the pathology dataset and to investigate the model's color robustness and edge detection capabilities. The detailed experimental results are shown in Table 2.

While self-supervised methods like MoCo and SimCLR excel in breast cancer image classification, they mainly focus on global features, potentially overlooking subtle edge and color differences. In contrast, our model integrates the Sobel operator and Dynamic Contrastive Loss, enhancing its ability to capture local features and improve generalization and representation.

The underlying reason for this performance improvement lies in the integration of the ResNet architecture with the Sobel operator and the three-view consistency enhanced momentum contrast loss function. The Sobel operator enhances the model's sensitivity to edge information, the ResNet architecture ensures the depth and diversity of feature extraction, and the momentum contrast loss function combined with multi-view consistency enhancement further improves training stability, model robustness, and the effectiveness of selfsupervised learning. The combination of these innovative methods enables our model to achieve outstanding performance across multiple datasets.

Subsequently, we conducted experiments on a large pathology image dataset to explore the model's classification performance on large-scale data. The specific experimental results are presented in Table 3.

From the perspective of capturing edge information and color sensitivity, Mymodel improves the sensitivity to image details, structure, and color variations by combining the Sobel operator and color jittering. This allows the model to not only accurately identify critical edge features on the NCTCRC-HE-100K dataset but also adapt to varying lighting and color conditions, resulting in a 99.98% accuracy and the best performance across other metrics. In comparison, while other methods perform well on certain metrics, they do not enhance edge information and color sensitivity as effectively as our model, which leads to slightly inferior performance in handling details and complex environments.

Finally, we conducted experiments on the large-scale dataset Food-101 to explore the model's performance in natural image classification, demonstrating its robustness and generalization ability. The specific experimental results are shown in Table 4.

Our model not only outperforms other models in terms of accuracy but also demonstrates equally outstanding performance across other key metrics, including recall, precision, and F1 score. This indicates that the model is highly effective in distinguishing between different categories, with extremely high sensitivity and specificity, resulting in very few false positives and false negatives. At the same time, SimSiam and other more recent models have also shown competitive performance.

The experimental results demonstrate that our model successfully addresses the common issues of edge blurring and color sensitivity in medical image analysis through hierarchical feature enhancement, dynamic optimization mechanisms, and the Sobel operator, while maintaining excellent performance in natural scenes.

4.4 Ablation Experiments

To better validate the significant roles of the (Dynamic Contrastive Loss, DCL)and(Momentum Contrastive Loss, MCL) within the model, we designed the ablation experiments shown in Table 5.

Based on the results of the ablation study, we can draw the following analysis:

After removing the dynamic loss module, all metrics slightly decline, with accuracy dropping by 1.96 percentage points. The equal distribution of loss weights across all samples leads to insufficient optimization of challenging samples, such as those with blurred edges, reducing overall detection accuracy.

When the momentum contrastive module is removed, the model performance drops slightly, with accuracy of 97.28%, a decrease of 2.09 percentage points. Compared to the removal of the dynamic loss module, the effect of removing the momentum update module is more pronounced, particularly with an improvement in precision (Pre) and F1 score. However, this improvement may come at the cost of stability.

When the SobelResNet module is removed, the model performance drops significantly, with accuracy of 97.16%, a decrease of 2.21 percentage points. The SobelResNet module plays a crucial role in edge feature extraction in images, and the lack of suppression of color sensitivity leads to a noticeable decline in performance.

4.5 Discussion on Backbone and Classifier

To further discuss the role of the backbone and classifier in classification, we conducted experiments on three datasets and provided a detailed discussion. The specific experimental results are shown in Table 6 and Table 7.

The classification results show that SobelResNet outperforms ResNet, mainly due to the strong reliance of pathological images on morphological structures, where staining inconsistencies can affect performance. SobelResNet reduces the dependency on color and emphasizes texture and structural features. Similarly, in natural images, factors like lighting variations can introduce noise. SobelResNet's focus on texture and structural features helps mitigate the effects of color variations, making it more robust and adaptable for both pathological and natural image classification tasks.

For larger datasets, the difference between linear and nonlinear models is relatively minor, and we consider the variation to be within an acceptable error margin. However, the significant disparity observed in the BreaKHis dataset is likely due to the presence of images captured at different magnifications. Nonlinear models can, to some extent, mitigate misclassifications caused by variations in magnification, providing better robustness under such conditions.

4.6 Visualization

In Figure 2, loss2, which incorporates Dynamic Contrastive Loss, exhibits a faster convergence rate: In the initial stages of iteration, loss2 demonstrates a quicker decline in loss compared to loss1, which is highly beneficial for saving training time and resources. Higher stability: Throughout the training process, loss2 shows greater stability, which helps to prevent performance degradation due to overfitting, particularly important in training with large-scale data sets. Lower final loss values: The lower final loss values indicate that under the same training conditions, loss2 may optimize model parameters more effectively.

As shown in Figure 3, this study employs the T-SNE visualization method to analyze the classification performance across 10 classes (Class 0-9), each represented by a different color. Figure 3(A) exhibits clear inter-class separability, where the data points of each class form distinct and welldefined clusters. This indicates that data points from different classes are likely to be highly distinguishable in the original high-dimensional space, providing strong evidence of the model's discriminative power and classification capability in the high-dimensional feature space. In panel (D), most of the classes exhibit compact clusters, suggesting that data points within the same class are highly similar in the original feature space. A few classes, such as the brown and orange clusters, show some overlap or proximity, which may indicate that these classes share certain similarities in the feature space.

As shown in the confusion matrix in Figure 4, most of the predictions are concentrated along the diagonal, exhibiting a clear pattern of centralized distribution. The values on the main diagonal are significantly higher than those in the off-diagonal regions, intuitively reflecting the model's high classification accuracy across the majority of classes. Although a few instances of inter-class misclassification are observed, their overall number is limited, further indicating that the model achieves good classification performance on this dataset.

5 Conclusion

The SVCMC algorithm developed in this study incorporates the Sobel operator and introduces a novel three-view consistency momentum contrast loss function, along with Dynamic Contrastive Loss. This approach has been rigorously tested across two pathological datasets and one natural image dataset, where it has demonstrated significant strengths in classification performance. The model excels in effectively managing issues related to color sensitivity and edge blur in images, showcasing its robustness and adaptability. Looking ahead, plans are in place to extend this research to include more comprehensive experiments on remote sensing imagery and a variety of structured medical datasets, aiming to further validate and enhance the algorithm's applicability and performance across diverse imaging contexts.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant no.62276265 and 62406326).