RepObE: Representation Learning-Enhanced Obfuscation Encryption Modular Semantic Task Framework

1 Introduction

With 6G technology advancing swiftly, traditional techniques are increasingly unable to meet the requirements for high bandwidth, low latency, and intelligent processing. Semantic communication, which prioritizes sending the meaning of data over raw bits, enhances both communication efficiency and task accuracy [Lu et al., 2024]. It also simplifies data transfer and boosts environmental awareness [Fu et al., 2024]. This approach not only enhances efficiency but also introduces novel methods for intelligent information processing and resource optimization. Research has progressed from semantic extraction in single-mode data (e.g., text, images [Sagduyu et al., 2024], audio [Chen et al., 2024], video) to multimodal integration [Guo et al., 2024], while addressing privacy and security issues, thus solidifying the theoretical and technical foundations of the field.

∗Corresponding author

Recent advancements in semantic communication have transformed both practical applications and academia. In the Internet of Things (IoT), it boosts device interaction and resource management, enhancing system efficiency [Sang et al., 2025]. For unmanned aerial vehicles (UAVs), it facilitates group coordination by extracting essential semantic data from missions, thereby reducing communication overhead and optimizing decision-making [Xu et al., 2025]. In digital twin technology, which requires accurate, real-time synchronization between physical and virtual entities, semantic communication is crucial for industrial production monitoring [Du et al., 2024]. Likewise, in vehicular networks, semantic communication tackles high latency and inefficiency issues in autonomous driving and V2V communication. By transmitting crucial semantic details about the environment and driving actions, it aids decision-making, improves safety, and optimizes traffic management [Xu et al., 2023].

Recent research on privacy and security within semantic communication has gained traction due to the complexity of extracting and processing deep semantic information. To counter data breach risks, several privacy-preserving methods are suggested, such as adversarial learning [Wang et al., 2025], location privacy protection [Qiu et al., 2023], and federated learning frameworks [Wang et al., 2024b]. Moreover, information bottleneck theory [Zhang et al., 2024] and variational inference [Li et al., 2024] have been examined to improve feature extraction and model training, thereby indirectly enhancing privacy. However, these predominantly focus on data privacy, often overlooking crucial model security challenges. Existing approaches fall short against sophisticated threats, including model inversion attacks, which reconstruct sensitive data [Wang et al., 2024a], and adversarial attacks, which disrupt systems and degrade performance using minimal perturbations [Zhang et al., 2025]. This paper introduces an innovative solution, with the primary contributions summarized below.

Innovative Framework for Privacy Protection. We propose the RepObE framework, which dynamically encrypts data during the stages of semantic extraction and feature transmission. This framework effectively prevents attackers from reconstructing data through eavesdropping, significantly enhancing the privacy protection of the system.

Enhanced Resilience in Image Communication Tasks. To further strengthen the resilience of the system, we introduce a collaborative adversarial training approach of prototype alignment based on learning. This approach leverages representation learning to extract and encode semantic features while integrating dynamic perturbation and robust optimization techniques.

Fusion Approach for Reliable Semantic Communication. Our fusion approach ensures reliable and stable semantic communication in complex scenarios. It maintains high performance while defending against attacks through feature obfuscation, adversarial training, and representation learning.

Experimental Results Demonstrating Superior Performance. Experimental results show that our method outperforms existing methods by more than 2% in resisting model inversion attacks on the MNIST dataset's classification task. Visually, our approach performs the best, making it difficult for attackers to obtain meaningful images. Our method also improves by 3% to 5% in resisting adversarial attacks on the CIFAR10 dataset's classification task.

The rest of this paper is organized as follows. Section 2 is related work. Section 3 proposes the framework design. Section 4 presents the experimental results. Finally, Section 5 summarizes the paper.

2 RelatedWorks

2.1 Image Semantic Communication

Advancements in AI and communication technologies have prioritized efficient image data transmission while retaining critical information. In image classification, Liu et al. employed semantic compression to reduce transmission load and latency while improving performance [Liu et al., 2021], and Hu et al. integrated masked VQ-VAE with adversarial training to enhance robustness and multitask optimization [Hu et al., 2023]. For UAV scene classification, Kang et al. used deep reinforcement learning to select key semantic blocks, optimizing efficiency [Kang et al., 2022]. In segmentation and autonomous driving, Pan et al. designed a system that significantly improved segmentation performance [Pan et al., 2023]. For image retrieval, Jankowski et al. [Jankowski et al., 2020] and Xie et al. [Xie et al., 2022] optimized single-user and multi-user scenarios, enhancing retrieval and transmission. In reconstruction tasks, Yang et al. leveraged the Swin Transformer to improve representation and efficiency [Yang et al., 2023]. For multitask scenarios, Zhang et al. utilized semantic encoding with a data adaptation network [Zhang et al., 2023], and Wu et al. introduced cross-task transfer to reduce storage needs and enhance performance [Wu et al., 2022]. Additionally, Liu et al. demonstrated a task-oriented framework that effectively reduced data volume and latency in classification and reconstruction tasks [Liu et al., 2022].

2.2 Privacy and Security of Neural Networks

Semantic communication enhances transmission efficiency but faces significant challenges in data security and privacy protection [Nan et al., 2023]. Neural network-based encoders and decoders are vulnerable to threats like model inversion, inference, and adversarial attacks, leading to potential privacy breaches and compromised security [Liu et al., 2024a]. To mitigate these threats, researchers have proposed defense strategies such as combining information bottleneck theory with adversarial learning [Wang et al., 2024a], employing adversarial training for robustness, and implementing hierarchical defenses for vehicular metaverse applications [Kang et al., 2023]. Random feature space perturbations [Xu et al., 2021] and encryption methods like homomorphic encryption and secure multi-party computation [Liu et al., 2024b; Patra et al., 2021] further protect sensitive data and model parameters. However, existing approaches still fall short in addressing privacy protection and security for image-focused semantic communication systems.

Existing literature on semantic communication for image data has improved efficiency but lacks adequate solutions for data security and privacy. Despite proposed defense strategies, neural network-based encoders and decoders remain vulnerable to threats. This paper introduces the Representation Learning-Enhanced Obfuscation Encryption Modular Semantic Task Framework to enhance both efficiency and security of image-focused semantic communication systems.

3 RepObE Framework Design

The proposed system model for this paper can be found in supplementary materials. In this section, we propose the RepObE framework for secure semantic communication, as shown in Figure 1. We dynamically encrypt data during processing to prevent eavesdropping and enhance privacy. For image communication, we introduce a representation learning-enhanced adversarial training approach. This approach encodes features and integrates techniques to strengthen robustness against attacks. It ensures reliable communication in complex scenarios while maintaining high performance. The training algorithm is shown in Algorithm 1.

3.1 Design of Semantic Encoder-Decoder

Design of Semantic Encoder. The input 224 × 224 image is processed through a 7 × 7 convolutional layer, followed by max-pooling, reducing the feature map to 56 × 56 with 64 channels. Two ResNet blocks extract deep features, maintaining the size and channel count. Subsequent residual blocks with an SE module increase channels to 96 and reduce the size to 28×28. A convolutional layer reduces the channels to 4, followed by 3×3 convolutions with an SE module, adjusting the channels between 32 and 4. A final channel obfuscation operation produces the output.

Design of Semantic Reconstruction Decoder. The feature map of size 4 × 28 × 28 first undergoes an unShuffle operation for channel rearrangement. It then passes through two ResNet blocks, producing a feature map of size 64×28×28. A transposed convolution increases the size to 64 × 56 × 56, followed by two more ResNet blocks that reduce the channels to 32 while maintaining the size. Another transposed convolution expands the size to 32 × 112 × 112. Two additional ResNet blocks further reduce the channels to 16, keeping the size unchanged. Finally, a transposed convolution adjusts the channels to C, outputting an image of size C × 224 × 224.

Design of Semantic Classification Decoder. The 4×28× 28 input is first unshuffled to rearrange channels. It then passes through two ResNet blocks, producing an output with 64 channels and a feature map size of 64 × 14 × 14. Next, two additional ResNet blocks increase the channels to 128, reducing the size to 128×7×7. Another two ResNet blocks further increase the channels to 256 and reduce the size to 256×4×4. Finally, global pooling is applied, followed by a fully connected layer, generating the final classification output with a dimension of 10.

Optimization of Semantic Encoder-Decoder. To ensure secure and practical transmission, the input image size I ∈

RC×H×W is set to at least 224 × 224. Both the encoder and decoder leverage convolutional neural networks, incorporating ResNet blocks and SE attention mechanisms for effective feature extraction. For an input image of resolution 224×224, the feature map resolution is progressively reduced to 28×28 through convolution, pooling, and downsampling, while the number of channels increases from 3 to 4. ResNet blocks enhance feature extraction, while SE blocks capture rich semantic features, improving the model's representational capacity. Multi-step downsampling minimizes computational complexity. A confusion layer, utilizing a shift algorithm for image confusion and deconfusion, is integrated at the sender's input and receiver's output. Each ResNet block comprises two 3 × 3 convolution layers and a residual connection. The formula for a ResNet block is

where the input feature map is F, Rn(F) represents the result of two convolution operations, and the output feature map is F ′ . The SE Block includes global average pooling, which compresses the spatial dimensions H×W to the channel dimension, extracting global features for each channel. Then, two fully connected layers are used to generate the attention weights for the channels, and finally, each channel is weighted to enhance the representation of important features. The formula is

where w1 and w2 are model weights, GAP denotes global average pooling, and w represents the computed weights. The

Algorithm 1 RepObE Training Algorithm

Require: A dataset Di, epochs e, batch size B, initialize semantic encoder network Sθ(·), semantic reconstruction/classification decoder network Gθ1 (·)/Gθ2 (·).

Ensure: The trained Sθ(·), Gθ1 (·) or Gθ2 (·).

• 1: The transmitter and receiver negotiate the key K = (Kr, Kc, r, c);
• 2: Set epoch counter e = 1;
• 3: while the training stop condition is not met do
• 4: The transmitter generates adversarial sample B′ based on PGD algorithm and dataset B;
• 5: The transmitter encrypts χ = Sθ(B′ +B) based on the key K and formulas (4), (5), (6), (7);
• 6: Transmit χ over the channel, receiving ˜ χ at the destination;
• 7: The receiver decrypts ˜ χ based on the key K and formulas (8), (9), (10), (11);
• 8: The receiver calculates the loss function Lrec (for the reconstruction task) or Lclass (for the classification task);
• 9: Update parameters of semantic encoder network Sθ(·) and decoder network Gθ1 (·) or Gθ2 (·);
• 10: e = e +1;
• 11: end while

feature transmitted over the channel is a k-dimensional vector z. Considering the output power p from the sender, we impose an average power constraint on the transmitted signal vector, expressed as

The encoder and decoder are jointly trained, with the channel modeled as a non-trainable layer.

3.2 Confusion Offset

Before entering the input channel, assume the semantic encoder at the sender needs to send a feature map of size Fs ∈ Ra×H′×W ′ , where a represents the number of channels in the feature map, H′ is the height, andW ′ is the width of the feature map. The sender and receiver negotiate a key K = (Kr,Kc, r, c), when Kr is the row permutation key of size H′ , Kc is the column permutation key of size W ′ , r is the row offset base, and c is the column offset base.

The encryption process consists of the following four steps.

Row Confusion. Rearrange the rows of each channel a of the feature map F according to the order specified by the key Kr. Let the i-th row of F be denoted as F[i, :], and the confusion feature map is denoted as F1 as follows.

where Kr[i] is the i-th element in Kr, indicating that the Kr[i]-th row of the original feature map is mapped to the i-th row in the confused feature map.

Column Offset. After the row confusion is completed, the feature map F1 undergoes a column shift for each row based on the column offset base c and the current row number i. The offset feature map is denoted by F2 as follows.

The column offset of each row increases incrementally. The 0-th row has an offset of 0 + c, the 1st row has an offset of 1 + c, and so on, until the (H′ - 1)-th row, where the offset is H′ - 1 + c.

ColumnConfusion. For each channel c, the columns of F2 are permuted according to the order specified by the key Kc. The permuted feature map is denoted by F3, and the operation is defined as

where Kc[j] is the j-th element in Kc, indicating that the Kc[j]-th column of the original feature map is mapped to the j-th column in the confused feature map.

Row Offset. After the column confusion is completed, the feature map F3 undergoes a row shift for each column based on the row offset base r and the current column index j. The offset feature map is denoted by Fen as follows.

The row offset for each column increases incrementally. The 0-th column has an offset of 0+r, the 1st column has an offset of 1 + r, and so on, until the (W ′ - 1)-th column, where the offset is (W ′ -1) + r.

The decryption process is composed of the following four steps.

Inverse Row Offset.

Inverse Column Confusion.

Inverse Column Offset.

Inverse Row Confusion.

Theorem 1. The encryption process consists of row confusion, column offset, column confusion, and row offset. These steps introduce significant complexity for attackers attempting to reconstruct the original feature map. The total key space size of the encryption process is given as follows.

Proof. The row confusion step uses a key Kr, which is a permutation of length H′ . The total number of possible row permutations is H′ !.

The column confusion step uses a key Kc, which is a permutation of length W ′ . The total number of possible column permutations is W ′ !.

The row offset r and column offset c are values within the range from 0 to H′-1 and 0 to W ′-1, respectively, providing H and W possible offset values for rows and columns.

Combining all the above steps, the total size of the key space is the product of all possible combinations, given by formula (12).

3.3 Representation Learning-Enhanced Prototype Alignment Collaborative Adversarial Training

To defend against adversarial attacks, we adopt an adversarial training strategy to improve the robustness of the model, as shown in Figure 2 Adversarial training aims to enhance the model's ability to correctly classify natural samples while resisting the interference caused by adversarial samples by incorporating adversarial examples into the training process. The goal of generating adversarial samples is to maximize the model's loss function by adding perturbations to the input sample I. We use the PGD algorithm to generate adversarial samples. PGD is an iterative optimization method that incrementally finds the most disruptive adversarial samples through multiple updates. In each iteration, the loss function is denoted as LCE(Mθ(Iadv), y), and the perturbation that increases the loss is denoted as formula (14).

Additionally, to enhance the representation learning capability of the model, we incorporate a contrastive learning framework during adversarial training. Contrastive learning leverages the relationships between adversarial and natural samples to enforce robust feature representations. Specifically, we treat adversarial samples Iadv and their corresponding natural counterparts Inat as positive pairs while ensuring that different samples form negative pairs. The contrastive loss is expressed as

where zadv and znat are the embeddings of adversarial and natural samples, sim(·) represents the cosine similarity, and λ is the temperature parameter. By minimizing Lcontrast, the model learns robust embeddings that are invariant to adversarial perturbations, improving both robustness and representation quality.

The goal of this paper is to maximize the model's adversarial robustness through adversarial training, while maintaining the classification performance on natural samples as much as possible. First, the adversarial loss is maximized to generate adversarial examples that can deceive the model, causing its predictions to deviate from the correct labels as follows.

where Inat represents the adversarial sample generated by maximizing the adversarial robustness loss. The next step is to input both adversarial and natural samples into the model, and optimize it to make the model's prediction on adversarial samples as close as possible to that of natural samples. The adversarial training loss function introduced is

In each training iteration, the sender inputs natural samples and generates corresponding adversarial samples based on the current state of the model. Then, both natural and adversarial samples are fed into the model for training. To minimize the loss, we have designed a new loss function as follows.

We introduce ˆw(Iadv)DKL (Mθ(Inat) ‖ Mθ(Iadv)) as the loss for adversarial samples, which aims to reduce the prediction discrepancy between natural samples and adversarial samples. ϵ is a hyperparameter used to adjust the magnitude of KL divergence loss. The KL divergence is used to quantify the difference in output distributions between natural and adversarial samples, and is given by the following formula.

where P, Q represent the output distributions of natural and adversarial samples, respectively. By minimizing KL divergence, the model's prediction distributions for natural and adversarial samples are consistent, thereby enhancing robustness.

Inspired by prototype alignment, suppose the dataset contains K classes. We first train an auxiliary model, and after training, the K-dimensional logits output for natural samples are denoted as the feature p(I) ∈ RK , where

and p(x) refers to the high-dimensional feature before the model's output layer (or the logits output), which can be used as a high-level semantic representation of samples during adversarial training and model optimization.

For each class y ∈ K, the prototype py ∈ RK is the mean of each dimension of the feature of all samples in that class. The calculation is listed as follows.

The goal is to bring the model's logits output for adversarial samples closer to the prototype of the corresponding class in natural samples, reducing the prediction discrepancy between natural and adversarial samples. The adversarial samples are assigned different weights w(Iadv) based on their deviation from the prototype of natural samples, where the weight is inversely proportional to the distance from the natural sample. A greater weight is assigned to adversarial samples that deviate further from the natural sample prototype, and a smaller weight is assigned to those closer to the prototype. The weight calculation formula is listed as follows.

where ‖Mθ(Iadv) - py‖2 is the squared Euclidean distance between adversarial sample Mθ(Iadv) and class prototype py.

The distance reflects how much an adversarial sample deviates from the natural sample distribution, with larger distances indicating harder-to-classify samples. The function 1-exp(·), a variant of the Gaussian kernel, maps the distance to a weight range of [0, 1). The smoothing parameter τ controls the weight's sensitivity to distance. A larger τ results in more uniform optimization, while a smaller τ increases the weight of samples farther from the prototype, emphasizing classification boundaries. Weights are normalized to ensure the average weight remains 1, dynamically enhancing optimization for difficult samples without diminishing the overall adversarial loss impact as follows.

4 Performance Evaluations

The experiments in this study utilized the MNIST and CIFAR-10 datasets, representing handwritten digit classification and more complex color image classification tasks, respectively. To simulate channel interference in a real wireless communication environment, the communication channel in the experiments was modeled as either an AWGN channel or a Rayleigh fading channel. The Signal-to-Noise Ratio (SNR) range was set from 7 to 23, covering typical communication conditions from low to high SNR. This experimental design aims to comprehensively evaluate the robustness of the proposed semantic communication framework across different datasets and channel environments, as well as its capability in privacy protection and resilience against adversarial attacks. Please refer to the supplementary materials for the experimental settings designed in this study.

4.1 Model Reversal Attack Experiment

As shown in Figures 3a and 3b, RepObE demonstrated superior performance on MNIST dataset under both AWGN and Rayleigh fading channels, with accuracy steadily improving as SNR increased (e.g., from 0.9943 to 0.9966 under AWGN and from 0.9920 to 0.9956 under Rayleigh). In contrast, NECST-DP [Choi et al., 2019] exhibited significantly lower performance, particularly NECST-DP0.05. While IBAL [Wang et al., 2024a] and ESCST performed well at low to medium SNRs, their improvements plateaued at higher SNRs, ultimately lagging behind ours. Additionally, NECST consistently outperformed NECST-DP across all SNR levels, reflecting the trade-off between privacy protection and performance. As shown in Figure 3c, RepObE exhibited robust performance on CIFAR-10 dataset under Rayleigh fading channel, improving from 0.9021 at SNR = 7 to 0.9151 at SNR = 23. While VAE [Mehrasa et al., 2019] consistently underperformed with a maximum of 0.8512, JPEG2000 [Christopoulos et al., 2000] slightly outperformed ours at high SNR (0.9203 at SNR = 23). Overall, ours demonstrated the best robustness under low to medium SNRs, while JPEG2000 showed a slight advantage at high SNRs.

In Figure 3d and Figure 3e, the first four columns show images reconstructed by attackers using IBAL, NECST, NECST-DP0.9, and ours, respectively, under model inversion attacks. The fifth column displays images received by legitimate users with ours. Figure 3d presents results for MNIST digits "3" and "4", while Figure 3e shows CIFAR-10 categories "ship" and "horse". Visually, attacker-reconstructed images with ours are almost unrecognizable, containing no meaningful information, whereas legitimate users received clear and usable images.

These results confirm the effectiveness of our method in resisting model inversion attacks while maintaining highquality image reconstruction and classification. Attackers failed to extract meaningful information, while legitimate users successfully received clear images, achieving a strong balance between privacy protection and task performance.

4.2 Adversarial Attack Experiment

As shown in Figure 4a the experimental results under the AWGN channel with MNIST dataset and adversarial training demonstrate that RepObE maintains high performance on both natural samples accuracy (Nat ACC) and adversarial samples accuracy (Adv ACC). Across all SNR conditions, the Nat ACC of our method remained stable between 0.9903 and 0.9945, closely matching the baseline model's Nat ACC range of 0.9944 to 0.9965, indicating that adversarial training has minimal impact on the classification performance of natural samples. Simultaneously, the Adv ACC of our method improved from 0.9832 at SNR = 7 to 0.9907 at SNR = 23, demonstrating strong robustness against adversarial attacks.

As shown in Figure 4b, the experimental results under Rayleigh fading channel with MNIST dataset and adversarial training (ϵ = 0.016) indicate that RepObE outperforms the baseline model in both Nat ACC and Adv ACC. Across all SNR conditions, the Nat ACC achieved by RepObE ranged from 0.9902 to 0.9931, closely approaching the baseline model's Nat ACC range of 0.9921 to 0.9957, demonstrating stability in classifying natural samples. Moreover, the Adv ACC of RepObE was significantly higher than that of the baseline model. The Adv ACC of RepObE improved from 0.9745 at SNR = 7 to 0.9924 at SNR = 23, whereas the baseline model's Adv ACC only increased from 0.7912 to 0.8023.

The experimental results in Figures 4c and 4d demonstrate the superiority of the proposed method (ours) over comparative methods AT and Trades on the CIFAR-10 dataset with adversarial samples (ϵ = 0.016, step size 0.0032, 5 iterations). Under the AWGN channel (Figure 4c), ours achieved steady performance gains, improving from 0.6904 at SNR = 7 to 0.7296 at SNR = 23, reflecting strong robustness and stability. In comparison, AT and Trades [Zhang et al., 2019] showed limited improvements, with maximum accuracies of 0.6756 and 0.6669 at SNR = 23, respectively, consistently lagging behind ours. Similarly, under the Rayleigh fading channel (Figure 4d), ours outperformed across all SNR levels, with accuracy increasing from 0.6849 at SNR = 7 to 0.7099 at SNR = 23. In comparison, the AT method exhibited relatively low performance under low SNR conditions, with an accuracy of only 0.5654 at SNR = 7, improving to 0.5943 at SNR = 23, but remaining consistently lower than ours. The Trades method showed relatively stable performance across SNR conditions, but its maximum accuracy was only 0.6656 at SNR = 23, which was significantly inferior to ours.

These results highlight the notable advantage of the proposed method in robustness against adversarial sample interference and under complex channel conditions.

5 Conclusion

This paper introduces the RepObE framework for secure semantic communication, emphasizing defense against attackers. By encrypting data dynamically during semantic extraction and feature transmission, the framework deters data reconstruction via eavesdropping, boosting privacy protection. We improved system resilience in image communication tasks using a representation learning-enhanced prototype alignment collaborative adversarial training method. This integrated approach, with dynamic perturbation and robust optimization, provides reliable semantic communication in complex situations. Experimental results show our method surpasses existing ones, with over a 2% increase in resisting model inversion attacks and a 3% to 5% improvement in adversarial attack resistance in classification tasks. Visually, it excels, making it difficult for attackers to acquire useful images. Future work will explore the RepObE framework's application beyond image communication and additional techniques to enhance system robustness and efficiency against evolving adversarial threats.

Acknowledgments

This work is supported by National Natural Science Foundation of China under grants 62171132 and U1905211, and Natural Science Foundation of Fujian Province under grant 2024J09032.