Band Together: Untargeted Adversarial Training with Multimodal Coordination Against Evasion-Based Promotion Attacks

1 Introduction

Multimodal Recommender Systems (MRSs) typically leverage visual and textual information from user-interacted items to capture users' fine-grained preferences, effectively alleviating the challenge posed by sparse interaction data [He and McAuley, 2016b; Wei et al., 2019; Zhang et al., 2021; Zhou et al., 2023c; Liu et al., 2024]. While auxiliary modal information enhances the personalization of recommendations, we find that MRSs exhibit heightened vulnerability to evasion-based promotion attacks, in which attackers imperceptibly perturb the modalities of targeted items to boost their rankings, as demonstrated by our motivation experiment in Figure 1.

∗Corresponding author.

Specifically, we conduct evasion-based promotion attacks (FGSM [Goodfellow et al., 2014] and PGD [Madry et al., 2017]) on both visual and textual modalities of unpopular items (defined as those with only 5 interactions) over various datasets (Amazon Baby, Sports and Clothing [He and McAuley, 2016a]) and architectures (VBPR [He and McAuley, 2016b] and MMGCN [Wei et al., 2019])). Figure 1 presents the results on VBPR under FGSM attack as a representative instance. The black numbers represent the average hit rate (Hit@50) of targeted items appearing in user recommendation lists. As one can find from Figure 1, the postattack Hit@50 always shows a significant increase, demonstrating the vulnerability of MRS to evasion-based promotional attacks.

Existing works [Wu et al., 2021; Zhang et al., 2024; Mu et al., 2025] have attempted to mitigate the vulnerability to promotion attacks; however, they suffer from two critical limitations when applied to MRSs. First, these approaches primarily focus on single-modal recommender systems, failing to address the complex vulnerability patterns that arise from interactions between visual and textual features. This oversight is especially problematic as attackers can exploit cross-modal correlations to amplify their attack impact. For example, the Re-writing Defense [Zhang et al., 2024] uses GPT-3.5-turbo to rewrite adversarial text as a defense for LLM-based recommendation models. Second, and more fundamentally, current defenses are almost exclusively designed for poisoning-based attacks, where adversaries compromise collaborative signals by injecting fake user profiles or interactions during the training phase. For example, APT [Wu et al., 2021] simulates the poisoning process by injecting fake user data to foster a more robust system. However, in MRSs, attackers can deliberately manipulate the description or image of a targeted item during the inference phase to influence the recommendation results-an aspect not addressed by existing defense methods. To bridge these critical gaps, we propose Untargeted Adversarial Training with Multimodal Coordination (UAT-MC), a novel adversarial training framework specifically designed to defend against evasion-based promotion attacks in MRSs. However, directly applying conventional adversarial training to this setting is non-trivial due to the following two challenges:

C1: Dynamic Attack Target. Unlike poisoning attacks where malicious targets are explicitly injected during training, evasion attacks dynamically select targeted items at the inference phase. This fundamental difference renders traditional targeted adversarial training approaches ineffective, as they rely on pre-defined attack targets to generate adversarial examples, which are unknown during the training phase.

C2: Cross-modal Gradient Mismatch. Under the multiuser promotion setting, MRSs present a unique challenge: combining perturbations across visual and textual modalities often yields suboptimal adversarial examples which degrade the robustness achieved through adversarial training. Crossmodal gradient mismatch arises when visual and textual perturbations in multi-user promotion attacks are dominated by different user groups, causing the two modalities to be optimized toward inconsistent objectives and resulting in misaligned gradient directions.

To tackle the challenge C1, we propose untargeted adversarial training for MRS, which treats all items as potential targets of evasion-based attacks. Our approach frames recommendation as a multi-label classification task over users, where items serve as labels. Untargeted adversarial training indirectly defends against targeted attacks by globally enhancing the robustness of decision boundaries and disrupting the local gradient information on which attacks rely. The core principle is that the model learns to resist perturbations from any direction, thereby covering attacks originating from specific directions. To tackle challenge C2, we propose a novel gradient-aligned multimodal perturbation method to address cross-modal gradient mismatch in adversarial training. Unlike perturbing visual and textual features independently, which can diminish attack strength due to misaligned gradients-our method enforces gradient synchronization across modalities by minimizing the cosine distance between visual and textual gradients through a joint loss term. This ensures perturbations coherently push items toward adversarial regions. As shown in later experiments, this coordinated approach maximizes attack potency during training and enhances adversarial robustness against evasion-based promotion attacks.

Our contributions can be summarized as follows:

• We identify evasion-based promotion attacks as a criti-
• cal threat tomultimodal recommender systems, inwhich adversaries perturb both the visual and textual features of targeted items.
• We propose a novel Untargeted Adversarial Training with Multimodal Coordination (UAT-MC) framework to enhance the robustness of MRSs against evasion-based promotion attacks. Specifically, the proposed untargeted adversarial training inherently defends against targeted attacks by universally hardening the decision boundaries, thereby covering all potential attack directions.
• We propose gradient-aligned multimodal perturbations to resolve cross-modal gradient mismatch in adversarial training. By minimizing cosine distance between modalities via a joint loss term, our method synchronizes perturbations to maximize attack potency.
• Extensive experiments conducted on real datasets verify the effectiveness of our method.

2 Preliminary

In this section, we conceptually define MRS and describe the evasion-based promotion attacks.

2.1 MRS

Let U and I denote user set and item set, respectively. R ∈ {0, 1}|U|×|I| is user-item interaction matrix, where an element ru,i is 1 if there exists interaction between user u ∈ U and item i ∈ I, otherwise 0. Each item i is associated with a visual modality embedding vi and a textual modality embedding ti which are generated by CNN and Transformer, respectively.

Generally, an MRS f infers the probability ˆ ru,i that user u will interact with item i based on given R and the modality embeddings vi and ti, i.e.,

Let D = {(u, i+, i-)} be a training set, where i+ is u's positive sample (ru,i+ = 1 ) and i- is u's negative sample (ru,i- = 0 ). Like traditional recommender systems, an MRS is usually trained with Bayesian Personalized Ranking (BPR) [Rendle et al., 2009] loss function,

where Θ represents the model parameters and σ(·) is the sigmoid function.

2.2 Threat Model

Attacker's Objective

Given a well-trained MRS, an evasion-based promotion attack is to imperceptibly perturb the multimodal embeddings of a targeted item i, so that i will appear in the top-K recommendation lists of as many users as possible. Inspired by [Wang et al., 2024b], we define the promotion utility function as:

where yu,K denotes the score of the K-th ranked item for user u. By maximizing Equation (3), the attacker encourages the targeted item score yu,i to surpass the top-K threshold yu,K.

Attacker's Knowledge

In this paper, we assume a white-box scenario for the generation of the adversarial examples in our UAT-MC, where the attacker has full access to the MRS f, including its parameters and gradients. This is because UAT-MC aims to train a more robust MRS, rather than to conduct attacks from the perspective of malicious merchants. This setting represents a worst-case attacker and allows us to evaluate the upper bound of the potential impact of evasion-based promotion attacks.

Attacker's Capability

Following the ideas of the related works [Tang et al., 2020; Zhang et al., 2021; Guo et al., 2024; Liu et al., 2024; Ong and Khong, 2025] , we perturb the multimodal embeddings instead of the raw inputs, which allows more fine-grained and efficient manipulations. Formally, the perturbed embeddings v′i = vi +∆iv and t′i = ti +∆it . For imperceptibility, the

perturbations ∆im (m ∈ {v, t}) are constrained by ∥∆im∥ ≤ ϵm, where ϵm is perturbation budget.

3 Cross-modal Gradient Mismatch Analysis

In this section, we demonstrate that under multi-user promotion settings, the gradients of visual and textual perturbations are inherently driven by distinct user groups due to varying modal sensitivities. Consequently, simply aggregating these gradients leads to conflicting optimization directions across modalities, which we term cross-modal gradient mismatch. This misalignment fundamentally limits the synergy of multimodal attacks and motivates our UAT-MC framework. We next provide a formal analysis to characterize and verify the existence of this mismatch.

3.1 Objective Inconsistency Across Modalities

Given the promotion objective Lpromotion defined over the user set Up (Equation (3)), the optimization problem can be formulated as:

where Lu denotes the promotion loss contributed by user u. Accordingly, the perturbations are updated by aggregating gradients from all selected users:

where gvu = ∂Lu ∂∆iv and gtu = ∂Lu ∂∆it denote the visual and textual gradients induced by user u, respectively.

To quantify how much each user contributes to the final update direction, we define the directional contribution of user u on modality m ∈ {v, t} as:

where cos(gm u ,Gm) measures whether user u's gradient direction is aligned with the aggregated update direction, and ∥gm u ∥ reflects the contribution of user u to modality m.

Based on this metric, we define the user set Uv and the user set Ut as the top-K users ranked by cvu and ctu , respectively. To measure the overlap between these two sets, we compute their Jaccard similarity:

A low Jaccard similarity indicates that the users dominating the visual and textual perturbation updates are largely different, suggesting that the two modalities are implicitly driven by different user groups and, consequently, toward different effective optimization objectives. As illustrated in Fig. 2(a), the visual perturbation ∆iv is primarily driven by users in Uv, while the textual perturbation ∆it is dominated by users in Ut. When such modality-specific perturbations are subsequently fused by the MRS, the resulting item representation fails to simultaneously benefit all users, leading to suboptimal promotion outcomes.

3.2 Empirical Evidence of Cross-modal Gradient Mismatch

As mentioned in Section 1, we conduct vanilla FGSM-based promotion attacks on the VBPR model across three datasets. For each dataset, we randomly sample 100 unpopular items whose interaction counts are no greater than 5. For each targeted item, we identify the user sets Uv and Ut using the directional contribution metric in Equation (6), and conduct the promotion attack variants by constructing the promotion loss over Up, Uv and Ut, respectively.

The results indicate that aligning the attack objective with modality-specific user groups is crucial, as visual and textual perturbations are driven by distinct user subsets. Specifically, Figure 3 reveals a consistently low Jaccard similarity between Uv and Ut, confirming their limited overlap. This distinctness is further corroborated by Figure 1, which demonstrates that optimizing over Uv or Ut yields significantly higher attack gains than using Up, as aggregating heterogeneous groups tends to dilute modality-specific optimization signals. (See Appendix B.4 for a detailed case study visualizing this phenomenon on a specific item.)

In summary, cross-modal gradient mismatch limits the effectiveness of multimodal perturbations. Since effective defense requires training against worst-case attacks, it is critical to harmonize optimization directions to maximize attack potency. This motivates our UAT-MC framework, which uses gradient alignment to generate stronger adversarial examples, thereby driving the model to learn more robust representations.

4 Methodology

Figure 4 shows the overview of UAT-MC. In Figure 4, the targeted MRS is divided into two parts, i.e., f = g ◦ h, where h is the encoder for collaborative filtering and g is the decoder. Usually, h is implemented as a GNN [Wei et al., 2019] or an MLP [He and McAuley, 2016b] for fusion of multimodal embeddings, while g is implemented as an inner product. A training instance consists of user ID embedding eu, a positive item's ID embedding e+, a negative item's ID embedding e-, the positive item's modality embeddings v+, t+, and the negative item's modality embeddings v- and t-.

During the k-th iteration, UAT-MC first generates the adversarial modality embeddings v′+, t′+, v′-, and t′-, by adding the perturbations generated in the last step. Then through the collaborative filtering encoder h, UAT-MC produces the user embedding hu by fusing with the multimodal content of the items interacted with u, the positive and negative item embeddings h+ and h- by fusing their respective multimodal embeddings, and their respective adversarial sample embeddings h′u , h′+ and h′-. Based on these embeddings, the decoder g computes the ranking losses LBPR and L′BPR on the clean embeddings and the adversarial embeddings, respectively. On these losses, UAT-MC conducts a min-max game to improve the robustness of the MRS f.

It is worth noting that to achieve worst-case adversarial robustness for f , UAT-MC promotes consistent perturbation directions across different modalities by maximizing the gradient alignment loss LAlign when generating adversarial perturbations. This approach realizes the multimodal coordination, thereby enhancing the aggressiveness of adversarial examples.

4.1 Untargeted Adversarial Training

As mentioned in Section 1, in evasion attacks, the specific targeted item at inference time is unknown during the training phase. To address this challenge, we propose untargeted an adversarial training for MRSs, which treats all items as potential targets of evasion-based promotion attacks. This approach is motivated by the fact that recommendation can be formulated as a multi-label classification task, where each item serves as a distinct label. Untargeted adversarial training improves the robustness of the model by strengthening the decision boundary against perturbations from arbitrary directions, thereby effectively defending against perturbations from any direction.

Specifically, given a user u, the attacker adds perturbations [∆v+ ,∆t+ ] to the positive item's multimodal embeddings [v+, t+] and the perturbations [∆v- ,∆t- ] to the negative item's multimodal embeddings[v-, t-], which result in the adversarial modality embeddings v′+, t′+, v′-, and t′-. Then the final adversarial embeddings are computed as:

Then the untargeted adversarial training is defined as:

where L′BPR (Θ,∆v,∆t) is defined as:

4.2 Multimodal Coordination

As analyzed in Section 3, independently perturbing each modality in multi-user promotion settings often results in cross-modal gradient mismatch, reducing the effectiveness of adversarial attacks. While identifying and optimizing over specific user subsets (i.e., Uv and Ut) can improve attack performance, it requires computing user-wise gradients for the entire user base, incurring a prohibitively high computational cost. To circumvent this bottleneck, we propose a lightweight multimodal coordination mechanism. Instead of explicitly partitioning users, we focus on directly rectifying the divergent optimization directions in the gradient space. By enforcing gradient-level alignment between modalities, we effectively mitigate the mismatch without the need for costly user identification. This approach achieves coordination with negligible computational overhead, requiring only minimal additional operations during backpropagation.

We first compute the gradients of the adversarial loss with respect to the perturbations in each modality:

Then, we define the alignment loss as the sum of the cosine similarities between the gradients of visual and textual modalities for both positive and negative items:

Maximizing LAlign encourages the perturbations in different modalities to be directionally aligned, thereby pushing the targeted item toward the adversarial region.

Algorithm 1 UAT-MC

Input: Training data D; Learning rate η; Perturbation budgets ϵt, ϵv; Hyperparameters λ, α, β

Output: Robust MRS model parameters Θ

• 1: Initialize Θ from normal-trained MRS
• 2: while not converged do

3: Randomly draw an example (u, i+, i-) fromD

• 4: // Max phase
• 5: L′BPR = -lnσ(h′u · h′+ -h′u · h′-)
• 6: Γv =∇∆v L′BPR , Γt =∇∆t L′BPR
• 7: LAlign = cos(Γv+ , Γt+ ) + cos(Γv- , Γt- )
• 8: Lmax = L′BPR +αLAlign
• 9: ∆v ← ϵv · ∇∆v L′max ∥∇∆v L′max ∥2 , ∆t ← ϵt · ∇∆t L′max ∥∇∆t L′max ∥2
• 10: // Min phase
• 11: Lmin = LBPR +λL′BPR (∆v,∆t) +β∥Θ∥2
• 12: Θ←Θ-η∇ΘLmin
• 13: end while
• 14: return Θ

4.3 Optimization

The training is divided into two phases:

• Attacking (Max phase): Generate the most disruptive perturbations ∆v and ∆t within ℓ2-norm budgets ϵv and ϵt by maximizing λL′BPR + αLAlign.
• Defending (Min phase): Update model parameters Θ by minimizing Lmin = LBPR + λL′BPR (∆v , ∆t) + β∥Θ∥2.

The overall training process is described in Algorithm 1, where the MRS is pre-trained using the standard BPR loss.

5 Experiments

The objectives of experiments are to answer the following research questions:

• RQ1: Can UAT-MC defend against evasion-based promotion attacks?
• RQ2: Does multimodal coordination enhance the effectiveness of adversarial defense?
• RQ3: How does the perturbation budget in adversarial training affect the defense performance?
• RQ4: How do the hyper-parameters affect the trade-off between recommendation performance and adversarial robustness?

All experiments are implemented using PyTorch 2.2.0 and run on an NVIDIA RTX 4090 GPU with 48GB of memory.

5.1 Experimental Settings

Datasets

We conduct experiments on three Amazon datasets Baby, Sports and Clothing [He and McAuley, 2016a], which have recently been adopted in MRSs research [Yu et al., 2023; Guo et al., 2024; Li et al., 2025]. The dataset statistics are summarized in Table 1. In our work, we use the pre-extracted visual features and textual features that have been published in [Zhou et al., 2023c; Zhou et al., 2023a].

Evaluation Metrics

To measure recommendation performance before and after adversarial training, we use Recall@10 and NDCG@10 under the leave-one-out evaluation protocol following standard evaluation settings in prior works [Zhou et al., 2023c; Guo et al., 2024]. To evaluate the effectiveness of the promotion attack, we adopt the hit rate Hiti@K = N i rec /|U| · 100% and the relative improvement ratio GainHiti@K = (Hitafter@K

-Hitbefore@K)/Hitbefore@K · 100% as evaluation metrics, where N i rec is the number of users for whom the targeted item i appears in the top-K recommendation list (with K defaulted to 50), Hitafter@K and Hitbefore@K denote the hit rate of the targeted item after and before the attack, respectively.

A larger GainHit@50 indicates a more effective promotion attack, suggesting weaker model robustness.

Targeted Items

At the MRS's inference phase, we randomly select 100 targeted items from unpopular items, whose interaction count is 5. We conduct promotion attacks on each targeted item individually and record the average Hiti@K and GainHiti@K.

Victim Models

We choose two mainstream and classical multimodal recommendation models as our victim models:

• VBPR [He and McAuley, 2016b]: As a representative of MLP-based models, VBPR incorporates visual features for user preference learning with BPR loss. Following [Zhou et al., 2023c; Guo et al., 2024], we concatenate the multimodal embeddings and the item ID embedding as the item embeddings.
• MMGCN [Wei et al., 2019]: As a representative of GCN-based models, MMGCN constructs three modalspecific graph to learn different modality features. It concatenates all modality embeddings to obtain the representations of users or items.

We follow MMRec [Zhou et al., 2023a] to save the model parameters at the point of best performance. The hyperparameters search spaces are provided in the Appendix B.1.

Promotion Attacks

We conduct promotion attacks by maximizing Lpromotion (Eq. 3) using two standard strategies: FGSM [Goodfellow et al., 2014] and PGD [Madry et al., 2017]. FGSM serves as a single-step baseline, whereas PGD functions as a stronger iterative attacker. In our experiments, the PGD attack is configured with 10 iterations and a step size of α = 1.25 · ϵm/10.

Defenders

To verify the effectiveness of the Untargeted Adversarial Training and the Multimodal Coordination, we compare UAT-MC with two baseline models: one is the victim model without any defense mechanisms, denoted as w/o AT, and the other is the variant of UAT-MC, denoted as UAT, which applies perturbations to both visual and textual modalities independently without coordination.

Implementation Details

During adversarial training phase, we set the maximum perturbation magnitude ϵm as 10% of the 2-norm of the input embedding for modality m, the coefficient α is searched in {0.1,1,2,3,4,5,6,7,8,9,10}, the coefficient λ is searched in {1,2,3,4,5,6,7,8,9,10}.

5.2 Defense Effect (RQ1 and RQ2)

We conduct a systematic evaluation of the performance of different defense methods on two attack scenarios (FGSM-based and PGD-based) on two mainstream MRSs (VBPR [He and McAuley, 2016b] and MMGCN [Wei et al., 2019]). The victim models take the original and perturbed multimodal embeddings as input, respectively. Meanwhile, we also report the recommendation performance on the clean test set to reflect the impact of adversarial training on overall recommendation quality. The results are shown in Table 2, from which we have the following observations:

• Across all dataset-model combinations, unpopular items consistently exhibit vulnerability to promotion attacks (w/o LAlign), which risk misleading the target model into treating them as popular items. It can be observed that perturbations generated by PGD are more effective in promoting the targeted items compared to those generated by FGSM. Statistically, on the Baby dataset, the average Hit@50 of unpopular items increases from 0.667% to 3.865% under FGSM-based perturbations, and further to 4.111% under PGD-based perturbations. This implies that more than 3.444% of users will be misled into recommending an item (e.g., targeted item) that should not appear in the recommendation lists.
• Attacks equipped with LAlign consistently achieve higher promotion gains than their counterparts without alignment, indicating that explicitly coordinating visual and textual perturbations leads to more effective promotion. This improvement is particularly evident for stronger attacks. For example, under PGD-based attacks on the Baby dataset, introducing LAlign increases the GainHit@50 of VBPR from 516.15% to 520.99%, and that of MMGCN from 446.76% to 450.30%. Similar trends are observed on the Sports and Clothing datasets, where alignment consistently yields additional promotion gains across both VBPR and MMGCN.
• After untargeted adversarial training (UAT), the MRSs exhibit significantly enhanced resistance to

evasion-based promotion attacks. The results show that the effectiveness of both FGSM-based and PGDbased attacks drops significantly after UAT. In the following, we analyze based on the average GainHit@50 results of the two attacks (w/ LAlign). On the Baby dataset, UAT reduces the GainHit@50 of VBPR from 503.39% to 288.83%, achieving a 42.62% relative reduction. For MMGCN, the value drops significantly from 315.72% to 8.42%, corresponding to a 97.33% reduction. Similar trends are observed on the Sports dataset, where VBPR and MMGCN experience reductions of 99.95% and 98.92%, respectively. On the Clothing dataset, UAT reduces GainHit@50 by 76.50% for VBPR and 99.71% for MMGCN.

• UAT with Multimodal Coordination can further enhance its defense capability. According to our observations, the GainHit@50 of MMGCN on the Sports dataset drops from 25.45% to 19.14%, while that of VBPR on the Clothing dataset decreases significantly from 898.38% to 27.70% after applying multimodal coordination. Notably, the improvement in robustness is achieved with minimal impact on recommendation performance.

5.3 Hyper-parameter Study (RQ3 and RQ4)

This section explores how the perturbation budget ϵm and the hyper-parameters λ and α in adversarial training influence the trade-off between adversarial robustness and recommendation performance.

• Impact of ϵd and ϵa To examine the robustness and generalization ability of UAT-MC, we conduct adversarial training with different defense perturbation budgets ϵd ∈ {2.5%, 5%, 7.5%, 10%}, and evaluate the trained models under FGSM-based promotion attacks with varying attack budgets ϵa ∈ {2.5%, 5%, 7.5%, 10%}. The results are visualized in Figure 5, where specific numerical values are detailed in the Appendix B.3. We observe a clear transition from dark blue to light blue/white as ϵd increases, indicating that our defense method effectively mitigates the promotion attack.
• Impact of λ and α We investigate the trade-off between adversarial robustness and recommendation performance by varying λ ∈ [1, 10] and α ∈ [0.1, 10]. Figure 6 (Baby dataset) and the Appendix B.1 (others) visualize this relationship, plotting recommendation performance (NDCG@10) against attack gain (GainHit@50). A clear trade-off is observed: higher recommendation performance typically comes at the cost of increased vulnerability.

6 Conclusion

In this work, we address the vulnerability of MRSs to promotion attacks. Crucially, we verify the existence of cross-modal gradient mismatch in multi-user promotion settings and proposed UAT-MC to mitigate it via a novel gradient alignment regularization. Extensive experiments demonstrate the effectiveness of UAT-MC in defending against evasion-based promotion attacks.

Acknowledgments

This work is supported by Natural Science Foundation of Sichuan Province under grant 2024NSFSC0516 and National Natural Science Foundation of China under grant 61972270.