Snap2Review: Vision-Grounded Retrieval and Pairwise Preference Alignment for Personalized Reviews Generation

1 Introduction

In the era of visual-centric e-commerce, users increasingly rely on images to narrate their consumption experiences, transforming product reviews from simple text descriptions into rich, multimodal stories. Automatically generating personalized reviews from these user-captured photos holds immense potential to enhance user engagement and streamline feedback loops. However, this task transcends the boundaries of traditional image captioning; it demands an AI agent that acts not merely as an observer describing visual content, but as a personalized ghostwriter capable of articulating the user's unique voice, preferences, and linguistic habits.

∗Corresponding authors.

Despite the rapid advancements in Multimodal Large Language Models (MLLMs), bridging the chasm between objective visual signals and subjective user personas remains a formidable challenge. Current approaches typically fall into the "Generic Trap": they prioritize factual correctness at the expense of stylistic authenticity. While Retrieval-Augmented Generation (RAG) has been employed to introduce context, standard methods often rely on text-to-text retrieval, failing to capture the subtle visual nuances-such as texture, color deviations, or specific defects-that trigger a user's commentary. Consequently, generated reviews often suffer from a dual failure: Visual Hallucination, where the model invents details unsupported by the image, and Stylistic Homogenization, where the output reads like a robotic, "one-size-fits-all" description rather than a genuine user reflection.

We argue that the root cause of these limitations lies in the coarseness of existing alignment paradigms. Personalization is not a binary objective; a review can be factually correct yet stylistically alien to the user. Standard alignment methods, such as Direct Preference Optimization (DPO) [Rafailov et al., 2023], typically utilize binary preference pairs (winner vs. loser), which are insufficient for teaching a model the fine-grained distinction between a "generic good review" and a "personalized authentic review." To truly mirror a user's persona, the model must learn to navigate a complex preference landscape, distinguishing between semantic errors, sentiment mismatches, and subtle phrasing discrepancies.

To address these challenges, we introduce Snap2Review, a visually-driven framework that harmonizes cross-modal grounding with fine-grained stylistic alignment. Our approach decomposes the personalization problem into two distinct yet interconnected stages.

First, Visual-Anchored Retrieval. To eliminate visual hallucinations, we optimize a Vision-Conditioned Retriever via contrastive learning on image-review pairs. Unlike standard retrievers, this module explicitly aligns visual embeddings with the semantic space of historical reviews, allowing it to retrieve precise textual evidence (e.g., specific mentions of product features) strictly grounded in the input image. This retrieved context serves as a reliable anchor, bridging the gap between raw pixels and semantic understanding.

Second, Fine-Grained Stylistic Alignment via Tri-DPO. To break free from stylistic homogenization, we propose a Persona-Aware Generator trained with our novel Tri-Level Direct Preference Optimization (Tri-DPO). Moving beyond binary preference pairs, Tri-DPO employs a stratified negative sampling strategy that constructs negative examples at three distinct difficulty levels: general negatives (irrelevant content), similar negatives (stylistically mismatched but factually correct), and near Negatives (minor semantic or factual errors). This "curriculum of errors" forces the model to discern fine-grained boundaries in user preferences, effectively steering generation from generic praise to authentic, personadriven expression.

Extensive experiments on real-world e-commerce datasets demonstrate that Snap2Review achieves state-of-the-art performance. By explicitly addressing the twin challenges of visual fidelity and stylistic nuance, our framework generates reviews that are not only relevant to the provided images but also indistinguishable from the user's historical writings in terms of tone and preference.

In summary, the primary contributions of this work are threefold:

• We propose Snap2Review, a unified framework that bridges the modality gap between visual perception and personalized text generation, explicitly addressing the limitations of generic MLLM outputs.
• We introduce a specialized Cross-Modal Retrieval mechanism that grounds generation in visually relevant historical context, effectively mitigating hallucinations caused by purely text-based retrieval.
• We design Tri-DPO, a novel alignment algorithm equipped with a Stratified Negative Sampling strategy. By learning from negative examples of varying difficulty, the model achieves superior alignment with finegrained user preferences and writing styles.

2 RelatedWork

2.1 Personalized ReviewGeneration

Automated review generation aims to assist users in articulating experiences and enhancing system interpretability [Li et al., 2020; Peng et al., 2025]. Early works focused on text-based generation, utilizing discrete attributes (e.g., user IDs, ratings) [Tang et al., 2016; Dong et al., 2017] or specific aspect-sentiment labels [Zang and Wan, 2017; Li et al., 2019] to guide RNN or Transformer decoders. While recent approaches leverage historical reviews to improve semantic coherence [Ni and McAuley, 2018; Kim et al., 2020], they largely operate within a unimodal textual space. In modern visual-centric e-commerce, existing Multimodal LLMs often struggle to translate user images into personalized feedback, typically yielding generic content that fails to capture individual writing styles [Truong and Lauw, 2019].

2.2 Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm to mitigate the hallucination issues of LLMs by grounding generation in external knowledge [Fan et al., 2024]. In the context of personalization, RAG has been effectively adapted to retrieve relevant user history or item profiles to guide the model. For instance, [Zhang et al.,

2025] utilized a transformer-based retriever to filter essential user preferences, while other works have focused on dynamic prompt selection mechanisms [Huang et al., 2024]. However, most existing RAG frameworks are restricted to textto-text retrieval, failing to bridge the semantic gap between user-uploaded images and textual history. Standard retrievers overlook critical visual nuances (e.g., texture, defects) essential for accurate reviews. To address this, we optimize a crossmodal retriever that maps visual features directly to historical text, ensuring the generation is strictly anchored in visual reality.

2.3 Preference Alignment in LLMs

Aligning LLMs with human intent is critical for ensuring generation quality [Yu et al., 2025; Huang et al., 2026]. While Reinforcement Learning from Human Feedback (RLHF) laid the foundation, recent research has rapidly expanded alignment techniques to diverse domains.

In specific applications, [Han et al., 2025] employed reinforced self-training to align robot planners with human preferences, while [Yang et al., 2024] focused on behavior alignment in conversational recommendation systems. Techniques have also evolved beyond simple feedback loops; for instance, [Trivedi et al., 2025] introduced a principled approach to prompt optimization for alignment, and [Shankar et al., 2024] investigated the alignment of LLM-assisted evaluation itself. Moreover, in retrieval-augmented scenarios, [Dong et al., 2025] proposed a dual preference alignment framework to better understand LLM needs within RAG pipelines.

Despite these advancements, most approaches rely on binary preference signals or task-specific rewards. For personalized review generation, a binary distinction is insufficient to capture the nuance between semantic correctness and stylistic authenticity. Our work addresses this by introducing a stratified alignment strategy to discern fine-grained stylistic differences.

3 ProposedMethod

The proposed Snap2Review framework operates through three progressive stages to achieve personalized, visuallygrounded generation. MR (Multimodal Retriever Finetuning): Aligns visual and textual feature spaces via contrastive learning for cross-modal retrieval; VGR (Visiongrounded RAG): Retrieves context to guide the LLM via supervised fine-tuning. TDO (Tri-DPO Optimization): Refines stylistic alignment using hierarchically structured negative samples. The overall framework is shown in the Figure 1. The pipeline begins by establishing a robust cross-modal retrieval mechanism, which provides precise semantic anchors for the initial generation phase. Finally, the generated outputs are rigorously polished through our novel preference optimization strategy to ensure deep alignment with the user's specific persona.

3.1 Multimodal Retriever Fine-tuning

To bridge the semantic gap between visual signals and textual descriptions, we employ a dual-encoder architecture that maps product images and historical reviews into a shared embedding space. This fine-tuned retriever serves as the cornerstone of our framework, ensuring that subsequent generation is grounded in vision-aligned textual evidence rather than generic context.

Domain-Specific Data Collection

We construct a domain-specific dataset D = {(vi, ti)}Ni=1 comprising N pairs of product images and their corresponding user reviews from historical interaction logs. Unlike general-purpose multimodal datasets, these pairs encapsulate e-commerce-specific visual attributes (e.g., texture, defects) and their linguistic descriptors. Training on this corpus enables the model to learn fine-grained visual-textual associations essential for the retrieval task.

Asymmetric Parameter Update

We initialize both the vision encoder fv and text encoder ft with pre-trained CLIP weights. To balance domain adaptation with generalization, we adopt an asymmetric fine-tuning strategy.

As shown in the left part of Figure 1, we freeze the parameters of the text encoder ft to preserve its robust linguistic capabilities while fully fine-tuning the vision encoder fv. This design choice is motivated by the observation that visual features in e-commerce (e.g., specific product angles, lighting) deviate significantly from general pre-training data, whereas linguistic patterns remain relatively stable. Updating only the vision branch efficiently aligns the visual embeddings to the fixed textual semantic space, reducing computational costs and mitigating overfitting.

Contrastive Learning Objective

We optimize the retriever using a symmetric InfoNCE loss to maximize the similarity between matched image-review pairs while minimizing it for unmatched ones within a batch. Given a batch of B pairs, let vi and ti denote the normalized embeddings of the i-th image and review, respectively. The similarity score is computed as the dot product vi · ti. The total loss Laft combines image-to-text (Lv2t) and textto-image (Lt2v) alignment objectives:

where Lv2t aims to match each product image to its corresponding review. For a batch of B image-review pairs, it is formulated as:

where τ is a learnable temperature parameter scaling the logits. The text-to-image loss Lt2v is defined symmetrically. This objective forces the vision encoder to produce embeddings that are semantically proximate to their corresponding textual descriptions, enabling accurate retrieval of historical reviews based on visual queries.

3.2 Vision-Grounded Retrieval-Augmented Generation

Based on the fine-tuned vision encoder, we design a retrievalaugmented pipeline to ground generation in both user history and product context. This process addresses a key limitation of standard LLMs, which often produce generic content, by grounding the generation in visually relevant and userspecific historical evidence. The framework is shown in the middle part of Figure 1, with three phases: knowledge base construction, joint retrieval, and Persona-Aware Generation.

Hybrid Knowledge Base Construction

We construct two specialized vector databases-a User Database and a Product Database-to capture personal writing styles and item-specific attributes, respectively. To ensure robust matching, we index historical data using a multimodal fusion strategy. For each historical entry (vi, ti), we generate visual embedding Vi and textual embedding Ti using our fine-tuned retriever. A unified index embedding ei is computed as the normalized element-wise sum:

By fusing modalities, Ei encodes both the visual appearance of the product and its linguistic description, creating a semantic-rich index that supports accurate retrieval. We index all historical reviews under this unified schema, segregating them by User ID to capture idiosyncratic stylistic patterns and by Product ID to encompass item-specific attributes. This dual-repository architecture provides a comprehensive knowledge base, enabling the system to retrieve evidence that grounds the generation in both the user's personal voice and the product's factual reality.

Joint Retrieval Operation

During inference, given a user-uploaded query image vq, the system first encodes it into a query vector vq using the finetuned vision encoder. We then perform a joint similarity search across both databases. The relevance between the query image and a historical entry is measured via cosine similarity:

We retrieve the top-K entries with the highest similarity scores from each database, yielding a total of 2K textual references. This dual-source retrieval ensures the subsequent generation is informed by both the user's historical preferences (Style) and the product's visual attributes (Content).

Persona-aware Generation

The final stage synthesizes the retrieved evidence into a personalized review using a Large Language Model (LLM). This process involves context-aware prompt construction followed by supervised fine-tuning.

Prompt Construction. We structure the input prompt to provide the LLM with comprehensive contextual guidance. The prompt x is composed of four segments:

• Instruction: A system-level directive defining the task of personalized review generation.
• Query Image: The visual features of the user-provided image vq.
• Product Context: The top-K reviews retrieved from the Product Database, providing factual grounding.
• User Context: The top-K reviews from the User Database, serving as stylistic references.

This structured prompt acts as a "persona blueprint," enabling the model to align its output with the user's voice while maintaining factual relevance to the image.

Supervised Fine-tuning (SFT). We first optimize the LLM (πθ) using standard supervised fine-tuning to adapt it to the generation format. The objective is to maximize the likelihood of the ground-truth review y given the constructed prompt x:

where L is the length of the target review sequence. This stage equips the model with the foundational capability to integrate visual cues and retrieved context into coherent text, setting the stage for the subsequent preference optimization.

3.3 Tri-Level Negative Preference Alignment

Standard preference alignment methods (e.g., DPO) typically rely on binary pairs (winner vs. loser), treating all nonpreferred responses uniformly. However, in personalized review generation, a response can fail in multiple ways: it might be visually hallucinated, stylistically mismatched, or completely irrelevant. To capture these fine-grained distinctions, we propose a Tri-Level Negative Sampling Strategy.

Hierarchical Negative Sampling

As illustrated in the right part of Figure 1, we construct a curriculum of negative samples (ynn, yns, yng) for each preferred response yp. This hierarchy forces the model to learn boundaries at varying levels of difficulty:

• Near Negatives (ynn, Hard): Constructed by applying minor factual/attribute corruptions to the preferred review (e.g., swapping a color/size/material attribute). These are the most challenging samples, teaching the model to learn fine-grained decision boundaries.
• Similar Negatives (yns, Medium): Sampled from reviews of the same product but written by different users. These provide correct visual context but incorrect user personas, forcing the model to distinguish between "what to say" and "how to say it."
• General Negatives (yng, Easy): Sampled from unrelated products and users. These serve as obvious baselines for relevance and coherence.

By contrasting yp against this spectrum of negatives, the model progressively refines its understanding from coarsegrained relevance (vs. yng) to fine-grained personalization (vs. ynn).

Tri-DPO Optimization Objective

We formulate the alignment objective to simultaneously penalize all three levels of negatives while maximizing the margin for the preferred response. Let πθ be the policy model and πref be the reference model. We extend the standard DPO loss to a multi-negative setting:

where β controls the strength of the KL divergence constraint. This formulation explicitly encourages the model to assign higher probability mass to the authentic review yp compared to the average log-probability of the negative set.

To ensure generation stability and prevent catastrophic forgetting of linguistic fluency, we integrate a regularization term based on the standard SFT loss. The final hybrid objective is defined as:Lall = Ldpo + λLsft, (8)

where λ balances preference alignment with language modeling fundamentals. This joint optimization ensures that Snap2Review not only aligns with complex user preferences but also maintains high coherence and visual fidelity.

4 Experiment

4.1 Experimental Settings

Dataset

We evaluate our proposed framework using two publicly available, real-world datasets from the Amazon review collection 1 : Toys and Games, and Arts dataset. Each data instance consists of a user ID, product ID, review text, and review images. To establish a chronological context for personalization, a user's/product's historical reviews are defined as all reviews with timestamps preceding the timestamp of the review currently being generated. Furthermore, we standardize the review length to L = 200 tokens, discarding reviews containing fewer than 30 words to mitigate the impact of overly short or uninformative reviews. The resulting datasets are then randomly partitioned into training (80%), validation (10%), and test (10%) sets.

Baselines

To rigorously evaluate the effectiveness of our proposed Snap2Review framework, we compare it against several strong baseline methods. First, we select representative strong Multimodal LLMs to assess the performance of general-purpose visual understanding and generation in this specific domain. The baselines include Qwen2.5-VL-3B, Gemma-3-4B, and InternVL3 5-4B, which are leading opensource models, as well as GPT-4o, representing the current state-of-the-art in commercial systems. Second, to verify the contribution of our specific modules, we formulate two ablated variants of Snap2Review. First, we evaluate w/o Retriever FT, where the domain-specific retriever is replaced with a standard off-the-shelf CLIP model. Second, we test w/o Tri-DPO (SFT only), a variant that removes the final preference alignment stage and relies solely on Supervised Fine-Tuning.

1 https://amazon-reviews-2023.github.io/

Hyper-parameters Settings

During the retrieval stage, we select the top N = 5 most relevant entries from the user-specific database. For the generation backbone, we employ Qwen2.5-VL-3B, a lightweight LLM chosen to ensure computational efficiency and facilitate practical deployment in real-world e-commerce scenarios. For fine-tuning, we use the AdamW optimizer with a learning rate of 2e-5, a batch size of 2, and train for a maximum of 2 epochs, selecting the best checkpoint based on performance on the validation set. For BERTScore calculation, we use the "best-base-uncased" model. The Student t-test is applied in our experiments to test the statistical significance.

Evaluation Metrics

We employ two complementary sets of metrics to assess generation quality comprehensively. First, to evaluate lexical overlap and fluency, we report the F1 scores of ROUGE1, ROUGE-2, and ROUGE-L [Lin, 2004]. Following established protocols in review generation [Tang et al., 2016; Li et al., 2019], these metrics measure the n-gram recall between generated content and ground-truth references. Second, to assess semantic fidelity beyond surface-level phrasing, we utilize BERTScore. By computing token-level cosine similarity, BERTScore captures the deep semantic alignment between the candidate and reference reviews, ensuring a robust evaluation even when wording differs.

4.2 Overall Performance Comparison

This section presents a comparative evaluation of our model against baselines on two datasets. The results are in Table 1, where P, R, and F1 denote Precision, Recall, and F1-score, respectively. And we could have the following observations.

Snap2Review consistently outperforms all general-purpose MLLM baselines (e.g., Qwen, InternVL, GPT) by a substantial margin. Notably, while strong baselines like GPT achieve relatively high BertScores, their ROUGE scores remain significantly lower compared to our method. This discrepancy reveals a critical insight: general MLLMs can capture the broad semantic meaning of the product (reflected in decent BertScores), but they struggle to mimic the user's specific lexical patterns and phrasing (reflected in low ROUGE). They tend to generate generic, "AI-like" descriptions that are semantically correct but stylistically impersonal. In contrast, Snap2Review achieves high scores in both metrics, indicating that it not only understands the visual content but also accurately replicates the authentic voice of the user.

Comparing the full Snap2Review model with its variants reveals the distinct role of each module in the generation pipeline. The variant Snap2Rev-MR (w/o retriever fine-tuning) exhibits the lowest BertScore among our variants, indicating that without domain-specific alignment, the retriever fetches less relevant evidence, directly degrading the semantic fidelity of the generated content. Conversely, Snap2Rev-DPO (w/o Tri-DPO) suffers a more pronounced drop in ROUGE compared to its BertScore. This trend offers a critical insight: while the SFT baseline (Snap2Rev-DPO) can maintain general semantic relevance (decent BertScore), it fails to mimic the user's specific lexical choices and phrasing habits(reflected in lower ROUGE). The full framework achieves the best overall performance, validating the synergy between robust visual grounding and fine-grained stylistic alignment

4.3 Performance of CLIP finetuning

To evaluate the effectiveness of our retrieval optimization strategies, we compare the performance of the original Multimodal Retriever (MR) against variants with Full Fine-tuning (MR+FT) and our proposed Asymmetric Parameter Update (MR+AFT). The results are shown in Table 2.

Specifically, the off-the-shelf MR yields suboptimal results due to the domain shift between general pre-training and ecommerce data. Comparing the fine-tuning strategies, the superior performance of MR+AFT over MR+FT reveals that user-provided images often contain significant visual noise (e.g., clutter, poor lighting) that can distort the embedding space during full fine-tuning. By freezing the text encoder as a robust semantic anchor, our asymmetric strategy forces the vision encoder to selectively extract meaningful features consistent with textual descriptors, effectively filtering out visual ambiguities. This precise visual-textual alignment is pivotal, as it prevents the retrieval of irrelevant evidence caused by visual ambiguity, thereby safeguarding the generator against hallucinations and ensuring the final reviews are strictly grounded in visual reality.

4.4 Influence of hyperparameters

Impact of Tri-DPO Training Steps

In this section, we verify the sensitivity of our model to the number of training steps during the Tri-DPO optimization phase. Figure 2 illustrates the performance trends across ROUGE and Similarity metrics under varying step counts.

As observed, before reaching 3200 steps, the consistent gain indicates that the model progressively masters the hierarchical curriculum, first securing basic relevance against general negative responses and subsequently resolving the finer stylistic boundaries presented by near-negative responses. However, the decline observed beyond this point points to the phenomenon of reward over-optimization. Excessive gradient updates cause the policy to drift significantly from the reference model, leading to high KL divergence where the model exploits statistical irregularities rather than learning generalizable patterns. This results in a loss of linguistic fluency, effectively degrading test performance. Consequently, we identify 3200 steps as the optimal equilibrium, effectively balancing aggressive stylistic alignment with the preservation of natural language capabilities.

Impact of Loss Balancing Weight

We further investigate the impact of the balancing weight λ, which regulates the critical trade-off between stylistic preference alignment and foundational language capabilities. Figure 3 illustrates the performance trends across various weight configurations.

As observed, the model achieves optimal performance when the two objectives are balanced equally (λ = 1). When the weight is skewed towards preference alignment (lower λ), the reduced regularization fails to constrain the model within a plausible language manifold, causing generation instability. Conversely, when the weight heavily favors supervised learning (higher λ), we observe a notable performance degradation. This suggests that an overly strong constraint on generic token likelihood suppresses the subtle personalization signals essential for mimicking user personas. Therefore, an equal weighting provides the ideal stability, allowing the model to internalize distinct stylistic traits without compromising general fluency.

4.5 Case studies

To intuitively verify the contribution of our alignment strategy, we present a qualitative comparison in Figure 4 between the full Snap2Rev model and its ablation variant Qwen+RAG.

In the first case (top row), Qwen+RAG adopts a generic, promotional tone (highlighted in blue), utilizing phrases like "thoroughly impressed" and "highly recommend" that resemble a marketing script. In contrast, Snap2Rev captures the authentic voice of a user (highlighted in red), making specific comparisons ("more than the name brand variety") and offering personal aesthetic judgments ("just look more like flowers"), reflecting a genuine consumer perspective. The second case (bottom row) highlights a critical difference in fine-grained visual grounding. Qwen+RAG suffers from visual hallucination, incorrectly stating that "The doll's hair is painted on." Conversely, Snap2Rev accurately identifies the specific material, describing the hair as "mohair and very soft." These examples demonstrate that while RAG ensures factual relevance, Tri-DPO is the key driver that steers the model away from generic descriptions towards truly personalized, human-like expression.

5 Conclusion

In this paper, we present Snap2Review, a visually driven framework designed to bridge the gap between objective visual signals and subjective user personas in automated review generation. To tackle the persistent challenges of visual hallucination and stylistic homogenization, we decomposed the task into two synergistic stages. First, we established a visualanchored retrieval mechanism, optimizing a cross-modal retriever to map user images to precise textual evidence, thereby ensuring a strict grounding of the generation in visual reality. Second, to capture the nuances of individual writing styles, we introduced Tri-DPO, a novel alignment algorithm leveraging a stratified negative sampling strategy. By learning to distinguish between obvious errors, stylistic mismatches, and near-authentic negatives, our model effectively moves beyond generic praise to mimic authentic user voices. Extensive experiments demonstrate that Snap2Review sets a new state-ofthe-art, generating reviews that are both visually accurate and highly personalized. In future work, we plan to extend this paradigm to interactive dialogue systems and explore its generalization across diverse e-commerce categories.

Acknowledgements

This work was supported by the Outstanding Youth Team Project of Central Universities (No. QNTD202504), the National Natural Science Foundation of China (Grant Nos. 62502036, 62572058, and U24A20244), the China Postdoctoral Science Foundation (Grant No. 2024M750199), and the Fundamental Research Funds for the Central Universities (No. PTJS202635, No. BFUKF202616).