Understanding Visual Detail Hallucinations of Large Vision-Language Models

1 Introduction

In recent years, large language models (LLMs) [Achiam et al., 2023; Touvron et al., 2023] achieve significant breakthroughs in natural language processing, excelling in tasks such as language understanding [Hendrycks et al., 2020] and generation [Fernandes et al., 2023]. These advancements also propel the emergence of large vision-language models (LVLMs) [Liu et al., 2024a; Achiam et al., 2023;

∗ Corresponding authors: Huishuai Zhang and Dongyan Zhao.

Caffagni et al., 2024], which further enhance performance on multimodal tasks, e.g., image captioning [Agrawal et al., 2019] and visual question answering [Goyal et al., 2017].

However, hallucinations in LVLMs, where generated outputs deviate from the input images, pose significant challenges to their deployment in critical domains such as healthcare and autonomous driving [Li et al., 2023b; Bai et al., 2024]. Addressing these hallucinations is essential for ensuring the reliability of multimodal models. First and foremost, establishing a reliable evaluation framework is the top priority. Existing benchmarks, such as CHAIR [Rohrbach et al., 2018], an early framework for assessing object hallucinations in image captioning tasks, and POPE [Li et al., 2023b] that frames hallucination evaluation as a binary classification task, primarily assess general object understanding. However, these benchmarks inadequately capture LVLMs' capabilities in scenarios involving small objects, which are critical for applications like video surveillance [Zhu et al., 2021] and autonomous driving [Pang et al., 2020]. Our preliminary experiments in Section 4 reveal that low hallucination rates for regular objects do not necessarily correlate with low hallucination rates for small objects, underscoring a critical gap in current evaluation methodologies.

To address this gap, we curate a specific dataset for evaluating and analyzing visual detail hallucinations. The dataset categorizes scenes into two types: detailed and regular, and includes three key hallucination types: existence, color, and position. This dataset enables a comprehensive analysis of visual detail hallucinations and supports meaningful comparisons between detailed and regular scenes.

Using this dataset, we perform extensive experiments on 11 state-of-the-art models, yielding several notable observations. First, the use of a mixture of visual encoders proves to be highly effective, consistently outperforming singleencoder approaches. Second, increasing the resolution of input images significantly enhances small-object recognition for models capable of processing images at varying resolutions. Third, the Chain of Thought (CoT) prompting method substantially improves performance on complex visual tasks, such as determining positional relationships. Additionally, we evaluate another training-free method, Scaffold [Lei et al., 2024], which adds anchor positions to input images. The results show mixed performance across the three hallucination types.

To gain deeper insights, we investigate the role of individual visual encoders within the mixture used by the EAGLEX5 [Shi et al., 2024] model, focusing on their impact on small-object recognition. Our findings reveal that the ConvNeXt encoder predominantly determines performance for existence hallucination questions. For other hallucination types, the model effectively integrates features from multiple visual encoders, leading to consistent improvements compared to using a single encoder. These results highlight the complementary strengths of different encoders in addressing various hallucination challenges.

In summary, our key contributions are as follows:

• A Specialized Dataset: We curate a dataset for visual detail hallucination assessment, encompassing two scene types and three hallucination categories, enabling robust evaluation.
• Model and Method Analysis: We evaluate multiple models and methods, revealing that most models struggle with detailed visual information. We show that a combination of visual experts and larger resolutions mitigate visual detail hallucinations.
• Analytical Insights: We investigate other design factors influencing visual detail hallucinations, including the contributions of various visual encoders in mixed architectures, the impact of image resizing, and the role of training data distribution, offering valuable guidance for future research.

This work establishes a pipeline for curating a dataset specifically to evaluate visual detail hallucinations in large visionlanguage models (LVLMs). We assess the performance of current LVLMs on this dataset and systematically evaluate several improvement strategies, providing in-depth analyses of their effectiveness. Additionally, we investigate underlying factors contributing to visual detail hallucination issues, offering valuable insights and identifying key directions for future research and efforts to enhance LVLM performance on small-object recognitions.

2 RelatedWork

This work focuses on evaluating the visual detail hallucination of large vision-language models. For clarity, we introduce the related work with the following three categories.

2.1 Large Vision-Language Models

Large Vision-Language Models(LVLMs) typically comprise three key components: an LLM backbone for user interaction, one or more visual encoders, and vision-to-language adapter modules [Caffagni et al., 2024]. The visual encoder is critical for perceiving visual information. Early LVLMs [Li et al., 2023a; Liu et al., 2024b] commonly use CLIP [Radford et al., 2021] as the visual encoder. For example, LLaVA [Liu et al., 2024b] projects CLIP-encoded visual features into the text space through linear projection or MLP, integrating them with textual embeddings for further processing. Subsequent research has explored LVLMs with a hybrid visual encoder structure[Shi et al., 2024; Lin et al., 2023; Fan et al., 2024], demonstrating that combining multiple visual encoders significantly improves the model's visual capabilities. However, these structures are typically limited to processing images at fixed resolutions. Recently, some studies [Wang et al., 2024; Wu et al., 2024] have enabled models to handle arbitrary resolutions. For instance, Qwen2-VL [Wang et al., 2024] enhances Visual Transformer(ViT) [Dosovitskiy, 2020] by replacing absolute position embeddings with 2D-RoPE, enabling the model to effectively capture the two-dimensional positional information of images. Overall, models with different structures exhibit varying capabilities in perceiving visual details. The effectiveness of methods for mitigating visual detail hallucinations depends on the model's structure. In this work, we conduct detailed experiments and analyses to explore these differences.

2.2 Hallucination Benchmarks for LVLMs

In vision-language models, hallucinations [Bai et al., 2024] refer to scenes where the generated output does not align with the input image. Existing research primarily focuses on object hallucinations, and so do the evaluation benchmarks. CHAIR [Rohrbach et al., 2018], an early work predating the advent of large models, evaluates object hallucinations in image captioning tasks. It evaluates how many generated words correspond to objects present in the image by leveraging ground truth sentences and object segmentations within a fixed set of objects. However, CHAIR's applicability is confined to traditional image captioning tasks with predefined object sets, making it unsuitable for assessing the diverse outputs of contemporary vision-language models. More recently, POPE [Li et al., 2023b] has emerged as a widely used benchmark for evaluating object hallucinations. POPE transforms hallucination evaluation into a binary classification task by asking yes-or-no questions about relevant objects. While POPE is reliable and flexible for evaluating existence hallucinations, it narrowly focuses on this specific type of hallucination, limiting its broader applicability. Additionally, several other benchmarks [Lovenia et al., 2023; Liu et al., 2023a] have been introduced to provide a more structured evaluation of hallucinations in multimodal models. In comparison, we focus on visual detail hallucinations and propose a pipeline for constructing related datasets and evaluation methods.

3 Evaluation Dataset Construction

In this section, we introduce our curated dataset, which evaluates hallucinations across three categories, further divided into small and regular objects. The small object dataset evaluates the model's ability to handle visual detail hallucinations, while the regular object dataset assesses hallucinations in standard scenarios. By comparing performance across both datasets, we quantify the model's performance degradation when handling small objects. We begin by defining the problem and categorizing hallucinations. Next, we describe the methods for data collection and filtering.

3.1 Task Definition and Hallucination Categories

We define Small Objects based on the small object detection task [Cheng et al., 2023] in computer vision to identify visual details. Specifically, objects with a bounding box area smaller than 1024 pixels or occupying less than 2% of the total image area are classified as Small (S), while all others are categorized as Regular (R). The classification of visual detail hallucinations aligns with that of regular object hallucinations. Since POPE [Li et al., 2023b] only evaluates existence hallucinations and does not assess other categories, we expand the categorization of object hallucinations into three types:

• Existence: Misperceptions about the existence of an object, such as describing an object that does not exist.
• Color: Misperceptions about the primary color of an object.
• Position: Misperceptions about the relative positioning of two objects.

3.2 Data Collection

Data Annotation Inspired by the previous work [Li et al., 2023b; Rohrbach et al., 2018], we choose MSCOCO [Lin et al., 2014] as the source for our dataset. Since the dataset lacks annotations for color and positional relationships, we generate this information separately. For the color category, we manually annotate the data by using bounding boxes to locate selected objects and label their primary colors. For positional relationships, we derive the information from the bounding boxes of the objects, specifically by analyzing the relative positions of their center points. The positional relationships are categorized as: ["bottom right," "top right," "bottom left," "top left"].

Small Object Data Filtering Our evaluation dataset is divided into two parts based on object size: small objects and regular objects. Additionally, it is categorized into three types of hallucinations. For each type, the number of data points is balanced between the two object categories, with the distinction based on whether filtering is applied. Different filtering methods are employed for each type of hallucination:

• Existence: All detected bounding boxes for each object are recorded. If the largest bounding box area falls below a given threshold, the image is classified as containing small objects.
• Color: An object is classified as a small object only if it has a unique bounding box and its area is below the specified threshold.
• Position: All unique objects in an image are paired, and at least one pair must include a small object for the combination to be considered valid.

Each small object must have a unique standalone bounding box (Color category), and only non-duplicate objects in the image will be paired (Position category). The validation set of MS COCO2014 contains fewer than 20 images with a total pixel count below 51,200 (1,024 pixels occupy 2% of the total 51,200 pixels). A bounding box area smaller than 1,024 pixels typically occupies less than 2% of the image area, reducing computational costs during filtering. We thus adopt the bounding box area as the filtering criterion. Through this filtering process, we obtain an annotated dataset of small object images and generate an unfiltered dataset of the same size for regular objects. Using these datasets, we create yes-or-no questions from the existing images to evaluate hallucinations.

Question Generation We employ a template-based approach to generate questions for each image by incorporating the corresponding annotated information: object name, color, and the positional relationship between two objects.

For positive examples, we directly use the correct information. To generate negative examples, we follow a method similar to POPE's [Li et al., 2023b] negative example generation. First, we collect all object names (or colors, or positional relationships) from the positive examples. Then, for each instance, we randomly or adversarially select an object name (or color, or positional relationship) from this collection that differs from the instance to create a negative example. The templates we used are as follows:

Existence: Is there {article} {object} in the image? Color: Is the main color of the {object} in the image {color}? Position: Is {object1} to the {direction} of {object0} in the image?

We create the evaluation dataset using this generation method. The statistical details for the data are summarized in Table 3. The data examples can be found in Appendix A.3.

Metrics We evaluate the model using four metrics: Accuracy, Precision, Recall, and F1 Score. Additionally, we calculate the total accuracy across the three hallucination categories, with separate scores for regular and small objects. The difference between these two scores highlights the performance degradation when processing visual details.

4 Evaluation of Models andMethods

In this section, we first evaluate the performance of 11 stateof-the-art LVLMs on the annotated dataset, leading to several notable observations. Next, we examine the effectiveness of three training-free data augmentation methods in mitigating hallucinations.

4.1 Evaluation of LVLMs on Visual Detail Hallucination

We evaluate several LVLMs, including LLaVA-1.5 [Liu et al., 2024a], DeepSeek-VL [Lu et al., 2024; Wu et al., 2024], Qwen-VL [Bai et al., 2023; Wang et al., 2024], CogVLM2 [Hong et al., 2024], SPHINX [Lin et al., 2023], Eagle-X5 [Shi et al., 2024], and GPT-4o [Achiam et al., 2023]. These models exhibit diverse architectural designs. Specifically, LLaVA, DeepSeek-VL, Qwen-VL, and CogVLM2 employ CLIP or its variants as visual encoders, which map images to a fixed resolution for processing. Sphinx and Eagle utilize a mixture of visual encoders with varying architectures. Deepseek-VL2 and Qwen2-VL adopt distinct approaches to enable models to process images of any resolution. Qwen-VL-MAX and GPT-4o are closed-source commercial models. We assess these models on our dataset to evaluate their performance in answering questions about visual detail hallucination. The results are shown in Table 1.

Our initial observations reveal that all models, including GPT-4o, experience varying degrees of performance degradation when transitioning from regular object scenes to those with smaller objects. Moreover, a model's ability to perceive small objects does not consistently align with its performance on normal scenes. For example, CogVLM excels in recognizing regular objects, outperforming Eagle in the existence and color categories. However, its performance on small objects declines significantly, particularly in the existence category, where it underperforms relative to other models. This discrepancy suggests that improving a model's performance on regular objects does not necessarily lead to enhanced performance on visual details, highlighting the importance of developing datasets specifically designed to evaluate visual detailrelated hallucinations. Furthermore, the performance of the same model varies across different categories, underscoring the necessity of categorical evaluation.

Among these models, Sphinx and Eagle, which employ a mixture of visual encoders, demonstrate strong performance on both small and regular objects. Eagle achieves the highest total accuracy score on the small object evaluation set. This success is largely attributed to the fact that most models typically rely on a single ViT-based visual encoder. While this approach excels at aggregating long-range interactions due to its training process, it often struggles to capture neighboring dependencies [Lin et al., 2023]. The mixed visual encoder partially addresses this limitation. We will provide further analysis about it in Section 5.1. Additionally, DeepseekVL2 and Qwen2-VL outperform models that can only handle fixed resolutions, as they reduce information loss when highresolution images are mapped to a fixed resolution.

4.2 Evaluation of Training-Free Methods for Hallucination Mitigation

We evaluate several training-free methods across different models. Scaffold [Lei et al., 2024] is a visual prompt method that overlays a dot matrix within the image to serve as visual information anchors, utilizing multidimensional coordinates as textual positional references. Chain of Thought(CoT) [Wei et al., 2023] typically prompts models to generate the reasoning process before outputting the final answer. In this task, we specifically prompt models to utilize bounding boxes of objects mentioned in the question as intermediate information for generating the reasoning process. For the color category, the model first detects the bounding box of the object and then determines whether the color within the bounding box is correct. For the position category, the model detects the bounding boxes of two objects, calculates their center points, and judges the positional relationship based on the relative positions of these points. For the existence category, detecting bounding boxes is more challenging than directly determining existence, so we did not conduct experiments in this category. The prompts we used can be found in Appendix A.1. Image Resizing is a straightforward approach to addressing small object hallucinations. We use bicubic interpolation to resize the images by a factor of 2 before evaluation. We conduct experiments on three methods using three distinct models: LLaVA, which employs a standard visual encoder; Qwen2-VL, which handles images of any resolution; and Eagle, which integrates multiple visual experts. The experimental results are summarized in Table 2.

The Scaffold method enhances performance in specific models and hallucination categories compared to the baseline, particularly in regular object scenes. However, its effectiveness is limited for small objects, often resulting in performance degradation. This decline occurs because adding visual prompts directly to an image has little impact on regular objects but can obscure small objects, making them harder for the model to perceive.

For Qwen2-VL, which handles images of any resolution, upsampling provides additional visual details, achieving the best performance across all three hallucination categories. The image resizing method also enhances Eagle's accuracy in answering existence and positional relationship hallucination questions. This improvement stems from Eagle's mix of visual experts, which includes models like SAM [Kirillov et al., 2023] that can process higher-resolution images. In contrast, the LLaVA model exhibits suboptimal performance with the image resizing method. This is primarily due to the visual encoder's preprocessing step, which resizes images to a fixed resolution. Upsampling the image before this resizing does not change the final resolution but degrade image quality, resulting in a lower-quality representation compared to the original.

The improvement from the CoT method is more limited across all models, partly because smaller models have less advanced language capabilities and struggle to comprehend complex instructions. To explore this, we test the CoT method on Qwen-VL-MAX, a commercial model with stronger language capabilities. The results in Table 4 reveal that the CoT method significantly improves performance in answering positional relationship questions on Qwen-VLMAX. However, it provides only minor enhancements for color-related questions and even exhibits a slight performance decline when addressing small object issues. We analyze the outputs and derive the following insights. First, Qwen-VLMAX performs poorly in object existence perception, with over 40% of responses in the color category stating 'can't detect the object.' This significantly degrades performance on small objects. Furthermore, for data points where the model with CoT answers "can't detect the object", it answers "no" 95% of the time without CoT, indicating that object detection capability directly impacts color perception. Additionally, we find that CoT with bounding box as intermediate information highly relies on the models' ability to perceive object existence. Overall, the CoT method does not directly enhance the model's visual perception capabilities. Second, the primary advantage of CoT lies in its ability to infer information beyond direct visual perception. For example, we observe a case where the model struggles to detect the bounding box of a very small phone. However, by identifying the position of the person holding the phone, it infers the likely location of the phone and correctly answers the question. This example can be found in Appendix A.2. In summary, the image resizing method mitigates hallucinations by providing the model with more detailed visual information, whereas the CoT method leverages the model's reasoning ability to infer additional information for hallucination mitigation.

5 Analysis of Design Factors

In this section, we analyze the architecture of mixed visual encoders and examine the distribution of object area across different training stages using LLaVA-1.5 as an example, to infer their potential effect on hallucinations related to small objects. Then, we analyze how the resize factor affects models that can handle images of any resolution.

5.1 Influence of Mixed Visual Encoders

In Section 4, we observe that models with mixed visual experts generate fewer hallucinations. To further explore this phenomenon, we conduct training-free experiments using the Eagle-X5-13B-Chat model, which exhibits superior performance in Table 1. This model processes visual information through a mixture of vision encoders combined via channel concatenation and utilizes a three-stage training recipe including vision expert pre-alignment. Firstly, we evaluate Eagle's performance using features from a single visual encoder across all three hallucination categories. Secondly, we combine information from multiple encoders and analyze their individual contributions to solving specific problems. To assess the impact of each encoder, we selectively mask its encoded features, disabling its contribution both individually and in combination. We test five visual encoders: CLIP, ConvNeXt [Woo et al., 2023], SAM [Kirillov et al., 2023], Pix2Struct [Lee et al., 2023], and EVA-02-L [Fang et al., 2024]. While ConvNeXt is based on a convolutional network, the others utilize the ViT architecture and are trained with distinct processes or for specific tasks.

From Table 6, we observe that task-specific models like SAM (segmentation) and Pix2Struct (OCR) perform poorly when relying solely on features from a single visual encoder for existence hallucination problems. While CLIP's encoding outperforms these two, it still underperforms compared to models trained exclusively with CLIP, such as LLaVAv1.5. In contrast, ConvNeXt's encoding yields the best results for existence hallucinations, even surpassing some models listed in Table 1. Moreover, augmenting ConvNeXt's encoding with other encoders' information does not substantially improve performance. This suggests that the model relies mainly on ConvNeXt's information and treats other encoder data as auxiliary. This also explains why performance drops more significantly when using only CLIP's encoding, compared to models trained with CLIP, as the model treats CLIP's information as secondary. For other types of hallucination problems, ConvNeXt's encoding still provides an advantage, though the effect is less pronounced compared to existence hallucinations. Incorporating information from encoders such as CLIP and EVA-02-L results in significant improvements, indicating their substantial contribution to solving these problems.

The results reveal several interesting insights. When multiple visual encoders process an image, each encoder generates distinct encoded features due to variations in architecture, training procedures, and objectives. These encoders can be likened to an LVLM equipped with multiple "eyes", akin to human vision, where each eye may perceive the same scene differently. Similar to the human brain, which processes visual information asymmetrically from both eyes [Khan and Crawford, 2001], a visual language model exhibits a "dominant" encoder after joint training. In such cases, the model prioritizes information from one encoder, analogous to the dominant eye, while treating input from the other encoders as supplementary. This structure shows potential for reducing small object hallucinations, suggesting that targeted training could enable the model to leverage different visual inputs for small object detection while using others for general scenes.

5.2 Small Object Distribution in Training Data

We analyze the distribution of small objects in its training data. The LLaVA-1.5 training process involves two main stages: pre-training for modality alignment and visual instruction tuning. The pre-training utilizes a 558K subset of the LAION-CC-SBU dataset with BLIP captions, while the instruction tuning phase incorporates from COCO2017, GQA [Hudson and Manning, 2019], OCR-VQA [Mishra et al., 2019], TextVQA [Sidorov et al., 2020], and VisualGenome [Krishna et al., 2017]. Since OCR-VQA and TextVQA focus on text within images, we exclude them from our analysis.

We extract the objects from the training data using part-ofspeech tagging methods and apply Grounding DINO [Liu et al., 2023b] to obtain their bounding boxes. We then analyze the distribution of bounding box areas for all mentioned objects. The results in Figure 1 indicate that the probability density of small objects is relatively high in both pre-training and instruction tuning data. Therefore, the proportion of small objects in the training data does not appear to be the primary cause of the model's tendency to hallucinate small objects.

5.3 Influence of Resizing Factors

Furthermore, we explore the influence of different resize factors on accuracy. Specifically, we set 16 factors ranging from

0.5 to 2.0 incremented by 0.1. We resize the image with a factor r: when r > 1, the image is upsampled with bicubic interpolation, and when r < 1, the image is downsampled with Lanczos interpolation. While LLaVA-1.5 preprocesses images by resizing them to a fixed resolution, we explore the use of Qwen2-VL here, which can handle images of any resolution, to provide a more flexible analysis.

The experimental results are summarized in Figure 2. For Existence category (Figure 2a), a clear positive correlation between the resize factor and accuracy is observed, particularly for small objects. As the image size increases, accuracy consistently improves, indicating that upscaling enhances the model's ability to detect small objects. In contrast, for regular objects, accuracy remains stable, suggesting that the resize factor has little effect on their detection. For Color category (Figure 2b), no significant correlation between resize factor and accuracy is observed for either small or regular objects. The performance fluctuates without a clear trend, implying that color recognition is less sensitive to changes in image size. For Position category (Figure 2c), accuracy improves as the resize factor increases, particularly for small objects. This trend highlights the importance of upscaling in enhancing spatial reasoning for small objects, whereas the performance for regular objects remains consistent.

In summary, the resize factor significantly affects the existence and position categories, particularly for small objects, but has minimal impact on the color category.

6 Conclusion

In this work, we curate a dataset to evaluate and analyze visual detail hallucinations, filtering data based on automatic or manual annotations for corresponding categories. We evaluate multiple models and methods, revealing that architectures like the mixture of visual experts and visual encoders capable of handling images of any resolution effectively mitigate these hallucinations. Additionally, the CoT and Image Resizing methods exhibit strong potential in mitigating visual detail hallucinations. Through an ablation study on model architecture, we find that ConvNeXt architecture is critical for object existence recognition tasks, and integrating information from multiple visual encoders is significantly more effective than relying on a single encoder for mitigating other types of hallucinations.