Multimodal Retina Image Analysis Survey: Datasets, Tasks and Methods

1 Introduction

The retina is a light-sensitive layer of tissue in the back of the eye that sends visual signals to the brain. As the only human tissue allowing direct, noninvasive visualization of the central nervous system and microvascular circulation, retina images play a vital role in clinical applications. Changes in the retina show signs of not only ophthalmic diseases, such as Diabetic Retinopathy (DR), Age-related Macular Degeneration (AMD), and GLaucoma (GL), but also systemic diseases, such as Alzheimer's disease (AD), cardiovascular diseases, and neurological diseases.

In practice, multiple imaging techniques are developed to examine various views of the retina. For example, Color Fundus photography (CF) provides views of the retina surface, Optical Coherence Tomography (OCT) delivers highresolution cross-sectional images of retina layers, and Optical Coherence Tomography Angiography (OCTA) visualizes microvascular networks without the need for invasive contrast agents. By integrating these techniques, clinicians obtain detailed images of anatomical and vascular structures for precise diagnosis and early disease intervention.

With the increasing availability of retina datasets [Hassan et al., 2022; Khan et al., 2023; Li et al., 2024], deep learning techniques [Liu and Yu, 2021; Wu et al., 2021; Xie et al., 2024; Chen et al., 2024] have proven effective in retina images analysis tasks such as disease classification, cross-modal registration, and lesion segmentation in clinical applications. As demonstrated in Fig. 1, we present the publication trend of multimodal deep learning research in retinal image analysis, along with the increasing proportion of such studies relative to the overall retinal imaging research publications. The adoption of multimodal deep learning algorithms in ophthalmology follows an increasing trajectory, reflecting its growing clinical promise.

In this paper, we revisit the multimodal retina datasets, tasks, and corresponding methods and discuss the limitations and challenges in retina image analysis. To be specific, we first categorize publicly available datasets based on their imaging modalities and analyze their characteristics. We then review the primary multimodal retina image analysis tasks, including classification, grading, registration, segmentation, image generation, and medical report generation, highlighting the associated datasets, evaluation metrics, methods, and the benchmarks established for each task. In addition, we further highlight some critical issues regarding the existing datasets and benchmarks, such as dataset overfitting by existing methods, inconsistent dataset splits, and incompetent evaluation of different modalities.

The remainder of this paper is organized as follows. Section 2 provides a revisit of existing datasets and the corresponding retina imaging modalities. In Section 3, we outline the tasks for retina image analysis and present the used dataset, evaluation metrics, and benchmarks. Section 4 emphasizes the challenges in the current research and explores future research directions. In the end, we conclude the survey with a summary of our findings and a call to action for further research in Section 5.

2 Multimodal Retina Image Datasets

Multimodal retina image datasets represent incredibly valuable resources for both the development and evaluation of deep learning models. Through the seamless integration of various imaging modalities, these datasets enable a wide array of fundamental yet diverse computational tasks, including disease classification, lesion segmentation, and vascular analysis. Such datasets exhibit significant diversity in terms of modality combinations, dataset size, annotation quality, and accessibility. In this context, we first present a systematic review of multimodal retina image datasets, focusing on their applications and support for various deep learning tasks.

2.1 Overview of Datasets

During our paper collection, we identified 17 publicly available datasets. These datasets cover a variety of retina imaging modalities, with CF and OCT being the most common.

Specifically, 13 datasets include CF images, while 11 utilize OCT. FA presents in 5 datasets, and OCTA appears in 2 datasets. In addition to these widely used modalities, other imaging techniques are represented in individual datasets: histology images (HI) in Project Macula, nearinfrared (NIR) images in OLIVES, and indocyanine green angiography (ICGA) images in APTOS2023. Notably, the FPRM dataset stands out for its diverse imaging modalities, including oxygen saturation images, pupillary light reflex videos, and retina blood flow (RBF) videos. However, despite its uniqueness, this newly published dataset has seen limited citations and lacks established benchmarking data or methods for comparison, similar to the GL-Dis dataset. In this section, we categorize these datasets by their imaging modalities and analyze their key characteristics and limitations.

2.2 Datasets with CF and OCT

Eight datasets that include both CF and OCT, namely CFOCT-REG-22, CF-OCT-OSD-50, Project Macula, BiomisaARMD, GL-CF-OCT, GL-Dis, GAMMA, INSPIRE-stereo, and APTOS2024. As the most common combination, the CF and OCT provide complementary structural information about the retina. CF captures a global view of the retina surface, while OCT provides high-resolution cross-sectional imaging of retina layers. This combination is particularly useful for analyzing morphological and structural changes in the retina associated with retina diseases, including DR, AMD, and GL. Several datasets in this category support disease classification and structural analysis tasks. Notable examples include GAMMA, Biomisa-ARMD, GL-CF-OCT, GL-Dis, and Project Macula dataset. Meanwhile, CF-OCT-REG-22 and CF-OCT-OSD-50 datasets are designed for registration tasks, facilitating multimodal alignment studies. Additionally, the APTOS2024 challenge introduces an image synthesis task, where models generate OCT images from CF inputs.

2.3 Datasets with CF and FA

Three datasets contain both CF and FA modalities, namely CF-FA, PRIME-FP20, and DeepEyeNet. FA enables highcontrast visualization of retina vasculature, while CF provides complementary contextual information, capturing retina surface features, including tissue morphology. These datasets are commonly used for tasks involving vascular abnormalities. For example, PRIME-FP20 is derived from a DR study project in clinical settings to segment vasculature. The CF-FA is proposed for DR detection and become an important resource for registration and image generation. Meanwhile, the DeepEyeNet covers 265 different disease conditions along with corresponding textual descriptions, making it a valuable dataset for medical report generation.

2.4 Other Multimodal Datasets

Some datasets incorporate imaging modalities beyond CF, OCT, FA, and OCTA, introducing unique imaging techniques that enable specialized tasks. These datasets provide valuable resources for exploring novel AI applications in retina images. For example, APTOS2023 includes ICGA images, which can provide additional vascular insights beyond FA. FOCTAIR combines FA and OCTA, making it one of the few datasets emphasizing vasculature. FPRM is particularly notable for its diverse range of imaging modalities, including oxygen saturation imaging, videos of pupillary light reflex, videos of retina blood flow (RBF), textual psychological assessment results and image quality reports, enabling research in quality assessment, disease classification, and psychological assessment. These above modalities, though relatively rare, provide unique perspectives for multimodal image analysis. Their diverse imaging techniques contribute to the scope of AI applications in retina research, beyond traditional structural and vascular assessments.

Additionally, the OCTA-500 dataset integrates OCT and OCTA to support not only multi-class disease classification, but also fine-grained segmentation tasks such as capillary segmentation, artery segmentation, vein segmentation, and FAZ segmentation. OLIVES incorporates near-infrared (NIR) imaging, a modality that assists in the analysis of retina tissue properties, particularly in detecting structural alterations and fluid accumulation associated with diabetic retinopathy (DR) and diabetic macular edema (DME). Both OCTA-500 and OLIVES are widely used datasets, supporting various retina AI research applications.

3 Multimodal Retina Image Analysis Tasks

We identify five distinct categories of tasks based on their input and output characteristics. This section discusses the benchmarks, evaluation metrics, and state-of-the-art methods associated with each multimodal retina image analysis task.

3.1 Classification and Grading

ProblemSetting. retina classification tasks primarily focus on detecting ocular pathologies by analyzing disease-specific features in retina images, such as GL classification and DME Classification. Grading derives from classification as it follows a predefined ordinal scale, where class labels have a set distance between them, rather than being distinct categorical labels. It follows an ordinal classification paradigm based on clinical needs, allowing the assessment of disease progression and severity (e.g., DR grading).

Associated Datasets. There are 10 datasets supporting classification and grading tasks. Five of them form available benchmarks, providing standardized evaluation frameworks. Among these, the GAMMA dataset comes with an online challenge. DEN, Biomisa-ARMD, and OLIVES conduct baseline benchmarks.

Evaluation Metrics. As shown in the Table 3, Cohen's kappa (Kappa) is served as an evaluation metric for grading problems to amplify the distance of the error:

where Po is the observed agreement, and Pe is the expected agreement by chance. The other metrics, such as Accuracy, Sensitivity (Recall, True Positive rate), Specificity (True Negative Rate), Precision, Area Under Receiver Operating Characteristic Curve (AUROC), F1-score are all commonly used metrics in classification tasks. Biomisa-ARMD originally provides a confusion matrix of normal, early, and advanced AMD as a baseline result.

Related Methods. Yoo et al. [2019] describe itself as the first to explore the use of both fundus and OCT images in a deep learning-based classification algorithm. Their preliminary findings suggest that multimodal integration of CF and OCT may enhance diagnostic accuracy. Notably, in recent years, the metrics on the Project Macular dataset reach 100%. Similarly, Zou et al. [2024] integrate uncertainty estimation into the fusion process, leading to a 100% outcome for the OLIVES dataset in the grading task. However, their models do not achieve top performance on the GAMMA dataset.

3.2 Registration

Problem Setting. multimodal registration aligns corresponding features across images from different modalities. It enables more precise comparisons, downstream multimodal fusion, and enhanced diagnostic decision-making

Problem Setting. The CF-FA dataset is one of the most used public datasets for retina registration. Other multimodal retina datasets offering registration annotations are CF-OCTREG-22, CF-OCT-OSD-50, INSPIRE-stereo, and FOCTAIR. INSPIRE-stereo dataset is rarely cited in recent registration works. Table 4 demonstrates recent research on datasets CFFA, CF-OCT-REG-22, CF-OCT-OSD-50, and FOCTAIR.

EvaluationMetrics. Success Rate (SR) quantifies the proportion of registration cases where alignment error falls within a predefined threshold, reflecting the reliability of the registration method. SR metrics, such as SRME(n) and SRMAE(n), measure the proportion of cases where registration error remains below a given threshold. While SRME relies on mean error, SRMAE uses absolute error, both providing insights into the preservation of anatomical structures. Mean Euclidean Error (MEE) quantifies the average registration error across all points, while Maximum Euclidean Error (MAE) highlights the worst-case misalignment by considering the largest observed discrepancy. Root Mean Squared Error (RMSE) further refines error measurement by emphasizing larger deviations through squaring, making it particularly sensitive to misalignments. Unlike these direct error metrics, Regional Error Propagation (REP) and Spatial Deformation Penalty (SDP) assess the smoothness and spatial consistency of transformations, ensuring that deformations remain locally coherent. In the context of similarity-based evaluations, Zeromean Normalized Cross-Correlation (ZNCC) measures the intensity correlation between registered images, making it particularly suitable for multimodal registration tasks. Dice Coefficient (DC) and Area Under the Curve (AUC) provide an overlap-based perspective, with Dice quantifying spatial alignment of segmented structures and AUC evaluating registration performance as a classification task. The Jacobian determinant constraint |Jϕ|≤0 plays a crucial role in assessing transformation field invertibility. A negative determinant indicates non-physical folding in non-rigid deformations, highlighting structural inconsistencies that could invalidate registration outcomes.

Related Methods. Sindel et al. [2022] adopt a keypointbased vessel structure alignment method to optimize vessel matching, enhancing the alignment accuracy of multimodal retina images. Mart´ınez-R´ıo et al. [2023] propose a weakly supervised deep learning approach within a deformable registration framework. This method achieves effective multimodal retina image alignment with minimal annotation requirements. Jahnavi and Sivaswamy [2018] discuss the challenges and optimization directions in cross-modal alignment and provide a generalized multimodal retina registration pipeline. Retina IPA [Wang et al., 2024a] introduces a self-supervised layer enhanced with keypoints as constraints while refining feature extraction across modalities.

3.3 Segmentation

Problem Setting. Segmentation in retina image partitions a raw image at the pixel level to delineate clinically significant anatomical structures or pathological regions, such as lesions, vessel boundaries, or fluid cavities. The output is a labeled mask that highlights these features, aiding in disease diagnosis, staging, and monitoring.

Associated Datasets. PRIME-FP20 dataset supports the vessel segmentation task, and OCTA-500 (two subsets: 6mm and 3mm) supports Capillary Artery Vein Fovea (CAVF) segmentation tasks. GAMMA dataset also provides annotations for fovea localization task but lacks followers.

Evaluation Metrics. AUCPR, Dice Coefficient (DC), Intersection over Union (IoU), and Correctly Classified Area over Lesion (CAL) each provide a unique perspective on segmentation performance. AUCPR measures model effectiveness in imbalanced datasets by evaluating the tradeoff between precision and recall, ensuring that high recall is not achieved at the expense of excessive false positives. Dice and IoU both assess spatial overlap between predicted and ground truth segmentations, with Dice emphasizing the harmonic mean of precision and recall, while IoU penalizes over-segmentation more strongly. CAL, in contrast, focuses specifically on segmentation accuracy within lesion regions, decomposing performance into correct lesion segmentation (C), accurate boundary classification (A), and lesion localization (L). These metrics balance global accuracy with lesion-specific performance, making them particularly useful in medical image applications.

Related Methods. The work of [Ding et al., 2021] proposes a weakly-supervised approach for vessel detection in ultra-widefield fundus photography by leveraging iterative multimodal registration and learning. It exploits angiography as an auxiliary modality for supervision without requiring extensive manual annotations. The model refines vessel detection iteratively, demonstrating improved performance over standard weakly-supervised methods in challenging ultrawidefield images. The authors of OCTA-500 [Li et al., 2020; Li et al., 2024] introduce a large-scale OCTA dataset with paired en face and volumetric angiography images. The study benchmarks several deep learning models, including their proposed IPNv2 for segmentation, highlighting the potential of their dataset for advancing OCTA image understanding.

3.4 Image Generation

Problem Setting. Image generation tasks, sometimes referred to as image translation tasks, are very popular in medical images. Image generation tasks inmedical images involve creating new or enhanced images to augment existing datasets or simulate realistic variations. Given a source image or latent representation, these approaches output new medical images with desired attributes. Such generation techniques can aid in data augmentation and improve diagnostic clarity.

Associated Datasets. In the context of multimodal retina images, only the CF-FA dataset, INSPIRE-stereo dataset, and APTOS2024 dataset are accessible. INSPIRE-stereo dataset has rare visibility in the deep learning community. APTOS2024 is a new challenge held by Asia Pacific TeleOphthalmology Society (APTOS), aiming to motivate CF to OCT generation.

Evaluation Metrics. Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), Fr´echet Inception Distance (FID), and Kernel Inception Distance (KID) are key metrics for evaluating image generation quality, each capturing different aspects of fidelity and realism. SSIM measures structural similarity between images by considering luminance, contrast, and texture, making it more perceptually relevant than pixel-wise comparisons. PSNR, in contrast, quantifies reconstruction quality based on logarithmic scaling of the pixel-wise error, favoring images with less distortion but often misaligned with human perception. MSE directly computes the average squared pixel difference, serving as a simple but less perceptually aligned metric, where lower values indicate better reconstruction fidelity. For assessing realism in generated images, FID and KID go beyond pixel-level comparisons. FID measures the distance between real and generated image distributions in a deep feature space, assuming a Gaussian distribution, making it effective for capturing global image quality but sensitive to dataset size. KID, as an alternative, leverages MaximumMean Discrepancy (MMD) to compare distributions without assuming Gaussianity, providing a more robust, unbiased estimate, particularly for smaller datasets. In a nutshell, SSIM, PSNR, and MSE focus on pixel-wise and perceptual fidelity, while FID and KID assess realism in a learned feature space. Kamran et al. [2018] use validation loss and a GAN-like module to demonstrate the validity of the pseudo-angiography images. FVD is an extension of the FID metric, specifically designed to evaluate the quality of video-like sequences by measuring their distributional distance from real data in a feature space.

In the APTOS2024 challenge, the FVD metric is applied by treating these six-frame sequences as mini-videos and comparing their learned feature distributions with those of real OCT frame sequences

Related Methods. Kamran et al. [2020] proposes an adversarial framework, named Fundus2Angio, to improve the realism of generated images, with a generator learning structural and vascular mappings while a discriminator enforces fidelity. Performance evaluation suggests that the synthesized FA images retain key vascular details, supporting potential clinical applications. Their following work [Kamran et al., 2021] proposes a semi-supervised approach by integrating Vision Transformers (ViT) with Generative Adversarial Networks (GANs). The key contribution lies in leveraging ViT for feature modeling to enhance both image generation quality and downstream diagnostic performance.

3.5 Medical Report Generation

Problem Setting. The medical report generation task automates the creation of medical reports from image data, covering descriptions, diagnoses, and recommendations. It can generate structured or free-text reports with disease classification, lesion localization, and clinical guidance. The report generation task enhances diagnostic efficiency, reduces reporting workload, and improves interpretability.

Associated Datasets. The DeepOpht dataset provides a standardized collection of multimodal retina images, enabling the development and evaluation of AI models for medical image captioning and disease classification. It provides a total of 15709 images, including 1811 FA and 13898 CFP. Clinical labels, keywords and clinical descriptions covering 265 different disease conditions are further provided.

EvaluationMetrics. This benchmark primarily focuses on assessing model performance in generating structured medical reports from retina images, ensuring clinical relevance and interoperability. As shown in Table 7, the performance of AI models on the DeepOpht dataset is evaluated using a range of metrics, each serving a distinct role in assessing model effectiveness, BLEU [Papineni et al., 2002], CIDEr [Vedantam et al., 2015] and ROUGE [Lin, 2004]. These are key metrics for evaluating text generation quality, particularly in machine translation, image captioning, and summarization tasks. BLEU measures n-gram precision by comparing generated text to reference text. BLEU-1 to BLEU-4 correspond to unigram to four-gram precision, respectively, with BLEU-avg representing their averaged score. Lower-order BLEU scores

(e.g., BLEU-1) capture basic word accuracy, while higherorder scores (e.g., BLEU-4) reflect fluency and coherence but can be overly strict due to exact match requirements. CIDEr, in contrast, emphasizes human-like relevance by weighting n-grams based on their importance in a given corpus. Unlike BLEU, which treats all matches equally, CIDEr assigns higher scores to rare yet meaningful phrases, making it particularly effective for tasks like image captioning. ROUGE evaluates recall-oriented overlap, commonly used in summarization. Unlike BLEU, which prioritizes precision, ROUGE measures how much of the reference text is retained in the generated output, making it well-suited for evaluating completeness in extractive and abstractive summarization. For text generation tasks, BLEU and ROUGE appear frequently, reflecting their importance in assessing medical report quality. In the original paper of DEN, the authors also introduce a human-involved visual evaluation named DNN Visual Explanation Module. This module compares the heatmap obtained from CAM [Zhou et al., 2016] with the manually labeled lesion sketch to qualitatively evaluate the effectiveness of the models.

Related Methods. DeepOpht [Jing et al., 2018] proposes the dataset DeepEyeNet (DEN) and explores the framework and central challenges of automated medical report generation. Li et al. [2019] integrate prior knowledge into encoding and retrieval processes, employing multi-stage paraphrasing to enhance report quality. Huang et al. [2022] introduce a non-local attention mechanism to capture more comprehensive image features, thereby improving the description generation for retina images. EyeGraphGPT [Wang et al., 2024c] is built on a multimodal large language model incorporating knowledge graphs, aiming to enrich medical knowledge integration and expression in ophthalmic report generation.

4 Discussion

4.1 Challenge of Current Multimodal Datasets

Dataset Constraints. Some datasets, such as OLIVES and Project Macula, have reached full accuracy, making it difficult to measure further improvements. Recent studies, such as EyeMoSt [Zou et al., 2024], achieve full accuracy even after adding Gaussian noise to OLIVES [2022], suggesting that some benchmarking datasets may no longer differentiate model performance effectively. These limitations reflect not only the saturation of current benchmarks, but also deeper issues in dataset design and availability. Many datasets remain small or contain biases, which limit the generalizability of models trained on them. This scarcity is often rooted in the challenges of clinical data governance, where privacy concerns, patient consent requirements, and institutional regulations restrict the ability to share data across centers. These challenges highlight the need for larger, more diverse, and well-standardized multimodal retina image analysis datasets.

Modality Misalignment. A common response to these constraints is to merge multiple datasets in order to construct larger and more diverse evaluation pools [Stankevi`eius et al., 2018; Pratap and Kokil, 2019]. While this strategy can improve sample diversity by introducing a larger sample pool, it also introduces practical challenges. In particular, these efforts often encounter inconsistent labeling standards and divergent annotation protocols, making it difficult to establish unified benchmarks and conduct reliable cross-dataset evaluations. These issues become more pronounced in multimodal settings. Beyond label inconsistencies, this complexity arises not only from device heterogeneity, but also from inconsistencies in modality composition. First, multimodal retina image datasets exhibit significant variation in acquisition devices, as summarized in Tables 1 and 2. Different datasets often use different imaging devices for the same modality. Such variability in imaging devices, including differences in hardware, resolution, contrast, and preprocessing pipelines, introduces domain shifts. The cross-domain shifts in the crossmodal scenarios may raise challenges to generalization. Second, imaging modalities vary across datasets. While some include OCT and CFP, others contain OCTA or FA, resulting in limited modality overlap and poor cross-dataset compatibility. In multimodal settings, each sample refers to one subject. Different subjects may have different sets of available modalities, particularly when data from multiple sources are combined. This makes it difficult to build data pipelines that rely on consistent modality combinations, and constrains the scalability of multimodal fusion strategies across large and heterogeneous data sources.

4.2 Issues in Benchmarks

EvaluationProtocols. The current benchmarks exhibit significant ambiguities regarding data splits for training and testing. Many datasets do not specify clear partitions for training versus testing, leading some studies to adopt 5-fold crossvalidation to compare their work against others. However, this method can introduce randomness [Varma and Simon, 2006], while other researchers opt for purely random splits, further complicating direct comparisons. Additionally, some work (4.1) mix heterogeneous public datasets and even private collections in their training or testing, often without clear documentation of the split strategies. Such inconsistencies limit the transparency, reproducibility, and comparability of downstream evaluations. A robust benchmark should feature a well-designed testing set that guarantees the training and testing data originate from the same distribution, ensuring more consistent and reliable evaluations.

Evaluation Metrics. This survey exposes 28 distinct evaluation metrics and 14 variations. Although assessing performance from various perspectives can be advantageous, an overabundance of metrics often results in redundancy due to overlapping characteristics. Moreover, the use of different evaluation metrics across studies directly hinders the ability to compare results effectively. Researchers in the retina registration field frequently reimplement existing methods using their own chosen metrics and datasets to ensure fair comparisons. This practice imposes additional computational and implementation burdens, and often results in incompatible results across studies. Benchmarks should therefore select metrics that are best suited to the dataset and task, providing a comprehensive reflection of model performance and covering all clinical-relevant aspects.

4.3 Evaluation of Individual Modality

Fusion Ambiguity. Multimodal fusion is a central objective in retina AI research, yet its practical benefits are not well-characterized. While some works (e.g., Uni4Eye [Cai et al., 2022]) report per-modality performance, they rarely quantify the incremental gains achieved through fusion. In some cases, such as IPNv2 [Li et al., 2024], models trained solely on OCTA even outperform their fused counterparts, challenging the common assumption that combining modalities always improves performance. These findings suggest that fusion may introduce redundancy, suboptimal architectural interactions, or additional noise. This reveals a lack of robust tools to isolate and evaluate individual modality contributions, limiting the ability to systematically optimize the fusion strategies.

Modality Variability. Beyond these methodological limitations, the modality contribution itself is inherently dynamic and context-dependent. First, the modality importance may vary by task, as different objectives rely on different imaging inputs. Second, the contributions may fluctuate across subjects, depending on disease characteristics or image quality. Third, the utility of a modality during training may not align with its value during inference, as some inputs enhance feature learning but contribute little to final predictions. These nuances call for dropout-based or ablation-based evaluations to assess modality-level robustness. Altogether, these challenges emphasize the need for principled evaluation frameworks that can disentangle, interpret, and benchmark modality-specific contributions in multimodal retina systems.

4.4 Future Direction

We believe that a dataset and the corresponding benchmark are essential to ensure a fair comparison and facilitate reproducibility by providing uniform evaluation protocols. To build such reliable and transparent evaluation protocols, several key improvements are necessary. First, the community must adopt shared protocols for dataset preprocessing, uniform training-validation-test splits, and explicit ablation studies to determine the impact of individual imaging modalities. Second, expanding dataset diversity, both in terms of pathologies and imaging devices, will mitigate the risk of overfitting specific data distributions. Third, federated learning offers a promising pathway for privacy-preserving data sharing across institutions, allowing for large-scale validation without compromising patient confidentiality. Finally, the development of standardized benchmarking frameworks, including robust cross-dataset validation, clinically interpretable evaluation criteria, and open-source reproducibility, is critical for transforming multimodal retina AI from promising research prototypes into clinically deployable tools. Emerging foundation models offer new potential for cross-modal reasoning and automated reporting in retinal analysis [2023b; 2024; 2024b; 2025]. Most existing foundation models are either limited to single-modality settings or trained on multimodal data without subject-level pairing. Foundation models aligned at the patient level hold promise for meaningful clinical integration and reasoning, a potential that is contingent upon the availability and quality of multimodal datasets.

5 Conclusion

This survey highlights the critical role of multimodal retina images in advancing ophthalmic diagnostics and AI-driven analysis. By integrating complementary structural and vascular modalities, multimodal approaches enhance disease detection and monitoring. However, the field lacks standardization in dataset usage, fusion methodologies, and evaluation frameworks, limiting reproducibility and comparability across studies. Our review systematically examines existing datasets, benchmarking methodologies, and deep learning strategies, emphasizing the need for standardized evaluation frameworks to ensure robust and clinically relevant AI models. Future research should prioritize the development of large-scale, well-annotated datasets and clinically interpretable benchmarks to bridge the gap between research advancements and real-world applications. Addressing these challenges will be pivotal in realizing the full potential of AIdriven multimodal retina analysis in clinical practice.

Acknowledgments

This research is funded in part by ARC-Discovery grant (DP220100800), ARC-DECRA grant (DE230100477). We gratefully thank all the reviewers and chairs for their constructive comments.