ActiveHAI: Active Collection Based Human-AI Diagnosis with Limited Expert Predictions

1 Introduction

Artificial intelligence (AI) demonstrates remarkable potential in healthcare [Rajpurkar et al., 2022], outperforming cardiology residents in identifying 12-lead electrocardiogram abnormalities [Ribeiro et al., 2020]. However, AI algorithms face challenges like out-of-domain inapplicability, bias, and lack of interpretability [Topol, 2019]. Motivated by this observation, prior work [Yu et al., 2021; Bansal et al., 2021; Gu et al., 2023] has explored human-AI collaboration to leverage their complementary strengths. Beyond the methods that assign inputs to humans or AI, like deferral [Madras et al., 2018; Keswani et al., 2021], combining human and AI decisions is emerging as a promising strategy [Wilder et al., 2021; Steyvers et al., 2022; Zhao et al., 2024]. Additionally, when medical AI is accessible only as a 'locked model' due to privacy constraints, effectively combining doctors' predictions with pretrained AI outputs becomes critical.

∗Corresponding author

Human-AI decision-making combinations [Steyvers et al., 2022] often require probabilistic distributions from both humans and AI. However, due to cognitive burden, asking doctors to provide confidence scores for all possible labels is impractical. Instead, additional predictions from doctors are needed to infer these scores, but extensive expert predictions incur high costs [Wang et al., 2024; Liu et al., 2024], limiting the feasibility of human-AI diagnostic systems. We aim to make accurate human-AI diagnostics with limited expert predictions. Figure 1 shows that doctor prediction is combined with AI probabilities to generate the final diagnosis.

Existing research [Kerrigan et al., 2021; Gupta et al., 2023; Singh et al., 2023] on combining human class-level predictions with AI probabilistic outputs often leverages confusion matrices, which quantify the relationship between expert predictions and ground truth labels. However, constructing confusion matrices requires substantial expert predictions to accurately estimate probabilities, which is time-consuming and costly, especially for large-scale medical imaging [Willemink et al., 2020; Alzubaidi et al., 2021]. Recent work [Hemmer et al., 2023; Mozannar et al., 2023; Dvijotham et al., 2023; Alves et al., 2024] on human-AI collaborative diagnosis has considered the constraints of limited expert predictions, using a small number of predictions to identify doctors' specific strengths. However, these studies primarily focus on deferring tasks to experts, overlooking the probabilistic distribution of expert predictions.

To improve the accuracy of human-AI diagnosis with limited human prediction labels, there are two challenges. First, given the high cost of inviting experts to predict samples that accurately represent their abilities, how can we design a strategy to collect a limited yet representative subset of expert predictions actively? Second, existing methods perform poorly when the number of available expert predictions is limited, how can we design an efficient method to evaluate human prediction probability distributions with limited predictions? To address these challenges, we propose two solutions: (i) For the first challenge, we introduce a median-window based active collection strategy. This strategy iteratively selects representative samples near the median of the estimated values and provides expert samples for prediction, thus reducing the cost of expert annotations on medical samples. (ii) For the second challenge, we propose an evaluator module that integrates sample features and expert predictions. This module combines the feature vectors from the pretrained AI with the encoded expert prediction vectors, enabling the evaluation of probability distributions with limited expert predictions.

In this paper, we propose ActiveHAI, an active collection based human-AI diagnostic combination method that enhances diagnostic accuracy with limited expert predictions. First, we use the proposed median-window sampling method to actively select samples for expert predictions, iteratively generating fully labeled data. Second, we train the evaluator module using data that includes expert predictions and ground truth labels, enabling improved evaluation by jointly encoding sample features and expert predictions. Finally, we combine the human prediction probability distributions generated by the evaluator module with the pretrained AI's probabilistic outputs to compute joint probabilities, ultimately producing the final human-AI combined prediction.

Our main contributions are as follows:

• Active Collection: We propose the median-window active collection algorithm to actively select expert predictions for human-AI diagnosis, enabling efficient evaluation of human prediction probability distributions.
• Human Prediction Evaluation: We propose an evaluator module that enhances the ability to transform expert predictions into probability distributions by leveraging pretrained feature layers and prediction embeddings.
• Experiment Study: Experiments on three real-world datasets show that the proposed method outperforms individual human and other human-AI collaboration methods by 16.3% and 3.6% in diagnosis accuracy, respectively. For reproducibility, we release the code and data in https://github.com/mercyzi/ActiveHAI.git.

2 RelatedWork

2.1 Medical Diagnosis with Limited Labels

Researchers have explored various techniques to improve predictions in data-scarce scenarios. For instance, [Hemmer et al., 2023] utilized semi-supervised learning to generate artificial labels with limited annotations, enabling deferred learning. [Chae and Kim, 2023] applied transfer learning in medical image analysis to enhance accuracy with small medical datasets. [Kotia et al., 2021] focused on few-shot learning to learn effective feature representations from a limited number of labeled samples in medical imaging. However, these approaches often rely on large amounts of unlabeled data and domain similarity, which may not always be feasible in realworld medical applications.

In contrast, active learning methods [Liu et al., 2020; Budd et al., 2021] aim to efficiently identify the most informative subsets of unlabeled samples, thus enabling more effective training of AI models and reducing the expensive annotation burden typically associated with medical image data. For example, [Tang et al., 2023] employed active learning to tackle label scarcity, achieving more accurate identification of gastrointestinal diseases. [Zhang et al., 2024a] proposed an interactive image annotation framework that improves prostate MRI image segmentation accuracy with fewer interactive annotations. However, few studies have explored how to actively select limited expert predictions in order to evaluate human expert capabilities effectively.

2.2 Human-AI Collaborative Diagnosis

In human-AI collaborative classification [Fragiadakis et al., 2024], humans and AI systems collaborate and complement each other to tackle more complex tasks. It is primarily divided into Learning to Defer (L2D) and Learning to Combine (L2C) [Zhang et al., 2024b]. L2D [Madras et al., 2018; Hemmer et al., 2022; Dvijotham et al., 2023] refers to assigning the final decision to either the expert or the AI. For example, [Mozannar and Sontag, 2020] proposed a framework based on classifiers and rejectors that defers the decision on chest X-rays to experts. [Verma and Nalisnick, 2022] introduced a calibrated L2D system that defers diagnosing skin lesions to experts. [Mao et al., 2024] extended the L2D model to multiple medical experts.

In this paper, we focus on L2C, which combines AI and human predictions for the final diagnosis, ensuring higher accuracy by leveraging the complementary strengths of both systems and capturing information that machines may miss [Groh et al., 2022]. For example, [Kerrigan et al., 2021] developed an algorithm that combines human predictions with machine model probabilities, improving accuracy and reliability. [Steyvers et al., 2022] proposed a Bayesian framework that uses varying confidence scores, demonstrating that a hybrid human-AI combination outperforms either prediction type individually. [Gupta et al., 2023] combined calibrated model probabilities with expert predictions, improving the alignment of model outputs with expert predictions. However, these studies do not address the limited availability of expert-predicted labels in the medical field, which may lead to suboptimal accuracy in human-AI diagnosis.

3 ProblemFormulation

In this section, we consider human-AI decision combination in medical diagnosis tasks. For a given medical instance x ∈ X, we assume that we have access to the probability vector ph(x) ∈ Ph predicted by a human expert and the probability vector pa(x) ∈ Pa predicted by a pretrained AI model. The goal is to accurately predict the true label y ∈ Y = {1, ..., K} with K denoting the number of classes. Based on Bayesian theory, assuming that the predictions ph(x) and pa(x) are conditionally independent, we compute their joint probability using the product rule. Similar to [Kerrigan et al., 2021], we represents the prediction probability of class j as:

where ph(x)j and pa(x)j represent the prediction probabilities of class j by humans and AI, respectively.

However, it is infeasible in practice to directly obtain ph(x), and it is costly for doctors to evaluate confidence levels when making predictions. Our premise for human-AI collaboration is to train an evaluator module E : X × H → Ph that models the probability distribution of individual doctor predictions h ∈ H = {1, . . . ,K}. Rather than directly converting h into a one-hot encoding, we use historical expert prediction data to fit a more fine-grained probability distribution. The capabilities of individual doctors vary across different disease feature spaces, and ground-truth labels in medical diagnosis datasets typically require consensus among multiple experts. For example, three expert pathologists independently annotate the slides in a colon slide dataset [Zhu et al., 2021], and the consistent results are used as the true labels. Therefore, doctors collaborating with AI models need to provide additional self-predictions to train the evaluator module for distribution fitting.

We define the binary data containing medical instances and the ground-truth labels as Du = {(xi, yi)}K·u i=1 , and the ternary data that additionally includes human expert predictions as Dl = {(xi, yi, hi)}K·l i=1 . Here, l represents the number of samples with human predictions for each class, u = n-l denotes the remaining number of samples per class, and n is the total number of samples in each class. Acquiring individual expert predictions incurs additional costs and human effort, particularly in the medical domain. Therefore, we consider achieving efficient human-AI diagnostic collaboration at a lower cost by evaluating the predictive probability distribution under a limited number of expert predictions l. The goal is to leverage the distribution derived from limited expert predictions l to approximate the distribution of full expert predictions n, thereby enabling a more accurate humanAI diagnostic combination.

In this study, the network parameters of the pretrained AI model M : X → Pa are fixed, and we redefine the human-AI combination probability of class j as:

where Eθ(x, h) and M(x) represent the outputs of the proposed evaluator module and the AI model, respectively, and θ denotes the trainable parameters of the evaluator module.

4 Approach

In this section, we describe the proposed method for fitting the probability distribution using limited expert predictions. The approach consists of two main parts: (1) collecting representative limited expert predictions to construct the fully labeled data Dl ; (2) training the proposed evaluator module on Dl to fit the probability distribution of the expert predictions.

4.1 Active Collection of Human Predictions

The first component of our method is the active selection of samples for annotation by medical experts. By selecting the most representative data samples, we effectively train the evaluator module to reduce the cost of expert predictions.

In active learning, the algorithm selects the most informative data samples to train AI models and efficiently reduce labeling costs. Unlike traditional active learning approaches: (1) They involve human annotation for samples with unknown true labels, whereas, in our task, the true labels of all samples are known. (2) Their goal is to improve the performance of the AI model itself. In contrast, our goal is to enhance the performance of the evaluator to estimate the human predictive probability distribution better. To this end, we propose a novel sample collection strategy, MedianWindow Active Collection (MWAC), to collect predictions frommedical experts.

As shown in Figure 2, we iteratively collect human predictions. Through median-window sampling, a subset of samples from Du is selected for expert annotation, generating a new fully labeled dataset Dl . Furthermore, the updated Dl from each iteration is used as training data for the evaluator module, which is retrained and utilized in subsequent median-window sampling iterations. Specifically, the median-window sampling consists of three primary steps: (1) Random Sampling: Since disease diagnosis is a multi-class problem, we aim to uniformly evaluate the predictive probability distribution of experts across different classes. To this end, we uniformly and randomly sample N samples per class from Du . (2) Estimation Generation: For the N samples in each class, in order to select the most representative samples, we use the evaluator module to calculate the estimates ei:

Algorithm 1: Median-Window Active Collection

Input: Du : data without expert labels. Niter: number of active learning iterations. l: number of human predictions for each class. QueryExpert: ask expert to label samples. TrainEvaluator: train the evaluator module. Output: Dl : fully labeled data after active learning. 1 Initialization: 2 for k ← 1 to K do 3 Randomly choose l1 samples for class k fromDu . 4 L1 ←QueryExpert(l1). 5 Add L1 to Dl , and remove samples fromDu . 6 E ← TrainEvaluator(Dl ). 7 for n← 2 to Niter do 8 for k ← 1 to K do 9 Randomly choose N samples for k from Du . 10 {(xi, ei)}Ni=1 ← Estimate(N, E). 11 Sort [(xi, ei)]Ni=1 by ei in descending order. 12 Construct median-window [Ws, Ws + Wl ]. 13 Randomly choose ln samples from window. 14 Ln ← QueryExpert(ln). 15 Add Ln to Dl , and remove samples fromDu . 16 E ← TrainEvaluator(Dl ). 17 return Dl . 18 Function Estimate(N, E): 19 S ← ∅ . 20 for each instance xi, yi from N do 21 ˜hi ← yi. 22 ei ← ∣∣∣E(xi, ˜hi)˜hi - 0.5∣∣∣. 23 S ← S ∪ {(xi, ei)}. 24 return S.

where, ˜hi = yi assumes that expert predictions align with the ground truth labels, and E(xi, ˜hi)˜hi denotes the probability output of the evaluator module for the pseudo expert prediction ˜hi category. (3) Window Sampling: We rank the N samples in descending order based on their estimated values and construct a window with a starting index Ws and a length Wl. This window corresponds to the interval [Ws,Ws +Wl] within the ordered sample list [0,N]. The MWAC algorithm is described in detail in Algorithm 1.

Since the estimated value e is defined as the absolute difference between the probability output by the evaluator for the true label and 0.5, the closer this probability is to 1 or 0, the larger the estimated value e. This reflects an extreme estimation of human predictions. Conversely, when the probability is closer to 0.5, the estimated value becomes smaller, representing a more ambiguous estimation of human predictions. We argue that the median of the estimated values better represents the collective characteristics of the sample set, enabling a more reasonable estimation of human predictions, which is validated in the experimental section. Therefore, we select the sampling window's starting index Ws based on the estimated values' median. Finally, we perform sampling within this window to obtain a limited number of expert predictions.

4.2 Design of Evaluator Module

As shown in Figure 3, for a medical instance with features x, expert doctors and the AI model independently provide predicted labels and probabilities. The evaluator module processes the expert's predicted label to generate a probability distribution, and the final human-AI collaborative prediction ˆ y is computed using Equation (2). The proposed evaluator module is designed to assess the probability distribution of doctors' decisions, represented as a probability vector over multiple categories.

Existing methods for combining human predictions and AI probabilities usually rely on confusion matrices, which quantify the alignment between expert predictions and ground truth labels. However, their reliability declines with limited annotations. To address this, we propose an evaluator module integrating expert predictions with sample features, enabling robust evaluation in scenarios with fewer annotations. Specifically, in the evaluator module, we first extract feature vectors from instance x using a feature layer:

where θx is the feature layer parameter from the pretrained AI model. Then, we pass the human prediction h through the embedding layer to obtain the embedding vector:

where θh is the embedding layer parameter. Finally, the feature vector and embedding vector are added together and then passed through a linear layer to obtain the final human prediction probability distribution:

where θl is the linear layer parameter. We can train the evaluator module by calculating the cross entropy loss between human-AI combination prediction ˆ y and true labels y:

where θ represents the set of θh and θl, excluding θx, as the feature layer network parameters are fixed.

5 Experiments

In this section, we conduct extensive experiments on realworld medical diagnosis datasets to evaluate our method.

5.1 Experimental Settings

Datasets. We extensively evaluate the proposed method on three datasets: MZ-10 [Chen et al., 2023], DR-5 [Ju et al., 2022], and Chaoyang-3 [Zhu et al., 2021]. MZ-10 is a medical consultation dataset, while Chaoyang-5 and DR-3 are medical imaging datasets.

MZ-10: A large-scale corpus for evaluating medical consultation systems. This dataset covers 331 symptoms and 10 pediatric diseases. As the dataset lacks doctor-predicted labels, we artificially generate three sets of expert prediction labels (D1, D2, and D3) to represent doctors with varying levels of expertise. For each set of artificial expert labels, we sample probabilities P from a uniform distribution with mean a, where a denotes overall accuracy and P represents accuracy for each class. Given the machine model's accuracy of 0.69, we set a to 0.65, 0.7, and 0.75 to simulate doctors with expertise slightly below, equal to, and above the AI model.

DR-5: A diabetic retinopathy dataset with five grading labels. It includes prediction labels from multiple ophthalmologists, and we select the ophthalmologist who provides the most significant number of expert predictions. Due to the limited number of samples for proliferative diabetic retinopathy, we randomly sample 200 samples from each category to ensure sufficient representation and unbiased evaluation.

Chaoyang-3: A colon slide dataset from Chaoyang Hospital, with prediction labels from the expert pathologist. Our study does not consider the diagnosis of adenocarcinoma, as the prediction accuracy for adenocarcinoma samples by experts approaches 100%, which limits the complementarity of human-AI collaboration in this category. We randomly sample 600 slides from each of the other three categories (normal, serrated, and adenoma).

Baselines. We compare our proposed ActiveHAI with the three following baselines:

CM [Kerrigan et al., 2021; Gupta et al., 2023; Singh et al., 2023]: A human-AI decision combination method that evaluates the probability distribution of human predictions using a confusion matrix, followed by Bayesian fusion of human and AI probabilities.

PCM [Hemmer et al., 2023]: A method that generates pseudo-human predictions to extend the confusion matrix, enabling a similar human-AI decision combination as CM.

Collab [Zhang et al., 2024b]: A human-AI decision combination method that directly combines human predictions and AI probabilities through a collaboration module.

Metrics. We use two widely used metrics to evaluate the proposed method's performance: accuracy and KL divergence. Accuracy measures the overall performance of the human-AI combination, while KL divergence quantifies the difference between the human probability distribution under limited predictions and the distribution under all predictions, thus evaluating the quality of the probability distribution fitting.

Implementation Details. We implement ActiveHAI using PyTorch on a single NVIDIA 3090 GPU. For the feature layer, we employ a two-layer Transformer [Vaswani et al., 2017] encoder for MZ-10, and a pretrained EfficientNet-B1 [Tan and Le, 2019] model for DR-5 and Chaoyang-3. The embedding layer dimension is set to 512. The evaluator module is trained for 100 epochs using the Adam optimizer with a learning rate of 3 × 10-4 . For MZ-10, we sample the probability P ten times for each set of artificial expert predictions, repeating each sampling experiment five times. For DR-5 and Chaoyang-3, we perform five-fold cross-validation, repeating each fold ten times.

We consider the following numbers of expert predictions: l ∈ {3, 4, 5, 6, 7, 8,All}. Initially, the number of expert predictions is set to 2, and it increases by 1 in each iteration. The random sampling size N is set to 100, and the medianwindow length Wl is set to 5. For D1, D2, and D3 in MZ-10, the window starting points Ws are set to 65, 50, and 50, respectively. For DR-5, Ws is set to 55, and for Chaoyang-3, Ws is set to 50.

5.2 Overall Performance

We present the accuracy performance of the human-AI diagnosis combination models on the MZ-10, DR-5, and Chaoyang-3 datasets in Tables 1, 2, and 3, respectively. Overall, when the number l ≤ 8 of human expert predictions, the proposed ActiveHAI method outperforms all baseline methods in diagnosis accuracy. Specifically, the results show that when l ≤ 8, our method achieves an average accuracy improvement of 17.8% (on MZ-10), 14.8% (on DR-5), and 13.2% (on Chaoyang-3) compared to human predictions. Moreover, our method also surpasses existing human-AI combination models by 4.6% (on MZ-10), 1.5% (on DR-5), and 2.8% (on Chaoyang-3) under the same conditions. These results highlight the significant advantage of our method under limited human predictions. Furthermore, when l = All, our method achieves accuracy comparable to other approaches, demonstrating its ability to maintain performance under complete human predictions.

Figure 4 shows the performance of human prediction probability distribution fitting for varying numbers of available human expert predictions on the MZ-10, DR-5, and Chaoyang-3 datasets. Since Collab directly combines human predictions with AI probabilities, these results highlight the quality differences between our method, CM, and PCM in converting human predictions into probability distributions. On the MZ-10 dataset, our ActiveHAI method reduces KL divergence by an average of 66.9% and 77.9% compared to CM and PCM, respectively, under limited expert predictions. On the DR-5 dataset, ActiveHAI reduces KL divergence by an average of 79.8% and 52.3% compared to CM and PCM, respectively. On the Chaoyang-3 dataset, ActiveHAI reduces KL divergence by an average of 62.7% and 56.8% compared to CM and PCM, respectively. These results indicate that our method significantly improves the effectiveness of fitting distributions under limited expert predictions.

5.3 Ablation Study

We conduct a series of ablation studies to validate the effectiveness of each component of our method. First, we introduce the MWAC algorithm to select limited samples for human prediction actively. Second, we propose the evaluator module to transform human predictions into probability distributions. Tables 1, 2, and 3 demonstrate that both the MWAC algorithm and the evaluator module improve the accuracy of the human-AI diagnostic combination when l ≤ 8. When the active collection strategy is replaced by random collection for obtaining human predictions, the accuracy decreases, particularly by 1.8% when l = 3. This result validates the effectiveness of the MWAC algorithm. Similarly, without the evaluator module and relying on the existing confusion matrix method, the accuracy drops significantly, highlighting the importance of the evaluator module.

5.4 Effect of Parameters Ws andWl in Median-Window

We use median-window sampling to implement the MWAC algorithm. Ws represents the starting position of the window, which defines the candidate sample interval. If Ws is close to 0 or N, the window primarily selects samples with extreme or ambiguous estimates. Conversely, if Ws is close to N/2, the window primarily selects samples near the median of the estimated values. Wl defines the size of the window. To assess the effect of the median-window on the accuracy of ActiveHAI across the three datasets, we set N = 100 and adjust Ws ∈ {0, 10, 20, 30, 40, 50, 60, 70, 80, 90} and Wl ∈ {5, 10, 15, 20, 25, 30}. The results are shown in Figure 4. The results indicate that as Ws increases, diagnostic accuracy initially improves, reaching a peak in the range of 40 to 70, and then decreases. In contrast, diagnostic accuracy declines as Wl increases. This aligns with our intuition that, under limited human predictions, selecting samples near the median of estimated values for human prediction evaluation improves distribution fitting and enhances the accuracy of human-AI collaborative diagnosis. Moreover, for different numbers of predictions l, the impact of Ws and Wl on accuracy remains consistent. This consistency suggests that the proposed median-window sampling method applies to various scenarios involving limited expert predictions and demonstrates a degree of generalization capability.

6 Conclusion

In this paper, we propose ActiveHAI, a human-AI diagnostic combination method designed to improve diagnostic accuracy with limited expert predictions. Through medianwindow active collection, we efficiently select human predictions, and our evaluator module enhances the evaluation of human predictions by integrating expert predictions with sample features. Experimental results on three real-world datasets demonstrate that ActiveHAI outperforms individual doctor performance and other human-AI collaboration methods. This method provides a promising solution for scenarios with high expert prediction costs and limited human resources, enhancing the potential of human-AI diagnosis.

Ethical Statement

We train a model in medical diagnosis to assess expert prediction probability distributions, aiming to improve the accuracy of human-AI diagnostic combinations. A broader contribution of this work is the proposed active collection strategy that selects limited representative samples to evaluate the probability distributions of doctors' decisions efficiently. The approach could theoretically be used to estimate the capabilities of individual expert doctors in disease diagnosis. However, real-world deployments must undergo training on larger-scale real-world data for ethical considerations. AI models learned from insufficient and incomplete medical datasets may pose considerable prediction risks. It is important to note that the public datasets used in our experiments have had all patient privacy-related information meticulously removed.

Acknowledgments

This work was supported in part by the National Key R&D Program of China (No.2021ZD0113305), the National Natural Science Foundation of China (No.62372381), and the National Science Fund for Distinguished Young Scholars (No.62025205).