Faster Annotation for Elevation-Guided Flood Extent Mapping by Consistency-Enhanced Active Learning

1 Introduction

Climate change is drastically increasing the intensity and occurrence of floods [Matgen et al., 2020]. In the past two decades, flooding has negatively impacted over 2.3 billion people [Wahlstrom et al., 2015]. Therefore, accurate and timely mapping of flood extent is crucial for effectively planning rescue and rehabilitation efforts [Oddo and Bolten, 2019]. Thanks to the rapid advancement in AI and the abundant geospatial data collected, such as satellite imagery from NASA and ESA and digital elevation model (DEM) data from USGS, it is the right timing to harness them to improve the performance of flood extent mapping.

Although annotated natural images are abundant, most Earth imagery data are unannotated. A popular annotated dataset for flood mapping is Sen1Floods11 [Bonafilia et al., 2020] which was curated by a startup called 'Cloud to Street' (now Floodbase) but it is very small, and [Bonafilia et al., 2020] reports that models trained on Sen1Floods11 have a quite low test mean IoU for the water class.

Two solutions can improve the performance of flood mapping. One solution is to train a more advanced AI model with a larger annotated dataset, and this paper aims to facilitate the productivity of flood annotation with a new strategy to enable the curation of larger annotated datasets. The other solution is to fine-tune a large geo-foundation model pre-trained with a vast number of satellite images in a self-supervised manner, but the fine-tuning stage still requires an annotated dataset of reasonable size. In fact, NASA and IBM recently released a pioneering geo-foundation model called Prithvi [Jakubik et al., 2023], whose fine-tuning on Sen1Floods11 has significantly improved the performance of flood mapping as compared with the results by training a model from scratch on Sen1Floods11 [Bonafilia et al., 2020].

In this paper, we propose to facilitate the annotation of more satellite imagery (e.g., via crowdsourcing) by designing a novel flood annotation tool based on active learning. Given a large pool of unlabeled data, active learning (AL) minimizes the amount of data to be labeled to train an underlying machine learning model, in order to yield a comparable performance as if using much more labeled data. AL operates in cycles: in each cycle, the most informative or valuable data points are selected for labeling (through an acquisition function) and then the model gets retrained. Our use of AL is different from conventional methods since:

• ÷ Conventional AL minimizes the annotation cost since finding an expert to annotate each data point is expensive, but there is no time restrictions. In contrast, we allow non-experts to annotate satellite images but we aim to minimize the annotation time; retraining should be fast to support interactive user annotating, so the underlying model should be lightweight.

• ÷ The problem domain in convention AL is the entire dataset, from which a small subset of data items are selected for user annotation. In contrast, our problem domain is each individual satellite image, wherein informative regions are selected for annotation. For different satellite images, the underlying model can be the same while the trained model parameters can be different.
• ÷ The end goal of conventional AL is the trained underlying model, but our end goal is annotating each image while the underlying model is just used to automatically label each pixel as 'flooded' or 'dry' (trained based on users' limited annotations), so it is desirable to overfit the underlying model (e.g., a lightweight U-Net variant) on the current image to minimize annotation efforts and improve annotation quality. The annotated images are collected usually to collectively train or fine-tune a more powerful model, such as a geo-foundation model often based on masked autoencoder (MAE) [He et al., 2022].

Besides active learning, we also utilize terrain guidance as illustrated in Figure 1 to improve the productivity and quality of flood annotation, as elevation data can be readily obtained from the digital elevation model (DEM) data downloadable from USGS [USGS, 2023].

The end product is an annotation tool called Active Learning for Flood Annotation (ALFA), which is open sourced at https://github.com/saugatadhikari/alfa. Our annotation tool takes the satellite image of a flooded area along with its associated elevation map (from DEM data), and visualizes the terrain in 3D. The tool supports rotation, movement, zoom-in and zoom-out of the terrain, so that users can annotate the pixels in red (to mean 'flooded') or blue (to mean 'dry') from different views. Elevation-guided breadth-first search (BFS) [Adhikari et al., 2022] is adopted to speed up annotation, where (i) when an annotator marks an individual pixel p as flooded, the label propagates to nearby pixels with lower elevations by 'pit-filling' BFS stopping when reaching pixels with elevation higher than that of p, and (ii) when an annotator marks an individual pixel as dry, the label propagates to nearby pixels by 'hill-climbing' BFS stopping when reaching pixels with elevation starting to drop. The labels of some pixels covered by tree canopy may be derived in this way from nearby pixels using the physical law of gravity.

Figure 2 shows the upper-left portion of our tool's interface from a particular view, and more elements of the interface will be introduced later in Figure 3 when we introduce our AL-based annotation pipeline. Figure 2(a) shows an area with many houses that are partitioned into multiple islands due to flooding, and such areas would need very careful and fine-grained user annotations with many flood/dry clicks, so directly annotating an entire satellite image is very timeconsuming, typically hours even for domain experts. Figures 2(b) and (c) show how the elevation-guided BFS can automatically label many pixels after just one click.

Our main contributions are summarized as follows:

• ÷ To keep the number of region candidates tractable (for computing acquisition functions), ALFA recommends superpixels (wherein pixels tend to have the same label) rather than individual pixels for annotation, but users can

• utilize elevation guidance to click on only a few pixels to cover many pixels with their labels automatically derived, including those outside the recommended superpixels.
• ÷ We identify two label-free consistency metrics that can guide active learning: (1) view consistency, which states that if the label prediction of a pixel changes with data augmentation, then it is more informative for annotation to retrain the underlying model; (2) temporal consistency, which states that if the output of the underlying model at a pixel location changes a lot across different cycles, then it is more informative for annotation. Both consistency metrics are integrated into the design of our acquisition function and loss function, and our solution is applicable to active learning in general (not specific to flood mapping) if we replace pixels with data items.
• ÷ Our acquisition function not only integrates the confidence (e.g., measured by entropy) and consistencies, but also considers a 'tree score' specific to our flood annotation domain to minimize the probability that regions covered by tree canopy get recommended (since they are ambiguous and users cannot give labels properly).
• ÷ We conduct extensive studies to verify that ALFA can significantly improve the annotation productivity and quality. ALFA is open sourced at https://github.com/ saugatadhikari/alfa.

2 RelatedWork

Due to page limit, we only briefly mention the related work. Appendix A online [Adhikari et al., 2025] gives more details.

Superpixel Algorithms. A superpixel is a group of contiguous pixels that share almost the same color, and representative algorithms include SLIC [Achanta et al., 2012] and SEEDS [den Bergh et al., 2015]. We adopt SEEDS since it generates higher quality superpixels than SLIC and is faster [den Bergh et al., 2015].

Image-based Active Learning. The image-based AL for semantic segmentation considers an entire image as the basic unit to be recommended for annotation. Related works include [Dai et al., 2020], [Sinha et al., 2019], [Yang et al., 2017] and [Huang et al., 2024]. All these works recommend entire images for annotation which is different from our setting where we recommend regions inside an image.

Region-based Active Learning. The region-based AL for semantic segmentation divides each image into nonoverlapping local regions and recommends regions for annotation. Some works divide the image into uniform patches for recommendation, such as EquAL [Golestaneh and Kitani, 2020] and DIAL [Lenczner et al., 2022]. However, a patch may contain pixels from different classes, so it is not an ideal unit for annotation. PixelPick [Shin et al., 2021] recommends sparse pixels for annotation based on uncertainty sampling using entropy measures. This method is inferior to ours since we recommend region units with label homogeneity, and each click can label many pixels (by elevation-guided BFS) to reduce the time to annotate a satellite image. To our knowledge, ViewAL [Siddiqui et al., 2020] is the only work that recommends superpixels. It studies the segmentation of 3D objects in multi-view datasets by enforcing that the same surface point in a scene should receive the same label when observed from different viewpoints. In contrast, we target a satellite image associated with a land surface derived from DEM data, rather than 3D objects in a multi-view dataset.

Geo-foundation Models. Recently, several geo-foundation models have been proposed based on MAE to pre-train with the vast amount of unannotated satellite imagery, such as Prithvi [Jakubik et al., 2023], SatMAE [Cong et al., 2022] and SpectralGPT [Hong et al., 2024]. The self-supervision during pre-training is achieved by masking out a fraction of patches for recovery. Our annotated images can be used to fine-tune these geo-foundation models.

3 Methodology

We regard each satellite image as a 2D grid I of size H ⇥W, and denote a pixel by p = (x, y), where x and y are the pixel coordinates in I. We also assume that an elevation map h(I) is available, and the elevation of pixel p is h(p). Our AL framework uses EvaNet [Sami et al., 2024] as the underlying segmentation model, which is a U-Net variant that (1) uses (de)convolution operations which integrates the elevation map h by a location-sensitive gating mechanism to regulate how much spectral features flow through adjacent layers, and that (2) uses a loss function Leva which intergates the physical rule that if a location is flooded (resp. dry), then its adjacent locations with a lower (resp. higher) elevation must also be flooded (resp. dry). We use EvaNet due to its lightweight training workload and its capability to utilize the elevation map to improve segmentation quality, and please refer to [Sami et al., 2024] for its detailed design.

Since EvaNet takes an input image of size 128 ⇥ 128, we partition I into 128 ⇥ 128 patches as in [Sami et al., 2024] and each patch is passed through EvaNet for segmentation. The segmentation results of all patches can be stitched back to obtain the segmentation results of the entire I.

Figure 3 overviews the workflow of ALFA, where the pretrained parameters of EvaNet are initially used, and the superpixels of I are precomputed using SEEDS [den Bergh et al., 2015]. In each AL cycle, each patch of I is passed through EvaNet to obtain an output tensor with 2 channels, one for 'flood' scores and the other for 'dry' scores. The results from all patches are stitched together to get a flood score map and a dry score map, both of size H ⇥W. Let the flood (resp. dry) score of a pixel p be sflood(p) (resp. sdry(p)), then a channel-wise softmax is conducted to normalize the score

maps into probability maps: pflood(p) = esflood(p) esflood(p)+esdry(p) (resp. pdry(p) = esdry(p) esflood(p)+esdry(p) ). The probability maps can

already be used to measure the confidence of model predictions to select the most uncertain superpixels for user annotation, but we can further improve the robustness of the uncertainty measure by considering view consistency and temporal consistency, which we will discuss in detail later.

As the lower-right corner of Figure 3 shows, the highuncertainty superpixels (highlighted in green) are recommended for user annotation, and users may click pixels within these superpixels to annotate with a 'flood' or 'dry' label. Since the label is propagated using elevation-guided BFS, the annotated pixels may propagate outside the recommended pixels. When a user finishes the annotation activities in the current cycle, it can click a "retrain" button so that the EvaNet will be retrained using all the annotated pixels so far, to be used by the next AL cycle. Note that (1) the loss function only considers annotated pixels for supervision, and it integrates both elevation guidance (i.e., Leva) and the consistency measures to improve training performance (we will discuss the details later), and that (2) the number of annotated pixels is small but increases after each cycle, so model retraining is very efficient with a GPU.

At any time during the annotation, users may visualize (1) the current prediction by EvaNet, (2) the recommended superpixels in the current cycle, and (3) all the annotated pixels so far, by clicking the corresponding button in the interface (or pressing the corresponding key on the keyboard) to facilitate annotation. Users are also allowed to erase old pixel annotations to fix annotation errors made previously.

We retrain for 3 epochs in each cycle, taking 80-120 seconds in total. To utilize the time during retraining, users can continue to annotate more pixels by comparing the previous EvaNet prediction and the satellite image to locate wrongly predicted pixels/regions to annotate with the correct labels.

In the sequel, we first introduce our consistency measures, then the acquisition function, and finally the loss function.

Pixel-level Uncertainty. Given the current predictions by the EvaNet model, we define the uncertainty score for each pixel. We consider three uncertainty measures: (1) confidence, (2) view consistency, and (3) temporal consistency, all defined based on the dry and flood probability maps.

For confidence, we consider two alternative measures based on probability offset and entropy, respectively:

Probability offset measures how much the flooding probability deviates from 0.5 (i.e., 50% flooding 50% dry). For uoff(p) (resp. uent(p)), the larger (resp. smaller) its value is, the more confident the EvaNet prediction is for p.

For view consistency, we penalize those pixels whose predictions vary with data augmentation. For each patch (i.e., input to EvaNet), we consider the 6 views (5 augmentations) as shown in Figure 4, all passed through EvaNet to get their probability maps. For a pixel p in the original patch, we denote the predicted flood (resp. dry) probability of its corresponding pixel in the ith view by p(i) flood (p) (resp. p(i) dry (p)), and denote the corresponding entropy by u(i) ent (p) as in Eq (2), then we can define the bias of p in the ith view in two alternative ways:

Based on our experiments (see Table 5 in Appendix C online [Adhikari et al., 2025]), b(i) ent performs better than b(i) prob when used to calculate view inconsistency, so we assume that b(i) ent is adopted from now on.

After we compute the biases in the 6 different views, we can aggregate them in two alternative ways:

which basically computes the variance of u(i) ent (p), and

which computes the maximum bias caused by data augmentation. Based on our experiments (see Table 6 in Appendix C [Adhikari et al., 2025]), ⇢max performs better when measuring view inconsistency, so we assume that ⇢max is adopted from now on.

For temporal consistency, we measure how much the predicted probabilities differ between two consecutive AL cycles:

where ♣c denotes the prediction in the cth cycle. As [Huang et al., 2024] shows, temporal output difference is an effective measure of model uncertainty about its prediction.

Besides these measures, we also consider the concept of 'tree score' to avoid recommending pixels covered by tree canopy, as their labels are hard to determine by human eyes. Specifically, we train a standard U-Net model (called as ForestNet) on satellite images carefully annotated with forest maps, and use it to predict the pixel-level forest pixel probabilities for our test regions. Let ptree(p) denote the tree probability score for pixel p by ForestNet, and let uconf be the confidence score that can be either uoff (Eq (1)) or -uent (Eq (2)), then the overall uncertainty score for p is:

where ⇢max(p) is the view inconsistency score defined in Eq (6), and utemp(p) is the temporal inconsistency score defined in Eq (7). Hyperparameters λ1, λ2 and λ3 balance the importance between the various uncertainty measures, and our goal is to recommend pixels with large u(p).

Superpixel-level Uncertainty Score Aggregation. Since ALFA recommends superpixels rather than pixels, we need to compute uncertainty score for each superpixel by aggregating the uncertainty scores of its pixels. Given superpixel S containing ♣S♣ pixels, we define its uncertainty score in two alternative ways, by average or maximum aggregation:

Based on our experiments (See Table 6 in Appendix C [Adhikari et al., 2025]), the average scheme works the better and is thus used by default. This is reasonable since averaging is more robust to noise. We recommend top-N superpixels for annotation by the users, and the default value of N is set to 25 which is observed to work well in our experiments.

Retraining by Semi-Supervised Learning. We retrain the EvaNet using (1) the supervised loss head Leva originally proposed in [Sami et al., 2024] that penalizes label predictions of pixel pairs that violate elevation guidance (e.g., a 'flooded' pixel with a nearby 'dry' pixel with a lower elevation), as well as (2) two unsupervised loss heads based on view consistency and temporal consistency, respectively.

Let the current AL cycle number be c, and let the set of all pixels not yet labeled by users be Uc, then both unsupervised loss heads are computed over pixels of Uc.

The view consistency loss of the current cycle is given by

which minimizes the prediction variance of every unlabeled pixel among the 6 different views.

Temporal consistency is enforced by minimizing the prediction difference between the current model at cycle c and a baseline model obtained by applying an exponential moving average (EMA) to the historical parameters as inspired by Mean Teacher [Tarvainen and Valpola, 2017]:

where wc-1 is the retrained EvaNet parameter at cycle (c - 1), w is the parameter of the baseline model, ↵ is the EMA decay rate, and Eq (12) updates w of cycle (c - 2) with the retrained parameter at cycle (c - 1). This design is found to perform better than minimizing the prediction difference at consecutive cycles c and (c - 1) [Huang et al., 2024].

Let us denote by utemp(p)♣EMA the sum of squared prediction differences as formulated in Eq (7) except that the model predictions at cycle (c - 1) is replaced by the predictions by the baseline model with parameter w. The temporal consistency loss at cycle c is hence given by

The overall loss objective at cycle c is hence given by:

where β1 and β2 are hyperparameters to balance loss terms.

4 Experiments

Datasets. We obtain high-resolution aerial imagery from NOAA NGS during Hurricane Matthew in North Carolina (NC) in 2016 [NOAA, 2016] and Hurricane Harvey in Texas (TX) in 2017 [NOAA, 2017] along with their accompanied DEM data. Table 1 summarizes our regions: R1-R4 are from Matthew 2016 and R5 is from Harvey 2017. The ground-truth annotations of these regions are obtained using the FloodTrace tool [Dyken et al., 2024], and some ambiguous pixels (e.g., those covered by tree canopies) are left unlabeled. See Appendix B [Adhikari et al., 2025] for more details.

Evaluation Metrics. Flood segmentation is a pixel-wise binary classification problem so we use commonly used measures such as accuracy, precision, recall, F1 score and intersection over union (IoU). The last four measures are calculated in two contexts: (1) "dry" is the positive class, and (2) "flood" is the positive class; we report mIoU as the mean of "dry" IoU and "flood" IoU. We only use the labeled pixels of test regions to compute these measures.

Setting. In each AL cycle, EvaNet is retrained on an A100 GPU for 3 epochs, and SEEDS [den Bergh et al., 2015] is used to compute superpixels for each satellite image. Appendix B [Adhikari et al., 2025] describes the detailed model hyperparameters, and Appendix C reports the detailed hyperparameter tuning results. Our user study includes five domain experts in the active learning pipeline, and all reported results are averaged over those of the experts.

Comparison with Baselines. While we are the first to apply AL for flood mapping, we here establish some reasonable baselines for comparison: (1) random sampling, where superpixels are recommended by uniform sampling; (2) using only the confidence measure for recommendation without considering consistencies; here, we use uoff (c.f. Eq (1)) as the uncertainty measure since it works generally better than uent (c.f. Eq (2)); (3) PixelPick [Shin et al., 2021], which recommends sparse pixels for labeling rather than superpixels. These baselines are compared against two ALFA variants: (i) ALFA-E, which uses uent as the confidence measure, and (ii) ALFA-PO, which uses uoff instead.

All methods initially use the off-the-shelf EvaNet model [Sami et al., 2024] for generating initial predictions, and the corresponding results are shown as 'Initial Metrics' in Table 2. The remaining metrics show the performance of these methods after 5 AL cycles. We can see that ALFA-PO consistently gives the highest accuracy, F1 score and mIoU on R2-R5, while ALFA-E is slightly better than ALFA-PO on R1. Among the baselines, random superpixel recommendation gives much lower performance than ALFA on R3-R5. PixelPick is frequently among the lowest performing methods since only sparse pixels are recommended and labeled, providing much fewer annotated pixels for retraining. Since ALFA-PO is the overall winner, ALFA-PO is used for ALFA by default hereafter.

Ablation Study. To verify the effectiveness of our consistency measures in both the acquisition function and the loss function, and the effectiveness of using ForestNet, we compare with 5 ALFA variants: (1) without view consistency in the acquisition function, (2) without temporal consistency in the acquisition function, (3) without tree score in the acquisition function, (4) without the view consistency loss term, (5) without the temporal consistency loss term. Without loss of generality, Table 3 shows the ablation study results on test regions R1 in NC and R5 in TX. We can see that the consistencies and ForestNet are all beneficial in ALFA.

Effect of AL Cycles. We study how the EvaNet performance as measured by the average F1 score (i.e., average of the flood and dry F1 scores) improves with the AL cycles. Figure 5 shows the results, where the x-axis corresponds to the percentage of pixels annotated so far at the end of an AL cycle, and the curve for each region has 5 data points corresponding to 5 AL cycles on the region. We can see that both the percentage of annotated pixels and the average F1 score improve with more AL cycles, and the average F1 score can reach a high value in just 5 cycles. Also, since each pixel click may cover different number of pixels via elevation-guided BFS, the percentage of increased annotated pixels varies with the cycles and regions. Other than R4 where the improvement seems to converge, the other regions show a clear performance improving trend so running more cycles is likely to further boost the performance.

Effect of AL in Speeding up Annotation. We show the total time taken by manual annotation (with elevation-guided BFS) and AL-assisted annotation (with ALFA) for all test regions in Figure 6. The time reported here is the median time obtained from the user study. We can see that AL assistance significantly speeds up the annotation. We also consider another baseline where users use AL-assisted annotation for 2 cycles and then fix EvaNet mistakes manually. Without loss of generality, the results on R1 and R5 are presented in Table 4, where we can see that to reach a similar level of average F1 score, using AL throughout the annotation process can reduce time from over 40 minutes to around 20 minutes.

Table 4: ALFA for 5 cycles (AL Only) v.s. ALFA for 2 cycles followed by manually fixing EvaNet prediction errors (AL + Manual).

5 Conclusion

We presented ALFA, an active learning framework for flood annotation on earth imagery. ALFA integrates prediction confidence, view consistency, temporal consistency and tree score to calculate the uncertainty scores of individual pixels, which are aggregated in the unit of superpixels to recommend the most uncertain (hence informative) ones by the underlying EvaNet flood segmentation model for user annotation by elevation-guided BFS. ALFA also expands EvaNet's supervised elevation-guided loss function with two unsupervised loss terms for view consistency and temporal consistency, respectively, to improve retraining performance. Extensive experiments show that all our techniques are beneficial and ALFA can obtain high-quality annotation for flood mapping in a much short time than manual annotation.

Acknowledgements

The authors at Indiana University Bloomington were supported by DOE ECRP Award DE-SC0025228, NSF OAC2414474, NSF OAC-2414185, the Radiance Technologies Innovation Bowl grand prize awarded as the competition winner in 2023, and 2024-2025 Luddy Faculty Fellow Award from Indiana University Bloomington. The research of Sidharth Kumar and Steve Petruzza were funded by NSF SHF2401274, NSF SHF-2221812, and NSF PPoSS Planning Award 2217036 and Large Award 2316157.