Detection and Geographic Localization of Natural Objects in the Wild: A Case Study on Palms

1 Introduction

Palms (family Arecaceae) are vital to tropical ecosystems, serving as essential resources for pollinators and frugivores and influencing the evolution of dependent fauna [Eiserhardt et al., 2011; Zambrana et al., 2007]. Beyond their ecological roles, palms are deeply integrated in the livelihoods of rural and indigenous communities, providing food, construction materials, fuel, and medicine while supporting sustainable non-timber forest product markets. Particularly in regions like the Amazon, palms support subsistence practices and enhance resilience to socio-economic and environmental changes, reflecting the intricate links between biodiversity, human well-being, and sustainability [Pitman et al., 2014; Malhi et al., 2014; van der Hoek et al., 2019; Terborgh, 1986]. While their distinctive star-shaped crowns make palms well-suited for automated mapping [Sutherland et al., 2013; Wagner et al., 2020], the absence of scalable, precise tools to monitor these species across vast forests limits timely conservation efforts and support for dependent communities.

Orthomosaic images are a valuable tool in remote sensing, with applications across various fields such as construction site monitoring, agricultural planning, environmental impact assessments, property development, and surveying, where accurate spatial data is essential [Kucharczyk and Hugenholtz, 2021]. These images are created by stitching together hundreds or thousands of geolocated aerial frames into spatially coherent maps, enabling consistent analysis across landscapes. By providing a top-down view of large geographical areas, orthomosaics allow for precise measurement and detailed analysis of land features. Unlike standard aerial imagery, which contains perspective distortions, orthomosaics offer an in-scale representation of the ground. Low-flying drones enable efficient data collection without the need for extensive ground surveying [Olson and Anderson, 2021].

However, orthomosaics are not error-free and can be challenging to process. They often contain noise, artifacts, and stitching errors due to limited camera perspectives, variable lighting, wind-induced movement, and dynamic cloud cover during image acquisition [Zhang et al., 2023b]. Additionally, resolution loss is a frequent byproduct of the stitching process. Typically stored in GeoTIFF format, orthomosaics can be exceptionally large, particularly in biodiversity conservation, forest health, and sustainable management applications, where a single file may range from 1 to 200 GB. As a result, image analysis must be performed in small patches that fit within the memory constraints of processing hardware.

Object-based detection techniques, widely used in remote sensing, are well-established in computer vision. However, detecting and localizing naturally occurring objects in orthomosaics is inherently challenging [Tagle Casapia et al., 2019; Cui et al., 2024; Qin et al., 2021; Jintasuttisak et al., 2022]. Unlike structured plantations, where trees are arranged in grid patterns with ample spacing (see Figure 1), palms in tropical forests grow irregularly, often obscured by dense vegetation that distorts key diagnostic features. The distribution of palm species is highly imbalanced, with only a few dominant species present in sufficient abundance to train reliable detection models [Li et al., 2016; Freudenberg et al., 2019]. UAVbased imagery also introduces substantial lighting variability, as shadows from thick canopies and shifting sun angles create significant fluctuations in brightness and contrast across detection targets, further complicating model training and evaluation [Gibril et al., 2021]. These challenges are amplified by the lack of high-quality labeled datasets, whose creation often requires extensive fieldwork and expert annotation [Muscarella et al., 2020; Hidalgo Pizango et al., 2022].

Lastly, manipulation of geographical information data requires domain expertise that is not readily available to computational scientists. A flexible pipeline that can process spatially referenced survey imagery and provide precise geographic target coordinates would support valuable downstream applications, leveraging state-of-the-art machine learning methods. These applications include monitoring palm abundance and distribution over time and across vast areas, and developing adaptive and modular survey methods. Additionally, balancing accuracy with interpretability and calibration is essential for trustworthy ecological decisionmaking [Selvaraju et al., 2017; Kuzucu et al., 2025]. This study therefore introduces a novel end-to-end pipeline that addresses a major need in environmental monitoring, is computationally efficient and reliable, and can support variableresource processing in field applications.

Our work presents the following main contributions:

1. We construct and provide the PAlm Localization in Multi-Scale (PALMS) dataset. PALMS contains data from extensive fieldwork across 21 sites in western Ecuador, spanning a rainfall gradient from the Choco's wettest forests to the dry edge of the Sechura desert, which captures corresponding gradients in palm species composition and canopy characteristics. For training, we annotated 1,500 image patches from 2 sites with 8,830 bounding boxes. For validation, we manually marked 5,026 palm crown centers from 4 reserves to direct compare detected georeferenced centers with ground truth.
2. We develop the PRISM pipeline, a unified and modular pipeline for natural object detection, segmentation, and
3. counting. PRISM integrates object detection with zeroshot segmentation to generate georeferenced palm coordinates. The modular design allows easy interchange of detection/segmentation models, while calibration analysis and saliency maps enhance trustworthiness and interpretability for ecological applications.
4. We apply PRISM to the PALMS dataset and validate generalization across four reserves with diverse environmental characteristics and species compositions. PRISM accurately localizes palm centers with strong ground-truth alignment. Benchmarking results show high inference efficiency across diverse hardware settings, supporting large-scale ecological monitoring.

2 RelatedWork

2.1 PalmDetection and Localization

Advancements in UAV technology, image stitching, and machine learning have driven significant progress in palm detection, segmentation, and localization from orthomosaic imagery. However, most studies have focused on commercially valuable species, such as oil and date palms, given their economic importance [Li et al., 2016; Gibril et al., 2021; Zheng et al., 2021; Jintasuttisak et al., 2022; Putra and Wijayanto, 2023]. For instance, Li et al. used CNNs with a sliding window for oil palm counting in Malaysia, while Gibril et al. developed a U-Net variant for enhanced date palm segmentation in UAE. Zheng et al. proposed a Faster R-CNN variant with refined feature extraction and a hybrid classbalanced loss to monitor individual oil palm growth. More recently, YOLO-based approaches have been adopted: Jintasuttisak et al. applied YOLOv5 for detecting date palms from UAV imagery over UAE farmlands, while Putra and Wijayanto employed YOLOv3 to detect and count oil palm trees for sustainable agricultural monitoring in Indonesia.

In contrast, the detection and localization of naturally occurring palms in tropical forests is largely underexplored. Tagle Casapia et al. pioneered palm crown identification using random forest, showing machine learning's potential for individual palm counting. Ferreira et al. applied a fully convolutional neural network with morphological operations to refine palm species segmentation. Wagner et al. used U-Nets and high-resolution multispectral imagery from the GeoEye satellite to map canopy palms over a large region in Amazon.

2.2 Object Detection and Zero-Shot Segmentation

Object detection, a core computer vision task, identifies and localizes objects via bounding boxes [Zou et al., 2023] and underpins advanced applications such as image segmentation and object tracking [Wang et al., 2022; Ma et al., 2024; Li et al., 2024]. The field is dominated by methods using You Only Look Once (YOLO) and Detection Transformer (DETR).

The YOLO family frames detection as a regression task balancing speed and accuracy, with overlapping predictions typically resolved via non-maximum suppression (NMS). YOLOv8 [2023] improves feature fusion and adopts an adopts anchor-free detection head optimized for accuracy and speed. YOLOv9 [2024b] introduces programmable gradient information and the generalized efficient layer aggregation network to address information loss. YOLOv10 [2024a] removes NMS through consistent dual assignments in training and one-to-one inference matching, coupled with a refined CSPNet backbone and a lightweight head to reduce computational cost. YOLO11 [2024] further enhances performance with a refined CSP bottleneck, hybrid attention, and adaptive anchors with extended IoU loss.

DETR [2020] directly predicts object sets using learned queries, bypassing the need for post-processing such as NMS. DINO [2022] enhances DETR with contrastive denoising and hybrid query initialization, while DDQ-DETR [2023c] introduces dense query assignment for improved one-to-one inference matching. RT-DETR [2024] optimizes DETR for realtime use via a hybrid encoder and multi-scale feature fusion.

Segment Anything Models (SAMs) are advanced segmentation models capable of segmenting any object in images using prompts such as points, boxes, or text [2023; 2024; 2023a]. Trained on the SA-1B (1 billion masks, 11 million images), SAM enables zero-shot inference and often surpasses fine-tuned methods in accuracy and efficiency [2023]. Its architecture features a ViT for image encoding, a prompt encoder to process input prompts, and a mask decoder that fuses features from both to generate segmentation masks. SAM 2 [2024], trained on the SA-V dataset (50.9k videos, 642.6k masks), enhances video segmentation and object tracking by refining multi-scale feature extraction. Mobile SAM [2023a] optimizes SAM for mobile use by simplifying the image encoder and using decoupled distillation, enhancing speed without compromising segmentation quality.

3 PALMS Dataset

In this section, we introduce the PALMS (PAlm Localization in Multi-Scale) dataset. The following paragraphs detail the study sites, data collection, and manual labeling process.

Study Sites. Data in this study (See Figure 2) come from western Ecuador's Choco tropical forest, including Fundaci´on para la Conservaci´on de los Andes Tropicales Reserve and adjacent Reserva Ecol´ogica Mache-Chindul park (FCAT; 00◦ 23'28" N, 79◦ 41'05" W), Jama-Coaque Ecological Reserve (00◦ 06'57" S, 80◦ 07'29" W), Canande Reserve (0◦ 31'34" N 79◦ 12'47" W), and Tesoro Escondido Reserve (0◦ 33'16" N 79◦ 10'31" W). FCAT is a high diversity humid tropical forest at elevation ∼500m, receiving ∼3000 mm yr-1 precipitation with persistent fog during drier period. Jama-Coaque ranges from the boundary of the tropical moist deciduous/tropical moist evergreen forest at the lower elevations (∼1000 mm precipitation yr-1 , ∼250 m asl) to fog-inundated wet evergreen forests above 580m to 800m. Canande (350-500 m elevation) and Tesoro Escondido (∼200 m elevation) are lowland everwet Choco forests, both receiving 4000-5000 mm yr-1 precipitation with no month experiencing drought stress or precipitation below 100 mm. These forests host several palm species with exposed canopy crowns, including the economically important Iriartea deltoidea, Socratea exorrhiza, and Oenocarpus bataua, with lesser amounts of Attalea colenda and Astrocaryum standleyanum, and species composition varying across study sites [Browne and Karubian, 2016; Lueder et al., 2022].

Data Collection. We collected UAV imagery in two stages, capturing 8,845 photos across 21 areas spanning 1,995 hectares, with a ground sampling distance under 6 cm. In June 2022, the first stage covered 95 hectares and produced 387 photos, while the second stage in February 2023 surveyed 1,900 hectares and captured 8,458 photos. Both missions used a DJI Phantom 4 RTK drone equipped with a 1" CMOS sensor and GS RTK for mission planning. The first mission flew at 90 meters above ground level with 70% sidelap and 80% frontlap, while the second operated at 150 meters. The collected images were processed for subsequent analysis. To enable landscape-level forest analysis, we created orthomosaics and conducted post-processing steps using Agisoft Metashape 2.0, including noise removal, edge trimming, and the generation of digital surface and terrain models.

Manual Labels. To enable fine-tuned palm detection and zero-shot segmentation, we curated a dataset of 1,500 images (800×800 pixels) from two FCAT reserve sites, capturing varied quality and palm density typical of natural forests. Manual annotation of palm crowns and isolated leaves faced challenges from vegetation overlap and orthomosaic distortions. Three trained experts initially labeled bounding boxes, followed by iterative refinement using a YOLOv8 detector trained on these annotations to identify and correct labeling errors or omissions. For landscape-scale validation, a trained expert annotated 5,026 georeferenced palm centers on four orthomosaics using ArcGIS Pro 3.3.1, with predicted coordinates from PRISM to refine annotations. This hybrid humanmodel workflow addressed the inherent complexity of labeling in natural forest environments, ensuring robust annotations for both detection and geospatial validation.

4 PRISMPipeline

PRISM (Processing, Inference, Segmentation, and Mapping) is an end-to-end pipeline that process orthomosaic images to generate georeferenced palm coordinates alongside bounding boxes and segmentation masks for visualization. Fine-tuned on the PALMS dataset, PRISM addresses challenges specific to dense rainforest environments, such as irregular palm distributions and overlapping crowns. By integrating fine-tuned detection models with zero-shot segmentation, PRISM ensures adaptability to a wide range of environmental tasks. Figure 3 illustrates the PRISM pipeline, with the details of each model component discussed in the following sections.

Detection Model. We selected YOLOv10 [2024a] for its speed and performance. YOLOv10 introduces a consistent dual assignment strategy for NMS-free training, which combines one-to-many and one-to-one label assignments for enriched supervision while eliminating post-processing NMS. This approach uses a unified matching metric:

where p is the classification score, ˆb and b are the predicted and ground truth (GT) bounding boxes, and s is the spatial prior, with α and β balancing classification and IoU. Architecturally, YOLOv10 adopts an efficiency-accuracy-oriented design, integrating lightweight classification heads, spatialchannel decoupled downsampling, rank-guided block design, and advanced features such as large-kernel convolutions and partial self-attention, achieving robust performance with faster inference and fewer parameters. Trained on PALMS dataset, the model processes orthomosaic patches to output bounding boxes and confidence scores for further processing.

Segmentation Model. We chose SAM 2 [2024] for zeroshot segmentation due to its superior segmentation quality and improved speed compared to its predecessor. SAM 2 uses a hierarchical image encoder for multi-scale feature extraction and an optimized architecture that reduces computational overhead while maintaining high precision. Its efficient prompt encoding and memory attention mechanism enable rapid mask refinement during inference. During inference, bounding boxes from the detection model undergo NMS to remove duplicates, as patches are cropped from the orthomosaic with a stride. The remaining bounding boxes and their surrounding image regions serve as input prompts for SAM 2 to generate segmentation masks. Final outputs include visualized bounding boxes and masks, alongside georeferenced coordinates derived from NMS-cleaned bounding box centers, which are subsequently used to quantify counting accuracy.

Calibration. Calibration ensures model trustworthiness by aligning predicted confidences with empirical IoU, critical for ecological monitoring where overconfident errors can mislead conservation decisions. We evaluate four calibration methods to improve palm detection reliability [2025]: (1) Linear regression (LR) fits a linear logit-probability mapping; (2) Isotonic regression (IR) fits a monotonic function: minˆ y1, ˆ y2,..., ˆ yn ∑ni=1 (yi - ˆ yi)2 , subject to ˆ yi ≤ ˆ yj, ∀i < j; (3) Temperature scaling (TS) divides logits by a learned temperature T before sigmoid; (4) Platt scaling (PS) applies logistic regression to map logits to calibrated probabilities.

Interpretability via Saliency Maps. Grad-CAM [2017] enhances interpretability by highlighting regions most influential to palm predictions. For a class score y, it computes gradients of y with respect to feature maps Ak , yielding weights αk = 1Z ∑i,j ∂y ∂Akij , where Z is the number of spatial positions. The heatmap: Lmap = ReLU (∑k αkAk ) is then generated to highlight regions influencing predictions, such as palm crown or leaf patterns. These visualizations help ecologists assess whether predictions rely on meaningful cues

like crown shapes rather than irrelevant artifacts.

5 Experimental Results

This section evaluates the proposed pipeline across various tasks and study sites. We numerically compare the detection performance and computational efficiency of different models, and visually compare the results of different zero-shot segmentation models. Calibration metrics are examined to measure model trustworthiness by analyzing performance before and after calibration. Saliency maps are analyzed to trace attention shifts during inference. Next, we apply PRISM to orthomosaics with distribution shift to examine the counting performance in practical scenarios. Finally, real-time simulations are conducted to assess the detection and segmentation speed, as well as the computational demands across devices.

5.1 Experimental Setting

The detection models were trained on PALMS dataset that split into 80% for training, 10% for validation, and 10% for testing. Data splitting was randomly performed five times to ensure robust evaluation, with training and testing data drawn from different ecological sites, and validation data mixed from the two sites. YOLO variants and RT-DETR were trained for 100 epochs, while DINO and DDQ used 30 epochs. Data augmentations, including hue, saturation, and brightness adjustments, along with rotations, scaling, translations, and flips, were employed in training. For inference on landscape orthomosaics, a stride of 400 was used to ensure complete coverage, and NMS was applied to eliminate duplicate predictions. Training was conducted on an RTX 4090 GPU, and testing was performed on multiple hardware platforms: RTX 3060 Laptop, RTX 4090 and H800 GPUs.

Detection was evaluated using precision, recall, AP50, AP75, and mAP. Consistency was assessed by reporting the standard deviation of these metrics across five random experiments. Computational efficiency was evaluated using GFLOPS and parameter counts (Params) to measure model complexity, and FPS to assess inference speed, illustrating their suitability for real-world applications.

We evaluated the calibration performance of detectors by

where J = 25 denotes the number of confidence bins, D and Dj represent the total detections and detections in bin j, and pj and IoUj are the average confidence and IoU within bin j. For LaACE0, pi and IoUi correspond to per-detection confidence and IoU [Kuzucu et al., 2025]. We evaluated the counting performance using retrieval ratios between predicted and GT centers within d = 5 meters and report the median shift of successful matches to quantify spatial deviation. For realtime analysis, we assess the inference speed on raw images that were directly taken from UAVs.

5.2 Detection and Segmentation Performance

Table 1 compares the detection performance when training and testing on different geographical regions within the FCAT reserve, introducing natural distribution shifts. YOLOv10 demonstrates superior efficiency with 177.04 FPS and 31.6M parameters while achieving competitive accuracy, particularly in higher-quality metrics like AP75 (67.94%) and mAP (61.73%). It marginally outperforms other YOLO variants by 0.17% AP75 and 0.11% mAP. DINO's lower AP50 suggests reduced detection capability under shifted distributions. DDQ attains the best recall (85.66%) and a solid AP50, but its lower FPS limits real-time applicability. RT-DETR achieves the highest precision (88.69%) but low recall (75.98%), indicating numerous missed palms. These results position YOLOv10 as optimal for speed-critical deployments, with DDQ reserved for high-recall scenarios.

Detection analysis under challenging conditions reveals model-specific capabilities (see Figure 4). All models detect large palms reliably, even when overlapping, but exhibit lower confidence on small palms. DETR-based methods demonstrate superior small-object detection performance (first-row top-left/bottom, second-row center), whereas YOLO-based methods excel in occluded scenarios with partial leaf visibility. In such cases, YOLO variants successfully detect individual palms while DETR-based approaches frequently merge detections (first-row bottom, second-row top-right). The sliding-window processing of orthomosaics in PRISM ensures that partially visible palms near edges are typically captured in adjacent patches, thereby compensating for missed edge detections.

We evaluated zero-shot segmentation performance using images from four reserves with distinct ecosystems, where bounding boxes generated by the detection model (trained on geographically distinct regions) introduce distribution shifts when used as prompts during inference, partly due to variations in palm species across sites. Figure 5 demonstrates robust zero-shot segmentation performance despite occasional errors from imperfect bounding box predictions. SAM occasionally produces fragmented palm leaves due to partial segmentation and background misclassification, while Mobile SAM exhibits background over-inclusion. SAM 2 offers more balanced results, handling areas near palm boundaries more effectively. Considering both segmentation quality and computational efficiency (analyzed in §5.5), SAM 2 is the optimal choice for the segmentation backbone in PRISM.

5.3 Calibration Analysis and SaliencyMaps

To enhance confidence reliability in palm detection, we evaluated calibration performance before and after applying calibration techniques. Confidence scores should align with IoU, and LRP-based thresholding (A) is used to filter out lowconfidence detections [Kuzucu et al., 2025]. As shown in Table 2, IR achieves the best LaECE0 for most models, while YOLOv10 benefits most from TS and DDQ from LR. For LaACE0, the optimal method varies - TS improves RT-DETR and YOLOv10, whereas LR is most effective for DDQ and YOLO11. Notably, no single method is universally best, and in some cases, lowering LaECE0 increases LaACE0. Figure 7 further illustrates the effect of LRP-based thresholding and calibration on YOLOv10's confidence distribution relative to IoU. Initially, low-confidence predictions (< 0.05) dominate, skewing calibration metrics. LRP filtering refines the distribution by removing unreliable detections. Post-calibration, both LaECE0 and LaACE0 further decrease, indicating improved alignment between confidence scores and IoUs.

Grad-CAM saliency maps illustrate YOLOv10's hierarchical feature extraction across layers. Early layers (Figure 6(bc)) capture fine-grained details, such as palm leaf boundaries and edge contrasts, progressively refining features from edges to full palm structures. Mid-level layers (Figure 6(d-e)) expand spatial context, enhancing palm localization. Deeper layers specialize in object scales: layer 16 emphasizes small features like individual leaves, layer 19 focuses on mediumsized palm crowns, and layer 22, designed for larger palms, shows limited relevance due to their absence in the scene.

5.4 Counting Performance

Table 3 shows site-dependent palm counting performance. Pred2GT ratios (proportion of predictions matched to GT) remains stable (0.8956-0.9361), while GT2Pred ratios (proportion of GT matched to predictions) vary markedly (0.7667-0.9253). Tesoro Escondido demonstrates strong bidirectional alignment with near-symmetric ratios, indicating balanced performance in localization and detection. FCAT, despite being a distinct sub-region, shows training-data familiarity through its high GT2Pred ratio and the smallest GT2Pred median shift. However, its elevated Pred2GT shift (1.10m) suggests overfitting to sitespecific features, yielding "precise but misplaced" predictions. Canande's moderate Pred2GT ratio masks severe recall failures (GT2Pred: 0.767), where unmatched GT palms are likely true misses rather than localization errors. JamaCoaque, with a low GT2Pred ratio (0.815) and the largest median shifts (Pred2GT: 1.50m, GT2Pred: 1.14m), reflects systematic challenges in both detection and localization. All sites maintain sub-1.5m median shifts, confirming robust localization despite occlusions or partial visibility.

Figure 8 illustrates these trends: Canande's smaller 90 th percentile Pred2GT shifts (Figure 8a) align with its precisebut-conservative predictions, while FCAT and Tesoro Escondido's dense canopies enable "proximal counting", where predictions align with clustered GT instances (e.g., overlapping crowns). This phenomenon partially inflates match rates in dense regions despite minor localization inaccuracies. FCAT's strong GT2Pred performance highlights training benefits, whereas Jama-Coaque's weaknesses underscore the need for targeted improvements in low-detection settings.

5.5 Real-Time Simulation

We evaluated the computational feasibility for real-time onboard processing by simulating the detection-segmentation pipeline on raw UAV imagery using three GPU configurations as shown in Table 4. Testing across 20 images with heterogeneous palm densities demonstrated robust detection performance: YOLOv10 achieves real-time inference speeds (1.21-5.70 s/image) with low temporal variance, confirming that mid-range hardware (an RTX3060 Laptop) remains capable of sustaining real-time detection tasks. Segmentation times, however, show substantial variability, as run-time scales linearly with detected palm count. Although segmentation provides valuable auxiliary visualization, its computational cost makes it optional for latency-critical deployments. These results confirm detection as the time-determining component, with YOLOv10's stability and speed meeting realtime UAV operational requirements.

6 Conclusion

We presented PRISM for automated palm detection and segmentation using UAV imagery, validated on the western Ecuador's ecologically diverse reserves. PALMS dataset (21 sites, 8,830 bounding boxes, 5,026 georeferenced palm centers) captures climatic and species diversity critical for biodiversity monitoring. The modular pipeline achieves real-time processing across GPUs, with sub-1.5m median localization shifts under environmental distribution shifts, while calibration analysis and saliency maps ensure trustworthiness and interpretability. Future work will deploy PRISM on edge devices for UAV-integrated field validation. PRISM's design addresses challenges specific to palms - occlusion, irregular spacing, and lighting variability - which are common to detecting trees in structurally complex environments (e.g., eastern white pines). This robustness allows adaptation to other ecologically critical tree species in wild ecosystems, and to lower-resolution satellite imagery (0.5-1m), which offers significantly reduced data acquisition costs (vs. frequent aerial/ground surveys) for scalable ecological monitoring. This positions PRISM as a useful tool for diverse stakeholders, enabling ecologists, conservation practitioners, and even local communities to apply advanced image analysis for better forest management and biodiversity protection.

Acknowledgments

We thank the IRSC Lab at WFU and CINCIA for their collaboration and expertise. We also thank Jordan Karubian and Paul Fine for their insights on palm human use and economic importance. This research was supported by the US NSF BEE 2039850, and CityU Starting Grant 9380162.

Contribution Statement

Kangning Cui (corresponding author) and Rongkun Zhu contributed equally to this work.