Statistically-Guided Deep Network Transformation to Harness Heterogeneity in Space (Extended Abstract)

1 Introduction

Spatial datasets are ubiquitous and collected at ever-growing scale, resolution, frequency and variety. Common types of spatial data include satellite/UAV imagery, points-of-interest (POI), GPS locations/trajectories, geo-tagged tweets, census data, maps (e.g., land cover, crimes, traffic accidents, COVID statistics), and many more. These spatial datasets are critical in a variety of societal applications, such as Earth observation (e.g., crop monitoring [GEOGLAM, 2021]), public health (e.g., COVID-19 mobility analysis [Kraemer and others, 2020]), public safety, transportation, etc.

†The original version of this paper [Xie et al., 2021] was published at IEEE ICDM 2021, and received the Best Paper Award.

‡Contact Author

While spatial datasets are both important and widely used, their intrinsic properties - spatial autocorrelation and heterogeneity - often undermine the independent and identical distribution (i.i.d.) assumption [Atluri et al., 2018; Shekhar et al., 2015]. Spatial autocorrelation means data samples are not independent as nearby ones tend to share higher similarity (e.g., landcover, temperature, mobility). Spatial heterogeneity, on the other hand, violates the identical distribution assumption as data are often generated by different processes over space. Even more challenging, such differences in distributions may not be reflected by variations in observed features [Goodchild and Li, 2021], and the spatial footprints of different generative processes could be arbitrary in shape due to complex social and physical contexts.

While the wide adoption of convolutional kernels [Krizhevsky et al., 2012] has explicitly filled the missing representation of spatial autocorrelation by local connections and maintained spatial relationships (i.e., non-vectorized), the complex spatial heterogeneity challenge has not been sufficiently addressed. In a recent study, a spatial-variabilityaware neural network (SVANN) was developed [Gupta and others, 2020; Gupta and others, 2021]. SVANN demonstrates the benefit (e.g., increase in accuracy) of separating out training data subsets belonging to known different distributions, but it requires the spatial footprints of heterogeneous processes to be known as an input, which is often unavailable in practice. Explicit spatial ensemble approaches aim to adaptively partition a dataset [Jiang et al., 2019], but the algorithm and its variation are specifically designed for binary classification problems and only allow two partitions. Outside recent literature on deep learning, a traditional approach to handle spatial heterogeneity is geographically-weighted regression (GWR) [Brunsdon et al., 1999; Fotheringham et al., 2017]. However, GWR is mainly designed for linear inference, and cannot handle complex prediction tasks commonly addressed by deep learning. Most existing methods also require dense training data across space to train models for individual partitions or locations, and cannot be applied to other regions outside the spatial extent of the training samples. Additional related work is discussed in [Xie et al., 2021a].

We propose a model-agnostic Spatial Transformation And modeRation (STAR) framework to automatically learn arbitrarily-shaped spatial footprints of different data distributions during network training, and generalize the learned partitioning scheme and model weights to other spatial regions.

Through experiments on real world tasks, i.e., satellitebased crop monitoring and COVID-19 human mobility projection, we show that the STAR framework can substantially improve model performance, capture flexibly-shaped spatial footprints of different processes, and can be effectively applied to prediction tasks in new test regions.

2 Deep Network Transformation

2.1 Heterogeneity in Space

Definition 1. Spatial process Φ: A function Φ : X ↦→ y governing data generation in a spatial region, which may involve observed and unobserved (or unknown) features as variables.

Definition 2. Spatial heterogeneity: An intrinsic property of spatial data stating that data are generated by different spatial processes {Φ} in different regions [Atluri et al., 2018; Shekhar et al., 2015; Goodchild and Li, 2021].

While deep networks can function as universal approximators for data following identical distributions [Kratsios and Bilokopytov, 2020], spatial heterogeneity commonly existed in spatial data violates this assumption (e.g., spatial data generated by two simple scalar functions y = x and y = -x across space cannot be approximated by a single network). As a result, the heterogeneous processes {Φ} will cause con-

fusion on data distribution during training, and hamper prediction performance and stability.

Moreover, another complicating factor we need to consider is the hierarchy of spatial processes across scales and their corresponding heterogeneity. For example, higher-level heterogeneity in the hierarchy may be caused by policies at larger scales, climate zones, major geographical barriers (e.g., mountains), whereas lower-level processes may vary by local policies, demographics, social/cultural contexts, and personal decisions. In addition, the spatial footprints of these different processes may be arbitrary in shape. Fig. 1 (a) and (b) show an example of mixtures of spatial processes at two different scales/levels, and this hierarchy is formally defined in Def. 3.

Definition 3. Spatial hierarchy of processes H: A multiscale representation of spatial heterogeneity [Xie et al., 2021b]. H represents the input spatial domain D as a tree; each node Hij ∈ H is a partition of D, where i is the level in the hierarchy, and j is the unique ID for each partition at level-i. Children of a partition Hij share the same lowerlevel processes (i.e., {Φ} at levels i′ < i). Φ is homogeneous within a leaf-node and heterogeneous across leaf-nodes.

2.2 Statistically-Guided Transformation

Based on Def. 3, we separate data samples belonging to different spatial processes Φ using a hierarchical structure. Specifically, at each step (Fig. 2), we identify an optimal space-bi-partitioning that maximizes the discrepancy of Φ between the partitions, and verify if the impact of separation is statistically significant for learning enhancement. The partitioning continues hierarchically until no significant heterogeneity can be recognized in new partitions. This transformation framework is a Dynamic and Learning-engaged generalization of the Multivariate Scan Statistic (DL-MSS) [Kulldorff and others, 2007; Neill and others, 2013; Xie et al., 2021c]. DL-MSS has three major components:

- Space-partitioning optimization with prediction error distribution: As spatial processes Φ : X → y are not directly observable, we leverage the function approximation power of a deep network F and use its prediction error distribution for a partition Hi j ∈ H as a proxy of Φij . Intuitively, if all data belonging to a partition Hi j are generated by a homogeneous Φij , we expect the error distribution for each target (e.g., class) predicted by a single model for Hi j to follow a homogeneous distribution over space; otherwise, heterogeneous. To obtain statistics on error rates, locations are first aggregated into unit local groups using a grid, where all samples in a grid cell form a group; a user may also choose a different grouping strategy. Using classification as an example, denote ˆyk,m as the predicted labels for samples with class m (i.e., true labels are m) at a cell sk. The number of misclassified samples of class m at sk is then ek,m = |ˆyk,m ̸= m|. Further, denote nk,m as the number of samples of class m at location sk; and Em and Nm as the number of misclassified and all samples of class m in the entire space. We identify an arbitrary set of cells S = {sk} that maximizes the error rate distribution discrepancy using Poisson-based likelihood ratio [Neill and others, 2013]:

where the null hypothesis H0 states that Φij is homogeneous, and H1 states that there exists a set S where the rate of having an error is qm times the expected rate under H0; and E(ek,m) = Em · nk,m Nm is the expected number of misclassified samples at location sk under H0. S∗ can be Phase-1: Spatial hypothesis by optimization with learned proxy of Φ

Data distribution

Figure 2: Illustrative example of the spatial transformation framework with dynamic and learning-engaged MSS.

• efficiently solved using the Linear-Time Subset Scanning (LTSS) property [Neill, 2012; Xie et al., 2021c] combined with coordinate ascent. More solution details and results on regression are available in [Xie et al., 2021a].

j1 and Hi+1

j2 , where one child

• Active significance testing with learning: Once the optimal S∗ is identified, the current node Hij will be temporarily split into two children Hi+1 corresponds to S∗ and the other for the rest of the space in Hij . To validate if the space-partitioning can lead to a statistically significant improvement on learning, we drop the traditional Monte-Carlo based descriptive test [Xie et al., 2021c] and use a learning-engaged active test. Specifically, we carry out two training scenarios with and without the split (Fig. 2) and perform an upper-tailed dependent T-test on the losses from the two scenarios [Xie et al., 2021a]. For the split scenario a new network branch will be created (e.g., adding a copy of the last L layers to the network) to allow private parameters (Fig. 1(d)). If the impact of S∗ is significant, we approve the new partitioning and network

branch; otherwise, the transformation on Hij is terminated.

• Spatial transformation via a dynamic and learningengaged spatial hierarchy H: As both the spacepartitioning and network parameters may be constantly updated during training, DL-MSS performs the first two key components as sub-routines for new partitions added to H to dynamically converge to the final H and heterogeneityaware network FH.

2.3 Spatial Moderation

The spatial hierarchy H and "spatialized" deep network FH learned and trained from the transformation step aim to capture heterogeneity in the spatial extent of the input X and y. However, the partitions cannot be directly applied to a new spatial region. To bridge this gap, we propose a spatial moderator, which translates the learned network branches in FH to prediction tasks in a new region. The key idea of the spatial moderator is to learn and predict a weight matrix W for all branches in FH (corresponding to all leaf-nodes in the spatial hierarchy H), and then use the weights to ensemble the

branches' predictions to get the final result (Fig. 3).

3 Beyond Spatial Data

The spatial transformation framework is designed for but not limited to spatial data. The model-agnostic approach can be applied to create explicit heterogeneity-awareness for general types of data, with the assumption that data samples can be combined into unit local groups (e.g., grid cells in this paper), as needed by the first component of DL-MSS in Sec. 2.2.

𝐗𝐗

Figure 3: Illustrative example of the spatial moderator.

4 Experiments

4.1 Datasets

California land-cover classification. We use multispectral data from Sentinel-2 satellites in two regions in Central Valley, California. Each region has a size of 4096×4096 (∼6711 km2 in 20m resolution). We first learn the spatial partitioning using the data from Region DA and then use the moderator to transfer it to Region DB. We use composite image series from May to October in 2018 (2 images/month) for time-series models, and one snapshot from August, 2018 for snapshot-based models. The labels are from the USDA Crop Data Layer (CDL) [CDL, 2021]. The training (and validation) set has 20% data at sampled locations in DA, and 1% data in DB is used for fine-tuning.

Boston COVID-19 human mobility prediction. Human mobility provides critical information to COVID-19 transmission dynamics models. We use the Boston COVID-19 mobility dataset from [Bao et al., 2020], which includes data from US census, CDC COVID statistics, and SafeGraph patterns. Here human mobility y is represented by the number of visits to points-of-interest (POIs; e.g., grocery stores, restaurants) and the counting is based on smartphone trajectories. The features include population, weekly COVID-19 cases and deaths, number of POIs, week ID and income. The dataset contains 12 weeks of data, and according to [Bao et al., 2020], we use the first 11 weeks for training/validation and the final week for testing.

4.2 Candidate Methods

For California land-cover classification, we have 12 candidate methods: (1) base F, spatially transformed FH, and FM (FH with moderator), each for three base architectures: DNN (snapshot-based and fully-connected), LSTM and LSTM+Attention [Jia et al., 2019]; and (2) F integrated with the model-agnostic meta-learning (denoted as meta)

Phase-2: Active significance testing with learning

Shared

…

…

Base

?

Split

…

…

…

𝑙𝑙𝑙𝑙𝑙𝑙𝑠𝑠𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏

Φଵ = Φଶ?

𝑙𝑙𝑙𝑙𝑙𝑙𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠

Pass test Phase-3: Repeat hierarchically Pass test Not pass Resolution ×2

after 2 splits

Repeat…

𝑯𝑯𝟐𝟐𝟏𝟏

𝑯𝑯𝟔𝟔𝟑𝟑

𝑯𝑯𝟐𝟐𝟑𝟑

Figure 4: Spatial hierarchy learned in region DA (first 3 levels).

[Finn et al., 2017; Yao et al., 2019] for fast adaptation in region DB with the available 1% of data (DA's data are clustered into tasks), for the three architectures.

For Boston COVID-19 human mobility regression, we have 5 candidate methods for comparison, i.e., ridge regression, geographically-weighted regression (GWR), COVIDGAN [Bao et al., 2020], DNN, and DNNH (spatially transformed version). As time-series is not used to construct additional features in [Bao et al., 2020] and some features are aggregated to week-levels, we follow the same strategy and only use week IDs as features, which also allows a more direct comparison with COVID-GAN. In addition, as training and test samples are from the same set of spatial locations (timestamps are different), we directly use spatial transformation and skipped the moderator in this comparison.

4.3 Results

Land-cover classification. As shown in Fig. 6, the "spatialized" network architectures overall achieved the highest F1-scores for different types of base models in both regions. For region DA, the general trend is that the results of a base model F gradually improve with the addition of spatial transformation FH, and the spatial moderator FM (e.g., the weighted F1 score increases from 0.49 to 0.59, and finally to 0.67 for DNN). In addition, Fig. 4 shows the hierarchical process of space-partitioning during spatial transformation for the first 3 levels. In the first level (largest scale), for example, H1 1 is a mix of urban and suburban areas, whereas H1 2 contains more rural and mountainous areas. Note that some partitions (e.g., H2 3 ) are not further split, as determined by significance testing. Finally, Fig. 5 visualizes the weights predicted by the moderator for two example network branches in FH for DNN for all locations in region DB. For each branch, the weight is averaged over all classes in the predicted W at each location. As we can see, in the new region DB, branch-2 is given higher weights for the left-side of the region, which is a mountainous area, whereas the weights for branch-7 shows the opposite spatial pattern.

COVID-19 mobility regression. Table ?? and Fig. 7 show the results of the candidate methods, where "Diff." refers to the difference between the total predicted and true mobility values (POI visits). Several potential causes of the spatial heterogeneity here include different mobility patterns in the more populous downtown area versus the suburban regions, and several "hotspot" areas of POI visits that are a bit abnormal compared to the rest. As we can see, overall DNNH achieved better results for the three measures. One interesting observation is that DNN (the base model F used for DNNH), while obtained better MAE than ridge regression and COVID-GAN, substantially underestimates the total mobility, which could be a result of incorrect predictions on several mobility hotspots, whose patterns do not follow the global pattern. Similarly, GWR shows a similar trend. Although GWR performs spatially-localized regression, it can only handle simple linear relationships using input variables and apply the same spatial neighborhood for all locations, which cannot well capture non-stationary mobility hotspots and variation in the data. Finally, DNNH automatically identified three heterogeneous partitions (other splits are statistically insignificant) and branched out downtown, suburban and several mobility hotspots (our method allows a single partition to contain multiple disjoint large-footprints), greatly improving the prediction performance.

Acknowledgments

Yiqun Xie is supported in part by NSF awards 2105133, 2126474 and 2147195, Google's AI for Social Good Impact Scholars program, and the DRI award at the University of Maryland; Erhu He and Xiaowei Jia are supported in part by NSF award 2147195, USGS award G21AC10207, Pitt Momentum Funds award, and CRC at the University of Pittsburgh; Han Bao and Xun Zhou are supported in part by the ISSSF grant from the University of Iowa, and SAFER-SIM funded by US-DOT award 69A3551747131.