OpenCarbon: A Contrastive Learning-based Cross-Modality Neural Approach for High-Resolution Carbon Emission Prediction Using Open Data

1 Introduction

Accurately accounting for high-resolution urban carbon emissions has become an increasingly critical issue [Stechemesser and Guenther, 2012]. Urban carbon emissions now account for more than 70% of global emissions [Agreement, 2015; Bank, 2021] and continue to rise. To fully understand the sources of these emissions and effectively design carbonreduction policies targeting high-emission areas, it is crucial to account for urban carbon emissions at a high resolution [Stechemesser and Guenther, 2012].

∗Corresponding author.

Despite the importance of emission accounting, traditional carbon accounting methods face a trade-off between accuracy and the effort required for data acquisition [Cai et al., 2018; Olivier et al., 1999; Dai et al., 2016]. Existing approaches to carbon emission accounting generally fall into two categories: bottom-up and top-down [Hutchins et al., 2017; Böhringer and Rutherford, 2008; Böhringer, 1998]. The bottom-up approach, which relies on point emission statistics from sensors or fine-grained activity data [Gurney et al., 2020; Gurney et al., 2009], is accurate but requires extensive data collection, making it unsustainable. In contrast, the topdown approach distributes regional sectoral emission totalscalculated from fuel consumption statistics-across highresolution areas using proxy factors [Olivier et al., 1994]. However, the ideal assumption of positive correlations between proxies and emissions often leads to inaccuracies. Given the strengths of machine learning and deep learning in identifying complex correlations, along with the vast availability of open data-which is widely used in socioeconomic prediction tasks [Yeh et al., 2020; Aiken et al., 2022; Xu et al., 2020; Wang et al., 2016]-developing a model that predicts carbon emissions using open data presents a promising solution. Once trained, the model can accurately infer a region's emissions using readily available open data, maintaining accuracy while significantly reducing costly data collection efforts.

Nevertheless, directly applying existing open data-based socioeconomic prediction methods to predict carbon emissions fails to capture the unique and challenging characteristics of carbon emission spatial distribution: the functional effect and spatial agglomeration effect. While existing socioeconomic indicators typically reflect one type of activity or urban functionality, carbon emissions result from the combination of diverse functionalities that have distinct effects on emissions [Dhakal, 2009; Liu et al., 2020] (Fig 1(a)). For instance, traffic roads generate transportation carbon emissions as vehicles burn petrol [Wang et al., 2015], while residential areas involve heating and cooking activities that consume natural gas and liquefied petroleum gas [Nejat et al., 2015; Zhang et al., 2015]. Learning the implicit and coupled relationships between such functionalities and carbon emissions is complex and challenging. Meanwhile, urban carbon emissions exhibit a strong and unique spatial agglomeration effect, originating from the continuity in city functionality layouts [Han et al., 2018; Wang et al., 2018; Wang et al., 2020]. As shown in Figure 1(b), adjacent areas in New York City have similar emission levels, and the overall carbon emission distribution in the city exhibits continuity. Characterizing such continuity is challenging but essential.

To model the unique characteristics of carbon emissions, we incorporate two types of open data-satellite images and point-of-interest (POI) data-to predict high-resolution spatial carbon emissions for two main reasons. First, satellite images and POI data provide complementary insights into a region's functionalities. Satellite images offer a relatively macroscopic yet comprehensive overview of how the city's static functional zones are distributed [Xu et al., 2020; Zhao et al., 2022], while POI data captures the fine-grained dynamic intensity of human activities, reflecting the rapid and continuous updates of facilities within urban functional zones [Fan et al., 2018]. Second, while satellite images convey information about the spatial layout of each functional zone, POI data is typically geo-tagged and contains valuable location-specific information. Together, these data types form the foundation for modeling the spatial correlations between urban functionalities and activities. Therefore, our primary goal is to effectively leverage these two data types to model the unique and challenging aspects of estimating high-resolution urban carbon emissions.

To bridge the gap, we propose our model, OpenCarbon, to predict high-resolution urban carbon emissions with open data of satellite images and POI data. To model the coupled and implicit functional effects from the complementary open data modalities, we propose a cross-modality information extraction and fusion module. This module first extracts information from each modality, explicitly modeling the distinct effects of different functionalities using a function-dimension-wise attention mechanism. It then extracts complementary functional information across the two modalities through a contrastive loss design, aiming to enable each modality to borrow underrepresented information from the other. To capture the spatial contiguity of functionality layouts underlying the spatial agglomeration effect, OpenCarbon features a neighborhoodinformed agglomeration modeling module that uses convolutional layers to extract the global context of the neighborhood and a cross-attention mechanism to model grid-neighborhood interactions. Our contribution may be summarized into three parts:

• We tackle the critical challenge of data collection in highaccuracy, high-resolution carbon accounting by developing an open data-based model that provides accurate carbon estimation while significantly reducing the data collection burden.
• Recognizing the unique functional and spatial agglomeration effects of urban carbon emissions, we propose a robust neighborhood-informed attentive neural network with contrastive learning for cross-modality fusion that achieves complementary modality extraction and spatial contiguity modeling.
• Extensive experiments on three large-scale real-world datasets validate our model's superiority over existing state-of-the-art methods, with an average increase of 26.6% in terms of R2 , showing our model's potential to facilitate convenient high-resolution carbon estimation and the carbon governance that follows.

2 RelatedWorks

2.1 Conventional High-resolution Carbon Emission Accounting

High-resolution accounting of carbon emissions is essential for effective, targeted carbon emission governance. Traditional carbon emission accounting methods are generally classified into two categories: bottom-up and top-down approaches [Böhringer and Rutherford, 2008]. Bottom-up methods aggregate monitored point emission data to the desired resolution [Gurney et al., 2020; Gurney et al., 2009], but they require extensive data collection and are not easily scalable across large areas. In contrast, top-down methods distribute regional carbon emissions, typically calculated from regional fuel consumption statistics, to the target resolution using proxy factors [Olivier et al., 1994], such as population. However, because carbon emissions are influenced by a range of factors, relying solely on proxy factors in top-down approaches often leads to inaccuracies. Therefore, we conclude that conventional high-resolution carbon accounting methods face a dilemma between accuracy and the substantial effort required for data collection.

2.2 Carbon Emission Prediction with Open Data

With the development of machine learning and deep learning and the boom in open data, researchers made a large trial in incorporating them to relieve the data collection pressure [Yang et al., 2020; Lu et al., 2017; Chen et al., 2024; Wu et al., 2022]. Yang et al. [Yang et al., 2020] ensembled several multi-layer perception models to predict city-level emissions with nighttime light satellite imagery. Chen et al. [Chen et al., 2024] constructed a graph neural network to model the spatiotemporal dependencies between nearby counties or districts, predicting temporally fine-grained emissions using available monthly emissions statistics. However, most of these methods target region-level or city-level carbon emissions and may not be suitable for high-resolution prediction due to the limited resolution of the data they involve, or because they inadequately characterize fine-grained functional and spatial interaction information. Therefore, we conclude that the potential of high-resolution open data for predicting carbon emissions remains underdeveloped.

2.3 Socioeconomic Prediction with Satellite Images and POI Distribution Data

Since our solution uses satellite images and POI distribution data as inputs, we review existing works that leverage these data types. Most studies incorporating POI data represent POIs as a distribution count vector. Wang et al.[Wang et al., 2016] developed a neural network model to predict neighborhood crime rates using the POI vector, while Yu et al.[Yu et al., 2018] used POI counts to predict smartphone application usage. For satellite imagerelated works, visual encoder networks have been applied to learn task-specific image representations [Yeh et al., 2020; Perez et al., 2017]. Researchers have also used unsupervised and self-supervised methods to create unified representations for various socioeconomic tasks. Jean et al.[Jean et al., 2019] designed a triplet loss function to enhance representation similarity for nearby grids, and Xi et al.[Xi et al., 2022] proposed similarity metrics for geo-adjacent grids and grids with similar POI distributions to improve grid semantics. However, to the best of our knowledge, no work has specifically explored the modeling potential of these two complementary data sources for high-resolution carbon emissions. The challenge of effectively extracting complementary information and modeling the implicit functional and spatial agglomeration effects remains underexplored.

3 Preliminaries &Problem Statement

3.1 Definitions

As our work incorporates two types of open data, the satellite image, and POI distribution data, we provide their definition and collection sources here.

Satellite Image: Satellite images are overhead-view pictures of the Earth's surface taken by satellites orbiting the planet. The recent development of remote sensing has enabled the acquisition of global, high-resolution, and safe satellite images [Campbell and Wynne, 2011]. Our data source is ArcGIS 1 , whose maximum pixel resolution is 1.19m.

POI Distribution: Point-of-interest(POI) data is the geotagged facility distribution data that is widely available in various map services and data providers. Some crowd-sourcing map service providers, such as OpenStreetMap, provide access to the global POI data [Zhou et al., 2022]. In this study, we use public POI data with location and category information provided from SafeGraph 2 and Tencent Map.

1 https://www.arcgis.com/home

2 https://docs.safegraph.com/docs/places

Notably, all two data modalities are sourced from publicly available, privacy-protected sources with no security risks.

3.2 Problem Statement

We target urban carbon emissions at a spatial granularity that has been paid much attention to in carbon emission governance, the 1km × 1km grids. For each grid i in the grid set A, given its corresponding satellite image and POI distribution data, our task is to predict its yearly carbon emission.

4 Method: OpenCarbon

In this section, we present our method, OpenCarbon (Figure 2), which predicts high-resolution carbon emissions using open data. OpenCarbon comprises two main components: (1) a cross-modality information extraction and fusion module that leverages complementary functional information from two modalities and models their interactions, and (2) a neighborhood-informed aggregation module designed to capture the continuity of city functionality layouts, thereby modeling the agglomeration effect. In the following sections, we provide a detailed introduction to these two modules and the training process.

4.1 Cross-modality Information Extraction and Fusion

Single-modality Representation Learning

Satellite images offer a broad overview of the functional layouts and their scale, whereas POI data provides complementary, detailed insights into the functionality and activity intensity within each layout. For example, while satellite images provide an immediate sense of a shopping mall's scale, they offer little insight into the composition of its internal store categories. POI data supplements this understanding. Therefore, aligning the observation scales of both modalities is crucial, allowing our model to effectively match corresponding information and develop a comprehensive representation of the area's functionalities.

We use satellite image pixels as the fundamental unit of granularity to organize the geo-tagged POI data into a threedimensional matrix with consistent pixel dimensions. The first two dimensions represent the spatial distribution of the POIs, while the third encodes their types. The matrix is constructed by mapping each POI to its corresponding matrix element based on its position within the grid. In this way, we obtain the grid-level matrix inputs of the satellite images and POI distribution.

For the satellite image modality, given its visual nature, we incorporate the ResNet-18 [He et al., 2016] architecture to obtain the embedding. Since the POI matrix is relatively sparse, we apply multiple layers of CNNs to obtain a representation of the POI distribution. However, since carbon emissions result from the aggregation of various activities, with distinct and implicit relationships between each activity type and its corresponding emission, we introduce a squeeze-and-excitation (SE) block [Hu et al., 2018] into every CNN layer to account for functionality heterogeneity in the POI distribution representation.

Specifically, for any input X ∈ RH×W×C to the CNN layer, a global average pooling layer serves as the squeeze function to capture the global information of each channel, which can be formulated as

An excitation function is then introduced to generate attention weights for different channels, scaling the input along the function dimension. The process can be expressed as follows:

Here, W1 ∈ R Cn ×C and W2 ∈ RC× Cn are the learnable projection matrices, and n represents the tunable compression ratio.

Cross-modality Complementary Information Fusion

Once each modality has been represented individually, the next step is to enable each modality to learn complementary information from the other and effectively fuse the two. To accomplish this, we employ contrastive learning, aiming to allow each modality to borrow and encode missing or underrepresented information from the other. Specifically, denoting the input from the satellite image modality as Xs and that from the POI modality as Xp, we introduce the well-acknowledged contrastive learning loss, NT-Xent (Normalized Temperaturescaled Cross Entropy) loss [Chen et al., 2020], which can be formulated as

Here sim(xi, xj) represents the cosine similarity between xi and xj. τ is the temperature parameter, used to adjust the distribution of similarity values. N represents the number of samples in a batch.

After obtaining the complementary representations, we apply the aggregate-attention mechanism to adjust the weights of the two modalities during the representation fusion. Specifically, the mechanismmaps the representations to a score space and adaptively learns the scores so that the more informative modality receives a higher weight. The attention weights are computed as:

Here W is a learnable matrix, and a and b are learnable vectors. The fused grid-level representation Xg is

4.2 Neighborhood-informed Agglomeration Modeling

While a graph structure can represent grid adjacency, it does not capture the relative position information between each grid, as the neighbors of a graph node are unordered. Therefore, we turn to convolutional layers to model the global representation of the neighborhood. Specifically, in single-modality representation learning, we obtain grid-level representations for satellite images Xs ∈ Rm and POI data Xp ∈ Rm. Considering the M ×M grids surrounding the target grid as the contextual range (where M is a hyperparameter), we concatenate the representations of the two modalities into two matrices, Xn,s and Xn,p, each of size RM×M×m. Each matrix is passed through several convolutional layers to learn the spatial agglomeration correlations of its neighborhood. The global representations are then aggregated using the previously introduced aggregate-attention mechanism. This process results in a neighborhood-level representation Xn of the grid.

To capture the spatial continuity underlying the spatial agglomeration effect, we introduce the standard cross-attention mechanism. We use the contextual neighborhood representation as the query vector to model the spatial correlations and interactions. In this way, we obtain the final grid representation Xf g , which can be formulated as:

4.3 Prediction &Training

By inputting the final grid representations Xf g into feedforward layers, we get the predicted grid carbon emission ˆ Y . During training, we employ the well-acknowledged mean absolute error loss as the prediction loss function, which can be formulated as

The overall loss function is a weighted addition of the contrastive learning loss and the prediction loss:

where A is the grid set of our target region and α is a tunable coefficient. Notably, in implementation, we add the contrastive learning loss during training after 100 epochs to prevent information collapse between the two modalities in the early stages of training.

5 Experimental Results

5.1 Experimental Setups

Datasets

We select three representative regions with varying levels of development for a comprehensive evaluation: Greater London (UK), Beijing (China), and Yinchuan (China). The first two regions are relatively well-developed, yet they exhibit distinct industrial structures and emission levels. In contrast, Yinchuan is a relatively underdeveloped region. The sources and basic statistics of the three datasets are provided in Table 1. The grid-level carbon emission statistics are all collected from ODIAC [Oda et al., 2018], a well-acknowledged carbon emission inventory constructed using the hybrid means of point emission calculations and top-down allocations. Consistent with ODIAC, we set our prediction resolution as 1km × 1km.

Baselines

We compare with both carbon emission prediction methods and satellite image-based socioeconomic indicator prediction methods. The carbon emission prediction methods typically incorporate related statistics to predict regional carbon emissions and we adapt them to the high-resolution grid level:

• SVM [Mladenovi´c et al., 2016]. Support vector machine method for emission prediction.
• Stacked-RFR [Zhang et al., 2022]. An ensemble two-layer stacked random forest regression model.
• BPNN [Zhang et al., 2021]. A classical back propagation neural network.
• CarbonGCN [Chen et al., 2024]. A graph neural network for neighborhood interaction modeling.

Since the inputs of these methods are vector-based, we transform satellite images into vectors by an acknowledged pretrained-encoder [Han et al., 2020] and concatenate them with the POI count vector as input. Satellite image-based socioeconomic indicator prediction methods include:

• READ[Han et al., 2020]: A pretrained satellite image model using transfer learning on a large-scale partially-labeled dataset to learn robust and lightweight representations.
• Tile2Vec[Jean et al., 2019]: Models the first law of geography by using triplet loss to maximize the similarity of geo-adjacent satellite image representations.
• PG-SimCLR [Xi et al., 2022]: Encourages geo-adjacent grids with similar facility distributions to have similar representations.

We adapt the unsupervised Tile2Vec and self-supervised PGSimCLR methods into a supervised approach by adding a carbon emission prediction loss term and balancing the multiple losses. We also incorporate POI count vectors at the prediction stage for fair comparison.

Metrics and Implementation

To measure the prediction performance, we adopt three commonly used evaluation metrics: mean absolute error (MAE), rooted mean squared error (RMSE), and coefficient of determination (R2 ) [Xi et al., 2022]. We performed a grid search on our hyperparameters, including learning rate, batch size, and neighborhood scale, and balancing coefficient. The search range for the neighborhood scale is {3, 5, 7}, and that for the balancing coefficient is {1e-1, 1e-2, 1e-3}. The hyperparameters of the baselines are all carefully grid searched. During training, we partition the dataset by regional divisions, selecting certain sub-regions for training and validation while reserving the remaining sub-regions for testing. As an example, for the Beijing dataset, we perform training and validation on five districts (including Xicheng and Chaoyang) and conduct testing on the Fengtai district. We perform experiment on an NVIDIA RTX 4090, with 12-20 GB GPU memory usage and < 4 hour training time.

5.2 Overall Performance

We conduct extensive experiments on three distinct datasets to comprehensively evaluate the performance of OpenCarbon. From Table 2, we can draw several insights:

• Our model's superior performance across cities with varying levels of development. As shown in the table, our model attains the highest performance across all three datasets, surpassing the best baseline by 28.4%, 11.3%, and 40.1% in R2 for Great London, Beijing, and Yinchuan, respectively. Across the three datasets, OpenCarbon attains an average R2 of 0.6997-a level that is highly satisfactory for practical applications. These improvements demonstrate the consistent superiority of our model, regardless of the region or the data source.
• Insufficient ability of existing methods in utilizing multimodality open data to capture carbon emission's unique characteristics. Existing carbon emission prediction methods struggle to utilize satellite images, missing their rich information. Meanwhile, current image-based approaches fail to effectively combine POI data, overlooking the complementary value of both modalities in modeling urban functionality. As a result, they cannot fully capture the functional impact on carbon emissions. Moreover, CarbonGCN's poor performance highlights the limitations of graph structures in representing the continuous layout of urban functions-a

challenge our neighborhood-informed agglomeration module addresses effectively.

5.3 Ablation Study

To precisely the contribution of each design in OpenCarbon, we conduct an ablation study on one developed region, London, and one developing region, Yinchuan. Specifically, we remove the see attention and contrastive loss design in the cross-modality information extraction module, together with the neighborhood-informed aggregation module respectively. As shown in Fig 3, removing the SE attention module brings about a performance decrease of 3.8% on R2 , indicating the necessity to consider the distinct importance of different categories of facilities. Also, removing the contrastive loss design decreases R2 by 6.49% and increases MAE by 9.49% on average. Such a performance decrease proves the essential contribution of the contrastive loss mechanism in extracting the complementary functional semantics from both the satellite image view and the facility distribution view. Further ablation studies on the neighborhood-informed aggregation module, which brings a 31.8% decrease of R2 , reveal the significant contribution of the module to characterize the agglomeration effect of emission distributions.

5.4 Generalizability Study

Unlike common socioeconomic indicators, emission factors vary across regions due to differences in energy structure and accessibility, making the same activity in different areas cause different carbon emissions. Therefore, there is an inherent distribution shift in carbon emissions that hinders the generalizability of prediction techniques. However, our experimental analysis shows that despite this distributional deviation, OpenCarbon's prediction results maintain their internal rank order, allowing governors to identify high-emission areas with significant mitigation potential. To evaluate our model's ranking ability, we perform direct transfer experiments: transfer the trained model to the new region directly without further finetuning, and use Spearman's Rank Correlation Coefficient to assess the consistency of emission magnitudes across grids. Since POI data from our three main datasets are not categoryaligned due to differences in their collection sources, we did not perform generalizability across our main datasets but instead collected data on four ceremonial counties in England for testing. As shown in Figure 3, OpenCarbon performed well across all twelve source-target pairs, achieving an average Spearman's rank correlation coefficient of 0.6204, outperforming PG-SimCLR by 26.56%. This demonstrates OpenCarbon's strong generalizability in ranking emissions, highlighting its effectiveness in modeling the relationship between land use, activities, and carbon emissions.

5.5 Case Study

Grid Representation Visualization v.s. Carbon Emission

This section examines the distribution of grids with varying carbon emissions in the representation space. We use t-SNE [Van der Maaten and Hinton, 2008] to project the representations from Equation 5 into 2D, with dot colors indicating carbon emission levels. The direction in the representation space corresponds to increasing emissions. We select six anchor points along this direction and show a representative satellite image for each in Figure 5. Carbon emission levels generally correlate with land use type and density. Grids near anchor point 1 represent underdeveloped farmland and forests, while those near anchor point 2 correspond to rural areas with basic infrastructure. Anchor point 3 shows denser structures and roads, anchor point 4 includes residential and industrial zones, anchor point 5 covers high-density urban areas, and anchor point 6 highlights highly urbanized, non-residential areas. These observations highlight the link between carbon emissions and grid functionality, emphasizing the importance of incorporating functional effects into our design.

Explanatory Insights

In OpenCarbon, we incorporate aggregate-attention to aggregate the representations of the satellite image and POI modalities. Since the aggregate-attention mechanism assigns softmaxed weights to these two representations and performs a weighted sum, the change in weight during emission rise reflects the relative importance of each modality. As shown in Figure 6, for grids with higher emissions, the weights for POI modalities consistently increase at both the grid level and the neighborhood level. This interesting phenomenon suggests that fine-grained functional information becomes more important for emission prediction modeling of high-emission areas, which tend to have a higher concentration of functional activities. Furthermore, one insight from this is that, while the overall layout of the city remains largely static and difficult to change, carbon emission mitigation efforts could focus on planning specific functional activities within these layouts, steering them toward more carbon-friendly options.

6 Conclusion and FutureWork

In this paper, we introduce OpenCarbon, a neighborhoodinformed attentive neural network with contrastive learning for cross-modality fusion, designed to predict high-resolution carbon emissions using open data from satellite images and POI distribution. By capturing the unique functional and spatial agglomeration effects of high-resolution carbon emissions, our model significantly outperforms existing methods for both carbon emission prediction and open data-based socioeconomic forecasting. Once trained, OpenCarbon can efficiently predict carbon emissions for new locations based on the corresponding open data inputs, greatly reducing the data collection burden of traditional carbon accounting, thereby enabling targeted carbon governance and mitigation planning, and driving progress toward urban sustainability.

Acknowledgements

This work was supported by the National Key Research and Development Program of China under 2024YFC3307603. This work is also supported in part by Tsinghua UniversityToyota Research Center. We also want to express gratitude to J. Zhao and S. Li for providing some processed data.