Towards an Integrated View of Semantic Annotation for POIs with Spatial and Textual Information

1 Introduction

Today's proliferation of geospatial data has fostered many successful stories in spatiotemporal data mining for various urban applications, e.g., venue recommendation [Sun et al.2021,Zhang et al.2022], and site selection [Liu et al.2021]. Among varying types of geospatial data, POIs have gained tremendous momentum, and they have been proven to be effective in many tasks of urban computing and services, such as online map search [G¨obel and Kiefer2019,Li et al.2020b], population mapping [Ye et al.2019, Zhao et al.2019], crime prediction [Wu et al.2022, Zhang et al.2020], and housing price prediction [Xiao et al.2017].

∗Corresponding author.

While abundant POIs are being generated, the quality of POI data is still questionable. The missing property problem of POIs is prevalent, and it is particularly challenging that the key information of POIs, i.e., categories, is often missing or incorrect due to the uncertainties in the human annotation processes for POIs. Categories are among the most important properties of POIs, which carry valuable semantic information to delineate the human activities that POIs bear [Bing et al.2022, Xu et al.2023]. Therefore, it is pivotal to complete missing POI categories, to empower POIs to be better used in various downstream tasks.

Several previous studies have investigated the problem of POI semantic annotation with human mobility data. For example, Li et al. [Li et al.2020a] capture the similarities among different users' check-in activities to learn the POI features and use a multi-class classifier for semantic annotation. Xu et al. [Xu et al.2022a] model the co-occurrences of POIs and categories in check-in sequences to embed them in the same latent space and infer categories for POIs based on the similarity between POI vectors and category vectors. In addition, several works leverage multi-source data such as check-in data, user reviews, and associated images to enhance the effect of POI semantic annotation [Giannopoulos and Meimaris2019,He et al.2016]. Such studies have yielded remarkable advancements to tackle the problem of missing or incorrect POI categories. However, they heavily rely on sequential check-in information of users, which can be hardly obtained in reality, especially for those newly created POIs. This raises a further demand for POI semantic annotation with only static POIs, which is the most common type of POIs with smooth acquisition.

In this paper, we focus on the problem of semantic annotation for static POIs. There are primarily two types of information that are pivotal for semantic annotation in our setting. (1) Spatial information, i.e., geographic locations of POIs. Intuitively, it is difficult to infer POI semantics from the coordinates of POIs directly. Nevertheless, geographically close POIs tend to be also semantically related, e.g., restaurants tend to appear in clusters. How to effectively leverage such information to understand the environment and background of POIs is a challenging problem. (2) Textual information, i.e., POI names. POI names are generally indicative of human activities taking place at the locations. For example, from the POI name "Master Bao's pastry(鲍师傅糕点)", one could readily understand that the POI is a bakery. Furthermore, many POI names have sparse information that is insufficient to reflect their actual categories. For example, given the POI name "Good neighbors(好 邻 居)", it is barely possible to derive its actual POI category Convenience Store only from this weakly correlated name. Thus the short POI names pose a challenge for modeling POI textual information.

To this end, we propose a Spatial-Textual POI Annotation (STPA) model that overcomes these challenges of limited spatial information and sparse textual information of static POIs. To solve the problem of limited spatial information, STPA uses Delaunay triangulation to build a spatial graph (with POIs being graph nodes) that incorporates spatial contextual information. A graph convolutional encoder is applied to the POI spatial graph to generate POI spatial embeddings. Meanwhile, STPA uses a semantic attention component to alleviate the negative impact of textual information sparsity, which captures the influence of POI textual contexts, i.e., neighboring POIs according to the semantic distance of POI names, and yields the POI textual embeddings. Finally, the two types of POI embeddings are fused to take account of the cross-view interaction for POI semantic annotation.

The contributions of this paper are as follows:

• We propose a Spatial-Textual POI Annotation (STPA) model which recognizes POI semantics from only static POIs with the information of geographic locations and names of POIs. To the best of our knowledge, STPA is the first deep model that simultaneously captures the spatial and semantic correlations of POIs from limited spatial information and sparse textual information to address the problem of semantic annotation for POIs.
• We leverage Delaunay triangulation and graph convolution to model spatial contexts of POIs to generate POI spatial embeddings. Moreover, we design an attentionbased text encoder to incorporate the textual knowledge captured from semantic POI neighbors based on POI names into the POI textual embeddings.
• We justify our STPA model on two POI datasets collected from AMap and evaluate its performance via the task of semantic annotation for POIs. STPA demonstrates significant performance gains over several baseline methods based on the paired t-test.

2 Preliminaries

2.1 Problem Statement

[Point of Interest (POI)] The information of a POI pi is composed of a POI ID, a geographic location (here, latitude and longitude of a POI are known), and a POI name textualized as a bag of words.

[POI Category] A POI category ci (e.g., Chinese Restaurant, University) represents the thematic topics of activities that are afforded at the POI pi.

Semantic Annotation for POIs. Given the POIs' geographical coordinates and names, our goal is to predict the category labels for those unlabeled POIs. Specifically, we divide all POIs into Ptrain and Ptest, where Ptrain contains the POIs with category labels and Ptest contains the unlabeled POIs. Our semantic annotation task is to find the category label for each POI p ∈ Ptest.

2.2 Empirical Data Analysis

The POI data used in this work is collected from AMap 1 , where the information of each POI consists of a POI ID, latitude and longitude, a POI name, and a POI category. We select the Haidian District of Beijing and the Lixia District of Jinan as study areas, whereas our semantic annotation method can also be generalized to other study areas. To reduce noise, we filter duplicate POIs by latitude and longitude. After this pre-processing, the Haidian dataset contains 105,577 POIs associated with 248 categories, and the Lixia dataset contains 45,280 POIs associated with 140 categories.

The First Law of Geography tells us that everything is related to everything else, but near things are more related than distant things. Therefore, we conduct the data analysis to investigate whether spatially adjacent POIs share the same category label. Specifically, for each targeting POI pi, we find its Nk nearest neighboring POIs according to the Euclidean distances with latitude and longitude information and compute the ratio of neighboring POIs which have the same category as pi. Similarly, we randomly choose Nk POIs from the POI set and compute the ratio of the Nk POIs which have the same category as pi. Finally, we average the ratios of all the POIs. As shown in Figure 1, we observe that the number of spatially adjacent POIs with the same category as the targeting POI is much more than that of those random POIs on the Haidian and Lixia datasets. This is an important motivation for modeling the spatial vicinity of POIs to improve semantic annotation performance discussed in the following section.

Further, we analyze the POI names of each category. Intuitively, we believe that the words of POI names with the same category are semantically similar. We study the word distributions of categories after removing stop words and punctuation marks of POIs' names. Taking the categories Tea house and Flower Shop as examples, many POI names of Tea house have the keyword "tea(茶)" and those of Flower Shop have "flower(鲜花)". Such observations encourage us to leverage the category information of semantically similar POIs to boost the performance of semantic annotation.

1 https://lbs.amap.com/api/webservice/guide/api/search/

௦௣௔

3 Spatial-Textual POI AnnotationModel

3.1 Model Overview

Figure 2 presents the framework of the proposed method. Let us suppose that we have POIs with latitude and longitude information as well as text names in a study area, where some POIs are labeled with categories and others are unlabeled. For a targeting POI (e.g., p8), on one hand, we model the spatial correlations of POIs based on the assumption that geographically close POIs tend to have the same/similar category labels. Specifically, we first construct a POI spatial graph based on their geographic locations (latitude and longitude) information, where the POIs are the nodes and the POI category information are the node features; we then aggregate the information from the spatial context via a graph convolution encoder on the spatial graph to generate the POI spatial feature vector hspa . On the other hand, we model the POI name information based on the assumption that POIs with similar textual information (names) are likely to have the same/similar category labels. Specifically, we first discover each POI's semantic neighbors through a pre-training language model and

propagate the category information of semantic neighbors to each POI through an attention mechanism. This step produces a textual feature vector htext for each POI. Finally, the two feature vectors (i.e., hspa and htext ) are fused and the entire model is optimized using the cross-entropy loss.

3.2 GCN-based Spatial Encoder

The structure of the GCN (Graph Convolutional Network)- based spatial encoder is shown in Figure 3. As the category information of POIs generally encodes POI semantics, which serve as important features for semantic annotation, we first generate category embeddings as the initial POI features. Next, we build a spatial graph for all the POIs using the latitude and longitude information based on Delaunay triangulation (DT), where the POIs become the nodes and the category embeddings become the node features. Finally, we apply a graph convolutional encoder on the POI spatial graph to generate POI embeddings, which aggregates the semantic information from POI's spatial contexts.

POI Category Embeddings

POI categories largely reflect POI semantics. We choose the simple one-hot representation of categories as the initial node features in the subsequent graph learning stage, as it provides intuitive semantic label information. That is, the embedding of a category ci is represented as

where the length of ⃗ ci is equal to the number of the categories and the ith element of ⃗ ci is 1.

GCN-based POI Spatial Embeddings

After obtaining the POI category embeddings, we further learn POI embeddings by modeling the spatial contexts. According to the First Law of Geography, we know that spatial adjacency entails a strong correlation among POIs. Adjacent POIs in space are naturally more semantically similar (cf. Figure 1), e.g., restaurants are usually spatially clustered. Therefore, for a POI, we propose to leverage the category information of its spatial contexts (i.e., nearby POIs) to enhance the POI embedding.

To this end, inspired by [Huang et al.2023], we model POI spatial relations using a graph structure and utilize the message passing mechanism in GCN to yield POI embeddings via aggregating the category embeddings in a POI neighborhood.

c1, ⃗

c2, · · · , ⃗

cN}). Note that, for

pipj )]

Specifically, we build a spatial graph for POIs based on DT, as the effectiveness of DT graphs for capturing the interactions among spatial vector data has been validated in a series of previous studies [Huang et al.2022,Xu et al.2022b]. The DT only generates edges between closely proximal (one-hop) POIs to avoid excessive linking of POIs and eliminate noise. Along this line, we connect all the POIs in a study area using the DT to build a spatial graph, where the POIs become the nodes and the corresponding POI category embeddings serve as the node features (X = {⃗ a POI pj without the category label, we average the category embeddings of its neighbor POIs as the initial node feature to ensure that all POI initial embeddings can be assigned anyhow, i.e., ⃗ cj = ∑i∈Nj ⃗ ci/|Nj |, where Nj is the spatial neighbor POIs of pj. In the POI graph, we follow [Calafiore et al.2021,Huang et al.2022] and define the weight of each edge between nodes pi and pj as Aij = log[(1+L1.5 )/(1+l1.5 to ensure that spatially closer POIs have larger spatial similarities, where L represents the diagonal length of the minimum bounding rectangle containing all the POIs, and lpipj denotes the actual spatial distance between two nodes pi and pj. Finally, we normalize the weights into the range from 0 to 1.

Based on the constructed spatial graph, we further apply a one-layer GCN encoder to generate POI spatial embeddings,

(2)

where A is the weighted adjacency matrix of the POI graph

without self-loops, D is the degree matrix of A, X is the initial node features of the graph, and Θ is a linear transformation with learnable parameters. Here we avoid self-loops in the graph, as we cannot utilize POIs' own category information in advance when aggregating neighbor information to generate POI embeddings. After graph convolution, the embedding of each POI encodes the semantic category information from its spatial contexts.

3.3 Attention-based Text Encoder

Figure 4 shows the structure of the Attention-based text encoder. We first find the semantic neighbors for each POI based on the POI names. After that, we design a semantic attention component that weights the influence of the category information of neighboring POIs based on the distance of the textual features between the targeting POI and its neighbors. Finally, it generates the attention-based textual vector htext i as the output.

Discovering Semantic Neighbors Based on POI Names

POI names usually contain semantic information related to the POI categories, and POIs with similar semantic names tend to have similar category labels. For example, "Master Bao's pastry(鲍师傅糕点)" and "Master Liang's pastry(梁师 傅糕点)") have similar text names and both are associated with the category Bakery. Therefore, we first find semantic neighbors with similar POI names for the targeting POI, which could largely mitigate the sparsity issue of POIs' textual information, and is a pivotal factor to facilitate POI semantic annotation.

Considering that the POI names are short texts and contain limited semantic information, we propose to learn features of POI names from both word and phrase levels based on the pre-trained word and phrase vectors from Tencent AI Lab Chinese and English Term Embedding Corpora 2 . Specifically, given a POI, we average the word vectors and the phrase vectors in the POI name respectively and concatenate the two vectors as the POI's initial textual feature. With these initial features, we find the semantic neighbors for each POI based on the cosine similarities.

Semantic Attention

As semantically similar POIs tend to have similar categories, we model this prior with semantic attention by weighting the semantic neighbor POIs based on the features of POI names. As depicted in Figure 4, we use the Bert encoder to learn finegrained textual features of POI names, and input the textual feature ⃗ ti of POI pi and the textual features ( ⃗ ti,1, · · · , ⃗ ti,k) of the k most similar POIs to the semantic attention component. The score between the textual features of a POI and its neighbors is calculated based on the Cosine distance,

Next, we calculate the attention weight for each neighbor POI pj based on the textual features. Specifically, we feed the distance vector ⃗D ∈ Rk containing the Cosine distance between the targeting POI pi and its neighbors to a fullyconnected layer to generate a hidden vector ⃗H,

where W ∈ Rk×k is a learnable parameter matrix and ⃗b is a bias factor. We then apply a softmax function on ⃗H to obtain the normalized weight for each neighbor POI pj, i.e., w(⃗ ti, ⃗ ti,j) = e ⃗Hj /(∑kj ′=1 e ⃗Hj′ ), where ⃗Hj means the transformed semantic distance value between the targeting POI pi and its neighbor POI pj .

2 https://ai.tencent.com/ailab/nlp/en/download.html

Finally, we compute the attention-based vector for the targeting POI pi,

where ⃗ cj is the category embedding of POI pj introduced in Section 3.2. Therefore, the element of htext i is the weighted sum of the category information of the semantic neighbors of pi, where the attention weights are adaptively learned. Note that, we merely encode the category information of the semantic neighbors into the attention-based vector htext i , neglecting the textual features ⃗ ti,j of neighbor POIs, as these neighbors' textual features contain implicit semantic information instead of explicit category information. Experimental results demonstrate that concatenating the textual features into htext i hinders the annotation performance.

3.4 Multi-view Fusion

As discussed previously, hspa from the geographical view emphasizes the spatial context of POIs (the First Law of Geography), while htext from the textual view emphasizes the semantic information about POIs. In addition, the importance of POI feature vectors from different views is still unknown for semantic annotation. Therefore, we propose a multi-view fusion layer to construct a more informative representation that preserves multi-view correlations for semantic annotation. Specifically, we consider three kinds of fusion methods:

• Concatenation. We simply concatenate hspa i and htext i to obtain the fusion representation for POI pi: hci = hspa i ⊕htext i .
• Attention-based Fusion. We adopt the attention mechanism [Xi et al.2022] to adaptively fuse the two vectors. That is,

where ⃗ wa , ⃗ ba , and V are learnable parameters.

• Wide & Deep Network-based Fusion. The Wide & Deep network contains a wide layer and a deep layer [Park et al.2019]. In the deep layer, hspa i and htext i are concatenated and fed to a MLP to yield a deep vector gi ,

whereWg and ⃗ bg denote the weight and the bias of the layer, respectively. In the wide layer, the outer product of hspa i and htext i is computed and then flattened to a wide vector ui,

where Wu and ⃗ bu denote the weight and the bias, respectively. F denotes a flattening function that converts a matrix into a single vector. Later, given a POI pi, the wide vector ui is directly concatenated to the deep vector gi to yield the fused vector: hwd i = gi ⊕ui.

3.5 Training Objective

After obtaining the fused vector, we directly concatenate it with the pre-trained text vector of the POI name to yield the final POI representation hfinal i for POI pi, as the POI name of pi contains the semantic information related to its category label. Next, we feed hfinal i into a three-layer MLP to generate the final output ˆyi, which is the probability that POI pi associated with each category label and mathematically calculated as follows,

Finally, we adopt the classification training objective and minimize the cross-entropy loss function,

where |P| and |C| denote the number of unique POIs and categories in the dataset respectively, yij denotes whether POI pi is labeled with the category cj, and ˆyij is the predicted probability that pi is labeled with the category cj.

4 Experiment

4.1 Experimental Settings

Datasets. We adopt the Haidian and Lixia POI data crawled from AMap as the datasets (cf. Section 2.2). For both datasets, we randomly split them into two collections in the proportion of 8:2 as the training set and test set. We run the model 5 times and report the mean of the results.

Evaluation metrics. Semantic annotation for POIs is a multi-class classification problem. We adopt the two wellknown metrics including Accuracy and Macro-F1 to evaluate the performance. In addition, based on ˆyi, we could generate the ranking list of the predicted category labels. Thus we also leverage the metric MRR which considers the position of real labels in the ranking lists. Specifically, it is defined as

where |Ptest| is the number of POIs in the test set Ptest and ranki is the rank of the real category in the predicted list.

4.2 Baselines

We compare the proposed method against the following feature learning baselines of three types.

I. Textual view methods

• Word-based Textual Feature (WTF): Based on POI names, we concatenate word and phrase embeddings pre-trained on massive Chinese corpus to yield wordbased features for POIs following [Liu et al.2020].

• Attention-based Text Feature (ATF): Given a POI, we use the attention mechanism introduced in Section 3.3 to obtain the weighted textual feature vector of semantic neighbors, and concatenate it with the word-based textual vector as the final feature.

II. Spatial view methods

• Grid-based Spatial Feature (GSF): Following [Liu et al.2020], we divide the study area into grids and use the TF-IDF transformation to obtain multi-scale geographic features for POIs based on the category distribution in the grids.
• GPS2Vec: Following [Yin et al.2021], we train a neural network to extract geo-aware features for POIs, which could learn the semantic embeddings from the initial GPS encoding of POIs.

III. Multiple view methods

• EHC: This is an Ensemble POI Hierarchical Classification framework, which mainly consists of a textual and geographic feature extraction component and a hierarchical classifier [Liu et al.2020]. As we do not include the category hierarchy in our model, we adapt EHC with a SVM classifier instead of the hierarchical classifier.
• WTF+GPS2Vec: We concatenate the word-based textual features and the geo-aware features learned with GPS2Vec as the fused features of POIs.

These baselines generate multiple types of features of POIs. Based on these POI features, we train a multi-class classifier (i.e., SVM [Chang and Lin2011]) to predict categories for those unlabeled POIs.

4.3 Comparison with Baselines

We report the comparative results in Table 1. It can be observed that:

1) The performance of spatial view methods is unsatisfactory on both datasets, as the limited spatial information of POIs cannot encode enough semantics for POI annotation. GPS2Vec directly uses a MLP network to encode the POI coordinates and performs the worst; GSF leverages the category information of other POIs in the same grid and outperforms GPS2Vec, indicating that spatial context contains useful, not enough though, information for inferring the category of the targeting POI.

1. Textual view methods perform better than spatial view methods, as POI names usually contain richer and more direct semantic information related to POI categories. In addition, ATF has better performance than WTF, as it leverages the information of semantic neighbors, which also proves the effectiveness of the designed attention mechanism.
2. Integrated view methods obtain decent results, and our proposed STPA performs the best. Compared with the best of the baseline methods (EHC), STPA achieves an average improvement of 2.7% on the Haidian data and 2.9% on the Lixia data in terms of the three metrics. Furthermore, the superiority paired t-test results show that the improvement of STPA over these baselines is of practical significance with p value < 0.01.

4.4 Ablation Study

Study of Performances in Different Views

Figure 5 shows the results of two single views and an integrated view on both datasets. We observe that the performances of the textual view are better than those of the spatial view, as POI names contain richer semantic information than geographic locations for our task. Further, it is interesting to observe that the performances of the integrated view are better than those of all the single view methods. Such observations indicate that the integrated view could make up for the deficiency of every single view and capture relatively complete semantics about POIs.

Study of Performances of Different Fusion Methods

Figure 6 shows the results of three kinds of multi-view fusion methods introduced in Section 3.4 on both datasets. We observe that the simple concatenation method outperforms the other two methods. A potential explanation is that the POI features learned with limited spatial information and sparse textual information are relatively simple, and fusing them using a complex function with many parameters may lead to the problem of overfitting.

4.5 Parameter Sensitivity

We investigate the sensitivity of results over the number of semantic neighbors (k) in the attention-based text encoder. We increase k from 5 to 40 with a step of 5 and report the results in Figure 7. Evidently, the performances improve when we increase k from 5 to 15, and then remain relatively stable when we increase it further.

5 RelatedWork

For the problem of POI semantic annotation, traditional methods usually model users' check-ins to construct POI-related features (e.g., distribution of check-in time and the number of check-ins) and train multi-label classifiers [Chang et al.2014, Li et al.2020a, Ye et al.2011]. Further, some studies learn representations of POIs and categories from checkin sequences based on embedding methods and predict the category labels of POIs based on the similarity between POI vectors and category vectors [Wang et al.2017, Rahmani et al.2019, Xu et al.2022a]. In addition, except for the checkin information, some works leverage external semantic information (e.g., users' ratings and reviews) of POIs to recognize POI semantics for better annotation [He et al.2016, Giannopoulos and Meimaris2019].

Besides considering the check-in sequential information, some studies model the spatial information of POIs for learning POI feature representations. For example, Yan et al. [Yan et al.2017] regard the spatial neighbors of a POI as its spatial context and adopt the word2vec method to model the POI co-occurrence information to learn semantic embeddings for POIs. Liu et al. [Liu et al.2020] leverage the category distribution in the neighborhoods and construct the multi-scale geographic features for POIs. Yin et al. [Yin et al.2021] present a model named GPS2Vec to extract geo-aware features for venues worldwide. Specifically, they divide the region into fine-grained cells and perform the initial GPS encoding; then they train a network to learn the semantic embeddings for the GPS encoding with geotagged documents (e.g., images and tweets) being the training labels. Huang et al. [Huang et al.2022] utilize random walks in a spatial network to capture the spatial contexts of POIs, and a manifold learning algorithm to capture POIs' categorical semantics.

There are also some methods that extract venue-related features from user-generated content (e.g., images) instead of check-ins and make semantic annotation for venues accordingly. For example, Meng et al. [Meng et al.2017] model the text-image pairs and predict the categories of venues based on a feature-level fusion method. Zhang et al. [Zhang et al.2016] construct features from visual, acoustic, and textual modalities, and label each unseen micro-video with venue categories based on a multi-task multi-modal learning model.

6 Conclusion

In this study, we investigate the problem of semantic annotation for static POIs with only POI names and geographic locations. We present a Spatial-Textual POI Annotation (STPA) model which fully captures the information contained in the locations and names of POIs. Specifically, we design a GCNbased spatial encoder to model the spatial correlations among POIs and generate the POI spatial feature embeddings by using the category information of spatial neighbors; we design an attention-based text encoder that models the POI names and weights the influence of the semantic neighbors' category information to yield the POI textual feature embeddings; we finally construct a more informative fused POI representation that preserves multi-view correlations for semantic annotation. We perform comprehensive experiments using POI data from AMap to demonstrate the effectiveness of STPA. In addition, we observe that textual information is more indicative than spatial information for deriving missing POI categories, while the latter is a useful addition to mitigate the sometimes sparse and weakly correlated textual information.

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant No. 61906107, the Young Scholars Program of Shandong University. W.H. was supported by the Knut and Alice Wallenberg Foundation.