Learning in CubeRes Model Space for Anomaly Detection in 3D GPR Data

1 Introduction

Ground Penetrating Radar (GPR), utilizing Electromagnetic (EM) waves, serves as a pivotal tool for identifying subsurface facilities or anomalies beneath urban roads [Chen and Cohn, 2011]. It functions by emitting high-frequency waves, with the reflected signals providing insights into the varying compositions and characteristics beneath the surface [Zhou et al., 2018]. Integrating multi-channel technology within GPR systems enables the simultaneous acquisition of EM waves through multiple antennas, obtaining the 3D GPR data for analyzing subsurface conditions [Goodman et al., 2013].

Underground anomaly detection in the real world involves segmenting GPR data to identify and confirm segments containing subterranean diseases [Zhou et al., 2023]. The further process involves locating and repairing these areas. However, manually processing 3D GPR data to detect and extract segments with underground anomalies is highly laborious and time-consuming. While image or signal feature extraction and classification algorithms assist in segmenting and categorizing GPR data, they struggle with the varying characteristics of subsurface media or anomalies, influenced by their composition, size, and environment. Recent developments have seen an increased focus on Deep Learning (DL) methods, particularly Convolutional Neural Networks (CNNs), in object and anomaly detection within 3D GPR data [Liu et al., 2021; Liang et al., 2022a]. However, DL approaches face notable challenges in this task. 1) Primarily, the scarcity of available GPR data, which involves various underground conditions, makes obtaining ample training data challenging. 2) Additionally, the varying underground environments complicate the generalization capacity of DL models, hindering their ability to adapt across different or novel subsurface scenarios. 3) Furthermore, the inherent complexity and extensive parameterization of DL models, especially the 3D-CNN-based ones, necessitate heightened computational resources.

∗Corresponding author: Huanhuan Chen.

In light of the challenges in analyzing 3D GPR data, the Model-Space Learning (MSL) framework presents a promising alternative [Chen et al., 2013]. MSL transforms the data from the data space to the model space by fitting the data with proper models that capture and describe the dynamics (i.e., changing information) within the data. Thus the fitted models could be used as more stable and parsimonious representations of the data, enabling learning algorithms to be effectively implemented on the models rather than the original data. Applied successfully in [Chen et al., 2015] using the Echo-State-Network-based (ESN-based) methods to fit and represent temporal signals, MSL has demonstrated its efficacy in various applications, including the diagnosis of the Barcelona water network [Quevedo et al., 2014] and the Tennessee Eastman Process [Chen et al., 2014], along with diverse time-series classification tasks [Gong et al., 2018; Wu et al., 2022]. Notably, MSL emphasizes data's intrinsic dynamics, requiring less training data and fewer computational resources than many DL techniques, especially when coupled with suitable model configurations [Ma et al., 2020].

For 3D GPR data processing, the MSL framework, while potent, encounters specific challenges. 1) MSL traditionally fits data uni-directionally to capture internal dynamics, predominantly for sequential data with strong contextual correlations. However, 3D GPR data exhibits both vertical continuity along EM waves and horizontal correlations due to the subsurface medium consistency. Effectively capturing these multi-directional dynamics is crucial for accurate fitting and representation of 3D GPR data. 2) Employing networks with fixed parameters for data fitting and dynamic capture, along with using non-adjustable model metrics to measure the difference between models fitted from diverse GPR data, prove inadequate for varying data-collecting scenarios. Such limitations pose significant challenges in both achieving accurate data fitting and representation, as well as in constructing a "category-discriminative" model distribution.

In this paper, we propose learning in the Cube Reservoir Network (CubeRes) model space for subsurface anomaly detection in 3D GPR data, as illustrated in Figure 1. We segment the 3D GPR data into blocks, with data segments in each block, referred to as GPR data blocks, being fitted by CubeRes. In 3D GPR data, each point is correlated with surrounding points in multiple directions. Different underground structures show different changing information in the GPR data. The proposed CubeRes integrates three reservoirs to comprehensively capture the multi-directional dynamics in horizontal and vertical directions. By constructing the connections between the points within the data in multiple directions, fitting the data block with CubeRes effectively captures the dynamics inherent in the data, obtaining a compact fitted readout model. A parameterized model metric is further introduced. Representing each original data block with a fitted model, coupled with the proposed distance metric between models, forms the CubeRes model space 1 .

We further implement an optimization algorithm in this model space, adjusting the parameters of both the CubeRes used to fit the data and the model metric, enhancing the fitting accuracy and bolstering the category discriminability of models, thus aiding in the distinction of models derived from different types of GPR data. Through the above, GPR data blocks formed by similar underground structures would be transformed into similar CubeRes readout models, while models fitted from GPR data blocks generated by different underground structures would have large differences due to the different dynamics captured. Finally, classification is conducted within this model space to categorize each readout model, thereby identifying corresponding GPR data blocks containing subsurface anomalies.

1 In this paper, "CubeRes" refers to the network for 3D GPR data fitting, yielding the fitted "CubeRes readout model" for data representation, also short-termed as "CubeRes model".

The main contributions of this paper are as follows:

• We introduce the Cube Reservoir Network (CubeRes), a novel network that adequately captures both horizontal and vertical dynamics inherent in the 3D GPR data into a compact fitted readout model. Representing the original 3D GPR data with the fitted model allows for further processing on the models rather than the original data.
• Our approach departs from fixed model metrics, employing an adaptive parameterized metric within the model space. We further complement the optimization of this metric and CubeRes parameters according to different underground scenarios, enhancing its ability for more accurate fitting and dynamic capture, also resulting in a more category-discriminative model space.
• The effectiveness of the proposed approach is demonstrated through experiments on real-world GPR datasets, which significantly outperforms the compared methods, particularly in scenarios with limited data availability.

2 RelatedWork

2.1 3DGPRData Analyzing

Multi-channel 3D GPR data represents an advanced approach in subsurface imaging. Unlike single-channel GPR systems obtaining the 2D GPR data, multi-channel GPR utilizes an array of antennas, transmitting and receiving EMwaves simultaneously across multiple channels, as illustrated in Figure 2.

In 3D GPR data, valuable changing information presents not only in the vertical direction (the direction of EM waves) and along the detection path, but also across different channels.

Recent DL methods primarily adopt two strategies for analyzing 3D GPR data. 1) The first involves the direct analysis of 3D data. For instance, Liu et al. [2022] employ multiple mirror encoding (MME) for data augmentation across diverse 3D GPR data sizes, integrating the C3D network [Tran et al., 2015] for spatio-temporal feature learning. 3DInvNet [Dai et al., 2023] utilizes a prior 3D CNN and a feature attention mechanism to suppress noise, followed by a 3D U-shaped encoder-decoder network with multiscale feature aggregation modules, mapping underground 3D permittivity maps. 2) The second extracts single-channel or cross-sectional profiles from 3D GPR data, further merging and analyzing the obtained results. For instance, UcNet [Kang et al., 2019a] integrates CNN with phase analysis for super-resolution (SR) GPR data. Similarly, Liang et al. [2022b] involve the comparison of VGG and ResNet for classifying GPR data containing underground anomalies. For 3D GPR data analysis, DL approaches face critical challenges: DL models tend to be complex and parameter-heavy, especially the 3D CNNs, while handling channels individually risks overlooking interchannel dynamics in 3D GPR data.

2.2 Model-Space Learning

The MSL framework was first proposed in [Chen et al., 2013] and applied to fault diagnosis in sequential data. Existing MSL-based methods employ ESN to fit individual data instances. Each instance is represented by a fitted model, and mapped into a model space with a defined model metric. Effective classification is then conducted within this space, leveraging the dynamics captured from the data. The versatility of MSL has extended to applications like time series classification, disease diagnosis [Bianchi et al., 2020], and tackling concept drift [Chiu and Minku, 2022].

Turning to the application of MSL in GPR data, existing studies [Zhou et al., 2023; Chen et al., 2023] proposed to detect anomalies in 2D GPR data, employing ESN-based network to fit data, capturing dynamics within the data both horizontally and vertically, and further classifying the fitted models. Though it has been explored to extend this to 3D GPR

data by processing each channel separately, some limitations remain: 1) 3D GPR data, offering extensive changing information beyond the aggregation from single-channel data, is not fully utilized in this way as it fails to capture the dynamics between channels, particularly in the horizontal dimension. 2) Existing studies utilize fixed metric and network parameters, and do not include adaptive adjustment, which could be vital for adapting to diverse subsurface environments. 3) The usage of ESNs combined with the multi-channel fitting strategy leads to extremely high-dimensional fitted models, significantly impeding computational efficiency.

2.3 Brief Introduction of Echo State Network

Echo State Networks (ESNs) represent a unique subclass of Recurrent Neural Networks, famed for the simplicity and efficacy in sequential data processing. The structure of an ESN includes an input layer, a hidden layer known as the reservoir, and an output layer. Figure 3 illustrates the ESN structure.

A notable aspect of ESNs is the randomized and fixed input weights and reservoir weights. It is also essential for ESNs to maintain the Echo State Property (ESP), ensuring stability and proper functioning. For sequential data fitting, after the computation of the hidden state for each point, the output layer maps these states onto the target sequence, with the output weights determined through ridge regression. Despite the efficacy of ESN in fitting sequential data, it exclusively captures the dynamics along the iterated direction, while overlooking the nuances in others, and fails to comprehensively capture the dynamics in multi-dimensional data.

3 Methodology

The entire process of our approach is divided into the following three stages:

• Fitting GPR data with CubeRes. Fitting each segmented GPR data block using CubeRes. The fitted readout model that captures the multi-directional dynamics inherent in the data block is then used to represent the original data block.
• Optimizing the CubeRes model space. A parameterized metric is defined on the fitted models to measure their pairwise difference. The CubeRes and the metric are optimized to enhance fitting accuracy and bolster the category-discriminability between the fitted models.

• Anomaly detection in the model space. Classifying the models derived from the data blocks within the model space, thus identifying the corresponding GPR data blocks containing subsurface anomalies.

The content of these stages are detailed in the Subsections 3.1, 3.2, and 3.3, respectively.

3.1 Fitting Data by Cube Reservoir Network

Similar to ESN, CubeRes comprises an input layer, a hidden layer, and an output layer. However, to adequately capture the multi-directional dynamics inherent in GPR data, it integrates three reservoirs in the hidden layer (Figure 1) to establish correlations among adjacent points in multiple directions.

Denoting a 3D GPR data block as U ∈ RX×Y ×Z , wherein a point within this block is located by (x, y, z), and the corresponding value at that point is u(x, y, z). As in Figure 4, the iteration of CubeRes begins at the point (1, 1, 1) and ends at (X, Y,Z). Sequentially, each point is sent into the hidden layer, with their hidden states h calculated as:

where g is the activation function tanh; Wx , Wy , andWz represent the reservoir weights 2 in the three respective directions; andWin denotes the input weights. These parameters within CubeRes, namely Θ = (Win ,Wx ,Wy ,Wz ), are randomly initialized and will be further optimized (detailed in Subsection 3.2).

During the iteration process as Equation (1), the hidden state at a point is determined by the value of the current data point as well as the preceding hidden states, establishing correlations among adjacent hidden states in different directions. As the iteration continues, these correlations persist, forming a network of connections. This links each point in the data to previously processed points across multiple directions, effectively retaining multi-directional dynamics within the data.

After the hidden states of all the points within the data block are calculated, the output layer computes the output value v for each point using the preceding hidden states:

2 The reservoir weights are required to satisfy the Echo State Property (ESP) [Jaeger, 2001].

In this paper, the fitting task is accomplished by the "next point prediction task" [Chen et al., 2013], that is to predict the value of the next data point using the processed points by associating hidden states with the input data points. Specifically, it requires that the output values (i.e., v) calculated from the preceding hidden states closely approximate the input values (i.e., u). In this case, the output weights Wout can be solved using ridge regression:

where H is a matrix formed by horizontally stacking hpre (x, y, z) at each point, that is, each column is a hpre at a specific point (x, y, z); u is a row vector comprising u(x, y, z), arranged in a sequential order that aligns with the column order in H; α is a regularization parameter; and I is identity matrix.

Through the whole fitting process, the correlation between adjacent points in the data block is effectively modeled by CubeRes's unique iterative approach. The multi-directional dynamic information within the data block is captured and encapsulated into a compact CubeRes readout model:

Following this, the readout model serves as a compact representation of the original data block. When a specific type of anomaly occurs in the data block, unusual dynamic information emerges, resulting in a fitted model that behaves distinctly versus those derived from normal data blocks. Consequently, further classification could be more effectively applied to these models, rather than the original data blocks.

3.2 Optimizing the CubeRes Model Space

We seek to construct a model space where the distribution of models is category-discriminative. This requires the CubeRes to accurately capture the dynamics inherent in the fitted data, and the metric between models effectively differentiates the dynamics captured from different types of data.

We first define a parameterized model metric, thus constructing a model space consisting of CubeRes readout models and the metric between models. After that, we conduct optimization within this space, employing two penalty terms to achieve the following two objectives. 1) Enhance the fitting accuracy to make the fitted models more accurately capture the inherent dynamics, thus more representative of the data blocks. 2) Bolster the category-discriminability in the model space by optimizing the parameters of CubeRes and training an appropriate model metric.

Metric between CubeRes Models

A model metric is defined to measure the differences between various CubeRes readout models. Formally, for any two data blocks Um and Un, their derived CubeRes readout models are denoted as fm and fn respectively (in the form of Equation (4)), and our model metric is defined as:

where ∥ · ∥2F is the square of Frobenius norm, defined as the sum of the absolute squares of the matrix elements. Unlike most MSL-based methods that employ a fixed model metric, in Equation (5), we introduce a parameterized matrix G, enabling adaptive weighting for elements in Wout . This innovation renders our metric flexible and trainable, with enhanced potential to differentiate models capturing distinct dynamics inherent in different types of data.

Enhancing Fitting Accuracy

To enhance the fitting accuracy of CubeRes, the Mean Square Error (MSE) between the output value v(x, y, z) and the approximate target u(x, y, z) should be minimized. Assume a training set composed of N data blocks, denoted as U = {U1, . . .UN}. We introduce the first penalty term to quantify the fitting error:

where M is the number of data blocks, and the subscript m indicates the index; the form of Hm and um is the same as in Equation (3); and β is a regularization parameter.

Bolstering Category-Discriminability

We optimize the parameters of CubeRes (i.e., Θ) and those of the model metric (i.e., G), aiming to maximally bolster the models' category-discriminability in the model space. Ideally, we desire models derived from the same type of data blocks to be closer, and those from different types of data blocks to be distant. This leads to the second penalty:

where tm indicates the type of the data block Um, and δ(fm, fn) is the model metric defined in Equation (5).

Overall Optimization Objective

By combining Equations (6) and (7), our overall optimization objective is as follows:

where η is the trade-off between Q1 and Q2. In practice, this optimization is implemented through gradient descent.

3.3 Anomaly Detection inModel Space

Real-word detection involves identifying the location and type of the subsurface anomaly in the collected 3D GPR data. Our approach involves the training and detecting phases.

Training Phase

Given 3D GPR data that is collected and manually labeled (normal or containing a specific anomaly), we employ a fixedsized 3D block to swipe and segment the data along the detection direction, obtaining a set of data blocks with label information. As shown in Figure 5, we fit these blocks and obtain a set of CubeRes readout models. Through the approach illustrated in Subsection 3.2, the CubeRes is optimized, and an appropriate model metric is also determined, forming a category-discriminative model space. After that, a classifier, such as KNN or SVM, is trained within this space given the labeled models and the determined model metric.

Detecting Phase

For GPR data subsequently collected in similar underground environments, the same-size block is swiped, obtaining a series of data blocks. These data blocks are fitted by the above-optimized CubeRes, obtaining the corresponding readout models that are further classified through the trained classifier in the model space, thus the corresponding data blocks are also identified to be normal or containing a specific type of anomaly. Therefore, the subsurface anomaly is detected by extracting and locating the classified data block. When a series of consecutive data blocks are detected with the same type of anomaly, the combined range of them is identified as the final area range of that anomaly.

4 Experimental Study

Experiments on real-world 3D GPR datasets are conducted. The results demonstrate the effectiveness of our approach, particularly under limited training data.

4.1 The Utilized 3D-GPR data

The data is collected along cement and asphalt roads, both the most common and widely used road types. A MALA 3D GPR system is adopted, equipped with an antenna of 16 parallel channels, and the interval of adjacent channels is approximately 0.15 meters. We segment the collected data using a 3D block with a size of 16×200×200. This corresponds to a physical area of 2.4m (width) × 2m (detecting direction) × 2m (depth). The amount of GPR data blocks for each category is detailed in Table 1.

Four major types of subsurface diseases appear in our experiments: cavity, looseness, water-rich, and crack. Besides, the collected GPR data includes data formed by pipelines and manhole covers (denoted as manhole) also should be identified. Figure 7 gives some examples of the GPR data blocks.

4.2 The Baseline Methods

We benchmark our approach against multiple baseline methods, broadly categorized into two types. 1) The first type encompasses methods that process 3D data either as images or by converting them into image-like structures for analysis. This includes renowned and effective network architectures, that is, VGG16 [Simonyan and Zisserman, 2014] and ResNet18 [He et al., 2016], widely recognized for their robust performance across various image processing tasks. Additionally, a novel UcNet [Kang et al., 2019b] is employed, which transforms 3D data into 2D grid figures before feeding it into networks. 2) The second type deals directly with 3D data. This includes 3D adaptations of traditional architectures, namely 3D-VGG and 3D-ResNet. Moreover, we explore M-C3D [Liu et al., 2022], which interprets 3D data as a sequence of images akin to a video stream, thus leveraging the temporal or sequential aspect of 3D information for enhanced contextual understanding. Lastly, we incorporate DS-3D-AlexNet [Bai et al., 2023], using depth-wise separable convolutions to reduce the parameter count and increase training efficiency.

We did not experiment with the traditional MSL method that employs ESN to fit GPR data blocks and classify the ESN readout model due to the impractical size of the ESN readout model. Fitting the data block along the detecting direction, the size of the ESN readout model is 200×16×reservoir size, fails to be directly classified. As for the CubeRes, along with a comprehensive capture of multi-directional dynamics, a more compact readout model size 3×reservoir size is obtained, facilitating efficient learning on the fitted models.

4.3 Experimental Results and Discussion

We varied the training and testing data splits at 90%, 70%, and 50% for training, with the remaining portions for testing, as detailed in Table 2. In our approach, the reservoir size of CubeRes is set to 50, and the default classifier in the readout model space is KNN 3 . The results demonstrate that 2D-based methods for 3D GPR data analysis underperform compared to direct 3D data processing. As the proportion of training data decreases, there is a uniform reduction in accuracy, recall, and F1-Score across all methods. However, our proposed method exhibits superior stability and performance, maintaining higher metrics across all data splits.

Discussion with Limited Training Data

Figure 6 illustrates the performance of several methods with better performance in Table 2 under decreasing training data (gradually reduced from 90% to 30%). DL methods depend heavily on extensive training to enhance feature extraction, necessitating ample training data to capture the discriminative information among different categories. Instead of training a deep network and using it as a classifier, our method focuses on the inherent changing information (i.e., the dynamics) within the data, represents the original data blocks with fitted dynamic-captured models, and accomplishes classification on these models rather than the original data.

CubeRes' comprehensive capture of multi-directional dynamics within the data and efficient optimization given specific data enable effectiveness even with limited training data. Taking the F1-score as an example, even when the amount of training samples dwindles to a mere 30%, that is about 60 data blocks for each type, our method achieves a best F1score over 85%. In such a case, as there is not enough data to improve its feature extraction and recognition capabilities, the effect of the DL method has dropped significantly.

The Effectiveness of Optimization

In examining the optimization's impact in our approach, Table 3 reveals that optimization enhances the F1-Score by over 5%. More intuitively, we map the CubeRes readout models to a 2D visualization using t-SNE [Van der Maaten and Hinton, 2008]. As Figure 8, each point represents a readout model, corresponding to a fitted GPR data block. Without optimization (w/o Opt.), while it is possible to differentiate models derived from normal and abnormal data, the distance between different types of anomalies is relatively close, with a lack of distinct clustering within each category. After optimization (Optimized), it is evident that the implemented optimization successfully reduces distances among models of the same category and amplifies differences between disparate ones, resulting in a more "category-discriminative" model distribution, thus allowing for more accurate model classification.

3 The impacts of using other classifiers like SVM and Random Forest are discussed next.

Performance Comparison of Different Classifiers

The performance of SVM and Random Forest (RF) within the CubeRes model space is given in Table 4. In conjunction with Table 2, it is evident that the KNN yields the best results, with RF performing better than SVM. As illustrated in Figure 8, models fitted from the same type of data are highly clustered, resulting in a category-discriminative model distribution, thus the above three methods have achieved considerable results, generally surpassing the other baselines.

5 Conclusion

This paper proposes learning in the Cube Reservoir Network (CubeRes) model space for anomaly detection in 3D GPR data. 1) We introduce a novel CubeRes, with three reservoirs, to comprehensively capture the GPR data's multi-directional dynamics into a compact fitted readout model for data representation; 2) The integration of a parameterized metric between models and the optimization of CubeRes significantly improved the fitting accuracy and category discrimination in the readout mode space, allowing for efficient classification on the models. Owing to the comprehensive capture of the data-inherent dynamics and the efficient optimization, our approach demonstrated notable effectiveness in the real-world GPR dataset, particularly in data-limited scenarios.

Acknowledgments

This work was supported in part by the National Key R&D Program of China (No. 2021ZD0111700), in part by the National Nature Science Foundation of China (No. 62206261, 62137002, 62176245), in part by the Key Science and Technology Special Project of Anhui Province (No. 202103a07020002), and the Special Foundation for Science and Technology Innovation and Entrepreneurship of CCTEG (No. 2020-2-TD-CXY006).