SemanticMask: A Contrastive View Design for Anomaly Detection in Tabular Data

1 Introduction

Anomaly detection has extensive applications across various domains, such as medical diagnosis, credit card fraud, and industrial equipment maintenance [Chandola et al., 2009; Ruff et al., 2021]. The goal of anomaly detection is to identify whether a test sample falls into the same distribution as the training data, which can be viewed as a one-class classification problem [Sch¨olkopf et al., 1999]. However, most anomalies in real-world applications are typically new and unknown, thus posing great challenges for supervised anomaly detection. By contrast, without the need to access to supervision of anomalies, self-supervised anomaly detection (SSAD) becomes much more preferred [Sehwag et al., 2021].

∗Corresponding Authors.

Specifically, contrastive learning (CL) based-SSAD utilizing data augmentation strategies, e.g., ContrastiveCrop [Peng et al., 2022], colorization [Tian et al., 2020a], and CutPaste [Li et al., 2021], has recently made significant progress for anomaly detection in the image domain [Cho et al., 2021; Tack et al., 2020].

Unfortunately, for data other than images, such as tabular data, despite its widespread usage, little consideration has been paid on what data augmentation technique can generate a useful representation for anomaly detection. It is also hard to design these transformations manually, because current skills heavily depend on the prior knowledge of the spatial structure of images. Existing augmentation skills for tabular data, such as mixup [Verma et al., 2021], internal contrastive learning (ICL) [Shenkar and Wolf, 2022] and random corrupt [Yoon et al., 2020; Bahri et al., 2021], primarily rely on randomness and overlook the semantic information of features, potentially degrading the quality of data representation.

In CL, selecting informative augmented views is crucial in producing a meaningful representation that helps downstream anomaly detection. The shared information between different augmented views controls the information that the representation extracts [Tian et al., 2020b]. Therefore, if anomalies are attributed to specific features, it is essential to include these features in the shared information for a responsive representation. In conventional anomaly detection tasks, the specific features with abnormal values that lead to anomalies remain unidentified. As a result, it is very important to balance the amount of shared information between views so that we can generate representations that are not only robust to insignificant variations but also still preserving the task-relevant information. One natural question arises: How can we design an effective data augmentation method to craft better contrastive views for anomaly detection in tabular data?

It is noteworthy that for real-word tabular data, the column names usually carry valuable semantic information [Wang and Sun, 2022]. This implies that features with semantically similar column names are typically correlated, exhibiting similar patterns of abnormal values when anomalies occur. For instance, anomalies that indicate the presence of diabetes in the Pima dataset [Rayana, 2016] may exhibit anomalous values in semantically related features, such as "insulin" and "glucose". To leverage such semantic connection, we propose a semantic-aware masking approach (dubbed as SemanticMask) for CL-based anomaly detection. Specifically, SemanticMask uses a language model and clustering algorithm to divide features into k clusters, grouping those with semantically similar column names together in the same cluster. In designing contrastive views, as anomalous features are unknown, it is crucial that the shared information includes full feature coverage across the entire semantic spectrum, i.e., the shared features between augmented views include at least one feature from each cluster. SemanticMask achieves this by performing a random and equitable division of these clusters into two distinct subsets. Each of the two augmented views then selects different subsets and applies a predetermined masking proportion to each cluster within its respective selected subset, while leaving unselected clusters unaffected. Consequently, SemanticMask lowers the likelihood of information loss of each cluster, while also preventing redundancy, thereby achieving balance in the amount of shared information between views. In contrast, traditional data augmentation techniques, such as randommask, overlook this semantic connection. They randomly select features as shared information, thus the shared features are more likely to exclusively come from partial clusters, resulting in an insufficient representation that fails to capture the crucial anomalous features. The main contributions are summarized as follows:

• We propose SemanticMask, a method that incorporates the semantic information of column names to create effective augmented views for anomaly detection in the tabular domain. Additionally, we introduce two extended variations: one incorporates a mask estimation module to address situations where categorical features naturally contain zero values that cause ambiguity; and the other integrates a one-sentence prompt describing the anomaly detection task as prior knowledge, assisting in the selection of shared features.
• We analyze the relationship between the amount of shared information and downstream anomaly detection performance in various settings of SemanticMask's mask probability. We also empirically demonstrate that good views for CL-based anomaly detection are featuredependent.
• SemanticMask surpasses state-of-the-art anomaly detection methods and augmentation techniques for tabular data. In addition, we also extend SemanticMask to multi-class novelty detection and further demonstrate its effectiveness and versatility.

2 RelatedWorks

Anomaly detection. Studies on unsupervised anomaly detection can be broadly classified into four categories [Sehwag et al., 2021; Yang et al., 2021]: (1) density-based methods [Eskin, 2000; Zhai et al., 2016; Li et al., 2020], which detect anomalies by assessing data points in low-density regions compared to nearby high-density areas.; (2) reconstructionbased methods [Pidhorskyi et al., 2018; Yan et al., 2021; Nguyen et al., 2019], which identify anomalies by comparing the reconstruction cost of a reconstruction model trained on the normal data; (3) classification-based methods [Ruff et al., 2018; Wang and Cherian, 2019; Reiss et al., 2021], which separate space containing normal data from all other regions; (4) self-supervised methods [Sehwag et al., 2021; Cho et al., 2021; Tack et al., 2020; Li et al., 2020; Golan and El-Yaniv, 2018], which utilize the strong representation learned from self-supervision [Sehwag et al., 2021; Sohn et al., 2021] or design surrogate tasks to help distinguish anomalies from normal samples [Tack et al., 2020; Shenkar and Wolf, 2022]. Our study belongs to category (4) the self-supervised approach, as we use CL to learn representations. Studies by Sehwag et al. [Sehwag et al., 2021] and Cho et al. [Cho et al., 2021] have shown that CL results in significant improvements in detection performance, as it enhances the quality of the learned representations.

Data augmentation for tabular data in CL. Data augmentation is essential in CL tasks that produce effective representations [Chen et al., 2020]. Most existing techniques apply only to images because their way of defining similarity rely heavily on spatial relationships [Tian et al., 2020a; Chen et al., 2020; Oord et al., 2018; He et al., 2020]. Due to the lack of a "common" correlation structure, designing data augmentation techniques for tabular data can be challenging [Yoon et al., 2020]. There are a few existing methods. Mixup [Verma et al., 2021] creates positive pairs by mixing data samples. Scarf [Bahri et al., 2021] forms augmented views by corrupting a random subset of features. ICL [Shenkar and Wolf, 2022] considers a subset of features from a sample, along with its remaining features, as positive pairs to train the network. However, these methods primarily depend on randomness and thus ignore the semantic information inherent in features, making it difficult to guarantee the quality of the augmented views.

Good views for CL. In CL, shared information between augmented views controls the information that the representation learns and thus significantly impacts the representation quality [Oord et al., 2018]. Tian et al. [Tian et al., 2020a] have demonstrated that there exists a sweet spot of mutual information between views that leads to the best performance in the downstream task. Views with either too high or too low mutual information result in inferior representations. Building on this, Tian et al. [Tian et al., 2020b] suggest that reducing the shared information between views and retaining the task-relevant information exclusively can enhance downstream performance. These previous studies primarily investigate the relationship between shared information and representation quality when the downstream task is classification for image data. This paper delves into the relationship when the downstream task is anomaly detection for tabular data.

3 ProblemFormulation

Given a table T with column description (e.g., column names) composed of n rows and d columns, where each row is a training sample x and each column represents a feature f , we can obtain the training dataset Xtrain ∈ Rn×d and the set of column names C = {c1, . . . , cd}. Column names are usually in the form of short phrases. The underlying distribution of the training data is denoted as Pin X . Anomalies not belonging to Pin X denote semantic anomalies coming from novel classes, characterized by abnormal feature values [Yang et al., 2021]. The task is to determine whether a new data sample belongs to the data distribution Pin X or not. Our detectors aim to address this task, with access to only Xtrain and C.

4 ProposedMethod: SemanticMask

In this section, we introduce an SSAD method for tabular data that incorporates a semantic-aware data augmentation technique called SemanticMask. The motivation behind this work is to utilize the semantic cues inherent in column names to extract crucial information on the correlation between features. This information can enhance the comprehension of the tabular data [Suhara et al., 2022], and ultimately improve downstream anomaly detection tasks.

4.1 SemanticMask: A Semantic-Aware Masking Scheme for CL-Based Anomaly Detection

The framework of SemanticMask includes three steps. In the first step, we employ Sentence-BERT (SBERT) [Reimers and Gurevych, 2019] to acquire sentence-level embeddings for column names. SBERT is a modification of the pre-trained BERT model that generates fixed-size, semantically meaningful vectors for input sentences. "Semantically meaningful" here means that semantically similar sentences are close in the vector space. Then we partition features into k clusters, designated as g1 through gk, based on the embeddings of the column names. This partitioning is performed using the k-means clustering method, due to its effectiveness and low computational complexity [Hartigan et al., 1979]. The value of hyperparameter k adapts to the feature dimensionality. As the dimensionality increases, k also increases. In Figure 1, we take the Pima diabetes dataset [Rayana, 2016] as an example. This dataset employs eight features to determine whether a female patient will develop diabetes, with k set to 2. After k-means clustering, features 2, 5, and 7 have been grouped into one cluster, while the remaining features have been grouped into the other cluster.

In the second step, we propose a data augmentation module that generates positive pairs for CL. Traditional tabular data augmentation techniques, such as RandomMask, indiscriminately mask features with equal probabilities, disregarding the inherent correlations between features and the impact of shared information on representation. These techniques often leads to insufficient representation. To tackle this issue, we design a semantic-aware augmentation technique called SemanticMask, which effectively adjusts the shared information between views to an appropriate level. In the process of generating augmented views, k clusters are evenly divided into two disjoint subsets, labeled G1 and G2, with each subset incorporating features from ⌊k/2⌋ clusters. Each of the two augmented views selects a different subset for masking. When generating one of the augmented view vi of the sample x, a subset of features denoted as Gi is initially selected. Subsequently, a binary vector mi = [mi,1, . . . , mi,d]⊤ ∈ {0, 1}d is produced by a mask vector generator. Specifically, for the j-th feature fj of x, if it belongs to clusters in Gi, then mi,j follows a Bernoulli distribution with a probability pm of being zero and 1 - pm of being one. For features not belonging to Gi, mi,j is set to one. The generation of vi is given by vi = mi ⊙ x. ⊙ here denotes element-wise multiplication. In the example shown in Figure 1, where there are only two clusters (k = 2), each subset Gi contains a single cluster gi.

In the third step, we pass the augmented view vi through an encoder network e to get representation zi. The InfoNCE contrastive loss lc is computed, encouraging representations of the same instance to be close and representations of different instances to be far apart [Chen et al., 2020]. Additionally, for tabular datasets that contain categorical features, it is common for many cells to have a value of zero. In order to differentiate between cells that have been masked to zero and those that are naturally zero, we add an optional mask estimation layer sm to predict which features have been masked in each augmented view. Specifically, this module utilizes the encoded representation of each view zi as input, and outputs a vector ˆmi that predict which features of x have been masked. The total objective loss lt of encoder is:

where the first loss function lc corresponds to the contrastive loss, and the second loss function lm, representing the reconstruction loss of the mask vector mi, is defined as follows:

In equation (2), e ◦ sm represents the entire model M in Figure 1, and vi is the input toM. The expression (e ◦ sm)j (vi) is the output of M, representing the value in the j-th position of the predicted mask. The trade-off between these two losses is determined by the hyperparameter λ.

Once the training of the encoder e is completed, we use Mahalanobis distance [Sehwag et al., 2021] to calculate the anomaly score sxt for the new sample xt:

where zxt is the representation of xt encoded by e, with µtrain and Σtrain being the sample mean and covariance of the training data's representation.

4.2 Incorporation of SemanticMask with One-Sentence Task Description

Based on SemanticMask, we propose an extension that leverages a single sentence s as prior knowledge to facilitate the early identification of important features for anomaly detection. The sentence s is structured using a prompt template, "The task is to detect {description}.", with a specific description relevant to the dataset. For instance, the sentence "The task is to detect the presence of heart disease." is used to describe the Heart disease dataset [Derrac et al., 2015]. We input s along with the column names C = {c1, . . . , cd} into the SBERT model. We then apply k-means clustering to divide {s, c1, . . . , cd} into k clusters, with one cluster containing s, and the remaining clusters without s. The features whose column names belong to the cluster containing s corresponds to the feature set gs. Aligned with SemanticMask, we proceed to partition clusters into two subsets.

In the phase of generating augmented views, a mask vector is constructed. Specifically, a mask probability of pm - ε is applied to the cluster gs, while other k - 1 clusters adopt a mask probability of pm+ε, where ε ∈ (0, pm) serves as a hyperparameter. By doing so, the shared information contains more features that are semantically relevant and closely related to the task description s. Incorporating the relationship between features and the task description into the SemanticMask framework can assist in feature selection, leading to improved performance in anomaly detection.

5 What Makes Good Augmented Views for Anomaly Detection

This section investigates the impact of view selection on shared information and downstream anomaly detection performance in two ways: (1) we combine detailed analysis and empirical evidence to show that the optimal performance for anomaly detection is achieved when the amount of shared information between views is at an appropriate level; (2) we demonstrate that good views of anomaly detection depend on the features that cause the anomalies.

5.1 View Selection Influences Shared Information and Detection Performance

After getting augmented views v1, v2 of the sample x by SemanticMask, we trace out how the amount of shared information between views affects the downstream anomaly detection performance. Previous studies have shown that for computer vision classification tasks, optimal performance can be achieved by a set of views that retains relevant information while eliminating all irrelevant information [Tian et al., 2020b]. However, in anomaly detection, views of training data lack anomaly information. We do not know which features are anomalous, so we want to preserve the shared information as comprehensively as possible, in order to include task-relevant information while avoiding excessive redundancy. In the SemanticMask approach, the amount of

shared information is controlled by adjusting the cluster-wise mask probability pm. In the following, we present an analysis that suggests that the optimal performance for anomaly detection is achieved when the shared information between views is moderate, neither too high (pm = 0) nor too low (pm = 1).

SemanticMask partitions the feature set F of sample x into two disjoint subsets, G1 and G2, based on the semantic information of the column names, where G1 ∪ G2 = F. Assuming the augmented view v1 selects G1 for masking, SemanticMask applies a mask probability pm ∈ [0, 1] to each cluster within G1 to generate the masked version G′1 , resulting in v1 = G′1 ∪G2. Similarly, v2 = G1 ∪G′2 . Adjusting the masking ratio pm from 0 to 1 controls the quantity of information in each view and the shared information between views. When pm = 0, both views are identical, v1 = v2 = G1 ∪ G2 = F. Conversely, when pm = 1, each view comprises distinct unmasked feature subsets: v1 = G2, v2 = G1.

The feature decoupling principle [Wen and Li, 2021] provides a theoretical explanation for the underperformance when pm = 0. Without data augmentation techniques, deep neural networks tend to learn representations that accentuate dense noise, which may overshadow the more semantically aligned sparse features. When pm = 1, v1 = G2, v2 = G1. Our analysis follows the concept of minimal sufficient representation zmin in CL, as defined in [Wang et al., 2022], which includes the shared information between views and disregards any non-shared information. In the context of CL applied to the downstream anomaly detection task T, the minimal sufficient representation of v1 is denoted as z1min . In the ideal case, we assume that there is no interrelation among different clusters; thus, G1 and G2 are semantically non-intersecting and mutually independent. Then z1min is found to lack sufficient task-relevant information, since I(z1min ; T) = I(v1; T) - I(v1; T|v2) = I(G2; T) - I(G2; T|G1) = 0, leading to a suboptimal performance. A conceptual illustration of the above analysis is shown in Figure 2. The optimal point, located at the curve's peak, corresponds to the optimal 0 < pm < 1 that balances the shared information between views to an optimal level, covering the required feature information for anomaly detection without redundancy.

Empirically, we evaluate the anomaly detection performance by adjusting pm on two public tabular datasets: the South African Heart dataset [Derrac et al., 2015] and the Heart disease dataset [Derrac et al., 2015]. The results, shown in Figure 5(a), reveal a reverse-U curve for both datasets, with the optimal point around 0 < pm = 0.4 < 1. This finding is consistent with our analysis illustrated in Figure 2.

5.2 Good Views Are Feature-Dependent

When anomalies are caused by certain anomalous features, we empirically validate that achieving a favorable detection performance relies on ensuring that the shared information between views includes relevant information about these features. Two anomaly detection tasks are conducted on the Pima diabetes dataset [Rayana, 2016].

In the first task, we generate a synthetic dataset by artificially modifying the values of the feature f2, corresponding to the column named "Plasma glucose concentration a 2 hours in an oral glucose tolerance test". Normally, this feature ranges from 0 to 197 with a mean of 179. We select a subset of normal samples, replace their f2 values with values generated from a Gaussian distribution with a mean of 190 and a standard deviation of 5, and labeled this subset as anomalous. We then combine this subset with a portion of normal data to create test datasets for anomaly detection during the testing phase. Four experiments are designed to manipulate shared information between views in different forms: one including solely the feature f2 itself, another involving the feature f1 with the column name "Number of times pregnant" which is semantically unrelated to the column name of f2, one comprising the subset G1 that contains f2, and the last encompassing the subset G2 that does not include f2. Results in Table 1 indicate a notable influence of the shared information between augmented views v1 and v2 on the anomaly detection accuracy (Acc) of the model. Specifically, it is crucial to ensure that the anomalous feature is included in the shared information, rather than other features. For instance, when v1 and v2 only share features like f1 or the subset G2 that are semantically unrelated to f2, the CL approach can hardly capture information of f2, leading to a performance decline.

In the second task, shown in Figure 3, we concurrently manipulate two features with semantically correlated column names, f2 and f5, to artificially induce anomalies in the dataset. Under normal conditions, the range of values for f5 whose column name is "2-Hour serum insulin", lies between 0 and 744, with a mean of 68.8. Based on the anomalous samples selected from the first task, we modify the value of f5 to values generated from a Gaussian distribution with a mean of 500 and a standard deviation of 5. Four scenarios of shared information between v1 and v2 are exhibited in this task. In Scenario 1, f2 is shared; in Scenario 2, f5 is shared; in Scenario 3, features of the subset G1 that contains both f2 and f5 are shared; in Scenario 4, features of the subset G2 are shared. According to Table 1, the accuracy in Scenario 1≈ Scenario 2≈ Scenario 3> Scenario 4, which suggests that good performance is achieved when the shared information includes at least a portion of the relevant information associated with the anomalous features. Performance declines when all anomalous feature information is absent. The outcomes explain the necessity of SemanticMask in including information from all clusters within shared information between views when the anomalous features are unknown, thus preventing the omission of cluster-specific information.

6 Experiment

In our experiments, we assess SemanticMask and its variants across various tabular datasets to validate their effectiveness. For anomaly detection, we compare the proposed method with the state-of-the-art anomaly detection methods and other existing augmentation techniques for CL on tabular data. Additionally, we expand the scope of SemanticMask to accommodate multi-class novelty detection by integrating the labels of training data into our framework. The source code and appendix are available on GitHub at https://github.com/TST826/SemanticMask.

6.1 Experiment Setup

Datasets. Aligned with the definition of anomaly detection, we refer to the class observed during training as "normal" data, treating samples from other classes as "anomalies". We conduct experiments on nine datasets with column names sourced from the Outlier Detection DataSets (ODDS) [Rayana, 2016], the KEEL datasets [Derrac et al., 2015] and the UCI datasets [Markelle et al., 2013]. We train our method on a random selected 50% subset of the normal data. The validation set, consisting of 25% normal data, is used to determine the threshold. The methods are then tested on the remaining normal data and all anomalous samples.

Hyperparameter setups and implementationdetails. For SemanticMask and its variants, λ is set to 0.5, pm is selected from the set {0.4, 0.5, 0.6}. For SemanticMask+description, ϵ is set to 0.1. We set k of k-means proportionally to the feature dimension d. For d < 18, k = 2. For 18 ≤ d < 100, k = 3. For complex datasets such as Arrhythmia [Rayana, 2016], where d ≥ 100, k = ⌊d/100⌋ + 3, features are partitioned into k clusters, forming two disjoint subsets with ⌊k/2⌋ clusters each. Contrastive loss uses a constant temperature τ of 0.01. The threshold for identifying anomalies is determined by the 85th quantiles of the Mahalanobis distance in the validation set. The encoder is a multilayer perceptron consisting of two hidden layers with 128 and 64 hidden units, along with the ReLU activation layer. The encoder is trained using the Adam optimizer with a learning rate of 0.001 and default values for other hyperparameters. Each experiment is repeated 5 times, and the mean and standard deviation (σ) of the results are reported.

Evaluation protocol and baselines. We evaluate the performance of all anomaly detection methods using two commonly used metrics [Han et al., 2022]: Area Under the Receiver Operating Characteristic Curve (AUCROC) and Area Under Precision-Recall Curve (AUCPR). We employ the following unsupervised anomaly detection methods for comparison: One-Class Support Vector Machines (OCSVM) [Sch¨olkopf et al., 2001], Local Outlier Factor (LOF) [Breunig et al., 2000], COPOD [Li et al., 2020], ECOD [Li et al., 2022], Feature Bagging (FB) [Lazarevic and Kumar, 2005], DeepSVDD [Ruff et al., 2018], internal contrastive learning (ICL) [Shenkar and Wolf, 2022] and OptIForest [Xiang et al., 2023]. We also evaluate five baseline data augmentation techniques. For Gaussian Noise [Verma et al., 2021], the mean is set to 0 and the standard deviation is selected from the set {0.001, 0.005, 0.01}. For Random Crop [Xie et al., 2022], the crop proportion is selected from the set {0.6, 0.75, 0.9}. For Mixup [Verma et al., 2021], the linear mixing ratio is selected from the set {0.5, 0.7, 0.9}. For Random Mask [Xie et al., 2022] and Vime [Yoon et al., 2020], the mask proportion is selected from the set {0.2, 0.3, 0.4}.

6.2 Performance Analysis

Results. In table 2, "SemanticMask" here denotes the framework proposed without the mask estimation layer. "SemanticMask+position" refers to the framework with the mask estimation layer. "SemanticMask+description" denotes the SemanticMask framework enhanced with a sentence-based description incorporating prior knowledge of anomaly detection tasks. The results demonstrate that SemanticMask and its variants outperform state-of-the-art unsupervised baseline methods and common data augmentation techniques for tabular data. Notably,"SemanticMask+description" achieves an average AUCROC of 0.752, which is 2.4% higher than the top baseline, OptiForest, and 2.9% higher than the leading existing augmentation technique, random mask. This improvement over the random mask is attributed to SemanticMask's effective design and utilization of semantic information to create views. Figure 4 displays the average values of AUCROC and AUCPR across all datasets, highlighting the superiority of SemanticMask and its variants in both metrics.

Hyperparameter sensitivity analysis. In addition to the analysis performed in Section 5.1 and Figure 5(a) for the hyperparameter pm, we examine two critical hyperparameters: k and λ. k denotes the number of clusters, and λ controls the balance between the mask estimation loss and the contrastive loss. Experiments are conducted on both the Drug dataset (d = 12) and the more intricate Arrhythmia dataset (d = 274). Specifically, we explore k values of {2, 3, 4, 5, 6, 7, 8} and λ values of {0.1, 0.3, 0.5, 0.7, 0.9, 1.0}. Figure 5(b) indicates that the choice of k impacts the results, with higher dimensions (such as Arrhythmia) benefiting from a larger k to maintain intra-cluster feature correlations. Conversely, in lower dimensions (such as Drug), an excessively large k can lead to sparse clusters, disrupting correlations and resulting in poor performance. Figure 5(c) reveals that SemanticMask is relatively insensitive to λ variations when 0.5 ≤ λ ≤ 0.9.

6.3 Extension toMulti-Class Novelty Detection

Setup. The proposed SemanticMask can also be applied to tasks beyond standard anomaly detection, such as multi-class novelty. Unlike anomaly detection with single-class normal samples, multi-class novelty detection aims to distinguish anomalies from normal samples in multi-class scenarios, each characterized by discriminative label information [Yang et al., 2021]. We evaluated our methods on ten multi-class datasets with column names, where specific classes are designated as normal and the rest as anomalies. In multi-class novelty detection, we incorporate label information to further improve the learned representations by using supervised contrastive learning (SCL) loss function [Khosla et al., 2020]. We assessed SemanticMask's performance with the aforementioned baseline augmentation techniques.

Results. SemanticMask demonstrates a significant advantage over other data augmentation techniques, achieving average AUCROC and AUCPR scores of 0.838 and 0.772, respectively. These scores represent increases of 3.2% and 3.4% over the top baseline, Vime. Please see Table 3 for details.

7 Conclusion

In this study, we propose SemanticMask, a data augmentation technique, specifically designed to generate more effective contrastive views for anomaly detection tasks in tabular data. SemanticMask incorporates semantic information from column names, and considers the correlation between features to adjust the shared information between views and learn an effective representation. We establish that to achieve a good representation for anomaly detection, the shared information between views should be appropriately balanced, avoiding both insufficiency and redundancy. Additionally, our empirical results reveal that good views for anomaly detection are feature-dependent. Extensive experimental results demonstrate the superior performance of SemanticMask compared to state-of-the-art anomaly detection methods and prevalent augmentation techniques for tabular data. Note that this study focuses exclusively on clear column names. In the presence of unclear or typo-laden column names, the performance of SemanticMask might deteriorate. To remedy the lack of semantic clarity, large language models like ChatGPT-4 [OpenAI, 2023] can be used for automatic standardization and column name interpretation, a task we leave as future work.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No.62276230 and No.62306277), Zhejiang Provincial Natural Science Foundation of China (LDT23F02023F02).