AADMIP: Adversarial Attacks and Defenses Modeling in Industrial Processes

1 Introduction

Correct Fault Detection and Diagnosis (FDD) allows to increase the efficiency and safety of enterprise production processes. Since the advent of Programmable Logic Controllers (PLC), this problem is usually solved at the hardware level by creating simple logical rules. However, this approach did not allow analyzing the states of technological processes characterized by complex behavior patterns. Nowadays, this issue can be solved by smart manufacturing technologies such as Artificial Intelligence (AI). AI and Deep Neural Networks (DNN) allows to analyze sensor data, increasing the efficiency in FDD task in supervised and unsupervised setting [Lomov et al., 2021; Golyadkin et al., 2023]. However, there are some restrictions on the widespread implementation of such systems for industrial process management. One of the restrictions is the vulnerability of DNN to adversarial attacks (Fig. 1). There is a potential threat in which an attacker will gain access to the data exchange system, and slightly changing the data, e.g., even by using domain knowledge [Ganeeva et al., 2024], will make the DNN predictions incorrect [Pozdnyakov et al., 2024]. Such situation is unsafe if DNN is involved in managing industrial equipment.

From a mathematical point of view, data from sensors is a multivariate time series consisting of observations x1, ..., xn, where xt ∈ Rd are sensor values at time t and d is the number of sensors. Each timestamp is labeled by a vector yt ∈ {0, 1}m+1 which indicates either the normal or one of the m faulty states of the technological process. To predict the state of a process at time t, a certain k-size time interval is usually used. It consists of xt-k+1, ..., xt observations that form a matrix Xt ∈ Rd×k . A machine learning model can be represented as a function f : Rd×k → {0, 1}m+1 that predicts the state by a given observation matrix.

An adversarial attack is a minimal malicious modification of the input data in order to violate the correct prediction of the model. During the attack, an adversarial sample X′t is created such that: f(X′t ) ̸= f(Xt), where X′t = Xt +N and N ∈ Rd×k is a perturbation matrix. To make changes in the data invisible to the human eye and other detection systems (Fig. 2), its maximum shift can be constrained by the ϵ parameter: ∥Xt-X′t ∥∞ ≤ ϵ. There are two types of adversarial attacks: white-box and black-box. White-box attacks require full access to a machine learning model to create a perturbation matrix N using auxiliary information, such as gradients of the backpropagation algorithm used in neural networks. In contrast, black-box attacks use only the inputs and outputs of a machine learning model. Potentially, white-box attacks are more dangerous than black-box attacks.

In this paper, we present the Adversarial Attacks and Defenses Modeling in Industrial Processes (AADMIP) system, which allows to evaluate the robustness of diagnosis methods against various types of adversarial attacks by applying different defense techniques. The system consists of 5 modules. Each module is implemented in Python language using popular packages for operating with data and machine learning, such as CatBoost [Prokhorenkova et al., 2018], PyTorch [Paszke et al., 2017]. In Section 2 we briefly review the modules, in Section 3 we consider some useful applications.

2 SystemOverview

2.1 Dataset Module

Dataset Module allows to connect datasets for training and testing diagnosis methods. Each dataset is represented by a set of source files. df.csv contains the sequence of sensor values at each point in time. label.csv contains the state number of the industrial process, where 0 is the normal state. train mask.csv contains the mask of the training set, where 0 is not a training sample, 1 is a training sample.

Dataset Module allows downloading data from the internet via a public link, unpacking and loading it into memory for later processing. Currently, there are 3 public benchmarks available in the system. rieth tep is a dataset of Tennessee Eastman Process based on the dataset [Rieth et al., 2017]. It contains 52 sensors, 21 states including 20 faults, 21000 runs, each run length is from 500 to 960 observations. reinartz tep is a dataset of Tennessee Eastman Process based on the paper [Reinartz et al., 2021]. It contains 52 sensors, 29 states including 28 faults, 2800 runs, length of each run is 2000 observations. lessmeier bearing is a dataset of an electromechanical drive system based on the paper [Lessmeier et al., 2016]. It contains a single vibration sensor, 3 states including 2 faults, 220 runs, length of each run is 256000 observations. Custom datasets can be easily added using the class inheritance.

2.2 Fault Diagnosis Module

Fault Diagnosis Module allows to manage the configuration of fault diagnosis methods, such as the size of the sliding window and model training parameters: batch size, learning rate, number of training epochs. During training of a machine learning model, data is supplied from Dataset Module. The data available for training is determined using a train mask. After training a model, testing on a test set and prediction by an incoming sample of sensor data are available by Application Programming Interface (API).

At this time, there are 5 models available in the module: Linear model [Pandya et al., 2014], Gradient Boosting [Prokhorenkova et al., 2018], Multi-Layer Perceptron (MLP) [Khoualdia et al., 2021], Temporal Convolutional Network (TCN) and Gated Recurrent Unit (GRU) [Lomov et al., 2021].

2.3 Adversarial AttackModule

Adversarial Attack Module allows to manage an adversarial attack parameters and perturb the input data to trick a diagnosis method. The module consists of white-box and black-box attacks. Each attack requires setting the value of the maximum acceptable deviation of the perturbed data from the input. The attacks use predictions of a diagnostic method to attack the data.

Available attacks are: Random Noise [Zhuo et al., 2022] (black-box), Fast Gradient Sign Method (FGSM) [Goodfellow et al., 2014] (white-box), Projected Gradient Descent (PGD) [Madry et al., 2017] (white-box), DeepFool [Moosavi-Dezfooli et al., 2016] (white-box), Carlini and Wagner (C&W) [Carlini andWagner, 2017] (white-box), Distillation Black Box [Cui et al., 2020] (black-box).

2.4 Defense Module

Defense Module allows to manage parameters of defense methods. During defense, a defense method encapsulates the diagnosis method and can be used for testing and prediction in the same way as the diagnosis method. In this way, a defense method becomes indistinguishable from a diagnosis method from attacker's point of view, allowing defense to be performed invisibly.

Available defense methods are: Adversarial Training [Goodfellow et al., 2014], Defensive Distillation [Papernot et al., 2016], Data Quantization [Xu et al., 2017], Gradient Regularization [Finlay and Oberman, 2021].

2.5 VisualizationModule

Visualization Module allows to run an interactive dashboard based on Bokeh [Jolly, 2018] to analyze the performance of adversarial attacks and defenses. To run the dashboard, a set of prepared files is used, where each file represents a combination of a selected dataset, diagnostic method, adversarial attack, and defense method. Each file contains an attacked data and the predicted process state in CSV format.

Currently all combinations are supported for the rieth tep dataset; diagnostic methods: Linear model, MLP, GRU; adversarial attacks: Noise, FGSM, C&W; protection methods: Adversarial training, Quantization, Regularization.

3 Use Cases

3.1 Benchmarking Fault Diagnosis Methods

To decide on the effectiveness of a particular defense method, it is necessary to compare their quality under different adversarial attacks for different diagnosis methods. For example, let us consider a diagnosis methods based on MLP, GRU, TCN neural networks. A model trained on the reinartz tep dataset is subjected to C&W attack with different levels of distortion of the original signal. We measure the quality of each diagnosis method at each distortion level and each defense method. The results obtained (Fig. 3) show that Gradient Regularization is suitable for defense of TCN, while it is significantly less effective for other models. In addition, it can be observed that Data Quantization shows the best protection for all models and for almost all distortion levels. The only defense that shows better performance is Adversarial training at distortion levels close to 0.1. From this we can conclude that Adversarial training is best for cases when we expect an attack at 0.1 level, for all other cases Data Quantization is better.

3.2 Interactive Dashboard

After preparing data for visual analysis, we can run an interactive dashboard to analyze the results. On the top left there are switches for diagnosis methods, adversarial attacks, and defense methods (Fig. 4). There are also sliders for selecting a unique process run and sensors displayed below. On the top right there is a scheme of the process. In the center we see current sensor values for a sliding window of size 30. On the left are the original values, on the right the attacked values. The data is updated dynamically, simulating real-time equipment monitoring. At the bottom, the real and predicted process state are available for analysis. By switching attacks and protection methods, we can analyze how sensor values and model predictions change.

3.3 Defending Fault Diagnosis Using API

The developed API allows to use diagnosis methods and defense methods available in the system both in isolation and together with each other. Let us consider an example of isolated use of Data Quantization defense method. Suppose there is some diagnosis method that is already used in the process monitoring system. To connect the Defense Module, we need to add this diagnosis method as a child class of the base model, implementing the interface of training and prediction. After that, when creating a Data Quantization object, we need to pass a diagnosis method in the initialization parameters. When the object is created, the model is retrained taking into account Data Quantization. After training, the object can be used for prediction using the predict method, which takes a numpy [Harris et al., 2020] array of dimension [B×L×D], where B is the size of the batch of data, L is the size of the sliding window, D is the number of sensors. An overview of the defending interface is shown in Fig. 5.

4 Conclusion

When integrating AI into industrial processes, it is necessary to at least estimate the existing potential threats. In this demo we present the AADMIP system that simulates adversarial attacks on sensor data and evaluate defense methods. The system has an open license for freely commercial use and can be integrated into various industrial processes through an API.

Acknowledgements

The work was supported by a grant for research centers in the field of artificial intelligence, provided by the Analytical Center in accordance with the subsidy agreement (agreement identifier 000000D730321P5Q0002) and the agreement with the Ivannikov Institute for System Programming of dated November 2, 2021 No. 70-2021-00142.