FD-UAD: Unsupervised Anomaly Detection Platform Based on Defect Autonomous Imaging and Enhancement

1 Introduction

Inspection of industrial product appearance quality is essential for ensuring product quality. Traditional inspections rely on manual labor, characterized by high labor intensity, poor stability, and low efficiency. Existing machine vision systems are designed for specific scenarios and struggle to adapt to the diversity of materials, size variations, and irregular shapes in complex industrial conditions. The challenge lies in achieving multidimensional feature modeling and image restoration of appearance defects in uncertain environments, and developing defect detection systems with limited data labeling.

Industrial anomaly detection aims to identify and locate abnormal samples and their regions in images. Traditional anomaly detection uses machine learning methods, such as k-NN [Angiulli and Pizzuti, 2002], isolation forest [Liu et al., 2008], OCSVM [Sch¨olkopf et al., 2001], principal component analysis [Shyu et al., 2003], etc. Recent years have seen major advancements in industrial image defect detection thanks to artificial intelligence and deep learning. The high cost of acquiring abnormal images has led to a reliance on unsupervised or semi-supervised methods, utilizing unlabeled

∗Corresponding authors

FD-UAD Platform in Diesel Engine Manufacturing Factory

normal data for training models, such as DRAEM [Zavrtanik et al., 2021], CutPaste [Li et al., 2021], PatchCore [Roth et al., 2022], RD4AD [Deng and Li, 2022] and CFA [Lee et al., 2022]. The MVTec AD dataset [Bergmann et al., 2019; Bergmann et al., 2021], a benchmark for unsupervised industrial anomaly detection, exclusively comprises normal samples in its training set, highlighting the gap from realworld industrial imagery. Zero-shot and few-shots learning approaches mitigate the issues of limited data labeling and anomaly diversity, thereby enhancing models' capabilities to identify novel anomalies, as demonstrated by WinCLIP [Jeong et al., 2023] and CLIP-AD [Chen et al., 2023b]. Nonetheless, in complex industrial settings, the effectiveness of defect detection models heavily depends on the quality of data, underlining the importance of optimizing image acquisition and restoration techniques.

The quality inspection systems within optical imaging and industrial vision inspection, such as those developed by Panasonic, IBM, SICK, Omron, Keyence, and Dalsa, are flourishing. These systems perform well in detecting regular defects and in relatively simple scenarios. These systems efficiently detect standard defects in simple scenarios but face difficulties in complex industrial settings with varied materials, textures, reflectivity, and uncertain lighting. These challenges have become bottlenecks in advancing industrial intelligent production.

To address these challenges, we propose FD-UAD: Unsupervised Anomaly Detection Platform Based on Defect Autonomous Imaging and Enhancement. This platform utilizes industrial AI vision perception modeling through multi-scale and multi-modal imaging and image fusion, establishing an AI perception system capable of image restoration and unsupervised anomaly detection. As shown in Figure 1, we have developed an integrated industrial machine vision system that combines multi-sensors collaboration for imaging and defect detection, successfully achieving demonstrative applications and commercial promotion within the top industrial company.

In this work, we make the following four contributions:

• Multi-Sensors Imaging and Image Fusion. We designed and implemented a multi-sensors imaging system with biomimetic optical imaging units, capturing multidimensional image information across RGB, infrared, and various focal lengths. The system employs image fusion to precisely merge multi-modal and multi-scale image data, ensuring features captured in complex conditions.
• Advanced Image Restoration Techniques. We introduced several state-of-the-art (SOTA) image restoration models into our system, including Retinexformer model for brightness enhancement, Uformer model for deblurring and denoising, HAT model for super-resolution. These integrations effectively tackle low light, noise, blur, and low resolution issues in complex industrial scenes, enhancing image quality.
• Mixed Noise-guided Mutual Constraint Framework For Unsupervised Anomaly Detection. We develop a novel mixed noise generation model that emulates real defects and a mutual constraint framework to augment the distinctiveness of the teacher-student network's characterization of anomalous features, achieving excellent results in defect detection.
• FD-UAD Platform. FD-UAD platform elevates the defect detection process through a pipeline that includes multi-scale and multi-modal imaging, image restoration, and defect detection, all embedded within an integrated software-hardware platform. Successfully deployed in a top industrial firm, it enhances product quality inspection with AI value addition.

2 FD-UADOverview

As shown in Figure 2, FD-UAD melds industrial insights and real-world applications, harnessing optical sensors, AI-driven computer vision, deep learning, alongside software and hardware deployment innovations. It crafts an all-encompassing imaging process for industrial products, embracing their material, size, and shape diversity, to capture and fuse multiscale, multi-modal images. Utilizing image restoration, it sharpens image quality for unsupervised anomaly detection. Custom-designed hardware and software cater to the unique demands of industrial settings. Demonstrated in leading companies, FD-UAD propels AI's impact on quality inspection, offering substantial value.

2.1 Multi-Sensors Imaging and Image Fusion

This study addresses imaging challenges in complex industrial environments, by using a multi-sensors framework. It features a Hikvision ME2P-2621-15U3M solid camera and an RS-A1500-GM60 NIR infrared camera for capturing RGB and infrared images. Then it uses Optotune ELM-12-2.8-18C liquid lenses, adjusting focus via electric current in 5mA steps, enabling variable depth imaging of product surfaces from blur to clear. The Knife-edge method calculates the Modulation Transfer Function (MTF) [Schroeder, 1981] for the Region of Interest (ROI), automating image clarity evaluation based on the MTF. The image fusion method combines Gradient Magnitude Awareness (GMA) deep learning optical flow with a deep fusion network to integrate multi-modal and multi-scale images efficiently.

In summary, this multi-sensors framework allows the system to mitigate image quality issues caused by material thermal changes and uneven surfaces, capturing high-quality features across modes and scales, effectively tackling industrial imaging's inherent challenges.

2.2 Image Restoration

In industrial production environments, imaging systems often face challenges such as low light, noise interference, image blur, and low resolution. These issues significantly reduce image quality. To address these challenges, this study designed an image restoration process that integrates the most advanced deep learning models in image enhancement, denoising, deblurring, and super-resolution. This significantly improves image quality.

To combat low-light conditions, this study employed the latest low-light enhancement model, Retinexformer [Cai et al., 2023]. This model combines retinal theory with estimating lighting information and its restoration, effectively enhancing low-light images while maintaining their natural appearance and details. For the common issues of noise and blur in industrial imaging, we introduced the Transformer structure for image restoration tasks, Uformer [Wang et al., 2022]. Uformer uses self-attention mechanisms in feature maps to restore image details, achieving excellent denoising and deblurring effects while optimizing detail retention and computational efficiency. For low-resolution issues, we used HAT model [Chen et al., 2023a], which combines channel attention and self-attention mechanisms. This leverages the global information processing capability of channel attention and the high representational power of self-attention to activate more pixels for reconstruction, resulting in outstanding super-resolution performance.

FD-UAD Platform Structure Overview

FD-UAD Algorithm Structure

Software & System Structure

Figure 2: FD-UAD Platform Structure Overview: Designed for complex industrial products with varied materials, sizes, and shapes, system utilizes RGB, infrared, and multi-focal auto focus imaging for image fusion across modalities and scales, extracting multidimensional features. To address low-quality images, it employs several SOTA AI models for image enhancement, deblurring, denoising, and super-resolution, achieving high-quality image restoration. The MNMC model facilitates unsupervised anomaly detection. FD-UAD software, running on Windows platform and developed with .Net Framework and WPF, integrates imaging, image restoration, and defect detection algorithms. Incorporating a variety of industrial equipment, it forms a complete industrial system framework.

2.3 Defect Detection

We develop MSC-AD dataset [Zhao et al., 2023] and MNMC model [Zhao et al., 2024] for defect detection. The MSCAD dataset serves as the training data, providing a multiscene unsupervised anomaly detection dataset for small defect detection. MNMC is an unsupervised anomaly detection model designed to tackle the challenges of anomaly detection in complex industrial scenes. It consists of three main components: a mixed noise generation module for simulating real defects; a mutual constraint module for enhancing the student network's ability to learn normal features; and an anomaly segmentation module for detecting anomalies at different scales. The model adopts a mixed noise model to generate features closer to real anomalies. Through the mutual constraint framework, this method further enhances the learning of normal features and proposes a new evaluation metric to balance the importance of normal and abnormal areas. As shown in Figure 3, this approach achieved desired detection outcomes.

2.4 Software & System Structure

We evaluated the needs and environment of factory settings, creating a combined software-hardware system. The system includes an industrial light panel (K-WELL KW-BK700-W) and polarizer (GCL-050003) for balanced lighting and reduced overexposure. An electric slide rail (FUYU FMC4030) allows precise camera and object positioning. The setup uses various cameras (solid, infrared, liquid) for detailed object imaging. The software, based on .NET Framework 4.5 and Windows Presentation Foundation (WPF), manages operations and enhances user interaction. Algorithms run on an Intel i7-8565u processor and GeForce RTX 3080, ensuring model efficiency.

FD-UAD Image Restoration and Defect Detection Result

(a) Image Restoration Result

(b) Defect Detection Result

Figure 3: FD-UAD Platform Results Display

3 Demonstration

From 2021 to 2023, we carried out several field surveys in different industrial manufacturing sites to understand their requirements and surroundings. During this period, we developed and iterated on AI algorithms, software, and hardware configurations. As is shown in Figure 1, in December 2023, we successfully implemented FD-UAD platform in a top diesel engine manufacturing factory, establishing it as a benchmark demonstration of AI system application in actual production environments.

4 Conclusion

This paper introduces the FD-UAD platform, combining multi-sensor imaging and image fusion with deep learning for image restoration and unsupervised anomaly detection, covering both software and hardware aspects. It addresses challenges from diverse materials, lighting, and irregularities in industrial environments, showing a holistic AI-enhanced system design and implementation. Successfully implemented at a top manufacturer, FD-UAD is extending to automotive, electronics, and precision instruments sectors, supporting automated quality control, improving production efficiency, and reducing costs.