Can We Find Neurons that Cause Unrealistic Images in Deep Generative Networks?

1 Introduction

Generative Adversarial Networks (GANs) have been exhibited excellent performance for creating photo-realistic images, such as human faces, cars, and buildings. [Karras et al., 2019; Brock et al., 2019; Karras et al., 2020; Chen and Yang, 2021; Esser et al., 2021; Karras et al., 2021]. Additionally, GANs have been used to synthesize a new realistic image from several real images. For example, a target human image can be transformed to obtain another's face [Kim et al., 2021]. These various uses of GANs have been utilized in many applications [Hung et al., 2021; Chen and Yang, 2021; Liu et al., 2021; Wang et al., 2021].

Even though it is known that GANs have sufficient representative capacity to generate high-quality images, recent studies show that GANs still produce unrealistic images [Odena et al., 2016; Bau et al., 2019] or only generate limited types of samples. In this context, even PGGAN [Karras et al., 2018] and StyleGAN [Karras et al., 2020], which are recognized as the high-performance models, may generate unrealistic images that include defective parts or unnatural objects, referred to as 'artifacts'. As long as artifacts in generated images are not detected and fixed, the application of GANs will be limited.

∗Corresponding Author

Recently, there have been several attempts to identify the artifacts. GAN-Dissection [Bau et al., 2019] analyzes internal units in a pre-trained GAN to identify which units contribute to generating a specific object in a generated image. In this work, the authors found several units that are responsible for typical artifacts. One method to identify the cause of an artifact is to find manually corresponding units creating artifacts. Another method for automatic identification of artifacts units is using unit-specific FID scores. They then turn off each identified unit, which means that activation values of all neurons in the selected unit are set to zero. Automatic Correction [Tousi et al., 2021] exploits an auxiliary classifier that is trained to distinguish artifacts from normal images and identifies the locations of defective parts by applying GradCAM [Selvaraju et al., 2019]. Handcrafted labels are required for fine-tuning this classifier. However, these studies lack analysis on the root cause of artifacts from the perspective of the generating process and they require manual sorting or additional training costs.

In this work, we focus on revealing which neurons make images distorted, how artifacts can be detected, and under which condition can make high quality natural images. In particular, we analyze PGGAN and StyleGAN2, which are known as high-performing models and have been widely adopted as the basic structures of generative models for highquality high-resolution [Chen et al., 2022; Kim et al., 2021; Yao et al., 2021]. We first analyze the frequency of activated neurons to reveal the following behavior of the generating process. Figure 1 illustrates the relation between defective parts of generated images and rarely activated neurons, which we will define as 'low frequency rate'. We then empirically verify that defective parts in the artifact images are corresponding to neurons with low frequency rates. From this analysis, we show that artifacts can be censored without any additional training cost. Furthermore, we propose an ablation method to remove defective parts in an unsupervised manner. This method doesn't require an additional training procedure. This experimental results support our hypothesis both qualitatively and quantitatively. The main contributions of our work are as follows:

• We analyze the frequency of activation in feature maps in order to reveal the properties of the generating procedure.
• We experimentally show that not only are the artifacts related to the rarely activated neurons, but also are the artifacts detected by this property.
• We provide qualitative and quantitative results. Furthermore, our analysis can be applied to detect and remove artifacts from generated images.

2 Related work

2.1 Generative Adversarial Networks

GANs [Goodfellow et al., 2014] have been leading a new paradigm of generative models. A GAN consists of a discriminator and a generator. The discriminator classifies between two images distributions: real images and fake images. The generator creates fake images to deceive the discriminator. One improved version of conventional GAN structure is Progressive Growing GAN [Karras et al., 2018], which is trained sequentially by increasing the resolution, and has emerged as a high-quality models. While recent models based on the GAN structure produce indistinguishable photo-realistic images, there have been few studies on the inner process of the GAN model.

2.2 Style-based GAN

StyleGAN [Karras et al., 2019] was proposed as a novel generating mechanism for GAN to generate high quality and resolution images. This model has two parts: a mapping network and a style generator. The former transforms an input latent code z to a style code w. The latter generates an image with the trained constant tensor and the style code w. Although StyleGAN affords good performance, blob artifacts can sometimes be found in the images. To address this, they suggested an advanced version, StyleGAN2 [Karras et al., 2020]. Despite their efforts to remove specific artifacts such as blobs, the general artifacts remain unresolved. In addition, they propose a method improving image quality, called 'truncation'. This forces the style vector located near the average style vector that is calculated on training time. However, the authors state that this truncation method negatively effects the diversity of output images.

2.3 Semantic Hierarchy Emerges in GAN

A recent study [Yang et al., 2021] revealed the hierarchy mechanism of generative models. The authors of this study noted that the generation process is the same as an artist drawing a picture by interpreting it from multiple abstraction levels. In the GAN models, the early layers create the layout of the output image. The middle layers create objects and attributes. Finally, the color scheme is rendered on the late layers. This implies that the artifacts are likely to occur in the early layers.

2.4 Detecting and removing Artifacts in Generative Models

In recent studies, GAN Dissection [Bau et al., 2019] and Automatic Correction of Internal Units in Generative Neural Networks [Tousi et al., 2021] have struggled to understand the inner mechanism of GANs. They suggest how the units where artifacts occur can be detedted and fixing internal errors in the GAN so that the generator can provide realistic images without artifacts. GAN Dissection removes the internal units that are expected to cause defective elements. The latter trains an auxiliary neural network classifier. For the case of an artifact image, the auxiliary classifier is used to search a region that is considered to be strange by applying Grad-CAM [Selvaraju et al., 2019].

3 Analysis for the Activation Frequency Rate

In this section, we attempt to shed light on the causes of the diversity of generated images by observing the frequency of activations in GAN, which may lead to the generation of artifacts. We first introduce notations for GAN structure to define the frequency rate of neurons. According to the frequency rate, we verify that rarely activated neurons are highly related to unusual features and defective parts in generated images by analyzing the effects of those neurons. Furthermore, we suggest an efficient detection and ablation techniques based on our analysis.

3.1 The Frequency Rate of Neurons

Notation. For convenience, we introduce some notations for GAN structure and define the frequency rate to measure the frequency of activations. GAN consists of two parts, a generator G and a discriminator D. The generator Histogram of the number of LR-neurons in PGGAN

G : Z → I generates an output image where Z is the latent vector space and I is the space of output images. Since we analyze the process of image generation and internal neurons, we determine the generator G as our main target. A function from the 0 th layer to the i th layer of the generator is denoted by G:i . The output of the i th layer is denoted by f i (z) = G(i-1):i (f (i-1) (z)), where f 0 (z) = G0 (z).

Definition 1 (Activated Neuron). Given a generator G and its featuremap f i (z) for the i-th layer, the activation value of the n-th neuron is defined by f in (z). We call it 'activated' if f in (z) > 0. In particular, the set of the activated neurons in the i-th layer is defined by [f i · ]+.

Definition 2 (Neurons with Activation Rate R). Given a featuremap f i , Rin is the probability that a neuron f in (·) is activated. f i ·,R is the set of neurons with an activation rate R such that Rin = R.

The activation rate Rin denotes how frequently the neuron n is indeed activated:

where I is the indicator function. Since the true probability for each neuron cannot be computed exactly, 30K latent codes are sampled to estimate the relative frequency Rin . We analyze the effect of neurons with low frequency rates and that of neurons with high frequency rates. Then, with the constant rate R, we denote the set of neurons which have Rin > R by f i·,R and denote the set of neurons that have Rin ≤ R by f i·,R.

When there is no confusion in the context, we simply call f i·,R 'HR-neurons' (high-rate neurons) and call f i·,R 'LR-neurons' (low-rate neurons).

Hypothesis. In featuremaps of generated images, some neurons are rarely activated whereas other neurons are frequently activated. In other words, neurons have different activation frequency, which are precisely defined as the 'frequency rate' in Definition 2. We observe that featuremaps of normal images among generated results are composed mostly of frequently activated neurons. For instance, the rare cases shown in Figure 3 have many neurons with low frequency rates in their featuremaps unlike general cases. Considering this, we hypothesize that rarely activated neurons are used to create diverse components which may include failures, but frequently activated neurons are used to construct general components that are necessary for realistic images.

Figure 2 provides empirical evidences of our hypothesis. The histogram shows the number of generated images according to the number of activated neurons that are rarely activated during the generating procedure. The images in the red box, of which featuremaps have many rarely activated neurons, have diverse components, such as images with phrases. Most of them, however, fail to construct realistic images, while the images in the blue box show general cat images. It is reasonable to suspect that rarely activated neurons are highly related to artifact images generated from GANs.

3.2 Relationship between Frequency Rate and Generations

To observe the effects of neurons according to their frequency rates, we first introduce an ablation method, namely 'Sequential Ablation', in Algorithm 1. Given a latent code z ∈ Z, it iteratively computes its featuremap while turning off activated LR-neurons for each layer. In other words, it makes values of activated LR-neurons as zeros. The target layers to remove LR-neurons are limited to indices in M, since neurons on different layers may have different effects on generated images. Previous research shows that each layer in

Algorithm 1 Sequential Ablation

Input: Generator G, A latent code z ∈ Z

Parameter: All Layer index list L, Layer index list M for ablation, Activation rate R

Output: Corrected Image Icorr

• 1: Let I ← z.
• 2: for i in L do
• 3: f i ←G(i-1):i (I)
• 4: if i inM then
• 5: [f i·,R(I)]+ ← 0
• 6: end if
• 7: I ← f i
• 8: end for
• 9: Icorr ← I
• 10: return Icorr

GANs plays a different role in the generating process in a hierarchical manner [Yang et al., 2021]. The authors note that the early and middle layers determine the spatial layout and the objects, and the last layers render the attributes and color scheme. Therefore, we set M to include the early or middle layers of the generator, since our work focuses on analyzing the root of unrealistic objects or defective parts.

Analysis for Layer

For a randomly selected artifact image generated from StyleGAN2, the first row in Figure 4 shows how LR-neurons are distributed in its featuremap for each layer. The generated image has a strange object on its chin and a grid-like background. The red part of the heatmaps indicates a high proportion of LR-neurons. The heatmaps on layers 5 and 7 have a large number of LR-neurons at the chin. The second row shows heatmaps and the third row shows generated images, when applying sequential ablation until the previous layers. In other words, the heatmap of layer 5 in the second row denotes the heatmap when applying sequential ablation at layers 0, 1, and 3. Through sequential ablation, LR-neurons at the defective parts of the chin are removed and the generated image is also successfully repaired. Therefore, we can identify that LR-neurons in the early layers are highly related to causing defective parts. However, when we apply sequential ablation for the higher layers, most of the background part disappears and the hairstyle become different. This implies that LR-neurons in layers 5 and 7 contributes to constructing detail features or the background. From this result, we conclude that applying sequential ablation until layer 3 is an appropriate choice to repair generated images without hindering the diversity of generations. In Section 4.2, the artifact correction results in Table 1 support our choice quantitatively.

Analysis for Rate

In this section, we provide qualitative analysis to verify our hypothesis and reveal the roles of neurons according to the frequency rate. Figure 5 shows the generation results of StyleGAN2 with sequential ablation applied in layers 0, 1, and 3 with different rates on FFHQ, LSUN-Cat,and LSUNChurch. In this analysis, we remove not only LR-neurons but also HR-neurons to identify the effects of HR-neurons even though sequential ablation originally denotes ablating LR-neurons. For each generated image, the first row shows results from ablating HR-neurons with a given R and the second row shows results from ablating LR-neurons.

When ablating HR-neurons, the basic structures of the objects, such as the shapes of faces, totally collapse. Uncommon characteristics, such as the blue artifact in the first cat image and the human-like artifact in the third cat image, still remain after ablating HR-neurons. Even though the remaining neurons, which are rarely activated, may be necessary parts to generate images with diverse properties, they easily hinder visual fidelity.

Artifacts in the generated images are removed when LRneurons are ablated. Distorted hands in the generated human images successfully disappear with R = 0.3 or 0.5. The watermarks in the second cat image and the second church image are eliminated, and their shapes become more natural. However, ablating [f i·,0.5 ]+ may remove too much information so that generated images have only fundamental components and lose their properties. In this case, R = 0.3 is a valid choice for sequential ablation to remove defective parts while maintaining the details.

3.3 Ablating Neurons to Repair Artifacts

From our previous analysis, we observe that LR-neurons in the early layers are highly related to generate defective parts in the results. Considering this, we propose an efficient ablation method to avoid generating artifact images. It follows the 'Sequential Ablation' method, which is described in Algorithm 1, and the early layers are set as a list of target layers M with R = 0.3. Experimental results in Section 4.2 will show that this setting is valid.

Figure 3 describes the overall correction process. Otherwise, we denote applying ablation is only denoted for a single layer by 'Single Ablation'. In Table 1, we use single ablation to compare the performance. Note that our ablation method does not require complex computations or manual analysis in the repair process. We can easily repair the image by setting activated LR-neurons as zero.

FFHQ

LSUN Cat LSUN Church

4 Experiment

In this section, we carry out the following qualitative and quantitative experiments to evaluate our approach. The main results show the relationship between LR-neurons and low visual fidelity (or artifacts). Deep generative models were reproduced in PyTorch from genforce github [Shen et al., 2020] which provides the official pre-trained weights; PGGANs trained on CelebA-HQ, LSUN-Cat, and LSUNChurch and StyleGAN2s trained on FFHQ, LSUN-Cat, and LSUN-Church.

4.1 Qualitative Results

50K latent codes were randomly sampled to generate images without the truncation method. The number of the total activated LR-neurons (R = 0.3) in the early layers was counted for each generated image, i.e., the sum of ∣∣[f i·,0.3 ]+∣∣. The target early layers were set to i = 1, 3, 5 for PGGAN and i = 0, 1, 3 for StyleGAN2. The dimensionality of each layer's output corresponds to R512×4×4 ,R512×8×8 , and R512×16×16 , respectively.

Low Fidelity Detection in Image Generations

To detect images with low visual fidelity, the generated images were sorted according to the number of activated LRneurons in decreasing order. the top 2K images and Bottom 2K images are then selected. As an example, Figure 2 shows the histogram of the number of activated LR-neurons. The blue region indicates that they have few activated LR-neurons and the red region indicates that the generated images have many activated LR-neurons. Consistent with our hypothesis, the generated images in the red box have more defective or distorted parts than the images in the blue box. The images in the blue box have a high visual fidelity, but lack diversity, whereas the images in the red box seem to have more uncommon properties. We provide abundant results of detection tasks in Appendix B-G.

Sequential Ablation

Figure 5 shows sequential ablation results of StyleGAN2 in FFHQ, LSUN-Cat, and LSUN-Church. The ablation method is conducted on layer 0, 1, and 3 with various rate R as described in Section 3.2. This supports that the LR-neurons induce the artifacts since removing them helps to generate realistic images. Furthermore, in Section 3.3, sequential ablation is conducted to correct artifacts generated from StyleGAN2 and PGGAN. As shown in Figure 6, we identify that our approach can repair the artifacts in various cases. In the case of StyleGAN2, a long crack on the face of the fifth human image was successfully eliminated. The buildings are not completely constructed and mixed with their backgrounds in the artifact church images generated from PGGAN. After sequential ablation, the buildings are repaired normally.

On the unaligned or noisy datasets such as LSUN, however, the HR-neurons have relatively less responsibility for creating a single representative object in the dataset. It is because the datasets that have sparse features can confuse the generative model to consider the objects to be mainly created StyleGAN2

PGGAN

as part of diversity. Therefore, our approach may work better on the well-aligned datasets. Additional results are provided in Appendix H (including the results of StyleGAN3 [Karras et al., 2021] with FFHQ).

4.2 Quantitative Results

Correction Study

Here, we evaluate our ablation method with various combinations of target layers and rate R in the FFHQ dataset with FID score [Heusel et al., 2017]. Table 1 shows FID scores computed with 10K real images and 1.3K generated artifacts images. Examples of hand labeled generated images are in Appendix A. 'None' denotes a case when we do not apply our ablation method and 'Random' denotes when we ablate 30% of randomly selected activated neurons. Note that we remove activated HR-neurons to identify the effects of HR-neurons on generations in the case of the 5th row. The ablation result when turning off [f i·,0.3 ]+ with i = 0, 1, 3 has the best FID score. Compared to the case [f i·,0.3 ]+, the case [f i·,0.7 ]+ has very high FID scores because HR-neurons play a significant role in generating essential parts of images. The most optimal setting is used ([f i·,R]+ with R = 0.3, i = 0, 1, 3) to conduct our qualitative experiments.

Diversity and Fidelity

We also quantify our detection method with the precision (fidelity), recall (diversity), and Realism Score [Kynk¨a¨anniemi et al., 2019] with 10K real images, the Top 2K and Bottom 2K samples for each dataset, and the number of neighborhood k = 3. Table 2 shows that the Bottom 2K samples that contain few LR-neurons have a high score in fidelity and realism score, but a low score in recall. On the other hand, the fidelity and realism score for the Top 2K samples that contain a large number of LR-neurons are lower than those of the Bottom 2K samples, while diversity is higher in all cases. These results are linked to the evidence of our hypothesis that encouraging diversity leads to artifacts. In addition, it may imply a tradeoff between fidelity and diversity in the generation procedure.

5 Conclusion

In this paper, we suggest a hypothesis that rarely activated neurons are highly related to generating artifacts, which might be induced as failures to create diversity. Our ablation study not only supports that low-rate neurons in the early layers induce defective parts but also shows that frequently activated neurons may affect essential parts. As a result, our approach can detect artifact images and correct defective parts by sequential ablation without any supervision.

Acknowledgements

This work was supported in part by the Institute of Information and Communications Technology Planning and Evaluation (IITP) grant funded by the Korea Government (MSIT) under Grant 2019-0-00075, Grant 2022-0-00184, and Grant 2022-0-00984 and partly supported by KAIST-NAVER Hypercreative AI Center.