ALL-E: Aesthetics-guided Low-light Image Enhancement

1 Introduction

1 Due to the limitations inherent in optical devices and the variability of external imaging conditions, images are often captured with poor lighting, under-saturation, and a narrow dynamic range. These degradation factors significantly impair the visual aesthetics of the images, and lead to detrimental effects on a wide range of downstream computer vision and multimedia tasks [Yu et al., 2021b; Cho et al., 2020; Guo et al., 2020]. Manually editing and enhancing low-light images is time-consuming, even for a professional. Therefore, many learning-based strategies have been proposed, introducing a series of potentially valid heuristic constraints for training image enhancement models [Lore et al., 2017; Wei et al., 2018; Zhang et al., 2019; Xu et al., 2020;

1 Dong Liang is the corresponding author. We thank Prof. Jie Qin from NUAA and Dr. Tianyu Ding from Microsoft Seattle for their comments. This work is supported by the National Natural Science Foundation of China under grant 62272229, and the Natural Science Foundation of Jiangsu Province under grant BK20222012.

(b) Low-light images with an average Aesthetic Score 5.694

Figure 1: Sample images on DPChallenge protect 1 . Each image receives dozens to hundreds of user ratings, ranging from 1 to 10. Higher scores indicate higher aesthetic quality.

Ren et al., 2019; Liang et al., 2022]. However, the aforementioned methods fail to account for the importance of subjective human evaluation in low-light image enhancement (LLE) tasks. On an online professional photography contest - DPChallenge 2 , the subjective human aesthetic scores of normal images are much higher than those of low-light images, and the former provides a significantly superior visual experience compared to the latter, as shown in Fig. 1. In light of this observation, we propose a novel LLE paradigm incorporating aesthetic assessment to effectively model human subjective preferences and improve both subjective experience and objective evaluation of the enhanced images.

However, embedding aesthetic assessment into LLE is non-trivial, as aesthetics are highly subjective and personalized evaluations that vary between individuals. These personalized user preferences can bring confusion to model learning. In addition, the manual aesthetic retouching of photographs is a highly causal and progressive process, which is difficult to replicate using existing LLE methods. To tackle these challenges, we propose the following solutions. First, we introduce a well-trained 'aesthetic oracle network' to construct an Aesthetic Assessment Module that can produce general aesthetic preferences, reducing penalization and increasing the method's versatility and aesthetic appeal. Second, we apply reinforcement learning to interact with the environment (i.e., the Aesthetic Assessment Module) to calculate rewards; we treat LLE as a Markov decision process, decomposing the augmented mapping relationship into a series of iterations through an Aesthetic Policy Generation Module, thus realizing progressive LLE adjustment. Third, we develop a group of complementary rewards, including aesthetics quality, feature preservation, and exposure control, to preserve better subjective visual experience and objective evaluation. All the rewards are non-reference, indicating that they do not require paired training images.

2 https://www.dpchallenge.com/

The main contributions of this paper are three-fold:

• We propose a new paradigm for LLE by integrating aesthetic assessment, leveraging aesthetic scores to mimic human subjective evaluations as a reward to guide LLE. To our knowledge, this is the first attempt to solve lowlight image enhancement using aesthetics.
• We devise aesthetics-guided LLE (ALL-E) through reinforcement learning and treat LLE as a Markov decision process, which divides the LLE process into two phases: aesthetic policy generation and aesthetic assessment.
• Our method is evaluated against state-of-the-art competitors through comprehensive experiments in terms of visual quality, no- and full-referenced image quality assessment, and human subjective surveys. All results consistently demonstrate the superiority of ALL-E.

2 RelatedWork

Low-Light Image Enhancement

Traditional Approaches. Early efforts commonly presented heuristic priors with empirical observations to address the LLE problems [Pizer et al., 1990; Land, 1977; Guo et al., 2016]. Histogram equalization [Pizer et al., 1990] used a cumulative distribution function to regularize the pixel values to achieve a uniform distribution of overall intensity levels in the image. The Retinex model [Land, 1977] and its multiscale version [Jobson et al., 1997] decomposed the brightness into illumination and reflections, which were then processed separately. Guo et al. [Guo et al., 2016] introduced a structural prior before refining the initially obtained illumination map and synthesizing the enhanced image according to the Retinex theory. However, these constraints/priors must be more self-adaptive to recover image details and color, avoiding the washing out details, local under/over-saturation, uneven exposure, or halo artifacts.

Deep Learning Approaches. Lore et al. [Lore et al., 2017] proposed a variant of the stacked sparse denoising autoencoder to enhance the degraded images. RetinexNet [Wei et al., 2018; Zhang et al., 2019] leveraged a deep architecture based on Retinex to enhance low-light images. RUAS [Liu et al., 2021] constructed the overall LLE network architecture by unfolding its optimization process. EnlightenGAN [Jiang et al., 2021] introduced GAN-based unsupervised training on unpaired normal-light images for LLE. Zero-DCE [Guo et al., 2020] transformed the LLE task into an image-specific curve estimation problem. SCL-LLE [Liang et al., 2022] cast the image enhancement task as multi-task contrast learning with unpaired positive and negative images and enabled interaction with scene semantics. However, all the above methods ignore the human subjective preferences of LLE.

Aesthetic Quality Assessment

Although there is no aesthetics-guided LLE method, and the focus of LLE is not on assessing the aesthetic quality of a given image, our work relates to this research domain in the sense that LLE aims at improving image quality.

Image aesthetics has become a widely researched topic in current computer vision. Image aesthetic quality assessment aims to simulate human cognition and perception of beauty to automatically predict how beautiful an image looks to a human observer. Previous attempts have trained convolutional neural networks for binary classification of image quality [Lu et al., 2014; Ma et al., 2017] or aesthetic score regression [Kong et al., 2016]. Assessing visual aesthetics has practical applications in areas such as image retrieval [Yu and Moon, 2004], image recommendation [Yu et al., 2021a], and color correction [Deng et al., 2018].

Since the highly differentiated aesthetic preference, image aesthetics assessment can be divided into two categories: generic and personalized image aesthetics assessment (GIAA and PIAA)[Ren et al., 2017]. [Ren et al., 2017] addressed PIAA problem by leveraging GIAA knowledge on userrelated data so that that model can capture aesthetic "offset". Later, research work attempted to learn PIAA from various perspectives, such as multi-modal collaborative learning [Wang et al., 2018], meta-learning [Zhu et al., 2020], multitask learning [Li et al., 2020] etc. Due to the high subjectivity of PIAA tasks, which pay more attention to the differences in personality factors, we aim to introduce "general aesthetic preferences"/generic image aesthetics assessment (GIAA) to LLE. To our knowledge, the proposed scheme in this paper is the first attempt to solve the LLE problem using aesthetics to preserve better both subjective visual experience and objective evaluation.

Reinforcement Learning for Image Processing

After deep Q-network achieved human-level performance on Atari games, there has been a surge of interest in deep reinforcement learning (DRL). For image-processing tasks, Yu et al. [Yu et al., 2018] proposed RL-Restore to learn a policy of selecting appropriate tools from a predefined toolbox to restore the quality of corrupted images gradually. Park et al. [Park et al., 2018] presented a DRL-based method for color enhancement and a distortion-recovery training scheme that only requires high-quality reference images for training.

While these methods focus on global image restoration, Furuta et al. [Furuta et al., 2019] proposed pixelRL to enable pixel-wise image restoration, which extended DRL to pixellevel reinforcement learning, making it more flexible in dealing with image problems. Similarly, Zhang et al. [Zhang et al., 2021a] proposed a novel DRL-based method for achieving LLE at the pixel level. In contrast, our DRL network learns with an image aesthetic reward to obtain LLE results that try to satisfy universal users.

3 Methodology

3.1 FromAesthetic Annotation to LLE

Many factors can affect the beauty of images, including the richness of color, correct exposure, depth of view, resolution, high-level semantics, and so on. Previous work [Lu et al., 2014; Ma et al., 2017; Kong et al., 2016; Yu and Moon, 2004] have preliminarily analyzed these factors to generate the aesthetic rating of an image, facilitating manual aesthetic annotation for downstream tasks.

As aesthetics is a highly subjective and personalized evaluation with individual differences, [Kang et al., 2020] modeled an aesthetic mean opinion score (AMOS) as a weighted sum of its four relatively independent attributes to generate more reliable aesthetic annotation,

where f is the scores of attribute index {1, 2, 3, 4} of an image. 1 is the light/color attribute, 2 is the composition and depth attribute, 3 is the imaging quality attribute, and 4 is the semantic attribute. To derive the specific importance (the weight values) in AMOS, [Kang et al., 2020] directly elicited from more than 30 observers the importance of each attribute in forming their overall aesthetic opinion. The observers had to indicate which factor(s) influenced their overall aesthetic score among the rated attributes. From Eq.(1), we observe that light/color is one of the three most important attributes, contributing approximately 30%. Another work [Yang et al., 2022] utilizes a Pearson correlation coefficient between the light condition of an image and its aesthetic rating, archiving a high correlation coefficient of 0.67.

Motivated by the above findings, we propose an aestheticsguided LLE. The first step is to select an aesthetic 'oracle' that can provide general aesthetic preferences with versatility and reflect popular aesthetics. In our implementation, we employ a lightweight aesthetic network [Esfandarani and Milanfar, 2018] with a VGG-16 backbone as the aesthetic 'oracle', trained on the AVA dataset [Murray et al., 2012]. Though NIMA [Esfandarani and Milanfar, 2018] (81.5%) stands at the middle level in terms of accuracy (DMA-Net [Lu et al., 2015] 75.4%; MTCNN [Kao et al., 2017] 79.1%; Pool-3FC [Hosu et al., 2019] 81.7%; ReLIC [Zhao et al., 2020] 82.35%), NIMA is with less computational complexity compared with other available models (ReLIC with a MobileNetV2 backbone has 2.7 times more parameters than NIMA with a MobileNetV1 backbone, while MobileNetV2 has fewer parameters than MobileNetV1). AVA is a database with 250,000 photos evaluated for aesthetic quality, with crowd-sourcing voting on the aesthetics for each image (from

78 to 549 people for each image). The aesthetic oracle network pre-trained on this vast dataset yields trustworthy preferences for broad aesthetics.

Since human aesthetic image retouching is a dynamically and explicitly progressive process that causally interacts with the current state of the image, we treat LLE as a Markov decision process, decomposing it into a series of iterations. To mimic this process, we design a reinforcement-learningbased Aesthetic Policy Generation Module, which interacts with the environment - Aesthetic Assessment Module to obtain rewards and provide optimized actions to realize progressive LLE adjustment. As shown in Fig. 2, our ALL-E consists of an Aesthetic Policy Generation Module based on an asynchronous advantage actor-critic network (A3C) [Mnih et al., 2016] to generate the action At , and an Aesthetic Assessment Module based on an aesthetic oracle network [Esfandarani and Milanfar, 2018] and a series of loss function to generate the reward rt . At the t-th step, given an image st , the Aesthetic Policy Generation Module generates an enhanced image st+1 via At , which is then fed to the Aesthetic Assessment Module to generate rt , then progressively complete image enhancement until n steps.

3.2 Aesthetic Policy Generation

The reason to use A3C [Mnih et al., 2016] is that it reports pixel-wise action performance with efficient training. A3C is an actor-critic method consisting of two sub-networks: a value network and a policy network denoted as θv and θp, respectively. Both networks use the current state image st as the input at the t-th step. The value network outputs the value V (st ), which represents the expected total discounted rewards from state st to sn , and indicates how good the current state is:

where Rt is the total discounted reward:

where γ i is the i-th power of the discount factor γ, rt is the immediate reward (will be introduced in Section 3.4) at t-th step. As some actions can affect the reward after many steps by influencing the environment, reinforcement learning aims to maximize the total discounted reward Rt rather than the immediate reward rt .

The gradient for θv is computed as follows:

The policy network outputs the probability of taking action At ∈ A.S. (the action space, will be introduced in the following subsection) through softmax, denoted as π(At |st ). The output dimension of the policy network is equal to |A|.

To measure the rationality of selecting a specific action At in a state st , we define the advantage function as:

It directly gives the difference between the performance of action At and the mean value of the performance of all possible actions. If this difference (i.e. the advantage of the chosen action) is greater than 0, then it indicates that action At is better than the average and is a reasonable choice; if the difference is less than 0, then it implies that action At is inferior to the average and should not be selected.

The gradient for θp is computed as follows:

3.3 Action Space Setting

Human experts often manually retouch photographs through curve adjustments in retouching software, where the curve parameters depend only on the input image. Usually, curves for challenging low-light images are of a high order. [Guo et al., 2020] suggests that this procedure can be realized by recurrently applying the low-order curves. In this work, we apply a second-order pixel-wise adjustment curve (PAC) at each step t. We enhance an input image st by iteratively applying a PAC-based action At (x) at the t-th step,

where x denotes the pixel coordinates. Eq.(7) models the enhancement of a low-light image as a sequential action-making problem by finding the optimal pixel-wise parameter map At (x) for light adjustment curves at each step t. Therefore, our optimization goal is to find an optimal light adjustment action sequence. To achieve this, we need a metric (the reward) to measure the light aesthetic of an image st . The exact calculation is described in the following subsection.

Fundamentally, low-light image enhancement can be regarded as the search for a mapping function F, such that sH = F(sL) is the desired image, which is enhanced from the input image sL. In our design, the mapping function F is represented as the ultimate form of multiple submappings {A1 ,A2 , . . . ,An} continuously and iteratively augmented, where At is constrained in a predetermined range of the action space (A.S.). The range of A.S. is crucial to the performance of our method, as too narrow a range will result in insufficient improvements, and too extensive a range will result in excessive search space.

Here, we empirically set the range of A.S. ∈ [-0.5, 1] with a graduation interval 1/18. This setting ensures that:

1) The brightness of each pixel is in the normalized range [0, 1], according to Eq.(7);

1. As the brightness of some pixels may need to be reduced, a negative range [-0.5, 0] is also set;
2. PAC is monotonous while also alleviating the cost of searching for suitable PAC for low-light image enhancement.

3.4 Reward Function

This section introduces three complementary rewards, including aesthetics quality, feature preservation, and exposure control, to preserve better subjective visual experience and objective evaluation. Note that all the rewards are non-reference, requiring no paired training images.

Aesthetics Quality Reward. As mentioned above, the aesthetic quality score of an image is closely correlated with several factors. In this work, we focus on dynamically adjusting and improving the brightness by the aesthetic score of an image. Therefore, utilizing the aesthetic score as a direct reward function would be an inadequate representation of the desired outcome. Instead, the difference in aesthetic scores between the original and enhanced images is employed as the reward for the currently selected action. The image aesthetics quality reward rtaes is:

where K denotes the range of ratings for the aesthetic scores of the images, i.e., [1, 10], and P denotes the probability of each rating. st denotes the state of the image at t-th step, st+1 denotes the state of the image at t + 1-th step.

Feature Preservation Reward. Since color naturalness is a concern in low-light image enhancement, we introduce a color constancy term incorporating an illumination smoothness penalty term as the feature preservation reward. It is based on the gray-world color constancy hypothesis [Buchsbaum, 1980; Guo et al., 2020], which posits that the average pixel values of the three channels tend to be of the same value. rtfea constrains the ratio of the three channels to prevent potential color deviations in the enhanced image. In addition, to avoid aggressive and sharp changes between neighboring pixels, an illumination smoothness penalty term is also embedded in rtfea =

where ξ = {R, G, B}, J p denotes the average intensity value of p channel in an image, (p, q) represents a pair of channels, n is the number of the steps, and ▽x and ▽y denote the horizontal and vertical gradient steps, respectively. We set λ to 100 in our experiments to achieve the best results.

Exposure Control Reward. The exposure control reward rtexp , which is a loss function widely used in the recent literature [Guo et al., 2020; Zhang et al., 2021a], measures the deviation of the average intensity value of a local region from a predefined well-exposedness level in RGB color space:

(j) Ours

Table 2: NIQE ↓ and UNIQUE (UN.) ↑ scores on DICM, LIME, MEF, VV, and NPE datasets.

where B represents the number of non-overlapping local regions of size 16×16, Yb is the average intensity value of a local region b in st+1 . According to [Guo et al., 2020; Zhang et al., 2021a], E is set to 0.6. For a given enhanced image, the immediate reward rt at a current state st is:

where w1, w2 and w3 are tunable hyperparameters. As introduced in Section 3.2, the goal of reinforcement learning is to maximize the total discounted reward Rt in Eq.(3) with this immediate reward rt .

3.5 Efficient Training Details

We use 485 low-light images of the LOL dataset [Wei et al., 2018] to train the proposed framework. We resize the training images to the size of 244×244. The maximum number of training epochs was set to 1000, with a batch size of 2. We train our framework end-to-end while fixing the weights of the aesthetic oracle network. Our framework is implemented in PyTorch on an NVIDIA 1080Ti GPU. The model is optimized using the Adam optimizer with a learning rate of 1e-4 . The total number of steps in the training phase is set to n = 6. In accordance with the training phase, the total number of steps is also set to n = 6 in the testing phase. Under these settings, training 1000 epochs costs about one day.

4 Experiments

4.1 Benchmark Evaluations

We compare our method with several state-of-the-art methods: LIME [Guo et al., 2016], Retinex-Net [Wei et al., 2018], ISSR [Fan et al., 2020], Zero-DCE [Guo et al., 2020], EnlightenGAN [Jiang et al., 2021], RUAS [Liu et al., 2021], ReLLIE [Zhang et al., 2021a], SCL-LLE [Liang et al., 2022], Uretinex-net[Wu et al., 2022], Zhao et al.[Zhao et al., 2021], Ma et al.[Ma et al., 2022]. The results of the above methods are reproduced by the publicly available models provided with the recommended test settings. To thoroughly evaluate the proposed method, a comprehensive set of experiments were conducted, including a visual quality comparison, image quality assessment, and human subjective survey, which are discussed in the following sections.

Visual Quality Comparison

We present the visual comparisons on typical low-light images in LOL test dataset [Wei et al., 2018] and LIME [Guo et al., 2016] dataset. We first investigate whether the proposed method achieves visually pleasing results in terms of brightness, color, contrast, and naturalness. We observe from Fig. 3 and Fig. 4 that the images enhanced by our method are aesthetically acceptable and do not cause any discernible noise and artifacts. Specifically, LIME causes color artifacts at strong local edges; Retinex-Net and EnlightenGAN cause local color distortion and lack of detail; ISSR and RUAS produce severe global and local over/under-exposure; Zero-DCE and SCL-LLE under-enhance extremely dark images, while ReLLIE over-enhances.

Fig. 5 demonstrates the visualization of the enhancing procedure of the proposed method. As the enhancement steps t progress, the image's brightness increases. In our test experiments, step t = 6 yields the best visual performance, which is rational as the total number of steps is set to n = 6 in the training phase. When continuing to enhance the image with an additional step (t = 7), the image may tend to be overenhanced, as shown in Fig. 5 (h). After many experiments, we found that t = 6 is the global best in most cases. in practice, the optimal enhancement step for a specific image may be different. The image sequences of all steps can be listed for people to choose their preferred one.

Image Quality Assessment (IQA)

For quantitative comparison, we use two non-reference evaluation indicators, Natural Image Quality Evaluator (NIQE) [Mittal et al., 2013], and UNIQUE [Zhang et al., 2021b]. NIQE is a well-known no-reference image quality assessment for evaluating image restoration without ground truth and providing quantitative comparisons. Since NIQE is regarded as poorly correlated with subjectivity, we also adopt UNIQUE, a metric for non-reference evaluation that is more rational and closer to subjective human opinion.

For full-reference image quality assessment, we employ the Peak Signal-to-Noise Ratio (PSNR, dB) and Structural Similarity (SSIM) metrics to compare the performance of various approaches quantitatively.

Table 1 and 2 summarizes the performances of our technique and thetechnique state-of-the-art methods on the test images of LOL test dataset [Wei et al., 2018], DICM [Lee et al., 2012], LIME [Guo et al., 2016], MEF [Ma et al., 2015], VV 1 , and NPE [Wang et al., 2013]. DICM, LIME, MEF, VV, NPE are ad hoc test datasets, including 64, 10, 8, 24, 17 images, respectively. They are widely used in LLE testing: SCL-LLE [Liang et al., 2022], EnlightenGAN [Jiang et al., 2021], Zero-DCE [Guo et al., 2020] et al.. Images in them are diverse and representative: DICM is mainly landscaped with extreme darkness; LIME focuses on dark street landscapes; MEF focuses on dark indoor scenes and buildings; VV is mostly backlit and portraits; NPE mainly includes natural scenery in low light. Note that only the PSNR of our method is not ranked first. We believe this is because the aesthetic reward focuses more on the global aesthetic evaluation and is insufficiently responsive to local noise.

Human Subjective Survey

We conduct a human subjective survey (user study) for comparisons. Each image in the LOL test dataset was enhanced by nine methods (LIME, Retinex-Net, ISSR, Zero-DCE, EnlightenGAN, RUAS, ReLLIE, SCL-LLE, and our approach), 25 human volunteers were asked to rank the enhanced images. These subjects are instructed to consider: 1) Whether or not noticeable noise is present in the images; 2) If the images contain over or underexposure artifacts; 3) Whether or not the images display non-realistic color or texture distortion.

1 https://sites.google.com/site/vonikakis/datasets

(f) t = 5

(g) t = 6

(h) t = 7

(e) Ground truth

Figure 6: Ablation study on the contribution of each component.

We assign each image a score between 1 and 5; the lower the value, the higher the image quality. The final results are shown in Table 1. We can see that our method achieves the highest score.

4.2 Ablation Study

To demonstrate the effectiveness of the aesthetic rewards and the action space configuration proposed by our technique, we performed several ablation experiments. Since rtfea and rtexp are demonstrated to be valid in the recent literature [Guo et al., 2020; Zhang et al., 2021a], we consider them to be the baseline rewards without ablation studies on them.

Action Space and Aesthetics Quality Reward. Regarding the the action space (A.S.) settings, the baseline follows [Zhang et al., 2021a]: the range of A.S. ∈ [-0.3, 1] with graduation as 0.05. In comparison, our settings fall within the range of A.S. ∈ [-0.5, 1] with a graduation of 1/18. Specifically, we design two experiments by removing the aesthetics quality reward component and keeping the baseline settings.

Subjective Experience. The visualization of the effects of action space and aesthetics quality reward rtaes are shown in Fig. 6. The absence of rtaes rendered the image gloomy and unappealing, while improper action space settings led to overexposure in certain portions of the enhanced image.

Objective Evaluation. Table 3 shows the NIQE, UNIQUE, PSNR, and SSIM scores under each experiment. Note that, from Table 3, the absence of rtaes does not appear to have a significant impact on the overall outcomes; however, the absence of a suitable action space (A.S.) setting appears to have a significant negative impact on performance. Since our experimental setup relies on the presence of different settings of the action space and image aesthetics quality reward, the absence of any component would result in subpar results. The aesthetics-guided action space setting can achieve the best results.

Our method can be seen as a compromise between aesthetics and Image Quality Assessment (IQA). In some specific scenarios, there would be situations where a high aesthetic score with poor IQA. The Aesthetics Quality Reward raes considers general aesthetics, while the Feature Preservation Reward rfea and the Exposure Control Reward rexp in our method ensure relatively good IQA.

5 Conclusion

We have proposed an effective aesthetics-guided reinforcement learning method to solve the LLE problem. ALL-E illustrates how we can leverage aesthetics to balance the subjective and the output. Unlike most existing learning-based methods, using the aesthetic policy generation and aesthetic assessment modules, our method treats LLE as a Markov decision process to realize progressive learning. With aesthetic assessment scores as a reward, general human subjective preferences are introduced, which aids in producing aesthetically pleasing effects, i.e. it interacts with the environment (aesthetic assessment module) to yield a reward to mimic a human's image retouching process.