DIIN: Diffusion Iterative Implicit Networks for Arbitrary-scale Super-resolution

1 Introduction

Single Image Super-Resolution (SISR), which aims to reconstruct high-resolution (HR) images from corresponding low-resolution (LR) observations, has recently gained great progress. Most exciting SISR methods [Dai et al., 2019; Liang et al., 2021; Dai et al., 2024; Dong et al., 2015; Zhang et al., 2018; Zhou et al., 2023] focus on fixed-scale setups, where models are optimized predefined up-sampling factors (e.g.g. 2×, 4×) during training. However, such fixedscale setup hinders the applications in real-world scenarios, where continuous arbitrary-scale factors are desired.

To upsample continuous arbitrary-scale factors, arbitraryscale super-resolution (ASSR) has recently received much attention. In particular, implicit neural representation (INR)

∗Corresponding author: Hang Guo (cshguo@gmail.com)

based SR methods [Chen et al., 2021; Lee and Jin, 2022; Chen et al., 2023] have achieved impressive performance by modeling image signals as continuous functions. Specifically, this type of method fits desired coordinates into the implicit neural function parameterized by a neural network to query image pixels at any position. In this way, the corresponding pixel signal can be generated directly at any coordinate, eliminating the constraints of pre-defined scales. However, most existing ASSR methods rely on point-wise multi-layer perceptions (MLPs) to parameterize the implicit neural function. However, such a design inherently treats different query positions independently, thus limiting the flow of contextual information between pixels. As illustrated in Figure 1, LIIF [Chen et al., 2021] and SRNO [Wei and Zhang, 2023] operate with restricted receptive fields, relying on limited contextual cues. In contrast, our proposed method expands the receptive field and adaptively incorporates relevant information from surrounding pixels.

The above observations motivate us to incorporate more contextual information in implicit neural representation. To this end, we rethink the modeling process of the implicit neural functions to inject neighborhood interaction and attempt to re-project the forward process of implicit neural functions as a signal diffusion process, where the signal transmission between different query locations adapts over time. Specifically, we define the diffusion rate to depict the mutual influence between any two nodes at a certain time step. The diffusion rate can measure the rate at which the signal flows from one node to another.

Inspired by the above analysis, we can derive the diffusion implicit function that describes the state updates of all nodes at each time step, and propose an efficient Diffusion Iterative Implicit Network (DIIN) with global signal flows for arbitrary-scale SR, which mainly consists of stacked diffusion iteration layers to help the sampled representations undergo iterative updates layer by layer. In this way, our method explicitly captures the dependencies between query locations and fully utilizes the contextual information in the image space. Furthermore, we introduce a set of adaptive token dictionaries, which interact with the latent code output from the encoder through a cross-attention module in an endto-end manner to generate sparse representations in the image feature space. These token dictionaries provide auxiliary information support for the iterative updates of each query point, effectively addressing the problem of incomplete contextual information due to the random sampling of HR coordinates. Besides, we introduce a position-aware Embedding Block, which adaptively modulates the sampled features based on query coordinates, thereby enhancing the spatial dependencies between subsequent input samples.

The main contributions are summarized as follows:

• We rethink the forward process of implicit neural functions as a signal diffusion process, and propose a novel Diffusion Iterative Implicit Network (DIIN) with stacked diffusion iteration layers to promote global signal flow with neighborhood interactions.
• We further introduce a set of adaptive tokens through cross-attention modules, interacting with the latent codes output by the encoder to generate sparse image feature space representations. This design effectively supports the iterative update of query points, addressing the issue of incomplete contextual information.
• Extensive experiments demonstrate the effectiveness of our method over other state-of-the-art methods across various in-distribution and out-of-distribution upsampling factors.

2 RelatedWork

2.1 Implicit Neural Representation

Implicit neural representations (INR) build the mapping between coordinates and their signal values using a neural network, enabling continuous and memory-efficient modeling for various signal types (e.g., 1D audio [Gao et al., 2022], 2D images [Tancik et al., 2020], 3D shapes [Park et al., 2019; Mildenhall et al., 2020]). In recent years, implicit neural representations have seen significant advancements in 2D applications, such as image representation [Sitzmann et al., 2020; Xie et al., 2023] and image super-resolution [Liang et al., 2021; Lee and Jin, 2022]. For image super-resolution, [Chen et al., 2021] proposed a method that uses implicit functions to model continuous image signals. This approach no longer relies on fixed-resolution storage of image signals, but can directly generate corresponding signal values at arbitrary coordinates, thereby overcoming the limitations of traditional methods that depend on predefined scales, offering greater flexibility and accuracy.

2.2 Single Image Super-Resolution

Single Image Super-Resolution (SISR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) observations, with the goal of improving perceptual quality while accurately recovering spatial details. Since the introduction of SRCNN [Dong et al., 2015], the first deep learningbased SISR model, various implementation architectures have emerged in the field with the development of new models. For example, CNN-based methods [Dai et al., 2019; Lim et al., 2017; Zhang et al., 2018; Kim et al., 2016], transformer-based methods [Liang et al., 2021; Dai et al., 2024; Zhang et al., 2024; Guo et al., 2024a], diffusion-based methods [Saharia et al., 2021; Xia et al., 2023; Wang et al., 2024b; Guo et al., 2024b; Wu et al., 2024; Wang et al., 2024a; Lin et al., 2024], and SSM-based methods [Guo et al., 2024d; Ren et al., 2024; Guo et al., 2024c] have driven the development of the field, each leveraging the unique characteristics of their respective architectures. However, these methods are limited to fixed scales, which makes them less suitable for real-world applications where continuous scaling adjustments are required.

2.3 Arbitrary-Scale Super-Resolution

To overcome the above limitation, Arbitrary-Scale SuperResolution (ASSR) has become a prominent research focus, allowing image reconstruction at any up-sampling factor with a single model. MetaSR [Hu et al., 2019] is among the first works to tackle this problem, introducing a meta-upscale module that predicts convolutional filter weights based on coordinates and scaling factors. Inspired by INR, LIIF [Chen et al., 2021] learns implicit features from local regions to predict the RGB value at arbitrary coordinates. Subsequent works have focused on improving performance by incorporating more features into the MLP-parameterized continuous function. For instance, LTE [Lee and Jin, 2022] proposed a local texture estimator that maps coordinates to Fourier domain information, enhancing the expressive power of its local implicit function. CLIT [Chen et al., 2023] combines local attention mechanisms with frequency encoding techniques to enhance the ability to capture fine details. Meanwhile, an accumulative training strategy and a cascaded framework are employed to optimize the training process. Although these INR-based methods achieve state-of-the-art results in arbitrary-scale super-resolution, they often parameterize the implicit neural function using point-wise MLPs, which results in the independence of different query positions, limiting the signal flow between pixels.

3 Method

3.1 Signal Diffusion Perspective

We propose the Diffusion Iterative Implicit Network (DIIN) to address the limitations of existing implicit neural representation methods, which either lack contextual interaction or suffer from high computational complexity. Our method rethinks the forward computation of implicit neural functions as a dynamic signal diffusion process, enabling efficient and adaptive information propagation between query positions while maintaining linear computational complexity.

Inspired by the analogy between neural network computation and heat conduction in thermodynamics [Wu et al., 2023], we model the signal flow between query positions as a dynamic diffusion process. This process is governed by a diffusion rate Sij(Z(t), t), which measures the influence of node. The diffusion process is expressed as:

Using a numerical finite difference method, we discretize the above equation into an iterative update form:

where α is a constant and S(k) ij is the diffusion rate matrix. By making different assumptions, this framework can generalize existing models, such as:

• MLP: When S(k) ij is an identity matrix, each query position is computed independently.
• Attention: When S(k) ij allows interactions between all positions, the model captures global dependencies but incurs quadratic complexity.

3.2 Diffusion Iterative Implicit Function (DIIF)

DIIF combines the strengths of MLPs and attention mechanisms by efficiently modeling contextual interactions with linear complexity. It represents the image signal as a continuous function, taking the latent code M extracted by an encoder and query coordinates xq as input to predict pixel values at specified coordinates:

For a set of query coordinates xq, the corresponding latent codes X are retrieved from M and passed through a neural network with layers f (l) :

where X represents the corresponding latent codes, and ˆY is the output prediction. To model dependencies between query coordinates, DIIF adopts the iterative update function from Eqn. 2. The diffusion rate ˆS(k) ij is computed as:

where f(∥∥∥˜z(k) i -˜z(k) j ∥∥∥2 2 ) measures the signal flow between nodes (i, j). To reduce complexity from O(N 2 ) to O(N), we approximate the diffusion rate using a dot-product with L2 normalization:

Furthermore, the computation of the aggregated signal flow is expressed as:

To enhance contextual modeling, we introduce an auxiliary adaptive token dictionary. This dictionary learns prior contextual information from the input data and aligns it with specific test images. The Diffusion Iterative Implicit Function is modified as:

This integration ensures robust predictions by leveraging both contextual and adaptive information, enabling efficient and accurate super-resolution across arbitrary scales.

3.3 Network Architecture

Based on the above discussion, we modeled the proposed Diffusion Iteration Implicit Networks (DIIN), as illustrated in Figure 2. In this section, we will provide a detailed explanation of the implementation of each corresponding module. Position-Aware Embedding Block. To enhance the spatial dependencies between input samples in the Diffusion Iteration Image Function, we generate conditional embeddings using coordinate and scale information. These conditional embeddings are used to adaptively modulate the sampled features, thereby strengthening the spatial correlation between input samples. This process provides more distinctive features for the subsequent diffusion process.

Specifically, given query HR coordinates xq ∈ xHR, the corresponding input features zq ∈ Rd for the Diffusion Iteration Layer are obtained through modulation by the PositionalAware Embedding Block (PAEB):

where xl and al represent the coordinates and corresponding features of the four neighbors of xq, respectively. ∆ ˆ coord(xq) denotes the relative coordinates between xq and its four neighbors. sl is the feature integration weight used to reduce blocky artifacts generated by direct interpolation of low-resolution feature maps. cell = (2/rx, 2/ry) represent a local region of size rx ×ry in HR image, where rx and ry are the scaling factors.

As depicted in Figure 2(c), ∆ ˆ coord(xq) and cell are processed through two separate MLPs to obtain their respective embeddings. These embeddings are subsequently fused to produce the corresponding conditional embedding. This conditional embedding is then used to adaptively modulate the sampled features, resulting in the initial embeddings Z (0) q for the Diffusion Iteration Layer.

Diffusion Implicit Block. As shown in Figure 2(b), we model the implicit iterative process at each layer based on Eqn. 7, where the input features are transformed into Q, K, and V , resulting in the following expressions:

where 1N×1 is an all-one vector, ˜Q = L2(Q) and ˜K = L2(K) denotes L2 normalization applied to the query and key vectors, respectively.

Specifically, given the input features of each layer Z ∈ RH×W×C , we first apply a 1 × 1 convolution layer to transform the features. The resulting features are then divided into three branches: (1) Two branches undergo L2 normalization to generate the query vector Q and key vector K, which are used to compute attention weights; (2) The third branch retains the unnormalized features V as the value for feature enhancement.

Subsequently, the diagonal matrix R and the weighted fusion of V are calculated based on Eqn.12 and Eqn.13, producing the final output P. The diagonal matrix R balances feature importance by aggregating global and local information, while the subsequent feature aggregation combines global pooling and local attention to effectively integrate global and local dependencies, yielding the enhanced features.

Dictionary Cross-Attention Block. Since query points are randomly sampled from the HR image, the input samples lack complete contextual information. To address this, we employ an adaptively refined token dictionary to provide auxiliary support for the iterative updates of each query point.

Inspired by [Zhang et al., 2024], we introduce an additional dictionary, D ∈ RM×d , initialized as network parameters to encapsulate external priors during training. As illustrated in Figure 2, the learned token dictionary D interacts with the latent code X via a cross-attention block (CSB) to adaptively capture global information, which serves as auxiliary input for the subsequent Diffusion Iteration Layer. The dictionary D is further used to generate the Key dictionary KD and the Value dictionary VD. Simultaneously, the input feature X ∈ RN×d generates Query tokens:

where WQ ∈ Rd×d/r , WK ∈ Rd×d/r , and WV ∈ Rd×d are linear transformation matrices for the query tokens, key dictionary tokens, and value dictionary tokens, respectively. Then, we use the key dictionary and the value dictionary to enhance query tokens via cross-attention calculation:

where τ is a learnable parameter used to adjust the range of similarity values, and Simcos(·, ·) denotes the calculation of cosine similarity between two tokens. The output of the DCAB is divided into two parts: S = A · V D, which represents the auxiliary information obtained from the token dictionary, used for the iterative update of each query point, and A, the attention map calculated, which is used for the subsequent adaptive refinement of the token dictionary.

Diffusion Iteration Layer. As shown in Figure 2(b), the DIL serves as the core module of our method, comprising the Diffusion Implicit Block (DIB) and the Dictionary Cross-Attention Block (DCAB). The first DIL layer takes two inputs: (1) features modulated by the Position-Aware Embedding Block, which integrates coordinate information and latent codes, and (2) a token dictionary adaptively refined through cross-attention with the encoder's latent codes. For subsequent layers, the input is the output of the previous layer.

Specifically, Z(l-1) is obtained by combining the DIB output P and the DCAB output S through addition, followed by a 1 × 1 convolution. Finally, the learned parameter α1 is applied to adaptively combine the results, producing Z(l) :

To adaptively refine the token dictionary, we select the corresponding enhanced feature Z(l) to reconstruct the new token dictionary D(l) :

where Norm is normalization layer to adjust the range of attention map, and σ is a learnable parameter.

4 Experiments

4.1 Experiments Setting

Datasets. Similar to [Chen et al., 2021; Wei and Zhang, 2023], we use 800 high-quality images with a 2K resolution from the DIV2K [Agustsson and Timofte, 2017] dataset as training data. During testing, the model is evaluated on the DIV2K validation set and several commonly used benchmark datasets, including Set5 [Bevilacqua et al., 2012], Set14 [Zeyde et al., 2010], B100 [Martin et al., 2001] and Urban100 [Huang et al., 2015]. Following previous works [Chen et al., 2021; Wei and Zhang, 2023], we evaluated HR image quality using PSNR, calculated on the RGB channels for DIV2K and on the Y channel (YCbCr space) for other benchmarks.

Implementation Details. We primarily follow the previous implementation [Chen et al., 2021; Wei and Zhang, 2023], using Bicubic downsampling [Boor, 1962] in PyTorch to obtain paired data for training an arbitrary-scale super-resolution model. Specifically, we crop 128s × 128s patches as ground truth (GT), where s is a scaling factor sampled from the uniform distribution U(1, 4). We use the existing SR models, such as EDSR [Lim et al., 2017] and RDN [Zhang et al., 2018], as backbones without their upsampling modules to evaluate various arbitrary-scale upsampling methods. We train all models with the Adam optimizer [Kingma and Ba, 2015], starting from an initial learning rate of 4 × 10-5 and minimizing the L1 loss for 1500 epochs using a batch size of 32. The learning rate is updated by a cosine-annealing schedule every 50 epochs. All experiments are implemented in PyTorch [Paszke et al., 2019] and executed on four NVIDIA RTX 3090 GPUs.

4.2 Comparisons With State-of-the-Art

Quantitative Results. Table 1-2 present a quantitative comparison of the proposed DIIN method with existing arbitraryscale SR methods, including MetaSR [Hu et al., 2019], LIIF [Chen et al., 2021], LTE [Lee and Jin, 2022], CLIT [Chen et al., 2023], and SRNO [Wei and Zhang, 2023]. We evaluate the performance of EDSR [Lim et al., 2017] and RDN[Zhang et al., 2018] as backbones across five test datasets, considering upsampling factors ranging from ×2 to ×30. Table 1 summarizes the quantitative results in terms of PSNR (dB) on the DIV2k dataset. It can be observed that our DIIN model consistently delivers outstanding super-resolution performance across all scaling factors, with up to a 0.05dB improvement over the second-best performer. Notably, our method demonstrates significant performance gains at the ×30 scaling factor, even surpassing some methods that use RDN as the encoder. Table 2 compares DIIN with prior works [Chen et al., 2021; Wei and Zhang, 2023] on widely-used benchmark datasets, including Set5 [Bevilacqua et al., 2012], Set14 [Zeyde et al., 2010], B100 [Martin et al., 2001], and Urban100 [Huang et al., 2015], using RDN as the backbone. We observe that DIIN consistently achieves either the best or second-best performance across these benchmark datasets, further demonstrating the effectiveness of our method.

Qualitative Results. In Figure 3, we present a qualitative comparison with other arbitrary-scale super-resolution (SR) methods. Our model demonstrates the ability to generate SR images with sharper and more coherent textures, particularly excelling in areas with regular patterns. In the first-row example, our approach effectively restores the smoothness and sharpness of lines while avoiding jagged artifacts. Additionally, it achieves more accurate recovery of hole edges and successfully suppresses artifacts in complex reflective regions. More visual results from the test set are provided in the supplementary material. Additional qualitative results can be found in the supplementary materials.

4.3 Ablation Study

In this section, we present a series of ablation studies to validate the design decisions proposed in this paper. All ablation experiments are conducted on the DIV2K validation set, using EDSR [Lim et al., 2017] as the encoder baseline, and evaluated using the PSNR metric.

Validation of the Design Choices. To validate the effectiveness of each component in the proposed DIIN model, we constructed three models and compared their performance on image super-resolution tasks. Table 3 summarizes the quantitative contributions of these components. The results show that incorporating the Diffusion Iteration Layer significantly improves performance. Removing the Token Dictionary leads to some performance degradation, indicating its role in effectively supplementing information. Additionally, replacing the Position-Aware Embedding Block with a simple 1×1 convolution results in noticeable performance degradation, further demonstrating that using coordinate information to modulate features enhances spatial dependencies among input samples, thereby improving overall performance.

Effectiveness of Different Designs of Diffusion Iteration Layer. We evaluated the model's performance, parameter count, and inference time at different depths (number of layers N) to analyze the impact of iterative layers on the performance of the Diffusion Iteration Layer. As shown in Table 4, increasing the model depth leads to improved performance (PSNR) across different scaling factors (×2,×3,×4), with particularly significant gains observed for smaller scaling factors (×2 and ×3). However, the marginal performance gains diminish as the depth further increases, while both the parameter count and inference time grow substantially. Balancing performance and computational cost, we ultimately selected N = 2 as the hyperparameter setting for our method.

5 Conclusion

In this paper, we rethink the modeling of implicit neural functions by introducing neighborhood interactions and redefining the forward process as a signal diffusion mechanism. To achieve this, we propose Diffusion Iterative Implicit Networks (DIIN), which efficiently promotes global signal flow with linear complexity. Key components such as the Dictionary Cross-Attention Block and the PositionAware Embedding Block enhance contextual awareness and spatial dependencies, respectively. Experiments on benchmark datasets demonstrate that DIIN achieves state-of-theart performance in arbitrary-scale super-resolution tasks, with significant improvements in detail restoration and contextual modeling. Ablation studies confirm the effectiveness of each module in the proposed framework. Looking ahead, DIIN could be extended to tasks like 3D reconstruction or video super-resolution, while further optimization may enhance its practicality in real-time applications. Our work establishes a promising direction for advancing implicit neural representation techniques.

Acknowledgments

This work is supported in part by the National Natural Science Foundation of China, under Grant (62302309, 62471310), Shenzhen Science and Technology Program JCYJ20220818101014030, and OPPO Research Fund.