Trajectory-Consistent Denoising Diffusion Codebook Models for Zero-Shot High-Fidelity Image Compression at Ultra-Low Bitrates

1 Introduction

The advent of Denoising Diffusion Probabilistic Models (DDPMs) [Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2020] has revolutionized generative AI, delivering unprecedented quality in image generation [Liu et al., 2023; Cai et al., 2025], restoration [Raphaeli et al., 2025; Zamir et al., 2022], and editing [Kawar et al., 2023]. Recently, these models have been successfully adapted for image compression. By leveraging extensive training [Yang and Mandt, 2023; Li et al., 2023], fine-tuning [Careil et al., 2024; Xue et al., 2025], or structural modifications [Zhang et al., 2025; Park et al., 2025], recent diffusion-based methods have demonstrated significant advantages over other models such as manually crafted codecs [Bellard, 2018], Generative Adversarial Networks (GANs) [Iwai et al., 2024; Muckley et al., 2023; Mentzer et al., 2020] and Variational Autoencoders (VAEs) [Theis et al., 2017]. However, these approaches lack zero-shot capability on unseen data distributions and require training separate models for specific compression rates. In contrast, the Denoising Diffusion Codebook Model (DDCM) [Ohayon et al., 2025] introduces a novel paradigm enabling flexible bitrate adjustment without additional training, achieving SOTA performance among zero-shot based methods [Xue et al., 2025; Elata et al., 2024]. Specifically, DDCM compresses a target image by sequentially selecting noise vectors that best match the target image from a fixed Gaussian codebook during the generative diffusion process. By recording the indices of these selected vectors, DDCM effectively converts the generative trajectory into a transmittable bitstream, the length of which scales linearly and log linearly with the number of inference steps and codebook size respectively.

However, as illustrated in Fig 1(a), standard DDCM and its variants (e.g., Turbo-DDCM) select noise vectors by minimizing the residual error relative to the target image x0. Given the substantial distance between noisy intermediate state and the clean target, this target-guided noise selection strategy provides only coarse guidance, inevitably leading to trajectory drift-a phenomenon where the generative path gradually diverges from the target. Consequently, DDCM requires massive codebooks to span the enlarged search space and extensive inference steps to correct accumulated deviations, resulting in significant compression inefficiency. We provide a detailed quantitative analysis of this target-guided noise selection in Sec 5.5.

To address these challenges, we propose the TrajectoryConsistent Diffusion Codebook Model (TC-DDCM). Our core insight is to shift the paradigm from approximating the distant target x0 to tracking a pre-computed inversion trajectory Tinv. As the inversion trajectory fits in the underlying generative distribution of diffusion models, TC-DDCM can naturally align the generative process with the reference trajectory through noise selection. Specifically, our method operates in two distinct stages: Inversion Trajectory Construction and Trajectory-Guided encoding.

During the inversion trajectory construction stage, we introduce two variants for this construction: TC-DDCM (DDIM), which utilizes standard deterministic DDIM inversion [Ho et al., 2020], and TC-DDCM (DDPM), which employs CycleDiffusion [Wu and De la Torre, 2023] as DDPMbased inversion. We leave the incorporation of other effective inversion methods for future work. Once the trajectory is established, our trajectory-guided noise selection mechanism minimizes the deviation between the predicted state and the proximal reference point at each step along the pre-computed inversion trajectory. As illustrated in Fig 1(b), this paradigm shift directly leads to two critical advantages:

• Small Codebook Efficiency via Precise Guidance: The minimal distance between the current prediction and the proximal reference point on the trajectory provides precise guidance (as shown in Tab 1) that drastically reduces the feasible search space for the noise vector, leading to significantly smaller codebooks.
• Few-Step Inference via Inversion Alignment: The inference steps T in TC-DDCM are strictly aligned with the chosen inversion method. This allows our framework to capitalize on efficient inversion techniques that achieve high-fidelity reconstruction performance in very few steps (e.g., T ≈ 20-30). Consequently, TC-DDCM reduces the number of required denoising steps substantially.

Benefiting from these two advantages, TC-DDCM achieves a substantial reduction in Bits Per Pixel (BPP) while maintaining reconstruction performance comparable to the original DDCM. It is worth noting that the theoretical upper bound of TC-DDCM is determined by the degree of alignment with the inversion trajectory, rather than the reconstruction quality of the inversion itself. As long as the trajectory terminates at the target image and adheres to the generative distribution of DPMs, precise alignment with the trajectory leads to highfidelity reconstruction.

Extensive experiments demonstrate that TC-DDCM can track inversion trajectories closely even with reduced codebook sizes and fewer inference steps. It surpasses other zeroshot methods in rate-distortion-perception performance especially at ultra-low bitrates (e.g. < 0.05 bpp) and achieves over 50× faster coding latencies, making zero-shot diffusionbased image compression practical.

2 RelatedWork

2.1 Diffusion Based Image Compression

Recently, diffusion-based generative compression has demonstrated impressive reconstruction quality, significantly outperforming traditional codecs in terms of perceptual fidelity. A predominant paradigm employs uniform coding strategies to quantize latent representations of Variational Autoencoders (VAEs) [Cemgil et al., 2020]. Building on this, several methods [K¨orber et al., 2024; Pan et al., 2022; Lei et al., 2023] leverage existing text-to-image diffusion models as powerful priors, conditioning the generation on both vector-quantized latents and textual descriptions. Furthermore, frameworks such as DiffEIC [Li et al., 2024] and RDEIC [Li et al., 2025] synergize compressive VAEs with pre-trained diffusion backbones to reconstruct photorealistic images, particularly at low bitrates. However, the substantial computational cost of iterative denoising remains a critical bottleneck. To address this, recent works have attempted to achieve one-step generation [Guo et al., 2025; Zhang et al., 2025; Park et al., 2025]. For instance, StableCodes [Zhang et al., 2025] modifies the encoder architecture, while DiffO [Park et al., 2025] employs a training strategy that models the residual between quantized and original latents.

2.2 Zero-shot Image Compression

Alternatively, zero-shot methods leverage pretrained diffusion models without training or fine-tuning. DiffC [Vonderfecht and Liu, 2025] utilizes Reverse Channel Coding (RCC) to achieve high-fidelity reconstruction. DDCM [Ohayon et al., 2025] compresses images by greedily selecting noise vectors during the generation process to minimize the denoising error at each step. Turbo-DDCM [Vaisman et al., 2025] further extends this by solving a closed-form sparse least squares problem to combine multiple noise vectors. Despite these advancements, all zero-shot methods are strictly limited by either requiring hundreds of denoising steps or excessive codebook sizes.

2.3 Image Inversion

Inversion approaches [Ho et al., 2020] map an input image into one or multiple latent noise vectors that reconstruct the original image during the generative process and hold significant potential for image editing [Wallace et al., 2023; Hu et al., 2022] and super resolution [Yue et al., 2025] tasks. Most existing inversion algorithms rely on DDIM sampling to establish a deterministic trajectory that maps an initial noise to the generated image. However, these methods inevitably introduce approximation errors at each step. These errors accumulate along the sampling chain, leading to substantial deviations from the target, particularly when Classifier-Free Guidance (CFG) is employed. Several approaches have attempted to mitigate the artifacts induced by CFG by optimizaing prompt embeddings [Dong et al., 2023; Mokady et al., 2023] or incorporating affine coupling layers [Wallace et al., 2023].

A distinct line of research focuses on DDPM-based inversion algorithms, which achieves perfect reconstruction by constructing a sequence of noise maps along the DDPM sampling trajectory. CycleDiffusion [Wu and De la Torre, 2023] constructs the auxiliary images that approximate the underlying generative process to extract noise sequences. The Edit Friendly DDPM inversion [Huberman-Spiegelglas et al., 2024] modifies the construction of auxiliary images, thereby yielding a noise space that remains statistically independent and is amenable to editing.

3 Background

3.1 Denoising DiffusionModels

Denoising Diffusion Models (DDMs) are trained with a denoising objective to predict the noise that progressively corrupts a clean image x0 into white Gaussian noise.

where ¯αt = ∏ts=1 αs, αt = √1-βt, and {βt}Tt=1 is a monotonically increasing noising schedule and ϵ ∼ N(0, I).

The reverse process of DDMs starts from a random noise vector xT ∼ N(0, I) and iteratively denoises it using the noise ϵθ(xt, t) predicted by the neural network:

Here ˆx0|t = ( xt-√1-¯αtϵθ(xt,t) √ ¯αt ) is the predicted x0 given xt 1 . The variance σt is determined by the hyperparameter η:

where η ∈ [0, 1]. η = 1 corresponds to the original DDPM sampling process, and η = 0 corresponds to the deterministic DDIM scheme.

3.2 Denoising Diffusion CodebookModels

Denoising Diffusion Codebook Models (DDCM) [Ohayon et al., 2025] build upon the DDPM framework but modify the reverse process by replacing stochastic sampling with discrete selection. Specifically, instead of sampling zt from a continuous Gaussian distribution, DDCM selects a vector from a fixed codebook Ct = [z(1) t , . . . , z(K) t ] containing K i.i.d. Gaussian vectors:

1 This is also referred to as the minimum mean-squared-error (MMSE) estimator.

where kt ∈ {1, . . . ,K}. The process begins with a fixed starting point xT = CT+1(kT+1) and ends at t = 1. To ensure reconstruction fidelity, DDCM employs a greedy selection criterion that pulls the predicted state ˆx0|t close to the target. It computes the residual error between ˆx0|t and the ground-truth x0, selecting the entry that maximizes the inner product:

The above equivalence is justified by the JohnsonLindenstrauss Lemma [Dasgupta and Gupta, 2003] in highdimensional spaces. By recording the indices (kt)Tt=2 , DDCM converts the generative trajectory into a bitstream. During decoding, these indices retrieve the noise vectors to reconstruct the image via Eq 5. To further reduce quantization error, DDCM employs Matching Pursuit (MP) [Mallat and Zhang, 1993] to approximate the noise as a linear combination of M vectors. The resulting bitrate (BPP) is:

3.3 Image Inversion

DDIM Inversion. DDIM inversion (η = 0) exploits the ODE consistency that can reverse the deterministic generation process from the initial noise xT . By rearranging the sampling update, xt can be derived from xt-1 as:

A critical limitation highlighted in Eq 8 is the dependency on ϵθ(xt, t), which is unknown at step t-1. DDIM inversion approximates this using ϵθ(xt-1, t-1). This local linearization assumption introduces approximation errors that accumulate across steps, leading to reconstruction degradation,especially in low-step regimes.

DDPM Inversion. DDPM inversion aims to find a specific noise sequence instead of a single noise map that perfectly reconstructs the image within the DDPM sampling process (η = 1). We adopt CycleDiffusion [Wu and De la Torre, 2023] as our baseline for DDPM inversion. This method constructs an auxiliary trajectory Tref = {xt}Tt=0 that adheres to the posterior distribution q(x1:T |x0). Concretely, for each step t = T, . . . , 1, they isolate the reconstruction noise ϵrec t derived from the current xt and the ground-truth x0 (Eq 1), and substitute this ϵrec t for ϵθ(xt, t) in Eq 2 to obtain xt-1. Through this iterative cycle, a DDPM inversion trajectory Tref = {xt}Tt=0 is explicitly constructed. The detailed procedure is summarized in [Wu and De la Torre, 2023].

4 Method

We propose the Trajectory-Consistent Diffusion Codebook Model (TC-DDCM), a unified framework that eliminates trajectory drift by strictly guiding the generative process along a pre-computed inversion trajectory. As illustrated in Figure 2, our approach comprises two phases: Reference Trajectory Construction and Trajectory-Guided Encoding.

4.1 Phase 1: Reference Trajectory Construction

In this phase, TC-DDCM constructs a deterministic inversion trajectory Tref that serves as the reference path for the entire encoding process. This trajectory fits in the distribution of generative process in DPMs so that it is natural for TC-DDCM to align the generative process with the reference trajectory. The pre-computed trajectory Tref provides a deterministic starting point x∗T and a sequence of intermediate proximal reference points {xinv t }1t=T-1 ending at the target image x0. We implement this phase using two distinct protocols:

• TC-DDCM (DDIM): Following DDIM [Song et al., 2021], we obtain a deterministic inversion trajectory where the starting point x∗T varies for each image.
• TC-DDCM (DDPM): We employ CycleDiffusion [Wu and De la Torre, 2023] as our DDPM-based inversion strategy. Crucially, to ensure determinism during construction, we initialize the inversion from a fixed initial noise state x∗T and predefine the random seed at each step, ensuring the inversion trajectory unique yet repro-

ducible for each image. Although DDPM inversion theoretically guarantees perfect reconstruction for arbitrary steps, the practical performance is limited by numerical precision. Empirically, we observe that it achieves nearlossless reconstruction at approximately T = 10 steps.

4.2 Phase 2: Trajectory-Guided Encoding Initialization with Inverted Start

In contrast to the fixed Gaussian initialization used in DDCM, TC-DDCM initializes the generative process directly from the inverted starting point of the reference trajectory, x∗T ∈ Tref . To transmit this starting point, we quantize it using the initial codebook CT+1:

Trajectory-Guided Noise Selection

The core innovation of TC-DDCM lies in its selection criterion. Recall that standard DDCM (Eq. 6) selects noise to minimize alignment with the distant target x0. As illustrated in Fig 1, this creates a substantial optimization gap, providing only coarse guidance that exacerbates trajectory drift.

Instead, TC-DDCM employs a trajectory guided noise selection mechanism. At each timestep t, given the current state ˆxt, the goal is to select a codebook vector Ct(k) such that the predicted next state ˆxt-1 lands as close as possible to the proximal reference state xinv t-1 ∈ Tref .

Mathematically, we substitute the random noise zt in the sampler with the codebook entry and solve:

where η is a hyperparameter controlling the variance schedule σt. Concretely, for the DDPM variant, we strictly synchronize η with the value used in the inversion trajectory, while for the DDIM variant, we set η ∈ (0, 1] to modulate the stochasticity of the generation.

By minimizing the distance to the proximal reference point on this inversion trajectory, our optimization objective provides precise guidance. Even when quantization errors occur at step t, this trajectory-guided strategy actively pulls the latent state back towards the inversion path, ensuring the generative process remains strictly consistent with the trajectory that eventually converges at the target.

4.3 Coding Latency and Bitrate Analysis

The coding latency comprises two stages: encoding and decoding. Specifically, the encoding phase incurs a cost of 2T function evaluations (NFE), accounting for both the reference trajectory construction (T steps) and the trajectory-consistent encoding (T steps). The decoding phase requires T steps, identical to DDCM. Consequently, the total NFE for the coding process amounts to Ttotal = 2T + T = 3T (compared to 2T in DDCM).

Although our method requires encoding the inverted starting point x∗T , this adds only a single codebook index. Compared to DDCM, this overhead is negligible. The total BPP is calculated as follows:

5 Experiments

5.1 Implementation Details

We conduct comprehensive evaluations on three standard benchmarks: Kodak24 [Franzen, 1999], CLIC2020 [Toderici et al., 2020], and DIV2K [Agustsson and Timofte, 2017]. For preprocessing, we apply a center crop of 512 × 512 for Kodak24 and CLIC2020, and 768 × 768 for DIV2K. We employ FID [Bi´nkowski et al., 2018] and LPIPS [Zhang et al., 2018] to evaluate perceptual quality, and PSNR for reconstruction fidelity. Following Mentzer et al. [Mentzer et al., 2020], we compute FID on extracted image patches. Both encoding and decoding times are measured on a single NVIDIA A100 GPU.

We benchmark our approach against existing zero-shot diffusion-based methods, including PSC [Elata et al., 2024], DDCM [Ohayon et al., 2025], DiffC [Theis et al., 2022], and Turbo-DDCM [Vaisman et al., 2025]. Additionally, we compare with non-neural, fine-tuning-based, and trainingbased methods, specifically including BPG [Bellard, 2018], HiFiC [Mentzer et al., 2020], PerCo (SD) [K¨orber et al., 2024; Careil et al., 2024], ILLM [Muckley et al., 2023], two

Algorithm 1. Trajectory-Consistent DDCM (TC-DDCM)

Input: Image x0, Codebooks {Ct}

Output: BitstreamB

• 1: Tref ← Inversion(x0) {Construct reference trajectory}
• 2: x∗T ← Tref [T]
• 3: cstart ← argminc∈CT+1 ∥c -x∗T ∥2
• 4: B ← {cstart}; ˆxT ← cstart
• 5: for t = T down to 1 do
• 6: kt ← argmink ∥µt(ˆxt) +σtCt(k) -Tref [t -1]∥2
• 7: ˆxt-1 ←µt(ˆxt) +σtCt(kt)
• 8: Append kt to B
• 9: end for
• 10: return B

configurations of CRDR [Iwai et al., 2024], and the one-step diffusion method StableCodec [Zhang et al., 2025].

All zero-shot methods utilize the same pre-trained diffusion backbones: the SD 2.1 Base [Rombach et al., 2022] model for 512 × 512 images and the SD 2.1 model for 768 × 768 images. The bitrate of TC-DDCM is modulated by varying the inference timesteps T, codebook size K, hyperparameters of matching pursuits M and C. Please refer to the supplementary material for further details.

5.2 Rate-Distortion-Perception Performance

As illustrated in Fig 3, our TC-DDCM significantly outperforms all competing methods across PSNR and FID metrics in the ultra-low bitrate regime. In particular, when compared to DDCM and its variant, Turbo-DDCM, our approach demonstrates superiority and stability across all the metrics with well-preserved fidelity, and reaches extreme bitrates as low as 0.002 bpp. As the bitrate increases, TCDDCM yields highly competitive results, surpassing the previous state-of-the-art zero-shot method, DiffC. It is worth noting that the DDPM-based variant consistently yields superior results compared to the DDIM-based variant across different bitrates, which may be due to inherent approximation errors in DDIM inversion.

5.3 Coding Latency

We compare encoding and decoding latencies against zeroshot baselines and one-step baseline StableCodec as shown in Fig 3. Our method achieves remarkable acceleration, performing approximately 50× faster than the standard DDCM, and consistently outperforming DiffC in both encoding and decoding speeds across all bitrate settings. Compared to Turbo-DDCM, our method exhibits a slight higher encoding latency due to the computational overhead of constructing the inversion trajectory, yet maintaining comparable decoding speeds. Crucially, this additional encoding investment yields significant gains in Rate-Distortion-Perception performance. Notably, these two approaches are orthogonal: Turbo-DDCM focuses on optimizing the matching pursuit parameters (the number of atoms M and coefficient bits C), whereas TCDDCM primarily optimizes the inference schedule (T) and codebook size (K). Since these optimization strategies are parallel, combining them to achieve even higher efficiency remains a promising direction for future work.

5.4 Qualitative Comparison

Fig 4 presents the qualitative comparison against state-of-theart zero-shot methods. Notably, TC-DDCM demonstrates superior capability in generating visually aligned details and realistic textures, even at ultra-low bitrates. As showcased in the first and second rows, our method faithfully reconstructs intricate structures, such as the murals on the wall and the model's teeth. In contrast, all other methods fail to maintain high realism and fidelity under such severe bitrate limitations. Specifically, DDCM yields blurry reconstructions with lost details, while DiffC and Turbo-DDCM suffer from distinct artifacts and structural deformations in complex regions.

5.5 Analysis on Noise SelectionMechanisms

To empirically validate the effectiveness of TC-DDCM, we conduct a comparative analysis between the standard targetguided strategy (DDCM) and our proposed trajectory-guided strategy. We focus on quantifying two critical aspects highlighted in our introduction: the magnitude of the optimization objective and the alignment precision of guidance.

First, we quantify the optimization distance by measuring the L2 and cosine similarity of the objective: ∥ˆx0|t - x0∥2 for DDCM versus ∥µt(ˆxt) - xinv t-1 ∥2 for TC-DDCM. The results are averaged across all samples on the DIV2K dataset over 50 sampling steps. As reported in Tab 1, the optimization distance of DDCM is consistently larger than that of TCDDCM. This validates our core insight: the distance between the current prediction and the proximal reference is significant smaller compared to the distant target image x0, as illustrated in Fig 1.

To further evaluate the efficiency of noise selection strategies, we introduce the Error Reduction Ratio (ERR), defined as:

This metric quantifies the compensatory capacity of the selected codebook vector. It measures the proportion of the optimization discrepancy that is effectively bridged by the selected noise, serving as a direct indicator of how precisely the generative process is consistent with the target image or inversion trajectory.

As shown in Tab 1, the improvement in ERR, stemming from larger codebooks and extended inference steps, validates our hypothesis that the former expands the search space while the latter corrects cumulative errors, ultimately yielding more precise guidance. However, standard DDCM continues to exhibit persistently lower ERR scores under all settings. This indicates that even with an expanded search space and extended inference steps, the codebook vectors cannot compensate for the vast distance to the remote x0. The limitation validates that the target-guided strategy is inherently capable of providing only coarse guidance. Conversely, TC-DDCM achieves significantly higher ERR scores. Since the inversion trajectory naturally fits in the underlying generative distribution of DPMs, our method can naturally align the generative process with the reference points through noise selection. This tight alignment proves that our trajectory-guided strategy provides such precise guidance that it alleviates the need for massive codebooks or excessive inference steps, enabling high-fidelity reconstruction even at ultra-low bitrates.

Ablation Study of Hyperparameter η

In diffusion-based generative processes, the hyperparameter η ∈ [0, 1] controls the degree of stochasticity injected at each sampling step by modulating the variance schedule σt(η). η = 0 corresponds to a deterministic DDIM process, while η = 1 corresponds to a fully stochastic DDPM process. To investigate its impact on our framework, we evaluate the PSNR performance of both TC-DDCM variants (DDIM and DDPM) across different η values, as shown in Fig 5. As depicted in Fig 5, the DDPM variant consistently outperforms the DDIM variant across all η values, attributed to the higher precision of its underlying reference trajectory. Specifically, when η is set too high (approaching 1), the injected stochasticity becomes excessive, where random noise dominates the signal. This causes the generative trajectory to deviate significantly from the reference path. Conversely, when η is set too low (approaching 0), the insufficient stochastic support fails to actively pull the generative process back towards the inversion trajectory, resulting in poor alignment.

6 Conclusion

In this paper, we introduce the Trajectory-Consistent Diffusion Codebook Model (TC-DDCM), a unified zero-shot compression framework designed to overcome the limitations of trajectory drift in existing DDCM methods. TC-DDCM reformulates the encoding paradigm from approximating a distant target to explicitly tracking a pre-computed inversion trajectory. Extensive experiments confirm that minimizing the deviation from the reference trajectory effectively shrinks the codebook size and minimizes the inference steps, achieving SOTA performance at ultra-low bitrates.

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Grant No. 62276245).