ADC-GS: Anchor-Driven Deformable and Compressed Gaussian Splatting for Dynamic Scene Reconstruction

1 Introduction

Dynamic scene reconstruction from multi-view input videos has received significant attention due to its wide applications. Beyond methods based on 3D Gaussian Splatting (3DGS) [Kerbl et al., 2023], 4D Gaussian Splatting (4DGS) has demonstrated substantial advances for dynamic scene reconstruction due to its impressive visual quality with ultra-fast training speed compared to neural radiance fields (NeRFs) based methods [Mildenhall et al., 2021].

Recent research on 4DGS approaches has primarily focused on two categories. The first category employs 4D Gaussians to approximate the 4D volumes of scenes [Li et al., 2024; Yang et al., 2024b; Yan et al., 2024] through temporal opacity and polynomial functions for each Gaussian. Another category focuses on temporally deforming a canonical space to target frames [Bae et al., 2025; Wu et al., 2024; Yang et al., 2024c; Lu et al., 2024a], utilizing a multilayer perceptron (MLP) with latent embeddings to predict changes in 3D Gaussian attributes over time.

Directly optimizing 4D Gaussians provides higher rendering speeds but requires a large number of Gaussians to model the entire sequence, resulting in both significant training time and substantial storage requirements. For example, [Yan et al., 2024] often leads to over 9 hours of training time and 2GB of data for a 10-second video. Although deformationbased methods address the above issues and reduce training time, they still require a large number of Gaussians to achieve high-quality rendering. Consequently, substantial storage and bandwidth requirements emphasize the need for more compact 4DGS representations and advanced compression techniques, which is the primary focus of this work.

Some recent methods have developed effective technologies to reduce 4DGS storage. For example, 4DGaussian [Wu et al., 2024] maps the 4D space onto six orthogonal planes as latent embeddings, effectively mitigating per-frame training redundancy and lowering storage costs. Since spherical harmonic coefficients dominate storage consumption, MEGA [Zhang et al., 2024] decomposes them into a per-Gaussian DC color component and a lightweight AC color predictor, eliminating redundant coefficients to reduce storage requirements. However, these methods compress each Gaussian independently in the canonical space, neglecting the strong similarity between local attributes (excluding opacity), as shown in Figure 2. Furthermore, existing methods rely on per-Gaussian deformation for dynamic scene modeling, overlooking the consistent deformations among neighboring Gaussians since local scenes can often be approximated as rigid motion. These limitations hinder both the compactness of dynamic scene representations and rendering efficiency.

Inspired by deformation-based dynamic scene reconstruction, this paper proposes Anchor-driven Deformable and Compressed Gaussian Splatting (ADC-GS), a novel framework for efficient and compact dynamic scene representation. Specifically, we organize Gaussian primitives into a sparse set of anchors, facilitating streamlined representation and processing. These anchors, with K neural Gaussian primitives predicted from each by shared MLPs, collectively construct the canonical space. To reconstruct dynamic scenes at any frame, we employ an anchor-based coarse-to-fine deformation strategy. In the coarse stage, the position, covariance, and color attributes of the anchors are deformed from the canonical space. Subsequently, the K Gaussian primitives are automatically updated based on the deformation of the associated anchor, notably reducing deformation redundancy. As the coarse stage focuses on capturing global deformation, a fine stage is introduced to dynamically refine the appearance of each primitive to recover fine-grained details. To further improve the compactness of the anchors, we implement a rate-distortion optimization scheme that incorporates a multi-dimension entropy model for accurate bitrate estimation and efficient compression. Besides, we propose a novel anchor refinement method that leverages the accumulated gradients of each Gaussian primitive's temporal significance to guide anchor growing, while using accumulated opacity to determine anchor pruning. This adaptive strategy robustly addresses both under-reconstruction and overreconstruction issues in dynamic scenes, ensuring a balanced and efficient anchor representation. As shown in Figure 1, ADC-GS achieves state-of-the-art (SOTA) storage size and rendering speed while preserving excellent rendering quality compared to prior works [Bae et al., 2025; Park et al., 2021b; Lu et al., 2024a; Wu et al., 2024; Kerbl et al., 2023]. Our contributions can be summarized as follows:

• We propose anchor-driven deformable and compressed Gaussian Splatting (ADC-GS) for dynamic scene reconstruction. By leveraging compact anchors to efficiently
• model 4D scenes, our approach achieves an extraordinary storage reduction of up to 200× over existing 4DGS methods.
• To accelerate rendering while preserving high reconstruction quality, we introduce a hierarchical coarseto-fine deformation method. We also propose an adaptive anchor refinement strategy to address underreconstruction and over-reconstruction issues.
• We develop a multi-dimension entropy model to enhance compactness through joint minimization of rendering distortion and bitrate consumption.

2 RelatedWork

2.1 Considering Spatial Relationships of Gaussian Primitives

In 3D space, Scaffold-GS [Lu et al., 2024b] and CompGS [Liu et al., 2024b] reconstruct scenes by synthesizing Gaussian primitives from anchors, exploiting the similarity of properties among neighboring Gaussians. HAC [Chen et al., 2025] utilizes a structured hash grid to take advantage of inherent consistencies among unorganized 3D Gaussians. In 4D scenes, some approaches [Yang et al., 2024c; Zhao et al., 2024; Chen et al., 2024a] introduce sparse control points combined with an MLP to model scene motion, based on the insight that motion can be effectively represented by a sparse set of basis. Inspired by the compact structures used in both 3D and 4D scenes, we design an anchor-driven deformable and compressed Gaussian splatting method to effectively eliminate intra-redundancy in both attributes and deformation among local Gaussian primitives.

2.2 Deforming 3DCanonical Space

Early research on deforming 3D canonical space to a target frame in both NeRF and 4DGS has explored various approaches. Nerfies [Mildenhall et al., 2021] and HyperNeRF [Park et al., 2021b] use per-frame trainable deformation rather than time-based conditions. D-NeRF [Pumarola et al., 2021] reconstructs dynamic scenes by deforming ray samples over time, utilizing a deformation network that takes 3D coordinates and timestamps as inputs. E-D3DGS [Bae et al., 2025] defines the deformation as a function of Gaussian and temporal embeddings, decomposing the deformation into coarse and fine stages to model slow and fast movements, respectively. Other methods [Yang et al., 2024c; Wu et al., 2024; Jiawei et al., 2024] learn implicit position mappings to deform primitives using latent embeddings, such as positional embeddings. However, these methods overlook the similarity in deformation among neighboring Gaussian primitives, resulting in suboptimal rendering speeds. To address this issue, we propose an anchor-based coarse-to-fine deformation method that accounts for deformation redundancy, achieving exceptional rendering speed while maintaining high reconstruction quality.

2.3 3DGS Compression

Recent studies on 3DGS compression can be categorized into processing-based and context-based methods. Processingbased methods include significance pruning [Fan et al.,

2023], scalar quantization [Niedermayr et al., 2024; Ma et al., 2024], codebooks [Lee et al., 2024]. Some point cloud compression techniques can also be adapted for Gaussian attributes due to their similar structure, such as graph-based signal transformation [Yang et al., 2024a], region-adaptive hierarchical transform [Huang et al., 2024], and octree encoding [Liu et al., 2024a; Fan et al., 2022]. Context-based methods leverage various contextual information to remarkably compress Gaussian attributes, including hash-grid features [Chen et al., 2025], hyperpriors [Liu et al., 2024b], and hybrid Gaussian features [Chen et al., 2024b]. Building on these advancements, we propose a multi-dimension entropy model for bitrate estimation and compression in this work, further eliminating redundancies within the anchors. By integrating the estimated bitrate into a rate-distortion optimization framework, our method effectively balances the trade-off between rendering quality and bitrate costs.

3 Method

3.1 Overview

As shown in Figure 3, ADC-GS begins with the initialization of a canonical space at t = 0, characterized by a sparse set of anchors with associated Gaussian primitives. Each anchor is equipped with latent features and explicit Gaussian attributes, including position, covariance and color. The explicit Gaussian attributes serve as the foundation for the associated primitives, while the latent features are responsible for predicting the residuals and deformation of each primitive. To reconstruct any subsequent frame t = T , we devise a coarse-to-fine strategy based on anchors. In the coarse stage, each anchor is deformed from the canonical space using lightweight multihead MLPs, with the associated primitives subsequently updated based on the coarse deformation. In the fine stage, the opacity and color of each primitive are further refined using the deformed position of the anchors and current timestamp information. By combining both coarse and fine deformations, all deformed attributes of Gaussian primitives are obtained to render the current frame using volume splatting [Kerbl et al., 2023]. In the subsequent rate-distortion optimization, rendering distortion and estimated bitrates from the multi-dimension entropy model are jointly minimized to balance rendering quality and storage requirements. Moreover, the temporal significance-based strategy is employed to prune and grow anchors for mitigating both under-reconstruction and over-reconstruction issues in dynamic scenes.

3.2 Canonical space initialization

Unlike prior deformation-based approaches [Yang et al., 2024c; Lu et al., 2024a] that directly use Gaussian primitives as the canonical space, we initialize our compact representation by employing a downsampled point cloud obtained via COLMAP [Snavely et al., 2006] to serve as anchors. These anchors are the foundation for local regions that improve overall efficiency by minimizing unnecessary duplication of information across the dynamic scene.

In ADC-GS, each anchor v is associated with K neural Gaussian primitives {g1, ..., gK}. The anchor consists of latent features Al = {fv ∈ RNv , fg ∈ RK∗Ng } and explicit Gaussian attributes Ae = {Xv ∈ R3 ,Σv ∈ R6 ,Cv ∈ R3 }. Among the latent features, the reference feature fv encapsulates the common characteristics between K associated primitives, while the residual feature fg captures the variations specific to each primitive. The explicit Gaussian attributes serve as the foundation for the K associated primitives. To generate associated primitives from anchors, we predict their explicit attributes in residual forms, as:

where ∆Xk ,∆Σk and ∆Ck are residuals of position, covariance, and color. Fθ represents the prediction network, which is modeled using a residual neural network. Since opacity Ok exhibits no apparent spatial correlation as shown in Figure 2, it is directly predicted without residuals.

Anchors and their associated primitives collectively define the canonical space that captures the global structure of the entire scene. Subsequently, the reconstruction of the scene in any frame t = T is performed by deforming this canonical space through a coarse-to-fine pipeline.

3.3 Coarse-to-Fine Deformation

Previous methods [Wu et al., 2024; Lu et al., 2024a] deform the geometric properties of each Gaussian primitive individually from the canonical space to the target frame T individually. However, this per-Gaussian deformation suffers from low efficiency due to the similar deformation among adjacent primitives, leading to slow rendering speeds. To mitigate the limitation, we propose an anchor-driven coarse-to-fine deformation strategy. In the coarse stage, we deform the anchor explicit attributes Ae by utilizing the concatenation of reference feature fv and time embedding ft, formulated as:

where Fω denotes the anchor deformation network, implemented as a tiny MLP. ∆Xv,t,∆Σv,t,∆Cv,t denote the position, covariance and color deformation of the anchors. ft encodes temporal information for different frames,

where Z ∈ R256 is a learnable weight for interpolation, t denotes the temporal index and Fs represents the grid sampling function to encode temporal variations. By deforming a small number of anchors, the associated K Gaussian primitives are automatically updated, substantially diminishing deformation redundancy to enhance rendering efficiency.

While the coarse stage captures the global changes of the associated Gaussian primitives, it lacks the capability to address finer details. Considering that opacity exhibits minimal local similarity and color significantly influences scene reconstruction [Bae et al., 2025], the fine stage dynamically refines these two attributes for each primitive, enhancing overall reconstruction quality. This refinement integrates the position embedding fp [Vaswani, 2017] of the deformed anchors and the time embedding ft,

where Fϖ represents the fine-stage deformation network similar to Fω and ∆Ok,t,∆Ck,t denote the per-Gaussian refinements of opacity and color, respectively. Therefore, associated Gaussian primitives are derived as a combination of coarse and fine-stage deformations:

where Xk,t, Σk,t, Ok,t and Ck,t are the position, covariance, opacity and color attributes used for volume splatting in the target frame T, respectively.

3.4 Rate-Distortion Optimization

The rate-distortion optimization scheme is designed to achieve a more compact representation by jointly minimizing both bitrate consumption and rendering distortion. While the anchor locations Xv are compressed using G-PCC [Liu et al., 2024a], the remaining four anchor features are modeled for bitrate estimation through a Multi-dimension Entropy Model (MEM), as demonstrated in Figure 4. Specifically, scalar quantization is first applied to the feature fi, where fi ∈ {fv, fg,Σv,Cv}. However, conventional rounding essentially performs quantization with a fixed step size, which is less flexible for features with different scales. To address this issue, we design an adaptive quantization module that better accommodates the diverse feature value scales as follows,

where Fq indicates MLP-based model to adjust the predefined quantization step size Qi. Note that Qi varies for fv, fg,Σv and Cv. Uniform noise U(, ) is injected during training to stimulate quantization loss in the testing stage as proposed in [Ball´e et al., 2018].

Subsequently, the probability distribution of fi is estimated to calculate the corresponding bitrate through the MLP-based context model Efi . The probability distribution p( ˆ fv) of reference feature is first parametrically formulated as a Gaussian distribution N(µfv , σfv ), where the parameters {µfv , σfv } are predicted based on hyperpriors [Ball´e et al., 2018] extracted from fv,

where ηfv denotes the hyperpriors. Furthermore, the decoded reference feature ˆ fv is used as contexts to model the probability distributions of covariance ˆΣv, color ˆ Cv and residual feature ˆ fg. The probability distribution of ˆΣv and ˆ Cv is modeled similarly to p( ˆ fv). The residual feature fg constitutes the largest portion of total bitstream, which is divided into M chunks. The decoded reference features ˆ fv and chunks are organized into a multi-dimensional context to guide the encoding of remaining chunks. This strategy effectively reduces channel-wise redundancy and enhances compression efficiency, formulated as:

where ch means the index of current encoding chunks and ⊕ denotes the channel-wise concatenation, effectively integrating the decoded context for improved compression efficiency.

Consequently, the total estimated bitrate consumption R of anchors is calculated as:

Therefore, the rate-distortion optimization process is formulated as:

where L1 and Lssim represent the L1 loss and SSIM loss used in [Kerbl et al., 2023]. λssim donates weighting coefficients for the SSIM loss and λe is the Lagrange multiplier to control the trade-off between rate and distortion.

3.5 Anchors Refinement with Temporal Significance

Anchors initialized from the sparse point cloud are often suboptimal, leading to under-reconstruction or overreconstruction in certain areas. The existing strategies in 3DGS [Kerbl et al., 2023; Lu et al., 2024b] focus solely on static scenes and neglect the temporal significance of Gaussian primitives in dynamic frames, making them unsuitable for 4D scene reconstruction. To address this issue, we propose a temporal significance-aware anchor refinement method, including anchor growing and pruning. The importance of Gaussian primitives across frames is calculated by computing a weighted accumulation of their positional gradients,

where ∇k,t is the 2D position gradient of a Gaussian primitive and ∇k represents the accumulated gradients over N frames. Ψ(k, t) [Huang et al., 2024] is the rendering weight of the α-blending in the target frame T. Gaussian primitives with ∇k values exceeding a predefined threshold are added as new anchors. These new anchors inherit latent features from their associated anchors while preserving their own Gaussian attributes. Conversely, anchors associated with Gaussian primitives whose opacity falls below a predefined threshold are pruned to avoid over-reconstruction.

4 Experiments

In this section, we present the implementation details of ADC-GS, followed by quantitative comparisons with SOTA approaches on two commonly used datasets. Finally, we provide in-depth analysis and ablation studies to evaluate the performance and effectiveness of the proposed ADC-GS.

4.1 Experimental settings

Our proposed ADC-GS framework is implemented based on Pytorch and trained on a single NVIDIA RTX 3090 GPU. For hyperparameter settings, the dimension of the reference feature Nv is set to 32, while the residual feature Ng is 16. Each anchor is associated with K = 10 neural Gaussian primitives. The Lagrange multiplier λe is varied from e - 2 to e - 4 for different bitrates, and λssim is fixed at 0.2. Quantization step sizes Qi are set to 0.1, 0.1, 0.01 and 0.01 for fv, fg, Σv and Cv, respectively. Meanwhile, fg is divided into M = 4 chunks. The prediction network Fθ and entropy estimation network Efi are both implemented as 2-layer residual MLPs. Similarly, the deformation network Fω and Fϖ are implemented as single 3-layer MLPs. The dimensions of time embedding ft and position embedding fp are 256 and 72, respectively.

Dataset. We evaluate our method on two multi-view video datasets: HyperNeRF [Park et al., 2021b] and Neu3D [Li et al., 2022]. HyperNeRF consists of videos captured using two phones rigidly mounted on a handheld rig, featuring four distinct scenes. Neu3D contains five distinct scenes, each comprising 20 multi-view videos. Data preprocessing follows the procedures outlined in [Wu et al., 2024].

Baselines. We compare our method against SOTA approaches from four categories. For NeRF-based methods, we include Nerfies [Park et al., 2021a], HyperNeRF [Park et al., 2021b], TiNeuVox-B [Fang et al., 2022], DyNeRF [Li et al., 2022], NeRFPlayer [Song et al., 2023], HexPlane [Cao and Johnson, 2023], MixVoxels [Wang et al., 2023], and HyperReel [Attal et al., 2023]. For 3DGS-based compression methods, we compare with 3DGS [Kerbl et al., 2023] and CompGS [Liu et al., 2024b]. In addition, we use 3DGStream [Sun et al., 2024], Real-Time4DGS [Yan et al., 2024], and STG [Li et al., 2024] as baselines for direct modeling of 4DGS methods. Finally, for deformation-based 4DGS methods, we benchmark against D3DGS [Yang et al., 2024c], DN4DGS [Lu et al., 2024a], 4DGaussian [Wu et al., 2024], and E-D3DGS [Bae et al., 2025].

Metrics. We report the quantitative results of rendered images using peak-signal-to-noise ratio (PSNR), structural similarity index (SSIM) [Wang et al., 2004], perceptual quality measure LPIPS [Zhang et al., 2018] and rendering speed (FPS). Model size is also reported to assess compression efficiency. Three bitrate points are displayed for each dataset due to space limitations. Moreover, rate-distortion (RD) performance is included to ensure a fair comparison.

4.2 Experimental Results

Quantitative results. The quantitative results presented in Tables 1, 2 and Figure 5 reveal consistent findings across all datasets: 1) our proposed ADC-GS achieves SOTA compression efficiency across all 4DGS methods, offering up to 32× storage reduction on the HyperNeRF dataset compared to deformation-based 4DGS while maintaining excellent rendering quality. This improvement stems from organizing Gaussian primitives into anchor-driven residual forms, effectively mitigating intra-redundancy among adjacent primitives. Moreover, entropy loss leverages the MEM model to compress anchors, further enhancing storage efficiency. 2) All NeRF-based methods render at extremely low speeds due to their computationally intensive volumetric rendering process. Although direct modeling 4DGS methods achieve the highest FPS, they demand 204× more storage compared to ADC-GS. Deformation-based methods demonstrate better convergence but still suffer from low FPS due to per-Gaussian deformation strategy. In contrast, our proposed anchor-driven deformation method delivers rendering speeds up to 3× faster than deformation-based methods, even surpassing some direct modeling 4DGS approaches. 3) ADC-GS only needs about 5MB of storage for both datasets, making it highly suitable for real-time transmission and practical applications.

Figure 7 presents a qualitative comparison between ADCGS and other approaches. The rendered images produced by ADC-GS exhibit unperceived distortion while saving lots of storage. Moreover, our method demonstrates clearer textures in certain datasets, attributed to the temporal significancebased anchor refinement that captures finer details, such as Flame Steak in Neu3D.

Bitstream. Our bitstream is summarized as four components: reference feature fv, residual feature fg, explicit Gaussian attributes (comprising covariance Σv, color Cv and position Xv) and networks F. Among these, the networks are directly stored in 32 bits, Xv is compressed using G-PCC, and others are encoded using the AE entropy codec [Witten et al., 1987] with estimated probabilities from our MEM model. As demonstrated in Figure 6, fg constitutes the largest portion of the total bitstream, underscoring the importance of employing the MEM to effectively reduce channel-wise redundancy. The explicit Gaussian attributes account for a small portion of the bitstream, approximately 2%. When analyzing the bit allocation of each component specifically, they are 0.60MB, 3.10MB, 0.15MB (0.10MB, 0.03MB, 0.02MB) and 1.47MB on Neu3D with λe = e - 3, respectively. Notably, a smaller λe results in a larger bitstream allocation for features to enhance rendering quality.

4.3 Ablation Study

In this subsection, we evaluate the effectiveness of each component in our framework on the HyperNeRF dataset. As shown in Table 3 and Figure 8, the framework's performance improves progressively with the addition of each module, demonstrating their individual and collective contributions.

The base configuration includes only the initialized canonical space, achieving a significant size reduction compared to existing 4DGS methods [Wu et al., 2024; Lu et al., 2024a]. While this setup provides high rendering speed due to the absence of deformation, the reliance on canonical space to represent the entire scene results in blurred rendering, particularly in motion-heavy regions. The addition of the coarse deformation module significantly improves the ability to capture the global motion structure. Although the rendering speed decreases compared to the baseline, it remains considerably faster than the per-Gaussian deformation approach in [Bae et al., 2025], showcasing the efficiency of the anchordriven strategy. The fine deformation module builds upon the coarse stage by refining appearance attributes, particularly in regions requiring higher detail. This refinement improves rendering quality by accurately modeling subtle temporal variations in color and opacity. The introduction of the rate-distortion optimization module leads to a remarkable reduction in bitstream size, from approximately 18 MB to 4.5 MB, without compromising rendering quality. This improvement can be attributed to the MEM module's ability to learn compact anchor representations by effectively balancing bitrate consumption and reconstruction fidelity. Finally, the anchor refinement module further improves the representation by dynamically growing and pruning anchors to better capture fine details. This adaptive strategy ensures that the model maintains high-quality reconstruction in complex areas while optimizing the anchor distribution across the scene.

Overall, each module contributes to achieving a balanced trade-off between compression efficiency, rendering speed, and dynamic scene reconstruction quality, resulting in a compact and effective representation of the scene.

5 Conclusion

This paper introduces Anchor-Driven Deformable and Compressed Gaussian Splatting (ADC-GS), a novel method for dynamic scene reconstruction. By organizing Gaussian primitives in an anchor-based compact format within the canonical space, ADC-GS effectively minimizes the intra-redundancy between adjacent primitives. For dynamic scene modeling, we introduce a coarse-to-fine pipeline based on anchors instead of traditional per-Gaussian deformation, significantly reducing deformation redundancy. Meanwhile, a multidimension entropy model is employed to estimate the bitrate costs of anchors, serving as the factor for rate-distortion optimization. To further enhance details, we employ a temporal significance-based anchor refinement strategy. Extensive experiments demonstrate the effectiveness of ADC-GS and its technical components. Overall, our approach not only outperforms deformation-based 4DGS methods in rendering speed but also achieves SOTA storage efficiency without compromising rendering quality.

Acknowledgments

This paper is supported in part by National Natural Science Foundation of China (62371290), National Key R&D Program of China (2024YFB2907204), the Fundamental Research Funds for the Central Universities of China, and STCSM under Grant (22DZ2229005). The corresponding author is Yiling Xu(e-mail: yl.xu@sjtu.edu.cn).

Contribution Statement

He Huang and Qi Yang contributed equally to this work. Yiling Xu is the corresponding author.